Faster method to retrieve pool keys [Raydium]

Question

Currently I am downloading the whole json file from here https://api.raydium.io/v2/sdk/liquidity/mainnet.json and save it into json file, then get the pool info. Code given below for further understanding:

def fetch_pool_keys(mint: str):
    amm_info = {}
    all_pools = {}
    try:
        # Using this so it will be faster else no option, we go the slower way.
        with open('all_pools.json', 'r') as file:
            all_pools = json.load(file)
        amm_info = extract_pool_info(all_pools, mint)
    except:
        resp = requests.get('https://api.raydium.io/v2/sdk/liquidity/mainnet.json', stream=True)
        pools = resp.json()
        official = pools['official']
        unofficial = pools['unOfficial'] 
        all_pools = official + unofficial

        # Store all_pools in a JSON file
        with open('all_pools.json', 'w') as file:
            json.dump(all_pools, file)
        try:
            amm_info = extract_pool_info(all_pools, mint)
        except:
            return "failed"

As you can see, I first check it in the downloaded file but if the key which is base mint and quote mint (SOL/USDC/USDT) is not there, then it downloads the file again.

My main concern: At the time of sending request, the code takes longer to download mainnet.json file and as we all know raydium updates the pools every minute or so. Due to which, I am asking for suggestions to speed up the download process or implement an alternative method to downloading mainnet.json

Answer 1

For a faster way to grab those pool keys on Raydium, especially if you're targeting Raydium pairs, the trick is to hit up the programAccounts directly. Then, just decode what you get with the layouts Raydium SDK hands out. I usually check both OpenBook and Raydium Market accounts for this.

import {
  LIQUIDITY_STATE_LAYOUT_V4,
  MARKET_STATE_LAYOUT_V3,
} from "@raydium-io/raydium-sdk";

// Define a function to fetch and decode OpenBook accounts
async function fetchOpenBookAccounts(connection, baseMint, quoteMint, commitment) {
  const accounts = await connection.getProgramAccounts(
    networkData.openbookProgramId,
    {
      commitment,
      filters: [
        { dataSize: MARKET_STATE_LAYOUT_V3.span },
        {
          memcmp: {
            offset: MARKET_STATE_LAYOUT_V3.offsetOf("baseMint"),
            bytes: baseMint.toBase58(),
          },
        },
        {
          memcmp: {
            offset: MARKET_STATE_LAYOUT_V3.offsetOf("quoteMint"),
            bytes: quoteMint.toBase58(),
          },
        },
      ],
    }
  );

  return accounts.map(({ account }) => MARKET_STATE_LAYOUT_V3.decode(account.data));
}

// Define a function to fetch and decode Market accounts
async function fetchMarketAccounts(connection, base, quote, commitment) {
  const accounts = await connection.getProgramAccounts(
    networkData.marketProgramId,
    {
      commitment,
      filters: [
        { dataSize: LIQUIDITY_STATE_LAYOUT_V4.span },
        {
          memcmp: {
            offset: LIQUIDITY_STATE_LAYOUT_V4.offsetOf("baseMint"),
            bytes: base.toBase58(),
          },
        },
        {
          memcmp: {
            offset: LIQUIDITY_STATE_LAYOUT_V4.offsetOf("quoteMint"),
            bytes: quote.toBase58(),
          },
        },
      ],
    }
  );

  return accounts.map(({ pubkey, account }) => ({
    id: pubkey.toString(),
    ...LIQUIDITY_STATE_LAYOUT_V4.decode(account.data),
  }));
}

Just plug in your connection details, the mint addresses you're interested in, and you're good to go. This should make your work with Raydium pairs faster and more efficient.

Answer 2

You could cache the pool keys in a nextjs app in the backend or in a database and add an api that only sends the keys you need. And then refresh the keys from the raydium json every hours or so asynchronously.

Answer 3

I just customised the Raydium-sdk to get the poolAddress from dexscreener and pass it to get the poolkeys to execute the swap.

You can try this: https://github.com/TrasherTravis/raydium-sdk-swap-no-mainnet.json-required.git

Answer 4

I was facing the same problem. Adding the repo, how I solve the problem.

Answer 5

You can now use @raydium-io/raydium-sdk-v2 and easily access it like so

const data = await raydium.api.fetchPoolByMints({ mint1: 'So11111111111111111111111111111111111111112', mint2: '4k3Dyjzvzp8eMZWUXbBCjEvwSkkk59S5iCNLY3QrkX6R' // optional })

https://www.npmjs.com/package/@raydium-io/raydium-sdk-v2