Best way to keep track of all the PDAs that has been created by my program in Solana

Question

Let's say I have something like Twitter, where a post can be liked/unliked, and you can see all the people that have liked the post.

Coming from Solidity, we can easily do this with an address[] that inserts the user's address if they like or removes if they unlike.

With Solana, I've learnt that we should use PDA for this case, and I think it can work to a certain extend (user knows if they like or dislike a post), but for the "query all users that like this post" requirement, I have not found the solution. To know it, we need to keep track of all the PDAs that have been created, but afaik Solana's Vec<PubKey> cannot have dynamic size (or in other words it can only contains a fixed number of PubKey). Unfortunately with that limitation it won't work for this case.

What should I do in this case? I'm pretty lost. Thank you in advance.

EDIT: If possible, I also want to query this kind of data on-chain (in my program).

Answer 1

You can use getProgramAccounts with filters. Here's Solana cookbook reference: https://solanacookbook.com/guides/get-program-accounts.html#filters

Here's an excellent blog specifically focused on querying tweets from a Solana program that replicates Twitter: https://lorisleiva.com/create-a-solana-dapp-from-scratch/fetching-tweets-from-the-program

The basic idea is that you use the memcmp filter (standing for memory comparison) to compare certain data fields stored in accounts. The filter would depend on how you have structured your accounts and program. For example if I have a user account and a tweet account in my program. And I want to fetch all tweets by a particular user, I can do something like:

const data = await program.account.tweetAccount.all([
  {
    memcmp: {
      offset: 8,
      bytes: anchor.utils.bytes.bs58.encode([5138]),
    },
  },
  {
    memcmp: {
      offset: 8 + 4 // memory size of the `owner` or similar field,
      bytes: new anchor.web3.PublicKey("user_pub_key").toBase58(),
    },
  },
])

Answer 2

If you really need to get the complete list of all users that liked the post on-chain, you might need to store all the pubkeys in a single account(i.e a likes(Vec<Pubkey>) field) and extend the account size if needed. You might need to keep track of the current capacity of the likes vector, or else calculate it dynamically from the account data's size whenever you need it. This is because Vec::capacity will always just return the size of the deserialized vector, essentially Vec::len.

Hence you'd just have to check for when capacity == length and then extend the account using the steps shown here.

An alternate way to keep track of the capacity is to always fill up uninitialized keys in likes with Pubkey::default so Vec::capacity gives the actual capacity. You'd then find and replace a default pubkey to insert an actual one, and know the capacity is filled up when there's no default to replace.

You can also just extend the account's size by 32 bytes for each like and make the user pay for their space. This would mean you don't need to track capacity as it's always used up.

P.S: I'm skeptical that you would need this information on-chain though. For most cases, the canonical way to handle stuff like this would be to use pdas, retrieve all pdas using the getProgramAccounts call on the client side, and only validate those accounts in your on-chain code(you might consider a merkle or hash-chain solution, but this might be expensive wrt compute). Do you mind sharing your use-case in this scenario?

EDIT: Based on additional context provided, it seems like what you'd need is a pda for each user that tracks how much SOL they purchased, and a shared pda for all users that tracks how much SOL has been purchased in total. Then you wouldn't need to always store and update the equivalent token amount, as how much tokens a user is entitled do be dynamically calculated during the claim/distribution step using the information outlined above.

If there's multiple rounds of claim/distribution, you should also consider adjusting your program to take previously claimed tokens into account in your calculations to prevent a double-allocation.

Answer 3

You actually can resize accounts.

How can I increase an existing account size?

Solana uses a Vec<(String, String)> for the additional_metadata of the TokenMetadata token22 token-metadata interface.

Link to the code: solana token-metadata interface additional_metadata github

It is recommended to put a dynamic sized data at the end of the account struct so any other data can be easily filtered, because once you get to vector data you have to do more work for filtering since the size will be dynamic.

I don't know how your Post account data looks, but including a vector of dynamic data every time you fetch a post doesn't feel like a great way to handle this. Instead I would add a couple more accounts PostLikes and PostDislikes which will be PDA's derived with ['post-likes', post.key()]. The likes and dislikes can be a counter stored in the post so you can still quickly get the amount of likes and dislikes without needing to get all of the pubkeys of the likers/dislikers. Then inside of PostLikes and PostDislikes you can have the Vec<Pubkey>.

Now if you want to get the number of likes/dislikes, just get the post data. If you want to get the pubkeys of the likes/dislikes, use the post to get the pda for the PostLikes/PostDislikes account.

Keep in mind account size and rent costs. A Pubkey is 32 bytes. 100,000 likes would be at least 3,200,000 bytes which is about 22 sol at current rent prices, which could be paid for and expanded by you as needed, or paid for by the user when they like the post.

Edit: Also note that accounts actually do have a max limit still. So if you really need 100,000 all in one account to be checked on-chain, you won't be able to use a PDA. If you don't need them all in one account, you can chunk them up and still use PDA's as well as another account keeping track of which chunk a likers/dislikers Pubkey is in. Just brainstorming here though, as it really comes down to what exactly you are trying to do.

At the time of this writing:

• Accounts have a max size of 10MB (10 Mega Bytes)
• PDA accounts have a max size of 10KB (10 Kilo Bytes)

solana cookbook account facts