Hey as the question states we need historical data to produce certain graphs by executing scripts based on historical data, apparently i was under the impression solana devnet and mainnet api serve historical data, it seems this is not the case atleast not for devnet Do there exist rpc services which gurantee such service if not what is the solana way of keeping tx history without running our own validator or rpc node?
-
i dont exactly need timestamp or account infos, but only program logs emitted out at those timestamp is there no rpc service or provider providing this currently if not how to dapps on solana accomplish this ?– Nabeel NaveedCommented Jul 22, 2022 at 12:19
Add a comment
|
2 Answers
It's not generally practical for an RPC node to store the full ledger history from genesis locally. Presently this is handled by so-called "warehouse nodes," run by the Solana Foundation, most RPC service providers and many other ecosystem participants with an interest in ensuring transaction history is available for their services. Warehouse nodes usually maintain one or both of:
- A BigTable instance: Which stores block data in a format suitable for serving RPC requests.
- A ledger archive: Which uploads raw ledger and accountsdb snapshots suitable for replay from scratch in the future
Most RPC service providers offer pools that are backed by a BigTable instance that goes back to genesis for mainnet beta. Devnet and testnet histories are not likely contiguous as they are second-class-citizens in this regard
-
how does the end user verify that the data provided by a ware-house node is valid , i assume some form of compressed proof of the data is stored on chain, for example ethereum will store merkle roots only of historical data, is there a similar feature on solana to help verify data retrieved via big tables via ticks ? Commented Jul 22, 2022 at 22:40
-
-
@trent.sol, do you have technical details on how to access these "warehouse nodes" run by Solana Foundation? Are they available to the public?– sohrabCommented Jul 23, 2022 at 2:40
-
the warehouse nodes themselves don't typically have anything to "access" directly. like i said in the answer, they simply follow the chain tip and populate other data stores which may or may not back other services. again, as mentioned in the answer, BigTable instances are typically accessible transparently via RPC Commented Jul 23, 2022 at 3:17
I'll try to answer this and your other question (How do validators maintain valid state without having the full state from genesis) in one go.
Solana network does not hold all historical data since genesis. Some historical data is kept off-chain by validators but I doubt it would be since genesis.
Apparently, Solana and Arweave struck some sort of deal to offload historical data to Arweave. This is done through what is called the Solar Bridge. I have found so very little information about this that I am starting to think this is vaporware (happy to be corrected by someone in the know).
In short, if Solar exists and is accessible, that would be the answer. Otherwise, unless some crafty validator has been keeping tabs since the beginning, then that data does not exist.
