Hey as the question states we need historical data to produce certain graphs by executing scripts based on historical data, apparently i was under the impression solana devnet and mainnet api serve historical data, it seems this is not the case atleast not for devnet Do there exist rpc services which gurantee such service if not what is the solana way of keeping tx history without running our own validator or rpc node?
It's not generally practical for an RPC node to store the full ledger history from genesis locally. Presently this is handled by so-called "warehouse nodes," run by the Solana Foundation, most RPC service providers and many other ecosystem participants with an interest in ensuring transaction history is available for their services. Warehouse nodes usually maintain one or both of:
- A BigTable instance: Which stores block data in a format suitable for serving RPC requests.
- A ledger archive: Which uploads raw ledger and accountsdb snapshots suitable for replay from scratch in the future
Most RPC service providers offer pools that are backed by a BigTable instance that goes back to genesis for mainnet beta. Devnet and testnet histories are not likely contiguous as they are second-class-citizens in this regard
I'll try to answer this and your other question (How do validators maintain valid state without having the full state from genesis) in one go.
Solana network does not hold all historical data since genesis. Some historical data is kept off-chain by validators but I doubt it would be since genesis.
Apparently, Solana and Arweave struck some sort of deal to offload historical data to Arweave. This is done through what is called the Solar Bridge. I have found so very little information about this that I am starting to think this is vaporware (happy to be corrected by someone in the know).
In short, if Solar exists and is accessible, that would be the answer. Otherwise, unless some crafty validator has been keeping tabs since the beginning, then that data does not exist.