Premature close error when fetching 25 or more accounts at once with @solana/web3.js connection.getAccountInfo

Question

Overview

Using the Connection class' method getAccountInfo with Promise.all and at least 25 requests will fail with:

Error: failed to get info about account Cfuy5T6osdazUeLego5LFycBQebm9PP3H7VNdCndXXEN: **FetchError: Invalid response body while trying to fetch <redacted RPC url> Premature close**
    at Connection.getAccountInfo (/home/dev/sandbox/test/node_modules/@solana/web3.js/src/connection.ts:3517:13)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)
    at async Promise.all (index 12)

I understand this is inefficient; ideally, I should use getMultipleAccounts instead, but that is not always possible. I am using multiple SDKs from different protocols that do not use getMutlipleAccounts method and I have to rely on fetching multiple accounts with Promise.all concurrency to make it faster. We use a non-rate-limited RPC in our backend applications so it should not be an issue.

Steps to reproduce

Using "@solana/web3.js": "1.75.0" version (also fails on 1.73.0, haven't tested on others):

import {Connection, PublicKey} from "@solana/web3.js";

const connection = new Connection(
    "https://api.mainnet-beta.solana.com");

const promises = [];
for (let i = 0; i < 26; i++) {
    promises.push(connection.getAccountInfo(new PublicKey('Cfuy5T6osdazUeLego5LFycBQebm9PP3H7VNdCndXXEN')));
}
Promise.all(promises).then(x => console.log(x.length)); // <-- this call will fail with the error described

To reproduce the error you can clone this repo: https://github.com/peroxy/solana-web3.js-premature-close-bug

and run commands:

    yarn install
    yarn start

I would expect this to work correctly and return all the account infos instead of failing with a very cryptic error.

Answer 1

This is because of an unpatched bug with chunked responses in node-fetch. You can read all about it here: https://github.com/node-fetch/node-fetch/issues/1767

You can work around the issue by disabling the HTTP agent in your Connection, though that might cause other performance problems.

const connection = new Connection('https://...', {httpAgent: false});