Why are the static account and instruction lists prefixed with a compact-u16 length instead of a simple u8?

Question

The header of a Solana transaction contains, among other things:

• a compact-array of account addresses
• a compact-array of instructions

A compact array is defined as follows:

A compact-array is serialized as the array length, followed by each array item. The array length is a special multi-byte encoding called compact-u16.

Compact-u16s exist so that you can express 16-bit values but save space whenever a value can be expressed in 7 bits instead (explanation).

Why is the length of the static account and instruction lists expressed as a compact-u16 (65536 options) instead of a u8 (256 options) when we can't even fit 256 accounts or instructions into a transaction to begin with?

Answer 1

A compact-u16 uses the same amount of space when storing a u8 < 128, so this is just future-proofing at the moment and we can think of the u16 as a u8. For example, if you want to store:

0000 0000 0010 1111

The compact u16 will be:

0010 1111

However if we wanted to store 129 or larger, we'd need to set the continuation bit and use the next byte:

000 0000 1000 0001

1000 0001 then 0000 0001

I might have flipped the bit-endianness but you get the idea. The next byte is only used if the last bit in the first byte has the "flag" set to 1.

The Solana network uses an MTU of 1280 bytes which leaves 1232 after headers, so you are correct that fits only ~38 keys that are 32 bytes each in the best case, and with overhead from signatures, etc in practice it's much less.

The MTU was chosen for faster network forwarding purposes, as this is the minimum/suggested IPv6 standard that will currently result in no fragmenting. Some networks will just discard fragmented traffic, and most will send it more slowly as they forward it, so this was an important compatibility choice. However that can change at any time, e.g. 1500 MTU is already pretty popular, and Solana could always allow larger packets with fragmenting in the future as well. More on big MTUs: https://blog.cloudflare.com/increasing-ipv6-mtu/