7

I'm having trouble reading data in the client from a program with a zero copy account. I've defined the account as follows:

#[account(zero_copy)]
#[repr(packed)]
pub struct EventQueue {
    header: EventQueueHeader,
    head: u64,
    buf: [Event; 100], // Used zero_copy to expand eventsQ size
}

The data appears to be shifted via padding and is appearing incorrect in the client when fetched as follows (note without zero copy, data is correctly fetched).

const eventQ = await program.account.eventQueue.fetch(eventQPda);
console.log(eventQ);

I've tried using #[repr(packed)] as per this question[https://solana.stackexchange.com/questions/2861/how-to-decode-anchor-zero-copy-account-with-idl], but the output is still wrong.

The on-chain values are being set and read correctly, and the program is able to utilise values stored in the EventQueue Account. Any thoughts on the best way to do this?

Edit:

Some additional info - EventQueue has an alignment of 0x10, based on Event which also has an alignment of 0x10. EventQueueHeader has an alignment of 0x8.

As suggested by one of the answers, I attempted to update the eventQueue to manually align with the most-significant alignment as follows: #[account(zero_copy)] #[repr(C)] pub struct EventQueue {

header: EventQueueHeader,
pub _reserved0: [u8; 8],

head: u64,
pub _reserved1: [u8; 8],
buf: [Event; 100], // Used zero_copy to expand eventsQ size

}` However, the misalignment persists. Any further suggestions? Also, could it be that I need to manually align the Event struct as well?

2
  • 2
    Have you tried #[repr(C)]? It's generally easier to handle padding/alignment issues with C.
    – Whiteseal
    Jul 12, 2023 at 15:56
  • Whats in the EventQueueHeader the only things that are serialized differently with different repr types are enums and option types afaik.
    – Jonas H.
    Jul 21, 2023 at 20:49

2 Answers 2

3
+25

tl;dr use repr(C) and manually align the struct to the most-significant alignment.

The problem is likely a struct alignment issue, which commonly manifests as fetch failing due to bad serialization or returning nonsense.

Consider this struct:

// Actual Size = 2 + 6 + 18_724 * 56 + 24 = 1_048_576
#[account(zero_copy)]
#[repr(C)]
pub struct AuditLog {
    pub next_slot: u16,
    pub in_order: bool,
    pub _reserved0: [u8; 5],
    pub log: [AuditEntry; 18_724],
    pub _reserved1: [u8; 24],
}

// Size 56 = 32 + 8 + 8 + 8
#[derive(AnchorSerialize, AnchorDeserialize, Clone, Copy)]
#[repr(C)]
pub struct AuditEntry {
    pub user: Pubkey,
    pub timestamp: i64,
    pub total_cost: u64,
    pub total_proceeds: u64,
}

Note that AuditEntry is aligned to 0x8 bytes (if you have Rust Analyzer in VS Code, hover over a struct to see its alignment). There's a 32-byte field (a multiple of 8), then three 8-byte values.

AuditLog is also aligned to 8 bytes, because AuditEntry is, and the highest alignment is used. If we didn't have _reserved0, it would fail to deserialize correctly, because next_slot is 2 bytes, then in_order is 1, we must manually reach the next 8-byte multiple, hence the use of the _reserved0 buffer that is 5 bytes large. The next field, log, aligns to 8 bytes, and so does the _reserved1 buffer at the end, so those are fine.

If we removed the AuditEntry array from AuditLog so it looked like this:

#[account(zero_copy)]
#[repr(C)]
pub struct AuditLogAlt {
    pub next_slot: u16,
    pub in_order: bool,
    pub _reserved0: [u8; 5],
    pub _reserved1: [u8; 24],
}

now it aligns to 0x2 bytes (the u16 dictates the alignment here). Here we could make _reserved0 a size 1 array, and it would still align correctly. If next_slot was u32, it would align to 0x4 bytes, or 0x8 if it was u64, and paradoxically u128 is 0x8 too (thanks compiler magic).

Now for your example, it depends on what the highest alignment is, which will probably come from EventQueueHeader or Event. You likely need to add padding between header and head and/or between head and buf to meet that alignment. For example, if EventQueueHeader aligns 16, and Event aligns 8, you would make it:

#[account(zero_copy)]
#[repr(C)]
pub struct EventQueue {
    header: EventQueueHeader,
    head: u64,
    pub _reserved0: [u8; 8],
    buf: [Event; 100], // Used zero_copy to expand eventsQ size
}

Or if Event and EventQueueHeader and both aligned to 0x4 bytes:

#[account(zero_copy)]
#[repr(C)]
pub struct EventQueue {
    header: EventQueueHeader,
    pub _reserved0: [u8; 4],
    head: u64,
    buf: [Event; 100], // Used zero_copy to expand eventsQ size
}

And so forth...

7
  • If you used #[repr(C)], why do you need the fields _reserved0 and _reserved1? You don't need these fields (and manual alignments) because there will be padding anyway because you used #[repr(C)]. If you had used #[repr(packed)], then what you did makes sense, because packed would remove the padding between members and some members such as AuditEntry would start at non-aligned memory! Aug 14, 2023 at 12:15
  • If you use [repr(C)] you don't get any automatic padding added, so you must manually align your values to the highest alignment of the struct. The reason is that the Anchor zero_copy implementation DOES add padding, and in my experience it doesn't always add the same padding as #[repr(packed)], because it uses a custom implementation with [repr(C)] (book.anchor-lang.com/anchor_references/space.html) that isn't exactly what you would expect from a POD type.
    – Whiteseal
    Aug 14, 2023 at 19:22
  • "The reason is that the Anchor zero_copy implementation DOES add padding". What is the basis of this claim? Can you show me the proof? Also, do you know what #[repr(C)] actually is? You're claiming too much without even verifying your assumptions. Aug 14, 2023 at 22:39
  • Or, can you tell me the difference between #[repr(C)] and #[repr(packed)]? Aug 15, 2023 at 5:40
  • "in my experience it doesn't always add the same padding as #[repr(packed)]". Do you even understand what you're saying here? Do you even understand what #[repr(packed)] does? It does NOT add any padding. Its name itself screams this fact. It's aptly named PACKED means no padding, no space between members. Aug 15, 2023 at 7:48
2

To be honest, I'm surprised that #[repr(packed)] isn't working, since that's exactly what you're looking to do. You have a couple of options:

  • update your client-side deserialization -- when you use zero-copy, there may still be padding introduced between fields and instances in the array. Normally, #[repr(packed)] should take care of that, but it can be finnicky. So just update the client to take padding into account.
  • remove all padding in your type to be sure that everything aligns to 1. For example, u64 aligns to 8, so if EventQueueHeader aligns to something smaller, then there will be padding between the two fields. You can get around this by using Pod-safe types, like PodU64 which is just a byte array under the hood. Here's an example doing that for u32 https://github.com/solana-labs/solana-program-library/blob/db5e69edb14e626e294f9f32e2526757b8605774/libraries/type-length-value/src/pod.rs#L48. The same applies to Event, since there can also be padding between array instances.

I believe Anchor uses bytemuck under the hood, and you can read more about its Pod types at https://docs.rs/bytemuck/latest/bytemuck/trait.Pod.html

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.