Table of Contents

Portable layout cookbook

This cookbook collects short layout patterns. For complete programs you can run, choose from the tested recipe catalog. Each pattern here is backed by either the compiled documentation runner or a named pair in manual-fixtures-v1.json, executed on .NET 8 and .NET 10.

Before copying one, replace widths, byte order, placement, pointer coordinates, and limits with facts from your format. Similar-looking bytes do not prove that two formats share the same rules.

Decode a packed fixed header

Use explicit-width primitives in declaration order:

struct header {
    uint16 kind;
    uint32 length;
};

Little-endian bytes 02 00 06 00 00 00 decode as kind 2 and length 6. Packed size is 6. In aligned placement, length would move to offset 4 and total size would become 8.

private static void DecodeHeader()
{
    var layout = new CStruct("struct header { uint16 kind; uint32 length; };");
    ReadOnlySpan<byte> bytes = [0x02, 0x00, 0x06, 0x00, 0x00, 0x00];
    StructValue header = layout.Parse(bytes, "header");
    Equal((ushort)2, header.Get<ushort>("kind"));
    Equal(6U, header.Get<uint>("length"));

    bool read = layout.TryReadValue<Header>(bytes, "header", out Header? typed);
    True(read && typed is { Kind: 2, Length: 6 }, "Typed header result differed.");
    True(!layout.TryReadValue<Header>(bytes[..1], "header", out _), "Truncated TryReadValue should fail.");
}

Verify the result by checking field widths, the root starting position, and the format's byte order.

Mix byte orders without moving fields

Use < or > only on fields whose encoding overrides the layout order:

struct root {
    uint16> network;
    uint16< device;
};

Bytes 12 34 78 56 produce network=4660 and device=22136. The fields remain at offsets 0 and 2 and total size is 4. The alignment-and-endian-overrides fixture checks this behavior.

Read a payload whose count comes from elsewhere

struct packet {
    uint8 kind;
    uint8 payload[COUNT];
};

private static void RuntimePayload()
{
    var layout = new CStruct("struct packet { uint8 kind; uint8 payload[COUNT]; };");
    var variables = new Dictionary<string, int> { ["COUNT"] = 3 };
    byte[] bytes = [0x7F, 0x10, 0x20, 0x30];
    StructValue packet = layout.Parse(bytes, "packet", variables);
    Equal((byte)0x7F, packet.Get<byte>("kind"));
    Equal(3, packet.Get<byte[]>("payload").Length);
    object? secondPayload = layout.ReadValue(bytes, "packet.payload[1]", variables);
    Equal((byte)0x20, (byte)secondPayload!);

    using var stream = new MemoryStream(bytes);
    stream.Position = 1;
    Equal(3, layout.GetArrayLength(stream, "packet.payload", variables));
    Equal(1L, stream.Position);
}

Pass the same COUNT to every later read, address, length, write, or update. Do not infer the count from remaining stream bytes.

Store fixed-capacity text

Use char[N] or wchar[N] when the format reserves exactly N code units:

struct label {
    char text[4];
};

private static void FixedText()
{
    var layout = new CStruct("struct label { char text[4]; };");
    StructValue value = layout.Parse(new byte[] { 0x41, 0x42, 0x43, 0x00 }, "label");
    Equal("ABC\0", value.Get<string>("text"));
    SequenceEqual(
        [0x58, 0x59, 0x00, 0x00],
        layout.Serialize("label", new Dictionary<string, object?> { ["text"] = "XY" }));
}

Writing "XY" produces 58 59 00 00. This is not a terminated scan; a longer value fails.

Put terminated text before another field

Use a strict named handler:

struct root {
    utf8_string_zero name;
    uint8 flags;
};

Bytes 41 00 7E produce name="A" and flags=126 at offset 2. Set MaxStringBytes so a missing terminator cannot scan beyond the format's expected maximum. The terminated-strings and bounded-failures pairs check success and a limit failure.

Preserve an enum number with no name

enum state : uint32 {
    Known = 1
};

struct root {
    state value;
};

private static void PreserveEnum()
{
    var layout = new CStruct("enum state : uint32 { Known = 1 }; struct root { state value; };");
    var value = (EnumValueResult)layout.ReadValue(new byte[] { 0xFF, 0xFF, 0xFF, 0xFF }, "root.value")!;
    Equal(new BigInteger(uint.MaxValue), value.Value);
    Equal(null, value.Name);
    Equal(32, value.BitWidth);
    True(!value.IsSigned, "uint32 enum should be unsigned.");
}

Check EnumValueResult.Name; when it is null, retain Value and RawBits instead of narrowing through int.

Preserve or choose a union member

union choice {
    uint8 small;
    uint16 large;
};

/// <summary>Reads every view of a union from the same storage bytes.</summary>
private static void PreserveUnion()
{
    var layout = new CStruct("union choice { uint8 small; uint16 large; };");
    UnionValue parsed = layout.ReadValue<UnionValue>(new byte[] { 0x34, 0x12 }, "choice");
    Equal("choice", parsed.UnionName);
    Equal((ushort)0x1234, (ushort)parsed.Members["large"]!);
    SequenceEqual([0x34, 0x12], layout.Serialize("choice", parsed));

    UnionValue selected = UnionValue.FromMember("choice", "small", (byte)0xA5);
    SequenceEqual([0xA5, 0x00], layout.Serialize("choice", selected));
}

Untouched raw storage 34 12 survives exactly. Selecting small=0xA5 starts from cleared two-byte storage and produces A5 00.

Decode compact flags

struct flags {
    uint8 low : 3;
    uint8 high : 5;
    uint16 next;
};

Bytes 8D 34 12 produce low 5, high 17, and next 4660. Both slices have byte offset 0. Confirm native C bitfield ABI before translating an existing header; Portable always allocates from the low bit.

Follow a relative pointer

Choose pointer width from the format and configure address mode:

private static void FollowPointer()
{
    var layout = new CStruct("struct root { uint8 *target; };", pointerSize: 1);
    using var stream = new MemoryStream([0x01, 0x2A]);
    StructValue root = layout.Parse(stream, "root");
    Pointer pointer = root.Get<Pointer>("target");
    Equal(1L, pointer.Address);
    True(pointer.IsDereferenced, "Pointer should be followed by default.");
    Equal((byte)0x2A, (byte)pointer.Value!);
}

For a relative format, use AddressingMode = Relative and the correct Origin. Zero stays null. Nonzero targets must fit the stream/memory region and configured traversal limits. Serialization writes a coordinate and does not relocate target data.

Paste a header as it is

A Windows SDK, Linux kernel, or dissect.cstruct definition compiles unchanged: the alias spellings are built in, typedef struct _X { ... } X, *PX; declares the tag and both aliases, a tagged inline union is a global type whose unnamed member is promoted, and _ fields are padding that never reach the result:

private static void WindowsHeader()
{
    // A header pasted from a Windows SDK or a dissect definition: SDK spellings, a repeated `_` padding
    // field, a tagged inline union whose members are promoted, and a pointer-to-string alias.
    const string definition = """
        typedef struct _RECORD {
            DWORD   Magic;
            WORD    Version;
            WORD    _;
            union version_information {
                DWORD Packed;
                struct { BYTE Major; BYTE Minor; WORD Build; };
            };
            DWORD   _;
            PWSTR   Name;
        } RECORD, *PRECORD;
        """;
    var layout = new CStruct(definition, pointerSize: 4, aligned: true);
    byte[] bytes = [0x4D, 0x5A, 0x00, 0x00, 0x02, 0x00, 0xFF, 0xFF, 0x0A, 0x00, 0x39, 0x30, 0xEE, 0xEE, 0xEE, 0xEE, 0x00, 0x00, 0x00, 0x00];
    StructValue record = layout.Parse(bytes, "RECORD");
    Equal(0x5A4DU, record.Get<uint>("Magic"));
    Equal((ushort)2, record.Get<ushort>("Version"));
    Equal((byte)10, record.Get<byte>("Major"));
    Equal((ushort)12345, record.Get<ushort>("Build"));
    Equal(0x3039000AU, record.Get<uint>("Packed"));
    True(!record.ContainsKey("_"), "padding is not a member");

    // The tag, the alias, and the pointer alias all name the same declaration; the promoted union writes
    // back through the member the data supplies and the padding as zeroes.
    Equal(20, layout.GetStructSizeInBytes("_RECORD"));
    byte[] written = layout.Serialize("RECORD", new Dictionary<string, object?> { ["Magic"] = 0x5A4DU, ["Version"] = (ushort)2, ["Packed"] = 0x3039000AU, ["Name"] = 0U });
    SequenceEqual([0x4D, 0x5A, 0x00, 0x00, 0x02, 0x00, 0x00, 0x00, 0x0A, 0x00, 0x39, 0x30, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00], written);
}

Only the long width (CLongWidth), the storage of an enum without a backing type (DefaultEnumStorage), and big-endian bitfield order (BitfieldAllocation) are choices a header leaves to the compiler; the migration guide lists the reading each option selects.

Read flags and data-terminated arrays

A flag decomposes into member names, entry entries[] reads until an all-zero element, and uint16 trailer[EOF] takes every whole element to the end of the input:

private static void FlagsAndDataSizedArrays()
{
    // A flag reads as a decomposed name list; `[]` on a struct is an array terminated by an all-zero
    // element (the exFAT/APFS habit) and `[EOF]` takes every whole element that remains.
    const string definition = """
        flag access : uint16 { READ, WRITE, EXEC, HIDDEN = 0x100 };
        struct entry { uint8 kind; uint8 size; };
        struct root {
            access mode;
            entry entries[];
            uint16 trailer[EOF];
        };
        """;
    var layout = new CStruct(definition);
    byte[] bytes = [0x05, 0x01, 1, 10, 2, 20, 0, 0, 0x34, 0x12, 0x78, 0x56];
    StructValue root = layout.Parse(bytes, "root");
    FlagValueResult mode = root.Get<FlagValueResult>("mode");
    Equal("READ|EXEC|HIDDEN", string.Join("|", mode.Names));
    True(mode.Has("EXEC") && !mode.Has("WRITE"), "EXEC is set and WRITE is not");
    Equal(2, root.Get<IReadOnlyList<object?>>("entries").Count);
    Equal((ushort)0x5678, root.Get<ushort>("trailer[1]"));

    // Writing appends the terminator element and nothing after the read-to-end array; a flag accepts names.
    byte[] written = layout.Serialize(
        "root",
        new Dictionary<string, object?>
        {
            ["mode"] = "READ|HIDDEN",
            ["entries"] = new object[] { new Dictionary<string, object?> { ["kind"] = (byte)3, ["size"] = (byte)30 } },
            ["trailer"] = new ushort[] { 1 },
        });
    SequenceEqual([0x01, 0x01, 3, 30, 0, 0, 0x01, 0x00], written);
}

Keep a header's defines and size expressions

Text and 64-bit #define values are published on CStruct.Constants, #ifdef selects declarations (also through CStructCompilationOptions.Defined), and a count may use %, ?:, sizeof, and offsetof:

private static void HeaderPreprocessor()
{
    // Preprocessor lines and expression forms that appear in real headers: text and 64-bit defines are
    // published as constants, `#ifdef` selects declarations, and counts may use sizeof, offsetof, % and ?:.
    const string definition = """
        #define MAGIC "CD001"
        #define BLOCK_MASK (1 << 40)
        #define LEGACY_VERSION 1
        #ifdef WIDE_COUNTS
        typedef uint32 count_t;
        #else
        typedef uint16 count_t;
        #endif
        struct header { uint8 kind; uint32 length; };
        struct root {
            count_t count;
            uint8 payload[count % 4 == 0 ? sizeof(header) : offsetof(header, length)];
        };
        """;
    var narrow = new CStruct(definition);
    Equal("CD001", narrow.Constants["MAGIC"].Value);
    Equal(BigInteger.One << 40, narrow.Constants["BLOCK_MASK"].Value);
    Equal(LayoutConstantKind.Integer, narrow.Constants["LEGACY_VERSION"].Kind);
    StructValue aligned = narrow.Parse(new byte[] { 4, 0, 1, 2, 3, 4, 5 }, "root");
    Equal(5, aligned.Get<byte[]>("payload").Length);
    StructValue unaligned = narrow.Parse(new byte[] { 3, 0, 1 }, "root");
    Equal(1, unaligned.Get<byte[]>("payload").Length);

    // The same source compiled with a -D style definition takes the other branch and is its own cache entry.
    var wide = CStruct.GetOrCompile(definition, compilationOptions: new CStructCompilationOptions { Defined = new HashSet<string> { "WIDE_COUNTS" } });
    Equal(4, wide.Layout.Declarations.Single(item => item.Name == "root").Fields[0].Size!.Value);
}

Register a codec of your own

A format-specific encoding (a protobuf varint, a SID blob) is an ICustomCodec the layout uses by name, including as an array count:

private static void CustomCodec()
{
    // A caller-defined type: a protobuf-style varint registered under the name a definition uses.
    var codecs = new List<ICustomCodec> { new Varint() };
    var layout = new CStruct(
        "struct entry { varint id; uint8 kind; }; struct root { varint count; entry items[count]; };",
        compilationOptions: new CStructCompilationOptions { Codecs = codecs });
    byte[] bytes = [2, 0x80, 0x01, 7, 0x05, 9];
    StructValue root = layout.Parse(bytes, "root");
    Equal(128UL, root.Get<ulong>("items[0].id"));
    Equal((byte)9, root.Get<byte>("items[1].kind"));
    SequenceEqual(bytes, layout.Serialize("root", root));
    using var stream = new MemoryStream(bytes);
    Equal(4L, layout.ResolveAddress(stream, "root.items[1].id"));
}

private sealed class Varint : ICustomCodec
{
    public string Name => "varint";

    public int? FixedSize => null;

    public int Alignment => 1;

    // The library hands the codec the bytes from the value's start: all remaining input for memory sources,
    // a growing window for other streams. The codec says how many it used.
    public OperationStatus Read(ReadOnlySpan<byte> source, out object? value, out int bytesConsumed)
    {
        ulong result = 0;
        for (int index = 0; index < source.Length && index < 10; index++)
        {
            result |= (ulong)(source[index] & 0x7F) << (7 * index);
            if ((source[index] & 0x80) == 0)
            {
                value = result;
                bytesConsumed = index + 1;
                return OperationStatus.Done;
            }
        }

        value = null;
        bytesConsumed = 0;
        return source.Length >= 10 ? OperationStatus.InvalidData : OperationStatus.NeedMoreData;
    }

    // DestinationTooSmall asks for a larger window; ten bytes always suffice for a 64-bit varint.
    public OperationStatus Write(Span<byte> destination, object value, out int bytesWritten)
    {
        ulong remaining = Convert.ToUInt64(value, CultureInfo.InvariantCulture);
        bytesWritten = 0;
        do
        {
            if (bytesWritten == destination.Length)
            {
                return OperationStatus.DestinationTooSmall;
            }

            byte next = (byte)(remaining & 0x7F);
            remaining >>= 7;
            destination[bytesWritten++] = remaining == 0 ? next : (byte)(next | 0x80);
        }
        while (remaining != 0);

        return OperationStatus.Done;
    }
}

A fixed size is a promise

ICustomCodec.FixedSize tells the layout how many bytes every value of the type occupies, or null when the length varies, as it does for the varint above. A fixed size is authoritative: the layout computes the offsets of the members that follow from it, so every value occupies exactly that many bytes, whatever Read or Write reports.

For example, suppose a codec named half declares FixedSize => 4, but its Read decodes a little-endian uint16 from the first two bytes and reports two bytes consumed:

struct rec {
    half a;      /* offsets 0-3: the declared size */
    uint8 tail;  /* offset 4 */
};

With the input 34 12 AA BB 09:

Offset Bytes Member Value
0 34 12 AA BB a 0x1234; the codec reads the first two bytes, and AA BB are skipped
4 09 tail 9

tail is at offset 4, not 2, in a parse, a selected read (ReadValue), an address (ResolveAddress), and an array length, and the same holds in generated code. Writing works the same way: when Write produces fewer bytes than the fixed size, zero bytes fill the rest, so serializing this value gives 34 12 00 00 09.

The rule also sets the limits of a fixed-size codec:

  • an input that ends before the declared size is a short read, even if Read would need fewer bytes;
  • a Read that reports more bytes consumed than the fixed size fails the read; and
  • a Write that answers DestinationTooSmall to ask for more room than the fixed size fails the write.

So return null from FixedSize for an encoding whose length varies, and otherwise make Read consume, and Write produce, exactly the declared size.

Patch one field in existing data

Use a path and Update when surrounding bytes and positions must stay fixed:

private static void PatchField()
{
    var layout = new CStruct("struct item { uint16 id; uint8 flags; }; struct root { item value; };");
    using var stream = new MemoryStream([0xEE, 0xEE, 0x34, 0x12, 0x01]);
    stream.Position = 2;
    layout.Update(stream, "root.value.flags", (byte)0xA5);
    SequenceEqual([0xEE, 0xEE, 0x34, 0x12, 0xA5], stream.ToArray());
    Equal(2L, stream.Position);

    byte[] before = stream.ToArray();
    Throws<CStructWriteException>(() => layout.Update(stream, "root.value.flags", 999));
    SequenceEqual(before, stream.ToArray());
    Equal(2L, stream.Position);
}

Path, range, shape, and limit failures detected by CStructSharp happen before destination writes. A physical stream failure during final commit may still leave an accepted prefix.

For new output, use owned or caller-provided serialization:

private static void RoundTrip()
{
    var layout = new CStruct("struct sample { uint16 id; uint8 flags; };");
    byte[] input = [0x34, 0x12, 0xA5];
    object parsed = layout.Parse(input, "sample");
    SequenceEqual(input, layout.Serialize("sample", parsed));

    Span<byte> destination = stackalloc byte[8];
    destination.Fill(0xCC);
    int written = layout.Serialize(destination, "sample", parsed);
    Equal(3, written);
    SequenceEqual(input, destination[..written].ToArray());
    Equal((byte)0xCC, destination[written]);

    var writer = new ArrayBufferWriter<byte>();
    Equal(3L, layout.Serialize(writer, "sample", parsed));
    SequenceEqual(input, writer.WrittenSpan.ToArray());
}

Continue with Choose an API for ownership and failure tradeoffs.

Edit this page