Table of Contents

Memory schemas and metadata import

A schema describes how bytes represent values: which member starts at which offset, how many bytes it occupies, and which codec turns those bytes into a number. For a file format you usually write that description yourself. For a memory image the description already exists, because a compiler made those decisions when it built the program. This guide shows how to state such decisions explicitly and how to import them from two common metadata formats.

Why a memory schema takes offsets instead of computing them

Consider this C declaration compiled for a typical 64-bit platform:

struct sample { uint8_t state; uint32_t count; };

The compiler does not place count at offset 1. Its platform's application binary interface (ABI) requires a four-byte integer to sit at an address divisible by four, so it inserts three padding bytes and places count at offset 4. The struct is eight bytes long. A different compiler, a #pragma pack directive, or a different CPU can produce a different answer, and a bit-packed member such as int flag : 3 adds compiler-specific bit placement on top. How C structs occupy memory explains these rules in depth.

An analyzer running on your machine cannot safely reproduce another platform's decisions from type names alone. MemorySchema therefore never guesses. Every composite member carries an explicit byte offset, every type carries an explicit size that includes padding, and bit-packed members carry an explicit bit offset and width. Where the Portable language computes placement from declaration order, a memory schema records placement as given.

Compile a schema once and reuse it in sessions that inspect many records or several address spaces. Validation happens at construction, so a bad description fails before any source is read. An invalid definition, such as a reference to an unknown type ID or two overlapping struct members, throws CStructLayoutException, the same exception an invalid layout declaration throws. Malformed BTF, ISF, or Portable metadata throws it too.

Three ways to obtain a schema

  1. Project an existing layout. PortableMemorySchema.Create(layout, "Root") converts a compiled CStruct with a fixed layout into descriptors. It copies the compiled offsets, pointer width, byte order, and codecs; it does not parse the declaration again. Use this when your own Portable declaration is already the authority.
  2. Write descriptors by hand. Construct MemoryTypeDefinition and MemoryField objects when your application knows the placement, for example from a hardware manual or a debugger.
  3. Import metadata. Feed BTF or ISF to the importers and use the returned MetadataImportResult.Schema and RootTypeId. Use this for operating-system structures whose placement was recorded by the build.

All three routes produce the same MemorySchema and support the same session operations. The Portable projection rejects conditional and runtime-sized members, because a memory schema must know every size in advance. For those layouts, open a bounded stream view and use the core API.

The building blocks of a schema

A schema is a graph of MemoryTypeDefinition nodes connected by IDs. Each definition has a Kind:

Kind Meaning Extra information carried
Scalar An integer, float, boolean, or enum decoded by a core codec ScalarType names the codec, for example "uint32"
Pointer An unsigned stored address Size is the pointer width; ElementTypeId names the target, or is null for an opaque pointer
Struct Members at explicit offsets that must not overlap Fields
Union Members at explicit offsets that deliberately overlap Fields
Array A fixed number of equally sized elements ElementTypeId and Count; Size is the total byte length
Incomplete A type with identity but no readable representation, such as a forward declaration Nothing; size is zero
RawBytes A type whose size is known but whose members are not usable; read and written as a byte[] Size only; no fields

An opaque pointer is a Pointer whose ElementTypeId is null. Like C's void *, it stores an address but says nothing about what lives there, so a session reads the address but cannot follow it. A RawBytes type is a different thing: it is a value stored in place whose size is trustworthy but whose members are not. Best-effort validation, described below, creates RawBytes placeholders.

Every definition has an Id and a Name, and they serve different purposes. Id is the reference key used by fields and by session calls. Name is a display string. Native metadata often contains several distinct types with the same name, so the importers generate unique IDs such as btf:7 and keep the original name for display.

A MemoryField connects a containing struct or union to a member: a Name, a TypeId, a byte Offset relative to the start of the containing record, and optionally a bit slice (BitOffset, BitWidth, Signed). A field marked Promoted makes the members of an anonymous nested struct or union visible in its parent, mirroring how C lets you write record.flag when flag lives in an unnamed inner union.

The MemorySchema constructor validates the whole graph before it returns:

  • every referenced ID must exist, and a scalar's declared size must equal its codec's size;
  • every member name must be an identifier (a letter or underscore, then letters, digits and underscores), because a path names members that way;
  • every member must fit inside its container, and an array's size must equal element size times count;
  • struct members must not overlap (two bit slices may share one storage unit as long as their bits are disjoint);
  • a type may not contain itself by value, directly or through other types, because that would need infinite storage. A pointer back to the same type is fine: the pointer has a finite size;
  • types nested by value may be at most 256 levels deep, the core's default layout nesting limit.

An array's element may have size zero only when that makes the whole array zero bytes wide - count times zero can only ever equal a declared array size of zero. This is a real case, not just an allowed edge: native metadata sometimes describes an empty marker struct with no members (Linux's lock_class_key, for example, exists only so the lock validator has a stable address to key on, never to hold data) and arrays of that struct. Any other size disagreement, zero-size element or not, is still rejected.

Describe a padded record with a signed bit slice

Suppose metadata says a record is eight bytes long. A three-bit signed state occupies bits 1 through 3 of byte zero; a four-byte count starts at offset 4. Bytes 1 through 3 are padding. We describe the state's storage unit as a one-byte scalar, even though the selected value uses only three of its bits.

Byte offset Input Meaning
0 8F State bits are 111; the neighboring bits are also present
1–3 AA BB CC Padding, preserved by an update
4–7 2A 00 00 00 Little-endian count 42 
// Every offset and size is stated, as metadata would state it; nothing is computed from declaration order.
var schema = new MemorySchema(new[]
{
    new MemoryTypeDefinition("byte", "byte", MemoryTypeKind.Scalar, 1, scalarType: "uint8"),
    new MemoryTypeDefinition("word", "word", MemoryTypeKind.Scalar, 4, scalarType: "uint32"),
    new MemoryTypeDefinition("record", "Record", MemoryTypeKind.Struct, 8, fields: new[]
    {
        // "state" is bits 1..3 of the byte at offset 0, read as a signed two's-complement number.
        new MemoryField("state", "byte", 0, bitOffset: 1, bitWidth: 3, signed: true),
        new MemoryField("count", "word", 4),
    }),
});
var session = new MemorySession(schema);

// Byte 0 is 1000 1111: bits 1..3 are 111, which is -1 in three-bit two's complement.
var source = new ByteArrayMemorySource("record", new byte[] { 0x8f, 0xaa, 0xbb, 0xcc, 42, 0, 0, 0 });
var region = new MemoryRegion(source, 0, 8);
Require((long)session.Read(region, "record", "state")! == -1, "Signed three-bit value");
Require((uint)session.Read(region, "record", "count")! == 42, "Member at offset four");

// Writing -2 (bits 110) touches only bits 1..3; bit 0, the high nibble, and the padding survive.
session.PlanUpdate(region, "record", "state", -2L).Commit();
Require(source.ToArray().SequenceEqual(new byte[] { 0x8d, 0xaa, 0xbb, 0xcc, 42, 0, 0, 0 }), "Neighbor preservation");

Byte 8F is 1000 1111 in binary. Bits are numbered from the least significant end, so bit 0 is the rightmost digit. Bits 1 through 3 are 111. A three-bit two's-complement number covers -4 to 3:

Bits Unsigned Signed
011 3 3
100 4 -4
110 6 -2
111 7 -1

Reading state therefore returns -1. Updating it to -2 stores 110 in bits 1 through 3, changing 8F to 8D; bit 0, the high four bits, the padding, and the count remain unchanged. Bit offsets count from the least significant bit of the decoded storage integer. Byte order decides how a multi-byte storage integer is assembled from its bytes; it does not change what bit index zero means.

Two rules from this example are worth remembering. Reads sign-extend a signed slice and writes check the signed range, so -5 is rejected rather than wrapped. And an update reads the storage unit first, so bits that are not part of the selected slice survive untouched; the update guide builds on this.

Import a small ISF description

ISF (Intermediate Symbol Format) is the JSON metadata format used by the Volatility 3 memory-forensics framework. Its symbol tables are generated from a kernel's debug information and describe base types, user types with member offsets, enums, and symbol addresses. This importer consumes the value-type subset: it turns types into descriptors. Choosing the profile that matches a capture, and finding the address of a record, remain the application's job. An imported type says how a record is laid out, not where one is.

The following independently authored metadata places value at offset 4 in an eight-byte record. The first four bytes are outside the selected member, so their values do not affect the result, 7.

const string json = """
    { "metadata": { "format": "6.2.0" },
      "base_types": { "u32": { "kind": "int", "size": 4, "signed": false, "endian": "little" } },
      "user_types": { "counter": { "kind": "struct", "size": 8, "fields": {
        "value": { "offset": 4, "type": { "kind": "base", "name": "u32" } }
      } } }, "enums": {}, "symbols": {} }
    """;
// The importer follows "counter" and the types it references; RootTypeId is the importer's ID for it.
MetadataImportResult imported = new IsfMetadata(Encoding.UTF8.GetBytes(json)).Import("counter", new MetadataImportOptions { PointerSize = 8, });
var session = new MemorySession(imported.Schema);

// Offsets 0..3 are outside the selected member, so their contents cannot influence the result.
var source = new ByteArrayMemorySource("capture", new byte[] { 0xaa, 0xbb, 0xcc, 0xdd, 7, 0, 0, 0 });
object? value = session.Read(new MemoryRegion(source, 0, 8), imported.RootTypeId, "value");
Require((uint)value! == 7, "Imported explicit offset");
Require(imported.Schema.GetField(imported.RootTypeId, "value").Offset == 4, "Metadata inspection");

Use using CStructSharp.Memory.Metadata; and using System.Text; for the importer and the UTF-8 conversion. Take RootTypeId from the result rather than guessing the importer's ID spelling. Review Diagnostics alongside Schema.Types when deciding which imported types can be read by value; a successful import can retain address-only types as pointer targets without making those targets readable, and a best-effort import lists the types it replaced with raw-bytes placeholders.

new IsfMetadata(json) parses the document once; Import(rootName, options) then compiles one root and everything it references, so several roots can share one parsed document. ISF import requires format 6.2.0. It supports base integers, floats and booleans, structs, classes, unions, arrays, enums, pointers, and member bitfields. A base type's explicit endianness overrides the schema default that the constructor's isLittleEndian sets. The constructor's maxBytes bounds the document (16 MiB by default; a full kernel profile is often larger, so pass a bigger limit for one). Relocations, profile matching, and guesses about older formats are outside its scope.

Symbols, type names, and forward declarations

A real profile also lists symbols: named addresses of kernel variables and functions, such as init_task, the first task record. TryGetSymbol(name, out address) looks one up and Symbols lists them all; UserTypeNames lists the struct, class and union names that Import accepts, in ordinal order. An address is reported exactly as the profile records it. If the running kernel was loaded at a randomized offset (kernel address space layout randomization), adding that offset is the application's job. Some profile generators write an address in the top half of the 64-bit space as a negative number; it is returned as the unsigned number with the same bits.

In C, a struct may hold a pointer to a type it has only declared, as in struct files_struct *files;. The compiler needs no layout for the pointed-to type, and profiles often omit it from user_types. The importer turns such a name, whether a user type, an enum, or a base type, into an address-only Incomplete type and adds a diagnostic that names it. A pointer to it reads as an address but cannot be followed. Embedding an undefined type by value is a different matter: the containing type has no size for that member, so a strict import fails with a message that names the missing type, and a best-effort import turns the containing type into a raw-bytes placeholder.

// "task" points to "files_struct", which the profile never defines: C only needs a forward declaration for that.
const string json = """
    { "metadata": { "format": "6.2.0" },
      "base_types": { "u32": { "kind": "int", "size": 4, "signed": false, "endian": "little" } },
      "user_types": { "task": { "kind": "struct", "size": 16, "fields": {
        "pid": { "offset": 0, "type": { "kind": "base", "name": "u32" } },
        "files": { "offset": 8, "type": { "kind": "pointer", "subtype": { "kind": "struct", "name": "files_struct" } } }
      } } },
      "enums": {},
      "symbols": { "init_task": { "address": 4096 } } }
    """;
var metadata = new IsfMetadata(Encoding.UTF8.GetBytes(json));
Require(metadata.UserTypeNames.SequenceEqual(["task"]), "User type names");
Require(metadata.TryGetSymbol("init_task", out ulong address) && address == 0x1000, "Symbol address");

// The missing struct becomes an address-only type, named in the diagnostics; the pointer still reads.
MetadataImportResult imported = metadata.Import("task");
Require(imported.Schema.GetType("isf:user:files_struct").Kind == MemoryTypeKind.Incomplete, "Forward declaration");
Require(imported.Diagnostics.Any(note => note.StartsWith("isf:user:files_struct:", StringComparison.Ordinal)), "Diagnostic");

// A toy capture whose byte offsets are addresses: the task record starts at the symbol's address.
byte[] capture = new byte[0x1010];
capture[0x1000] = 1;
capture[0x1008] = 0x40;
var session = new MemorySession(imported.Schema);
var task = new MemoryRegion(new ByteArrayMemorySource("capture", capture), address, 16);
Require((uint)session.Read(task, imported.RootTypeId, "pid")! == 1, "Record at symbol");
Require(session.Read(task, imported.RootTypeId, "files") is StoredPointer { Address: 0x40 }, "Opaque pointer");

How ISF bitfields are placed

ISF describes a bitfield as an integer of its storage type, read at the member's offset in that type's byte order, and a slice of bit_length bits starting bit_position bits above the integer's low bit. Profile generators disagree about which byte the offset names. Some record the start of the compiler's storage unit; others record the byte that holds the slice's first bit. In the second form, the last bitfield of the eight-byte x86 segment descriptor desc_struct appears as a two-byte integer at byte 7, which would end past the struct.

Both forms select the same physical bits, so the importer places each slice in the storage unit a C compiler would use: an integer of the storage type's size whose offset is a multiple of that size. That descriptor field becomes bits 8 to 15 of the two-byte unit at byte 6, and its neighbors in the same unit share offset 6. A packed struct can put a slice across such a unit; the slice then keeps its recorded place when that fits the struct, and otherwise moves to the widest smaller aligned integer that holds it. A C _Bool bitfield, and any slice moved to a smaller integer, is read through an unsigned integer of the unit's size, so it reads as a number rather than as a Boolean.

An ISF enum is read with the width of its own size. Its base type contributes the byte order, and the constants decide the sign: a negative constant makes the storage signed even when the recorded base type is unsigned, as a C compiler does for such an enum.

Both importers take the same MetadataImportOptions:

Option Default Meaning
PointerSize 8 The analyzed image's pointer width in bytes. Set 4 for a 32-bit capture, even on a 64-bit host.
BestEffort false Demote a type that fails validation to a RawBytes placeholder instead of failing (see below).
MaxTypes 100,000 The most type descriptors one import may create, bitfield storage and generated types included.

An import walks the type graph with an explicit work list rather than recursion, so a long chain of pointers does not run into a depth limit. Types nested by value (a struct inside an array inside a struct, and so on) may be 256 levels deep, the same default limit that applies to a compiled layout; a deeper graph fails with CStructLayoutException. ISF documents are likewise limited to 256 levels of JSON nesting.

Import BTF and split tables

BTF (BPF Type Format) is a compact binary type description emitted for the Linux kernel and its modules. A BTF blob has a small header, a table of numbered type records, and a string table. Each record has a kind (integer, pointer, struct, and so on), a size or referenced type, and a variable-length payload such as struct members with their bit offsets. Tools usually read the blob from the .BTF section of a kernel image or from /sys/kernel/btf; this API does neither. You obtain the bytes and pass them in.

First supply the blob to new BtfMetadata(bytes), which validates and indexes it. Then FindType("task") locates a unique named type and Import(typeId, options) compiles that type and everything it references. FindType rejects duplicate names rather than picking one, because two kernel types can share a name while having different layouts. When names are ambiguous, use the numeric ID from the metadata producer.

The blob's magic number tells the parser its byte order (9F EB for little-endian, EB 9F for big-endian). The target's pointer width is a separate setting, because BTF describes types, not the machine word size.

Here is a complete synthetic blob with two types: a four-byte unsigned integer and an eight-byte record whose member starts at byte 4. BTF stores member offsets in bits, so the record says 32 and the importer produces byte offset 4. The comments identify each section; applications normally receive these bytes from a build tool rather than writing them by hand.

// BTF header: little-endian magic, v1, 24-byte header, 40-byte type table, 18-byte string table.
byte[] blob = Convert.FromHexString(
    "9FEB01001800000000000000280000002800000012000000" +
    // Type 1: name offset 1 (u32), INT kind, size 4, unsigned 32-bit encoding.
    "01000000000000010400000020000000" +
    // Type 2: name offset 5 (record), STRUCT kind with one member, size 8.
    "050000000100000408000000" +
    // Member: name offset 12 (value), type 1, bit offset 32 (byte offset 4).
    "0C0000000100000020000000" +
    // UTF-8 strings: empty, u32, record, value; each is zero-terminated.
    "00753332007265636F72640076616C756500");
// Parsing indexes the table; FindType demands a unique name; Import compiles the reachable graph.
var metadata = new BtfMetadata(blob);
uint rootId = metadata.FindType("record");
MetadataImportResult imported = metadata.Import(rootId, new MetadataImportOptions { PointerSize = 8, });
var session = new MemorySession(imported.Schema);
var image = new ByteArrayMemorySource("BTF record", new byte[] { 0xaa, 0xbb, 0xcc, 0xdd, 9, 0, 0, 0 });
Require((uint)session.Read(new MemoryRegion(image, 0, 8), imported.RootTypeId, "value")! == 9, "BTF member placement");
Require(imported.Schema.GetField(imported.RootTypeId, "value").Offset == 4, "BTF bit-to-byte offset");

The decoded value is 9; the first four image bytes are padding. Because the blob was written independently of the importer, the example checks that offsets are interpreted rather than inferred.

A split BTF table extends a base table; the kernel uses this so each module only ships the types it adds. Construct the base BtfMetadata first, then pass it as baseMetadata when constructing the split table. Base type IDs and string offsets keep their identity. Do not concatenate the blobs or renumber IDs yourself.

Supported representations are integers, supported floating-point sizes, arrays, pointers, structs, unions, enums and ENUM64, qualifiers such as const, typedefs, and member bit slices. Function and forward-declaration targets remain address-only. Older BTF encoded bitfields inside the integer type itself; the importer normalizes such legacy slices when they are used as members, but does not offer them as standalone scalars. Any other reachable value kind fails explicitly instead of producing a guessed layout.

BTF gives each bitfield an absolute bit offset counted from the start of the containing struct, not from the storage word it happens to live in. When two bitfields share one storage word (a compact C idiom - for example unsigned long value : 11; unsigned long tasks : 53; packed into a single eight-byte unsigned long), the importer places both at the same byte Offset - the start of that shared word - and tells them apart by BitOffset, the position within the word. A member whose bit offset merely happens to fall in the word's second byte does not get its own byte-1 storage slot; it is still part of the same word its neighbor started.

Describe one type without importing its graph

Import compiles a type and everything it references, recursively - so a single incompatible type anywhere in a large graph fails the whole call, even if the type you actually asked for is perfectly fine. A real kernel's BTF easily reaches thousands of types from one root, so this matters in practice, not just in theory.

Describe(id) answers a narrower question: what does this one type record look like? It reports the kind BTF resolves to (following typedef/const/volatile/restrict/type-tag chains transparently, the same way Import does), its size, and - for a struct or union - its direct members' names, declared type IDs, and placement. It never imports or validates anything a member merely points to; that member's own type ID is reported as-is.

This makes Describe the right tool for finding out what a specific ID actually is - typically while diagnosing why Import rejected something deep in a large graph, by walking one level at a time from the root down toward whichever type is causing trouble - rather than a replacement for Import when you actually want to read values.

// The same blob ImportBtf uses: one integer type and one struct with a single member.
byte[] blob = Convert.FromHexString(
    "9FEB01001800000000000000280000002800000012000000" +
    "01000000000000010400000020000000" +
    "050000000100000408000000" +
    "0C0000000100000020000000" +
    "00753332007265636F72640076616C756500");
var metadata = new BtfMetadata(blob);
uint rootId = metadata.FindType("record");

// Unlike Import, Describe never follows a member into its own type - it reports the declared ID as-is,
// so this succeeds even if "value"'s own type would fail to import.
BtfTypeDescription description = metadata.Describe(rootId);
Require(description.Kind == BtfKind.Struct, "Describe reports the struct kind");
Require(description.Members.Count == 1, "Describe reports one member");
BtfMemberDescription value = description.Members[0];
Require(value.Name == "value" && value.Offset == 4, "Describe reports the same placement Import would");
Require(value.BitOffset is null && value.BitWidth is null, "Describe reports a whole-value (non-bitfield) member");

Import best-effort, tolerating one locally broken type

By default, Import (whether from BTF, ISF, or a hand-built MemorySchema) is all-or-nothing: if any type reachable from the root fails validation - a member placed past its container's extent, a bitfield that does not fit its storage, and similar defects in the source metadata, not in this library - the whole import fails, even when the type you actually asked for is perfectly consistent on its own terms. For a real kernel, where one root like task_struct can reach several thousand types, that means one bad type anywhere in a huge graph blocks everything.

Passing bestEffort: true to MemorySchema's constructor, or BestEffort = true in the MetadataImportOptions of a BTF or ISF import, changes this: a type that fails validation is demoted in place to a same-sized MemoryTypeKind.RawBytes placeholder instead, and the substitution is recorded as a diagnostic rather than thrown. A placeholder keeps the type's real, checked size, so every other definition that embeds it - by pointer or directly by value - still places its own members correctly; only that one type's own contents become unreadable, as raw bytes rather than a decoded value. This is squarely aimed at forensic memory captures, which are often torn or partial - one subsystem's metadata can be locally corrupt while the rest of a large graph remains perfectly readable - rather than at a healthy, complete kernel, which should not need it.

var scalar = new MemoryTypeDefinition("byte", "byte", MemoryTypeKind.Scalar, 1, scalarType: "uint8");

// "inner" is intrinsically broken: its one member sits past inner's own declared one-byte extent. This is
// the shape a torn or partial forensic capture produces - not something a real, complete kernel emits.
var inner = new MemoryTypeDefinition("inner", "inner", MemoryTypeKind.Struct, 1, new[]
{
    new MemoryField("x", "byte", 4),
});

// "outer" is fine on its own terms; it just happens to embed the broken "inner" by value.
var outer = new MemoryTypeDefinition("outer", "Outer", MemoryTypeKind.Struct, 1, new[]
{
    new MemoryField("field", "inner", 0),
});

MemoryTypeDefinition[] types = { scalar, inner, outer };

// Strict validation (the default) fails the whole schema for "inner"'s sake, even though "outer" is fine.
bool strictFailed = false;
try
{
    _ = new MemorySchema(types);
}
catch (CStructLayoutException)
{
    strictFailed = true;
}

Require(strictFailed, "Strict import fails outright");

// With bestEffort, "inner" is demoted to a same-sized raw-bytes placeholder instead, so "outer" still imports.
var schema = new MemorySchema(types, bestEffort: true);
Require(schema.GetType("inner").Kind == MemoryTypeKind.RawBytes, "The broken member becomes raw bytes");
Require(schema.Diagnostics.Count == 1, "The demotion is reported as a diagnostic");

// A raw-bytes value reads and writes as a byte[] - the size is trustworthy, its layout is not.
var session = new MemorySession(schema);
var image = new ByteArrayMemorySource("capture", new byte[] { 42 });
var read = (StructValue)session.Read(new MemoryRegion(image, 0, 1), "outer")!;
Require(((byte[])read["field"]!)[0] == 42, "Raw-bytes member reads as its raw byte");

Semantic metadata versus compiled scalar codecs

Schema.Types, GetType, and GetField present the imported model: real names, IDs, offsets, bit slices, and Provenance strings that say where a definition came from. Use these in an analyzer's user interface. GetField looks up immediate members; session paths additionally resolve uniquely promoted members.

The schema never compiles a layout for a struct, union, or array. Their placement is already recorded, so the constructor checks it directly with the rules listed above: plain byte arithmetic on sizes and offsets. A struct can therefore be as large as an int byte count allows, and a member may sit millions of bytes into it.

Only the leaves need the core compiler. To decode and encode bytes, the schema compiles one tiny core layout per scalar, such as struct __memory_scalar { uint32 value; };, and one per bit slice, such as struct __bits { uint32 :12; uint32 value:5; }; for a 5-bit slice that starts at bit 12 of a four-byte unit. The memory APIs reuse the core's scalar and bitfield codecs through these layouts instead of a second decoder. The generated names (__memory_scalar, __bits, value) are not part of the API: results and diagnostics always use the metadata's own names and IDs.

Prepare repeated field reads

A prepared reader resolves a path against one schema before applying it to records. It keeps member, index and codec decisions; it keeps no bytes, addresses or operation context. Bind once for a repeated field such as pid, then reuse that reader with independent contexts. The session must hold the identical schema instance.

// struct Entry { uint32 pid; int8 name[4]; }; bytes are little-endian and member order fixes offsets 0 and 4.
var schema = new MemorySchema([
    new("u32", "u32", MemoryTypeKind.Scalar, 4, scalarType: "uint32"),
    new("i8", "i8", MemoryTypeKind.Scalar, 1, scalarType: "int8"),
    new("name", "name", MemoryTypeKind.Array, 4, elementTypeId: "i8", count: 4),
    new("Entry", "Entry", MemoryTypeKind.Struct, 8, [new("pid", "u32", 0), new("name", "name", 4),]),
]);
var session = new MemorySession(schema);
MemoryFieldReader<uint> pid = schema.Bind<uint>("Entry", "pid");
var source = new BorrowedMemorySource("record", new byte[] { 7, 0, 0, 0, 65, 255, 0, 0, }, 0xffff800000001000);
var region = new MemoryRegion(source, source.BaseAddress, 8);
var context = new MemoryAccessContext();
Require(pid.Selection.Offset == 0 && pid.Read(session, region, context) == 7, "Prepared pid read failed.");
Span<byte> name = stackalloc byte[4];
session.ReadBytes(region, "Entry", "name", name, context);
Require(name.SequenceEqual(new byte[] { 65, 255, 0, 0, }), "Signed byte storage changed.");

// A readable prefix is evidence; the excluded suffix must not decode as zero-filled success.
var partial = new BorrowedMemorySource("partial", new byte[] { 7, 0, 0, 0, }, source.BaseAddress, readableLength: 2);
Require(!pid.TryRead(session, new MemoryRegion(partial, partial.BaseAddress, 8), out _, out MemoryReadFailure? failure), "A truncated pid succeeded.");
Require(failure!.MemoryFailure == MemoryFailure.MissingBytes, "Missing bytes lost their category.");

The declaration in this example has pid at byte offset 0 and four signed bytes at offset 4. Bytes 07 00 00 00 decode as the unsigned integer 7. The byte-array copy returns 41 FF 00 00 unchanged: signed element interpretation does not change raw storage. The final example excludes half of the integer's storage and therefore returns MissingBytes, even though a larger backing allocation exists.

schema.Select(typeId, path) inspects a path without any memory source. Its Offset is the byte offset from the root when every step is known from the schema. A path that follows a pointer's .value has a null offset, because its target address is stored in each record. MemoryPath.Resolve applies the prepared decisions and returns full selection and containing-record metadata. Pointer targets are resolved afresh on every read.

Prepared selection validates the entire schema path before reading memory. TryResolve uses this preparation too. If a path is invalid beyond a pointer step, it reports the schema failure without reading that pointer. Ordinary Resolve walks the path while accessing memory, so an earlier unavailable pointer can be reported first.

Typed bindings support built-in 8-, 16-, 24-, 32- and 64-bit integers, booleans, float32, float64, enum results, bit slices and StoredPointer. The requested managed type must match: for example uint32 binds as uint, and a signed bit slice binds as long. Other scalar codecs, including caller-defined codecs, bind as object and keep their ordinary decoding behavior. Binding never calls a custom decoder. An incompatible requested type is a WrongType outcome from TryBind; Bind raises its path exception.

Browse and reuse metadata

IsfMetadata.Describe(name) returns one user type's recorded kind, byte size and direct members. Each member retains its original JSON type descriptor, including unsupported kinds and unresolved references. Description does not import those referenced types and does not validate unrelated entries. Malformed direct fields still raise a layout error. BTF descriptions retain numeric identities; ISF descriptions retain their user-table names and descriptor syntax.

Imports from one parsed ISF document share successful scalar and bit-slice compilation through a bounded cache, as BTF imports do. Each root still builds and validates its own graph and ordered diagnostics. Different settings remain distinct. Failures and cancellation are not cached. Keep the parsed metadata alive while importing related roots; do not merge whole root schemas to obtain compilation reuse.

Import selected fields

A field-directed import prepares only the values an operation needs. Give ImportProjection a root and ordinary session paths. ISF roots use their user-table name; BTF roots use their declared numeric type ID, including a typedef ID. The result keeps the original root size, member offsets and array element stride. It shares successful scalar compilation with other imports from the same metadata document.

var metadata = new IsfMetadata(Encoding.UTF8.GetBytes("""
{"metadata":{"format":"6.2.0"},"base_types":{"u32":{"kind":"int","size":4,"signed":false,"endian":"little"}},"user_types":{"Entry":{"kind":"struct","size":16,"fields":{"pid":{"offset":0,"type":{"kind":"base","name":"u32"}},"next":{"offset":8,"type":{"kind":"pointer","subtype":{"kind":"struct","name":"Other"}}}}},"Other":{"kind":"struct","size":4,"fields":{"value":{"offset":0,"type":{"kind":"base","name":"u32"}}}}}}
"""));
MetadataImportResult selected = metadata.ImportProjection("Entry", ["pid", "next"]);
MemorySchema schema = selected.Schema;
Require(schema.GetType(selected.RootTypeId).Size == 16, "Projection changed record size.");
Require(schema.Select(selected.RootTypeId, "pid").Offset == 0, "Projection changed field placement.");
Require(schema.GetType("isf:user:Other").IsUnselected, "Unrequested target was imported.");
Require(!schema.TrySelect(selected.RootTypeId, "next.value.value", out _, out MemoryReadFailure? unavailable), "Unselected pointer target was traversed.");
Require(unavailable!.Kind == MemoryReadFailureKind.UnselectedType, "Projection availability was lost.");

// A wider import is a new schema; the earlier projection keeps its original decisions.
MetadataImportResult wider = metadata.ImportProjection("Entry", ["next.value.value"]);
Require(wider.Schema.Select(wider.RootTypeId, "next.value.value").Offset is null, "Pointer path has a constant offset.");
Require(schema.GetType("isf:user:Other").IsUnselected, "Import mutated an existing schema.");

The example's record occupies 16 bytes. pid stays at byte 0 and the eight-byte next pointer stays at byte 8. Input 07 00 00 00 at byte 0 still decodes as 7. The selected next path includes its stored address and target identity. It does not request the target's fields. next.value.value requests the target representation explicitly. Its final field has no constant root-relative offset because each record supplies its pointer address.

Selections are combined by type identity. A terminal composite, or an empty path selecting the root, includes its whole by-value shape: embedded composites and arrays are included, while pointers stop at their stored address. An indexed path such as items[2].pid retains the array's original count and stride and imports the element's pid. Every element shares that projected type. If another selected path needs the same type, their fields are combined. A target already needed by a selected embedded value is therefore available to pointers to that identity too.

Inspect these properties before presenting a projected schema as a general type description:

  • MetadataImportResult.SelectedPaths contains the distinct requested paths, or null for a full import.
  • MemoryTypeDefinition.IsProjected identifies composites imported with a narrower validation scope. Fields contains the imported members; UnselectedMembers lists omitted immediate names in metadata order.
  • MemoryTypeDefinition.IsUnselected identifies a pointer target whose representation was deliberately excluded. Its kind is Incomplete, but this marker distinguishes deliberate exclusion from a diagnosed missing or forward-declared type. It does not establish whether the omitted type is defined.

TrySelect and TryBind report UnselectedMember for a name outside a projected shape, and UnselectedType for an excluded representation. An absent projected member does not establish whether the full metadata defines that name. Whole-record reads return only imported members. Whole-value encoding rejects composites with omitted members so it cannot fill unknown storage with zeros. A selected-field PlanUpdate still preserves the surrounding bytes.

Projection validates the selected descriptors, their containing sizes and placements, direct member names and the anonymous promotion candidates needed to detect ambiguity. It does not validate omitted ordinary descriptors or walk unrelated pointer graphs. Missing, ambiguous and malformed selected paths fail explicitly. BestEffort keeps its usual behavior for selected layouts that fail schema validation; a demoted type remains visibly projected. Use full Import when the operation needs general inspection or full reachable-graph validation.

Each import owns its selected field sets. A later wider projection or full import cannot mutate an earlier schema. Create a new schema and matching session to read more fields. Paths have the ordinary syntax and at most 256 steps; MaxTypes bounds distinct shallow and emitted descriptors. MaxProjectionWork separately bounds supplied path entries, characters and steps, native member inspection, promotion searches and by-value closure work (default 1,000,000). Promotion searches memoize shared subgraphs and stop once ambiguity is established. Cancellation is observed during planning and compilation. No operation context or captured bytes are retained.

Check your understanding

  1. Metadata describes struct pair { uint8 a; uint8 b; } with b at offset 0 and a at offset 0. The schema constructor throws. What is wrong, and how would you describe a type where both really do share byte 0?
  2. A four-byte storage unit holds a signed 5-bit slice at bit offset 12. What is the range of values the slice can store, and what does an update to 20 do?
  3. FindType("list_head") throws CStructLayoutException saying the name is ambiguous. What does that tell you about the blob, and what should your program do?

Answers: struct members may not overlap; declare the type as a Union, whose members are expected to share storage, or give the fields different offsets. -16 to 15, so an update to 20 throws CStructWriteException before any byte is staged. The blob contains two or more types named list_head, perhaps one from the base table and one from a module; ask the metadata producer for the numeric ID you want and call Import with that ID instead of a name.

The overview links the source format specifications and summarizes their limits. Next, use the schema to follow pointers or create and update records.

Edit this page