Nemo’s NTFS parser

This page covers internal/filesystem/ntfs: detection, mounting, MFT record parsing, directory traversal, and the in-place named-stream write path.

At a glance

Detection

Sniff in ntfs.go checks for the NTFS OEM ID string "NTFS " (eight bytes, padded with spaces) at boot-sector offset 3. It handles a signature shorter than 11 bytes without panicking.

Mounting

New(img) reads and validates the boot sector. Everything else waits for lazy, per-call MFT record reads. There is no up-front tree walk.

1. Read the boot sector

The code is in bootsector.go. readBootSector requires the image to be at least 512 bytes and re-checks the "NTFS " magic at offset 3. Then it reads:

In that encoding, a positive value is a cluster count. A negative value -n means 1 << n bytes. Zero, an exponent over 31, or a size that does not fit a uint32 is rejected.

2. Locate MFT records on demand

The code is in mft.go.

The first read of any record past record 0 calls loadMFTBytes. It reads record 0, parses its $DATA attribute, and caches the data runs (f.mftRuns) and size (f.mftDataSize).

$MFT’s own $DATA must be non-resident, uncompressed, unencrypted, and non-sparse (attribute.flags == 0). Anything else is refused.

Every later record read finds its offset through those cached runs with readRunsAt. That is the same run-walking code used for stream data.

3. Apply fixups

Every record read goes through applyFixups. It validates and reverses NTFS’s per-sector update-sequence protection.

MFT records and attributes (attribute.go)

parseMFTRecord reads the 48-byte record header. parseMFTRecordHeader validates usedSize <= allocatedSize <= len(record) and a sane first attribute offset. Then the parser walks attributes from firstAttributeOffset until the 0xFFFFFFFF end marker.

Each attribute is resident (value bytes inline in the record) or non-resident (a run list describing clusters elsewhere). A byte at offset 8 of the attribute header says which.

parseDataRuns decodes the run-list byte stream. Each run is a length-then-offset pair of variable-width integers, and LCN deltas add up to a running absolute cluster number. Before a run is trusted, nemo checks VCN coverage, cluster-count overflow, and a negative absolute LCN.

The parser recognizes these attributes:

Attribute Type
$STANDARD_INFORMATION 0x10
$FILE_NAME 0x30
unnamed $DATA 0x80
$INDEX_ROOT 0x90
$INDEX_ALLOCATION 0xA0
$BITMAP 0xB0

It refuses $ATTRIBUTE_LIST (0x20) outright. A record that needs one to describe itself, because it has more attributes than fit in one MFT record, is unsupported. A duplicate unnamed $DATA or $STANDARD_INFORMATION is also an error, since real NTFS never produces one.

Directory traversal (directory.go)

A directory’s children come from its $INDEX_ROOT attribute. That attribute is always resident and keyed on $FILE_NAME. A large directory also has an $INDEX_ALLOCATION attribute and a $BITMAP.

parseIndexHeader walks the fixed-size index-entry array the same way in both structures. It reports whether any entry pointed at a child node (hasChildren, INDEX_ENTRY_NODE).

When the root is not enough, directoryEntries reads one INDX record per set bit in the bitmap. For each record it:

A file can have more than one directory entry, such as a short 8.3 name plus a long name. preferDirectoryNames removes the duplicate. It keeps the long-namespace entry (namespace 1 “Win32” or 3 “Win32 & DOS”) over the short-only namespace 2 “DOS” entry.

Named streams

The code is in namedstream.go. A file’s named streams are its named $DATA attributes, which means every $DATA after the first unnamed one. Entry implements filesystem.NamedStreamCapable.

Reading

streamRecord reloads and re-validates the entry’s MFT record on every call. It does not cache it. It refuses two cases:

NamedStreams lists every named $DATA attribute’s name, sorted.

ReadStream returns a resident stream’s value directly. For a non-resident stream it walks the data runs with the same bounds-checked readRunsAt used for MFT records.

A compressed, encrypted, or sparse stream is refused, not misread. streamFlags checks the attribute’s flags and, for a non-resident stream, its compression-unit field.

Writing

WriteStream builds a fresh resident $DATA attribute header and packs the UTF-16 name and value into it. Then repackStream splices it into the record where NTFS’s type-ordered attribute layout expects it. It replaces any resident stream of the same name.

For an existing non-resident stream, WriteStream calls replaceNonresident. That can overwrite the value only within the stream’s existing cluster runs (capacity from streamRuns).

Two cases need cluster allocation, which this parser does not do. Growing a stream past its allocation, and creating a new non-resident stream, both return a clear error.

DeleteStream refuses a non-resident stream, because freeing its clusters needs deallocation support that does not exist.

Checks before every change

Every change goes through prepareStreamRecord before any byte is written. It:

writeStreamRecord and commitStreamRecord write the planned physical extents. Then they re-read the record and compare it byte for byte with the expected result. Any mismatch fails loudly, so a record never ends up in a state that differs from what was intended.

Timestomping (timestomp.go)

In image mode, Entry implements filesystem.TimestompCapable. It reads timestamps through Timestamp.

It updates the creation, modification, metadata change, and access times in the resident, unnamed $STANDARD_INFORMATION (0x10). Values must fit an unsigned FILETIME (100 ns ticks since 1601-01-01) without rounding. $FILE_NAME timestamps, DATA streams, ADS, and all other attribute bytes are preserved.

The operation holds the same MFT lock as named-stream edits. It reuses their record loading, validation, mapped writes, FILE fixups, and read-back verification.

Slack space (slack.go)

In image mode, Entry implements filesystem.SlackSpaceCapable.

Slack is the allocated cluster bytes minus the unnamed DATA attribute’s initialized size. Resident files and directories expose no cluster slack. Fragmented non-resident DATA is mapped in logical order, with one physical slack region per affected run.

The slack technique reads, writes, and clears framed payloads through the custody-wrapped image. Each frame must fit in one physical region.

Before exposing a region, NTFS reloads the MFT record and validates:

Sparse, compressed, encrypted, and ATTRIBUTE_LIST-based files are refused.

Nothing else changes. No clusters are allocated, and initialized content, file sizes, runlists, and MFT bytes stay as they were.

Bytes between the initialized size and the logical EOF count as slack. NTFS keeps treating that uninitialized range as zeros. As with named streams, nemo does not infer volume-wide allocation ownership.

Limitations

Live mode (Windows)

Accepted targets:

Rejected targets: physical drives, UNC paths, relative paths, stream paths, and arbitrary device names.

Local paths resolve to their volume GUID, including junctions and nested mount points. The existing parser validates the NTFS boot sector.

Live reads are not a consistent snapshot. Concurrent filesystem changes can cause inconsistent results or parser errors. The existing parser limitations apply. Native ADS and timestamp writes are not implemented.

On non-Windows hosts, OpenLive returns filesystem.ErrUnsupported without touching the target. NTFS image mode works on every supported host.

Safety against crafted images

Testing

These tests build synthetic boot sectors and MFT records byte by byte:

They do not depend on an image from a real Windows install. They pin down the exact on-disk layout assumptions: fixup geometry, resident and non-resident attribute headers, INDEX_ROOT and INDEX_ALLOCATION traversal, and resident and non-resident stream writes.

Further reading