Prerequisite

These past few months have been a whirlwind: waking up, diving into my screen to study, applying for jobs, researching, and refining my knowledge. My days are a cycle of study, the photography studio, more research, and rest.

In the process, I’ve encountered several core concepts that clicked for me. Realizing that the human brain loves to play tricks on us, I decided it was only right to document them. This is an archive of the concepts and implementations that caught my eye and piqued my interest.

CORE CONCEPTS

1. Structural vs. Semantic Parsing

Parsing is often a multi-stage process where we separate the "shape" of data from its "meaning."

Structural Parsing
This phase concerns itself with the correctness of the structure. It ensures the data conforms to the expected format without yet worrying about the logic of the input.

For example, in a binary protocol where the first 4 bytes denote a header (version, flags, and length), structural parsing simply ensures we have at least 4 bytes to read. If we do, the structure is "valid" even if the values inside are nonsense.

[0x01, 0x00, 0x00, 0x09] // Valid structurally (it's a 4-byte header), but potentially invalid semantically.

Semantic Parsing
This is the logical phase. It validates the "security layer" of the system—checking for truncated bodies, valid payload types, and message integrity. A slip-up here often leads to protocol-level vulnerabilities.

Without rigorous semantic checks, you are liable to build a failing system. It is critical to write procedural and adversarial tests to ensure that even "correct-looking" data doesn't crash your logic.

// Structurally correct, but semantically incomplete (missing the 9 bytes promised in the header)
[0x01, 0x00, 0x00, 0x09]

// Valid both structurally and semantically
[
    0x01, 0x00, 0x00, 0x09, // Header: length = 9
    0x02,                   // Msg_type = Ping
    0x00, 0x00, 0x00, 0x00, 0x69, 0x61, 0x1A, 0x80, // 8-byte payload
];

IMPLEMENTATION TECHNIQUES

Slice-Based Parsing

A high-performance technique where you view portions of a memory buffer as they exist (&[u8]) rather than copying data into new variables. In Rust, this is often achieved by moving a cursor across a buffer and "slicing" the exact range needed.

Length-Delimited Framing

The practice of prefixing a payload with its size (common in binary protocols). This allows the parser to know exactly how many bytes to wait for before attempting to interpret the message, preventing "fragmented message" errors during streaming.

Enum Construction from Discriminants

Mapping raw bytes (discriminants) to high-level Rust enums. Instead of working with 0x01 or 0x02, we use Message::Handshake or Message::Ping. This turns raw data into type-safe structures early in the pipeline.

Zero-Copy Borrowing

Taking the slice-based approach further by ensuring the final parsed message holds references to the original buffer rather than owned copies. This minimizes memory allocation and CPU cycles, which is vital for high-throughput systems.

Validation Before Interpretation

The "Parsing, not Validating" philosophy. We design our types so that once a message is parsed, it is guaranteed to be valid. We catch errors (like unknown message types or invalid lengths) at the boundary, ensuring the rest of our system only ever sees "clean" data.

Periodic Compaction

In a streaming parser, the buffer continuously grows as data arrives. To prevent infinite memory growth, we "compact" the buffer by draining processed bytes and resetting the cursor once it crosses a certain threshold (e.g., MAX_FRAME_SIZE).

The Concept of Unsafe

While Rust provides safety guarantees, sometimes we use unsafe (like get_unchecked) for micro-optimizations when we can manually prove an invariant—such as knowing our cursor will never exceed the buffer length. It is a powerful tool that demands extreme caution.

// Invariant:
// - cursor <= buffer.len() at all times
// - cursor only moves forward
// - buffer is only truncated when cursor == 0
// Therefore buffer[cursor..] is always valid.
impl StreamParser {
    #[inline(always)]
    // SAFETY:
    // cursor is always <= buffer.len()
    // and never decreases
    fn available(&self) -> &[u8] {
        unsafe { self.buffer.get_unchecked(self.cursor..) }
    }
}

SECURITY & RELIABILITY

Declarative Parsing + Adversarial Tests

Adversarial testing is a proactive security measure. We deliberately feed the system misleading, malicious, or truncated inputs (like a header claiming 1GB of data) to expose vulnerabilities, buffer overflows, or logic failures. By combining declarative parsing logic with these "stress tests," we build systems that are resilient by design.

Important: This is a growing archive of my concepts and implementations, and it is not exhaustive. I plan on touching base on more concepts and implementations in the future and also most concepts written here are yet to be touched.

Systems Programming: An Archive of Concepts & Implementations