AI Generative Market Data Protocol Decoding - Part 3
Building a verifiably accurate Order-Entry Decoder
Parts 1 and 2 of this series covered market data — decoding a real NASDAQ ITCH session, then building dashboards on top of it. Market data is the easy protocol category, though. It's public, standardized, and every vendor in this space already knows how to decode it. Order entry is a different animal, and it's the one that actually matters to most of our users.
Order entry is how your trading strategy talks to an exchange — it determines whether you were fast enough, not just what happened on the tape afterward. It's also genuinely hard to get real data for from external sources: you can't download a capture from the Internet, you can only have one by capturing what your trading strategy sends to the exchange — exactly the position most of our users with proprietary order-entry protocols find themselves in. This post walks through building a decoder for exactly that kind of protocol: NASDAQ's OUCH, order entry rather than market data, and structurally different enough from ITCH to be a genuine test rather than a repeat of Part 1.
Decoded OUCH data visualising Order Flow
Setting the model up properly
FMADIO isn't shipping AI-generated decoder code. The decoder architecture underneath all of this — the classify/parse/emit pipeline, the Parquet output layer, the coding conventions — was written by our own engineers, the same architecture already proven out on ITCH. Writing the OUCH decoder meant extending that existing, human-built structure to a new protocol, not starting from scratch.
Before the decoder code was written, the an Ubuntu LXC container was created, and within it, the model was given what a competent engineer would want on day one: the official OUCH and SoupBinTCP protocol specifications as source PDFs, a copy of tshark, a reference PCAP that had already been manually checked in Wireshark against those same specifications, and a copy of the FMADIO public repository so the new decoder would follow the same conventions as the ones already built. Real primary-source protocol documentation, a real independent packet-inspection tool, and a pre-verified reference dataset to check its own output against, sitting alongside the architecture it needed to extend.
This starting point, and especially a known-good PCAP, provide a framework for the LLM to cycle through a write -> check -> debug -> rewrite process, just as a human dev would, just quicker.
Getting from spec to a working, verified decoder
With that in place, the actual build followed a shape any engineer would recognize. OUCH is TCP, point-to-point, per-participant gateway sessions, with orders that get replaced and cancelled and chained together across a session — structurally different from ITCH's UDP multicast broadcast in almost every way that matters.
The new decoder was created and tested in a one-shot prompt:
In the Github repo https://github.com/fmadio/fmadio_markets/ you will find a decoder that turns ITCH market data into parquet files. I'd like to do the same thing for the OUCH order-entry capture — Using the existing ITCH decoder as a template I’d like to build an OUCH decoder that works the same way and outputs data we can use the same way.
PDFs of the OUCH 5.0 and Soupbin protocol specs are in the ~/Git/protocols folder. Please cover the whole OUCH 5.0 spec, not just the message types in our test capture — I'd rather it be complete than just work for this one file.
Before you start writing anything, talk me through your plan so I can sanity check it.
tshark is installed on this container along with the OMI dissectors for OUCH 5 and Soupbin. A sample OUCH PCAP is in the ~/Git/captures folder. Please use this to test the decoder and compare to the tshark output with the OUCH dissector, ensuring all messages match.
The output is a single C decoder file that fits the framework of the FMADIO Markets project.
Getting a decoder to run without crashing proves nothing about whether it's correct, so it was checked three separate ways, each catching a different class of error:
A synthetic unit test covering every field and message-type edge case — 34 out of 34 correct.
A coarse sanity pass against the independently-verified reference capture — message counts and per-type distributions checked against tshark's own read of that same capture, before drilling into anything field-level — 7 out of 7 categories matching exactly.
A full independent field-by-field cross-check against that same tshark output, every single field of every single message, decoded completely separately from the new decoder — 3,283 out of 3,283 messages, zero field-level differences.
That third check is the one that actually matters, and it wasn't entirely trivial to get right. Building it surfaced two real bugs in the validation query itself — a NULL-matching failure across a join, and a legitimate one-order-many-fills relationship that inflated the match count — both needing to be found and fixed before the result could be trusted. A test harness is code too, and it earned the same scrutiny as the decoder it was checking.
Tick to trade calculations - based on a combination of ITCH and OUCH decoded capture
Why this matters beyond one protocol
The end result is a decoder that agrees exactly with an independent, third-party tool on every field of every message in the reference capture — built by extending an existing, human-designed architecture rather than starting over, and checked at every step against something external rather than against its own assumptions. That's a genuinely different process from handing a model a one-line prompt and shipping whatever comes back, and the numbers above are there so you don't have to take our word for that difference — you can see it.
It's also not a one-off. The architecture is protocol-agnostic and the validation approach doesn't depend on OUCH specifically — point it at a proprietary protocol, and the same shape applies: extend the existing structure, ground it in independently-verified reference data before writing a line of decoder code, then check the result multiple independent ways. None of it is Claude-specific either — the same process works with whichever LLM you're already running, including an internal deployment, which matters if your proprietary protocol is exactly the kind of thing that shouldn't leave your own infrastructure. For a category of protocol that's normally the most expensive, least accessible thing a firm has to license, that's the part worth sitting with.
In the next part of this blog series, we’ll be discussing how to achieve lossless, near-real-time decoding of market and order entry data.
If you’d like to discuss anything from this series, or would like to know more about FMADIO products, please see our Contact page: https://www.fmad.io/contact-us.
All code from this series remains public on our GitHub project: fmadio_markets
