parsarium

0x0000about

A shared set of targets for parser research.

I'm José Antonio Zamudio Amaya, a PhD researcher at CISPA in Saarbrücken. My work is on fuzzing and test generation for programs whose inputs have structure: files, requests, and protocol messages. Parsarium is the target set I built for that work, made public so other people can use it too.

0x0010why

Why I built it.

Most studies on parser testing start the same way: pick some programs, install them, write an adapter that feeds them bytes, and check that they work on a valid input. Each group does this again for every paper, and each group picks differently. That makes results hard to compare and hard to reproduce.

I wanted to do this step once and do it well. For each format and language, Parsarium searches the package registries, verifies the parsers it finds against a valid control, ranks them by adoption, and wraps the ones that pass in a pinned harness. The result is a set of targets you can inspect, rerun, and cite.

What it is for.

Comparing fuzzers and test generators on the same targets. Checking how many implementations of a format accept or reject a given input. Studying how parser behaviour differs across languages. Teaching: the manual uses Parsarium for all of its exercises.

It also works as a starting point for security research. A generated corpus can be replayed through every parser for a format with one command, and any target can go straight into an AFL++ campaign.

0x0020a target

What a target contains.

Every runnable target lives in projects/<format>/<language>/<package>/. Taking Pillow's PNG target as an example:

Pinned release
pillow 13.0.0, installed from a lock file with its source hash recorded.
Input API
The documented call that parses the bytes, here Image.open(), Image.load(), and Image.getpixel(). Generic file I/O or MIME sniffing does not count.
Valid control
controls/png/red-green.png, a 71-byte image with one red and one green pixel.
Expected output
What the parser must report for the control: width 2, height 1, and both RGBA pixels.
Docker harness
A Dockerfile on a pinned base image and a harness that reads one input and prints what the parser observed.
Verification receipt
The image ID, toolchain versions, file hashes, the observed output, and how many parser source lines the control covered.
Fuzzing
An AFL++ overlay with its own feedback receipt, plus the ecosystem's native engine (Atheris for Python).

0x0030selection

How parsers are chosen.

The catalog works one format × language cell at a time, keeping the most adopted verified packages in each cell, up to the limit the selection protocol sets. A parser qualifies when it is open source, has a pinned release, and parses local bytes through a documented API with an observable, format-specific result. Wrappers are allowed and their underlying parser family is recorded.

Packages are ranked by one adoption metric per cell, usually registry downloads, and only against other packages in the same cell. Standard-library APIs are verified the same way and listed separately. Every search query, result, and unresolved lead is saved, so you can see exactly how a cell was built. The full rules are in the selection protocol.

0x0040notebook

Lab notebook.

Each harness writes a receipt when it is verified. These are the dates of the current 986 receipts, and the most recent ones. The site is rebuilt from the repository on every change.

  1. 36
  2. 722
  3. 227
  1. yaml/c/libfyamllibfyaml 0.9.6
  2. yaml/cpp/rapidyamlrapidyaml 0.16.0
  3. toml/cpp/cpptomlcpptoml 0.1.1
  4. toml/php/php_collective_tomlphp-collective/toml 0.1.5
  5. toml/javascript/toml_eslint_parsertoml-eslint-parser 1.0.3
  6. toml/python/tomlitomli 2.5.0
  7. yaml/javascript/yamlyaml 2.9.1
  8. toml/java/jackson_dataformat_tomljackson-dataformat-toml 2.23.0

0x0050cite

Citing Parsarium.

If you use Parsarium in a paper, please cite the package version you installed (parsarium --version) or the commit you used. When you work with a subset, include the selection ID from the bench as well.

@misc{parsarium,
  author       = {Zamudio Amaya, Jos{\'e} Antonio},
  title        = {Parsarium: An Open Testbed of Real Parsers},
  year         = {2026},
  howpublished = {\url{https://parsarium.com}},
  note         = {Commit 2f739f07}
}

0x0060contact

Contributing.

Suggestions for parsers, formats, or fixes are welcome as GitHub issues. If you find a bug in a parser using Parsarium, report it to the upstream project first, through its security policy when it has one. For research collaborations, you can find me at pepezamudio.com.