Chapter 04
Choosing targets
Cells, adoption ranks, parser families, and standard APIs: how to pick a target set that fits your question.
The targets you choose shape the results you get. A fuzzer that does well on C image decoders may do very differently on Java archive readers. This chapter explains how the catalog is built, so you can pick a set that answers your question and describe it clearly.
Cells and the working goal
The catalog is built one cell at a time, where a cell is one format in one language. For every cell, Parsarium searches the package registries with the format name and terms like parser, decoder, or reader, inspects the results, and verifies the candidates against a control. Each cell keeps the most adopted verified packages, up to a limit set in the selection protocol. A cell can stay below the limit when the search finds no more candidates with enough adoption evidence. Every search query, its date, and its hit count is saved next to the cell, together with any leads that are still open.
Adoption ranks
Inside a cell, packages are ranked by one adoption metric: registry downloads when the registry publishes comparable numbers, otherwise a documented alternative such as GitHub stars. The ranking says which packages people use most. It is not a quality score, and it is only meaningful inside the cell, because downloads on npm and stars on GitHub are not the same unit.
Ranks are useful for picking a smaller set. Taking the top five per cell gives you the libraries most likely to sit in real software.
Packages and families
Many packages are wrappers. In PHP, most image packages call GD or ImageMagick underneath. In JavaScript, many PDF packages build on pdf.js. Each catalog entry records its parser family: the code that actually reads the format.
Whether to count packages or families depends on what you want to measure:
- If you study ecosystems (how PHP developers parse images, for instance), every package counts, because each one is a separate dependency with its own API and wrapper code.
- If you study parser engines (how many distinct implementations crash on an input), count families. Otherwise one bug in GD shows up once per wrapper.
- If you compare fuzzers, one target per family keeps the comparison from leaning toward whichever engine happens to have many wrappers.
Standard-library APIs
Some languages ship parsers in their standard library: Go's image/png and net/http, Python's zipfile and http.client, Java's java.util.zip. These are verified the same way as packages and have their own harnesses, but they are listed separately and are not part of the adoption ranking. Where they exist, they are often the most used parser in their language, so most studies will want to include them.
Feedback kind
Targets also differ in how AFL++ sees their coverage. C, C++, and Rust targets are compiled with instrumentation. The others use a bridge that reports parser source coverage. If your experiment depends on fast, native feedback, filter on it. The next chapter covers this in detail.
Writing it down
The bench turns these choices into a selection. It gives the selection an ID derived from its target list, a manifest with releases and controls, and a table for your report. With the ID and the Parsarium commit, anyone can rebuild the same set.
Key ideas
- The catalog is built cell by cell, keeping the most adopted verified packages up to the limit in the selection protocol.
- Adoption ranks compare packages only inside one cell, because each cell may use a different metric.
- Many packages share a parser family. Count packages to study ecosystems; count families to study engines or compare tools.
- Standard-library APIs are verified like packages but listed separately and not ranked.
- A selection ID plus the Parsarium commit is enough for someone else to rebuild your exact target set.
Check yourself
Answer all five to see your score
Can you compare rank 1 in the Python PNG cell with rank 1 in the Rust PNG cell?
- Yes, both are the most popular.
- Yes, if both use downloads.
- No. Ranks are only meaningful inside one cell.
Each cell uses one documented metric, often from a different registry. Downloads on PyPI and on crates.io are not the same unit.
You want to compare two fuzzers on image formats. Which selection fits best?
- One target per parser family
- Every package in every image cell
- Only the top-ranked package in each cell
One target per family keeps the comparison from leaning toward engines that happen to have many wrappers, such as GD in PHP.
Eight PHP PNG packages crash on the same input inside the same GD function. How many bug reports should you write?
- Eight, one per package
- One, to GD
- None, PHP bugs are out of scope
The bug lives in the shared engine. One report to the engine fixes it for every wrapper.
How are standard-library APIs treated in the catalog?
- They are ranked with the third-party packages.
- They are verified the same way but listed separately and not ranked.
- They are not included.
Go's image/png or Python's zipfile go through the same control check and have harnesses, but they sit outside the adoption ranking.
What lets someone rebuild exactly the target set you used?
- The number of targets
- A screenshot of the bench
- The selection ID and the Parsarium commit
The ID is derived from the sorted target list, and the commit pins every release, harness, and control. The manifest records both.
Exercises
Exercise 4.1Warm-up
Open the PDF cells and compare packages with families.
$ parsarium catalog pdf javascript $ parsarium catalog pdf go
Then open the bench with PDF selected, and turn on One target per parser family. How many targets are left? Which languages lose the most? Note the two selection IDs.
Hint
Look for repeated family names in the last column of each ranking, such as mozilla/pdf.js.
Solution
JavaScript loses the most: most of its PDF packages build on pdf.js, so they collapse into one target. Go also shrinks, because several of its packages descend from the same PDF reader. The two IDs differ because the target lists differ; write both down if you report results for both.
Exercise 4.2Practice
Build a selection for a fuzzer comparison on image formats: PNG and ASCII PNM, one target per family, at most three per cell. Then turn on Only compiled-edge feedback. How does the selection change, and what would you lose by keeping that filter on?
Hint
Compiled-edge feedback comes from C, C++, Rust, and a few native targets in other languages.
Solution
With the filter on, the selection shrinks to compiled targets, mostly C, C++, and Rust. You gain fast, precise feedback for every target, which suits tools that depend on native instrumentation. You lose the managed languages, and with them parsers that many applications actually use. A fair report states which choice you made and why.
Exercise 4.3Challenge
Some cells are much smaller than others, and PHP × ASCII PLY is one of the smallest. Open its data and find out why.
cat "$PARSARIUM/data/php/ascii_ply/cell.json" cat "$PARSARIUM/data/php/ascii_ply/leads.json"
Hint
Read the uncertainty field in cell.json; it summarises the search and what was found.
Solution
The searches over Packagist, GitHub, and Sourcegraph returned hundreds of hits, but very few were PHP code that actually parses PLY: most were writers, unrelated names, or JavaScript embedded in PHP pages. Some of the packages it did find share one engine. The cell is small because the search surface is documented and exhausted, not because nobody looked. This is what the protocol means by a recorded search limit.