James Summerill

← All work

Public repo 2026

agents-fyi miner

Finds real failures by executing one release of a package against the next.

A Ruby client that installs two releases of a package, asks each what its public callables accept, and executes the same call against both. When the older release accepts the call and the newer one refuses it by name, the interpreter's own traceback is filed — but only after the corpus says the finding is new. It also answers open reports with an executed fix and confirms other operators' reports by reproducing them, as one of two independent methods.

Documented example

A differential, run on one package

./bin/agents-fyi-miner mine --package httpx --from 0.27.2 --to 0.28.1
httpx 0.28.1: httpx 0.28.1 removed the app= argument (2 call site(s), 0 repaired)
httpx 0.28.1: httpx 0.28.1 renamed proxies= to proxy= (11 call site(s), 11 repaired)
httpx 0.28.1: httpx 0.28.1 removed the http2= argument (1 call site(s), 0 repaired)
httpx 0.28.1: httpx 0.28.1 removed the cert= argument (9 call site(s), 0 repaired)
found 4 finding(s); nothing was filed

What it does

  • A finding is arithmetic: the same call was executed against both releases, and the traceback filed with the report is the interpreter's output from the refusing run.
  • mine needs no corpus and files nothing; publish asks the service whether each finding is new, and --dry-run asks without writing.
  • Findings are clustered by mechanism, because one upstream decision can rename an argument across eleven callables.
  • A replacement argument is executed before it is offered; a rename is only claimed when a candidate is built from every word of the old name, and a tie is refused.
  • The two verifiers run the same image under their own accounts: one executes the call sites the report names, the other re-derives the claim from the two releases and ignores the report's prose.
  • The ledger is state, not a cache: a pair that failed is retried, a pair that was examined is not, and an unbuildable release is stamped with the interpreter and whether a compiler was present.

Built with

  • Ruby
  • Python
  • PyPI
  • RubyGems
  • Docker

Writing