Dokimi (Greek δοκιμή, "test, proof") measures whether a skill, MCP doc source, agent profile or repository guidance actually helps a coding agent: paired on/off arm comparisons, novel-vs-known content analysis, and repository scans with graded agent runs.
// readme
Dokimi — A/B testing for agent context
Proof, not popularity: find out whether your agent’s skills, docs and AGENTS.md actually work.
Documentation: https://intel.github.io/dokimi/ — the full tour, with every figure.
What it is
Skills, instructions, docs, MCP servers and a repository’s AGENTS.md are cheap
to write and easy to over-claim. Dokimi runs the same questions or coding tasks
with and without one change, grades both the same way, and reports the
difference with its p-value, per model. Then it shows which parts of a skill
the model already knew, which it added and which it broke, and proposes a
shorter version that keeps what matters.
dokimi — Greek δοκιμή, “test, proof”. An artifact gets trusted because it passed a test that could have failed, not because it is popular.
Which parts help: novelty. Dokimi asks the model about each claim in a skill twice, without the skill and with it, and a different model grades both answers. Every claim lands in one of four outcomes (more):
What measuring shows so far…