The problem
Agents now write outreach, publish content and try to shape how brands appear in AI answers. Nobody can tell whether they learn a market or just add noise to it. Real outcomes are slow, noisy and costly to learn from: every bad email burns a sending domain and every bad page is public. LLM judges can be gamed, and LLM-simulated buyers are too cooperative.
Why it matters
Without an honest way to grade go-to-market decisions, agents in this space are judged on demos and anecdotes. A trustworthy grader is a precondition for improving them, and for letting them act with less supervision.
Research question
Can go-to-market agents be graded honestly, on their market beliefs, decisions and experimental discipline, inside simulated markets where the right answer is known?
Working hypothesis. Discipline learned in simulated markets with planted truth produces better real-world outcomes. This is unproven. If gym scores do not predict real movement, the gym is an evaluation tool, not a training ground, and we will say so.
Approach
- Each world has hidden parameters. Tactical truths, such as which levers work and how fast, are randomised across families of worlds, so agents are rewarded for learning each market rather than memorising one theory of marketing. Ethics and budgets are fixed.
- Rewards are the expected value of the agent’s committed actions under the hidden truth, not the noisy samples it observes.
- Every task ends with a structured belief report, scored with proper scoring rules. Claiming a false effect and missing a real one both cost.
- Consent, fabrication, disclosure and budget rules are hard gates. A violation ends the episode, and performance cannot buy it back.
- Scores are normalised between an honest agent that does nothing (0) and a planner that knows the hidden truth (1).
- It does not grade taste, such as whether copy is witty or on-brand. That stays with people and real outcomes.
Methodology
- The first gym covers answer-engine optimisation: how a brand appears in AI-generated answers.
- Worlds are deterministic and seeded: the same seed gives the same world and the same random draws.
- Training and test seeds are kept separate, and some world families are held out entirely.
- A release gate checks determinism, the ranking of reference agents, and leaks of hidden state before any version is used for results.
Findings so far
No results yet. The engine, the first two task types and a set of reference agents are built, and the validity checks are being written. No external model or agent has been evaluated.
Limitations
- A simulator can be wrong about the world. How well gym scores track real outcomes is unmeasured until Propel has live data to compare against.
- Competitors in the first version are scripted, not adaptive.
- Only the answer-engine gym is in scope. Outbound and go-to-market operations come later.
Artifacts
The repository is not public yet. The engine is planned to be released under the Apache-2.0 licence once it passes its own release gate.