Discuss research

NeoLab

Researching and engineering the next generation of AI systems.

NeoLab investigates difficult problems in AI systems and turns hypotheses into experiments, prototypes, benchmarks and reusable capabilities. We are early, and this page says exactly how early.

See the research betsDiscuss research
Each orbit is a research track; each point is a project on it. The solid orbits are the tracks we are committed to, the dashed ones are directions we are only exploring. Hover a label to open it.

Where each project stands

Every project carries one of five labels, and only moves up when the evidence does. We publish negative and inconclusive results alongside positive ones. The line fills as far as our most mature project, and no further.

  1. Proposed

    A question we intend to work on. Nothing built yet.

    Nothing here now

  2. In progress

    Being built now. No results to report.

  3. Prototype

    A working artifact exists. Not yet evaluated against baselines.

  4. Evaluated

    Measured against stated baselines, with results and limitations written up.

    Not reached yet

  5. Published

    Write-up, code and data released for others to check.

    Not reached yet

Research agenda

One primary track, one secondary, and two directions we are only exploring. Each track is a question we can test, not a topic we follow. The moving pictures show how each approach works.

Primary track

Verifiable intelligence

How do you tell whether an agent did real work well, when the outcomes are slow, noisy or partly subjective?

Evaluation environments and benchmarks for real-world agent work, such as marketing, sales and product management, that grade only what can be graded honestly.

Grading a belief against a hidden truthIllustration, not results

Secondary track

Reliable and coordinated agents

What does an agent need to survive production: durable state, approvals, recovery, and runs a person can follow?

Protocols and runtime patterns that keep long-running agents recoverable, interruptible and legible to the people supervising them.

A run that survives a crashIllustration, not results

Future direction

Context and memory

How should agents read, cite and remember the documents and decisions they work from?

Exploratory. One open artifact so far, with no measurements published.

From document to citable contextIllustration, not results

Future direction

Efficient AI infrastructure

When can a cheaper model do the job, and how do you know before routing to it?

Exploratory. Early routing and serving code, with no results published.

Routing work to the right modelIllustration, not results

Research bets

Each bet has a page with its problem, question, approach, findings so far and limitations.

Open code you can run today

Quickstarts are taken from each repository’s README. Licences are as stated in the repository.

  • abhichat85/pmevalMIT

    PMEval. A benchmark for AI agents doing the actual work of a product manager.

    git clone https://github.com/abhichat85/pmeval && cd pmeval
pip install -e .
pmeval run --agent echo --agent-name baseline
    Verifiable intelligenceSource
  • abhichat85/agent-streamMIT

    agent-stream. An open event protocol for streaming what an agent is doing (tokens, tool calls, thinking, progress) to any interface.

    pip install agent-event-stream
npm install @agent-stream/react
    Reliable and coordinated agentsSource
  • abhichat85/papyrusApache-2.0

    Papyrus. Turns documents into Markdown that agents can read and cite, with structure and page anchors preserved and no LLM in the conversion path.

    git clone https://github.com/abhichat85/papyrus && cd papyrus
pip install -e .
papyrus convert report.pdf
    Context and memoryLive demoSource
  • abhichat85/photonNo licence

    Photon. An inference gateway with model routing, quality gates and a fine-tuning pipeline. No results published.

    Efficient AI infrastructureSource

Discuss the research

If you work on agent evaluation, reliability or grounding, we would like to compare notes, share results or plan an experiment together.

Start a research conversation

Contribute to open work

Issues and pull requests are welcome on the open repositories. For anything larger, write to us first.

Talk about contributing