AI TRAINING SYSTEMS

Atomos Sales

Find the behavior behind a playbook score.

01 / THE SYSTEM

The problem.

A single score gives a coach little guidance on what to edit. Reviewing a revision means looking at both the conversation-level penalties and its performance across simulated runs.

Role I built the simulated buyer and seller, evaluation workflow, version tracking and parallel work claiming.

Approach I gave the evaluator a versioned rubric and recorded results by generation. The score report marks retained and reverted revisions, while the insights command breaks out the behaviors that attracted deductions.

PythonSQLiteClaude APIOpenAIElevenLabs
Training report

Review a playbook revision

Loading imageTerminal session showing ranked simulation results and the penalty-frequency report.
The reviewer moves from the ranked simulation runs to the insights command. Its penalty report distinguishes invented features from unsupported claims and shows how often each drew a deduction.Open full-size still (opens in a new tab)
02 / ARCHITECTURE

How it connects.

Adversarial training / evaluation / multi-provider inference

  1. 01BUYER AGENTS
  2. 02PLAYBOOK AGENTS
  3. 03LLM JUDGE
  4. 04EVOLVE + EVALUATE
03 / THE DETAILS

Implementation evidence.

01

Read the score and its causes

The saved chart shows the training history. The command-line report lets a reviewer inspect how often a particular penalty occurred before choosing what to revise.

02

Keeping parallel work distinct

In the lead-research pipeline, a worker claims a pending row in SQLite before starting research. That gives parallel workers distinct assignments when they pick up work at the same time.

PROOF

Evaluation is part of the product

Simulated buyers exercise a playbook. A model scores those conversations against the current rubric so the reviewer can compare revisions and roll one back.

Outcomes

  • A reviewer can compare conversations against the rubric used for that training run.
  • An unsuccessful revision has a rollback path to the earlier playbook.