Experiments — The AI Pipeline Engine

An Experiment is a single run of an AI workflow. It moves through up to 20 states — from Draft all the way to Completed — with automatic checkpointing, budget enforcement, human approval gates, and a full audit trail at every step.

Scenario: A SaaS company uses experiments to detect churn signals every week. The experiment pulls usage data, scores accounts by risk, drafts personalised win-back emails, and waits for a human to approve before sending.

The 22-state pipeline

Every experiment progresses through an explicit state machine. Transitions are validated before execution — you can never skip states or create inconsistent data.

State	What happens
Draft	Created, not yet submitted for processing.
SignalDetected	Created automatically from an inbound signal via a Trigger Rule.
Scoring	AI evaluates the goal and assigns a feasibility score.
Planning	Breaks the goal into a step-by-step execution plan.
Building	Constructs the execution environment and validates resources.
AwaitingApproval	Paused for human review before execution.
Approved / Rejected	Human decision recorded. Rejected experiments loop back to Planning.
Executing	Agent is actively running the plan. Live logs available.
AwaitingChildren	Waiting for parallel workflow branches (dynamic_fork nodes) to complete.
ExecutionFailed	A stage failed during execution. Retryable from any checkpoint step.
CollectingMetrics	Output gathered, token costs settled.
Evaluating	Quality check — did the output meet the goal?
Iterating	Auto-revision loop when output quality is insufficient.
Completed	Terminal — success.
Paused	Temporarily suspended. Can be resumed.
Killed	Terminal — manually terminated.
*Failed states	ScoringFailed, PlanningFailed, BuildingFailed — each retryable.

Terminal states (Completed, Killed, Discarded, Expired) are irreversible. Use Pause if you want to hold an experiment temporarily.

Creating an experiment

Create from the UI at /experiments, or via API:

bash

curl -X POST https://fleetq.169.58.89.204.sslip.io/api/v1/experiments \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Churn Risk Analysis — Week 12",
    "agent_id": "01jf4a2b-...",
    "goal": "Analyse usage data and identify top 10 churn-risk accounts",
    "budget_cents": 5000
  }'

Monitoring in real time

The experiment detail page shows:

Timeline — state transitions with timestamps and actor
Execution Log — live streaming output from the agent
Stage progress — which playbook steps have completed
Cost tracker — credits reserved and settled in real time
Artifacts — versioned outputs you can preview or download

Retrying from a specific step

Instead of rerunning the entire experiment, you can retry from any step. Only that step and its downstream dependencies are reset — completed steps are preserved.

bash

# Retry from step "generate_email" onwards
curl -X POST https://fleetq.169.58.89.204.sslip.io/api/v1/experiments/EXPERIMENT_ID/retry-from-step \
  -H "Authorization: Bearer YOUR_TOKEN" \
  -d '{"step_id": "STEP_ID"}'

Experiment tracks

Tracks classify what kind of business outcome an experiment targets. This enables filtering, reporting, and metric attribution by business goal:

growth

Acquisition, activation, and new revenue experiments.

retention

Churn prevention, win-back, and engagement experiments.

revenue

Upsell, expansion, and monetisation experiments.

engagement

Product usage, content, and community experiments.

debug

Internal diagnostics, testing, and platform experiments.

Artifacts

Every experiment produces one or more Artifacts — versioned output files (documents, code, data, or media). You can:

Preview them inline at /artifacts/{id}/render
Download via GET /api/v1/artifacts/{id}/download
Pipe them into outbound connectors (email, Slack, webhook)