# LoopSpec tooling integration for Flue

Guide version: 0.1.0
Last reviewed: 2026-08-13
Tested reference: `@flue/runtime@2.0.3`, `@flue/github@2.0.3`, Node 22, Python 3.11/3.13
Status: experimental tutorial

## What to build

Add an evidence-to-eval layer to an existing Flue application. It should turn privacy-minimised
runtime events, verified channel ingress, explicit application outcomes, and human-in-the-loop
capability probes into:

1. versioned episode records;
2. separate performance, capability, and authority assessments;
3. an evidence-linked candidate LoopSpec revision;
4. canonical LoopSpec validation and semantic diff output; and
5. a removal-probe regression eval that runs in CI.

This integration is a design-review tool, not a claim generator. It may propose a change, but it
must not claim that a real person gained or lost a capability, approve its own proposal, or deploy
a behavioural change.

## If the host project does not use Flue

Do not add Flue solely for this integration. Preserve the evidence-to-eval strategy and map its
framework-specific edges onto the host project:

- use the host runtime's lifecycle hooks or existing OpenTelemetry instrumentation instead of
  `observe(...)`;
- use a stable, application-owned case or run identifier instead of Flue's `submissionId`;
- join verified ingress through the channels the application already trusts;
- implement event projection and episode assembly in the host's language; and
- keep `.loop.yaml` plus the LoopSpec CLI as the canonical design and semantic layer.

The episode boundaries, privacy rules, separate assessment planes, explicit human probes, review
gate, and regression-eval pattern below still apply. Treat the Flue code as a worked adapter, not
as a required dependency.

## Inspect the Flue project first

1. Read the project's `AGENTS.md` and package-manager files.
2. Determine the Flue source root. Flue selects the first existing directory in this order:
   `.flue/`, `src/`, then the project root.
3. Inspect `app.ts`, `agents/`, `workflows/`, `channels/`, `cloudflare.ts`, `flue.config.ts`, and
   `wrangler.jsonc` where present.
4. Use the project's installed Flue types and current public APIs. Do not copy version-specific
   event fields without checking them.
5. Determine whether the deployment target is Node or Cloudflare before choosing persistence and
   where the LoopSpec semantic engine runs.

## Suggested source layout

Put ordinary support modules beneath the selected source root; do not place them in a discovery
directory unless they are intended to be discovered by Flue.

```text
<source-root>/
├── loopspec/
│   ├── types.ts
│   ├── privacy.ts
│   ├── observer.ts
│   ├── domain-evidence.ts
│   ├── assess.ts
│   └── engine.ts
└── channels/
    └── github.ts              # only when the project already uses this channel
tests/
├── loopspec-observer.test.ts
├── loopspec-channel.test.ts
├── loopspec-engine.test.ts
└── loopspec-removal.eval.ts
```

## Episode event contract

Define a small, versioned TypeScript union. Keep values JSON-serialisable.

```ts
type LoopEvent = {
  schemaVersion: "1.0";
  id: string;
  type:
    | "episode.queued"
    | "episode.running"
    | "episode.settled"
    | "channel.delivery.accepted"
    | "model.completed"
    | "action.completed"
    | "outcome.observed"
    | "capability.assessed"
    | "authority.window.measured";
  occurredAt: string;
  episodeId: string;
  source: "flue" | "channel" | "application" | "human_probe";
  specId: string;
  specDigest: string;
  trace: {
    instanceId?: string;
    conversationId?: string;
    operationId?: string;
    turnId?: string;
    toolCallId?: string;
  };
  data: Record<string, string | number | boolean | null>;
  contentPolicy: "structural-only" | "explicit-assessment";
};
```

Use the Flue dispatch receipt's `submissionId` as the episode join key. Do not reconstruct episode
identity by matching message text or timestamps. Record the LoopSpec semantic digest on every
event so later analysis can identify the exact design that produced it.

## Observe Flue without recording content

Register `observe(...)` from `@flue/runtime` once at module scope. Project the installed
`FlueObservation` union into the smaller event contract and return immediately for events that
are not needed.

The observer callback is synchronous on the emission path. Keep it cheap and non-throwing. Append
to an in-memory batch or enqueue to application-owned persistence, then flush through the host's
lifecycle. The observation stream is a signal, not a durable ledger.

Record structural facts only:

- submission lifecycle and outcome;
- model provider/name, duration, usage, and error boolean;
- tool name, origin, duration, and error boolean;
- task/operation kind and duration;
- compacted message counts; and
- hashed correlation identifiers.

Exclude `turn_request`, prompts, model output, reasoning, tool arguments and results, application
log messages, error messages, stacks, credentials, sender identity, and raw provider resource
names. Content export must remain an explicit opt-in outside this integration.

## Join an existing GitHub channel

If the project has no GitHub channel, start with Flue's current blueprint:

```bash
flue add channel github
```

Keep the channel's existing signature verification, routing, and tools. Inside the verified
callback, pass the provider delivery ID as the dispatch idempotency key, retain the receipt, and
append one structural channel event after admission:

```ts
const receipt = await dispatch(Assistant, {
  id: channel.instanceId(issueRef),
  idempotencyKey: delivery.deliveryId,
  initialData,
  message,
});

appendLoopEvent(projectAcceptedGitHubDelivery(delivery, receipt, {
  specId: "support_triage",
  specDigest: loopspecDigest,
  instanceId: channel.instanceId(issueRef),
}));
```

`projectAcceptedGitHubDelivery` must only be callable after signature verification. Hash the
instance ID and repository/issue identity. Do not record sender, title, body, or model-visible
message content. Resolve `loopspecDigest` from the target's validated configuration or binding;
do not hardcode it. Test one valid signature and one invalid signature; the invalid request must
be rejected before the application callback runs.

## Add evidence that telemetry cannot infer

Application outcomes and human assessments must enter through explicit typed functions. Never
infer human capability from the agent's trace.

For the tutorial, add a short delayed-removal probe: a flash-card-like exercise in which the human
must notice and escalate a severe support case without agent help. Record the score, threshold,
collection condition (`unassisted`), instrument (`delayed-removal-probe`), capability bearer, and
assessor. In production, make consent, access, retention, and interpretation explicit.

Also measure the authority window: seconds between the point when the accountable person can see
the proposed action and the point after which they can no longer prevent its consequence.

## Keep three assessment groups separate

Run the same complete episodes through three deliberately different views:

| Group | Question | Tutorial transition rule |
| --- | --- | --- |
| Performance | Did the human-agent system achieve the outcome? | Mean assisted score below `0.70` |
| Capability | Can the human still perform the critical act without assistance? | First-to-last score change at or below `-0.20` |
| Authority | Does the accountable human still have time to intervene? | First-to-last window change at or below `-30s` |

The synthetic reference sequence is intentionally contradictory:

```text
assisted outcome       0.88  0.91  0.90  0.93  0.92
human-only recovery    0.82  0.78  0.68  0.56  0.43
intervention window     120   110    92    58    35 seconds
```

Expected result: performance is stable while capability and authority transitions are detected.
The disagreement is the feature. A successful trace cannot establish that a person learned,
retained, or lost a capability.

## Generate a candidate, never an approved design

When capability declines, propose an unaided recovery checkpoint. When the authority window drops
below one minute, propose pre-execution confirmation. Link each change to the exact episode event
IDs that motivated it.

The proposal must include:

- `status: requires-human-approval`;
- the base `specId` and `specDigest`;
- all source episode IDs;
- the three assessment-group results;
- evidence IDs for every proposed pattern; and
- a claim ceiling stating that synthetic effects are hypotheses.

Write a candidate `.loop.yaml`; do not mutate the production spec in place.

## Keep canonical LoopSpec semantics in one engine

The Flue integration should be TypeScript. Use the versioned LoopSpec CLI JSON protocol for
canonical parsing, expansion, linting, and semantic diff instead of reimplementing semantics in
TypeScript.

```ts
interface LoopSpecEngine {
  check(spec: string): Promise<LoopSpecCheckResult>;
  expand(spec: string): Promise<LoopSpecExpandResult>;
  diff(before: string, after: string): Promise<LoopSpecDiffResult>;
}
```

Implement a Node adapter with `spawn()` and an argument array, never a shell string. Invoke:

```text
loopspec check <path> --json
loopspec expand <path> --json
loopspec diff <before> <after> --json
```

Require `protocol_version: 1` and fail closed on mismatch. Treat an invalid spec response as a
typed result; reserve transport errors for malformed JSON, incompatible protocol, spawn failure,
or an unexpected non-zero exit.

Do not attempt to spawn Python in a Cloudflare Worker. On Cloudflare, collect and enqueue the
privacy-minimised events at the edge, then run episode analysis and the LoopSpec engine in Node CI,
an MCP server, or a dedicated service. A future native TypeScript engine can replace the adapter
behind `LoopSpecEngine` only after cross-language fixtures prove exact IR, finding, semantic-hash,
and semantic-diff parity.

## Standards mapping

- Use Flue's OpenTelemetry integration for model, tool, task, and agent operation spans instead of
  creating duplicate spans.
- Represent point-in-time LoopSpec evidence as OpenTelemetry events or structured log records with
  low-cardinality names such as `loopspec.capability.assessed`; put identifiers in attributes, not
  in event names.
- Propagate W3C `traceparent` and `tracestate` across HTTP boundaries. Never place personal data in
  `tracestate`.
- If episodes cross a service or queue boundary, a CloudEvents 1.0 envelope is a reasonable
  transport wrapper; keep the LoopSpec event schema version independent of that envelope.
- Treat GenAI input/output content as sensitive and opt-in. This integration defaults to
  structural-only evidence.

## Evals and CI

Generate a deterministic removal-probe scenario that compares baseline and candidate designs.
Keep the scenario marked `synthetic: true`, retain its source episode IDs, and assert both the
human-recovery threshold and intervention-window threshold.

For a live preview deployment, install Flue's Vitest Evals blueprint and replace the deterministic
harness body with its public HTTP SDK harness:

```bash
flue add tooling vitest-evals
```

Add one CI job that runs, in order:

```bash
npm run lint
npm run typecheck
npm test
npm run tutorial
loopspec check generated/support-triage.candidate.yaml --fail-on-findings
loopspec diff loops/support-triage.base.yaml generated/support-triage.candidate.yaml
npm run evals
```

Use `npm ci`, a supported Node version, a Python matrix covering the project's supported versions,
dependency caching, least-privilege workflow permissions, and a job timeout. Do not give the eval
job production credentials.

## Acceptance criteria

- The project type-checks against its installed `FlueObservation` union.
- A signed channel delivery and its runtime observations share one `submissionId` episode.
- An invalid webhook signature never reaches the application callback.
- No forbidden content or raw provider identity appears in emitted events.
- The synthetic sequence produces stable performance plus capability and authority transitions.
- The generated proposal requires human approval and cites evidence IDs.
- The candidate spec passes the canonical LoopSpec check.
- The semantic diff is generated through protocol version 1.
- The removal-probe eval fails for the baseline and passes for the synthetic candidate assumptions.
- The application still operates when optional telemetry export credentials are absent.

## Primary references

- Flue CLI URL blueprints: https://flueframework.com/docs/cli/add/
- Flue project layout: https://flueframework.com/docs/guide/project-layout/
- Flue observability: https://flueframework.com/docs/guide/observability/
- Flue OpenTelemetry integration: https://flueframework.com/docs/ecosystem/tooling/opentelemetry/
- Flue GitHub channel: https://flueframework.com/docs/ecosystem/channels/github/
- Flue Vitest Evals: https://flueframework.com/docs/ecosystem/tooling/vitest-evals/
- OpenTelemetry event conventions: https://opentelemetry.io/docs/specs/semconv/general/events/
- W3C Trace Context: https://www.w3.org/TR/trace-context/
- LoopSpec source and CLI: https://github.com/morrisclay/loopspec