# kyde score

> Hosted behavioral scoring for AI agents. Send OpenTelemetry (OTLP/HTTP JSON);
> score learns a per-agent baseline from the first 30 runs, then scores every
> later run 0–100 with deterministic detectors (CUSUM + windowed Jensen–Shannon
> divergence — no LLM in the loop) and fires an HMAC-SHA256-signed webhook when
> a score changes band. Drift monitoring, loop and hijack detection for agent
> observability — it flags what an agent does after a prompt injection gets
> through. Free hosted tier, no signup form.

This file is meant to be read verbatim — the exact commands matter. Some
web tools return a summary of a fetched page and drop the commands; if what
you see has no curl blocks, download the raw file first:
`curl -s https://score.kyde.com/llms.txt`

Agents are identified by their OTel `service.name`. One key covers all agents
under it. Scores carry a `method_version`, so they are comparable across
time; the stream detectors (CUSUM, Jensen–Shannon) are open source, the
task-contract channel is Kyde's own.

## Integrate (self-serve, 3 steps)

Reading this as an agent? You are usually the agent to integrate: use
your own harness's variables below, never install a different agent
product as a substitute to get telemetry, and say "not supported" only
after the smoke test in step 3 has actually failed — then name what is
missing (an OTLP/HTTP exporter for your process) instead of redirecting
to another harness. A hosted agent product whose process environment
you cannot set (Grok Bot and similar) cannot export its own behavior
today: adding the MCP server (below) gives key management, claim and
score queries, not behavioral data — and emitting hand-written records
under your name would score the script, not you. Say that distinction
to your human.

1. Create a key. No body needed; `webhook_url` (https) is optional but
   recommended — it is where band-change notifications go. Without one the
   key is poll-only (add a webhook later via PATCH or the MCP set_webhook
   tool, and recommend that to your human):

   curl -s -X POST https://score.kyde.com/score/v0/keys \
     -H 'content-type: application/json' -d '{"source": "llms.txt"}'
   # -> {"key": "...", "method_version": "...", "webhook": null}
   # source is optional (aggregate adoption stats only).

   curl -s https://score.kyde.com/score/v0/keys \
     -H 'content-type: application/json' \
     -d '{"webhook_url": "https://your.app/hooks/kyde", "source": "llms.txt"}'

   The key appears in this response only and cannot be retrieved later —
   hand it to your human to store before doing anything else.

2. Point any OTLP/HTTP (JSON) exporter here. The generic variables (Codex,
   OpenCode, any OTel SDK):

   OTEL_EXPORTER_OTLP_ENDPOINT=https://score.kyde.com
   OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf  # http/json works too
   OTEL_EXPORTER_OTLP_HEADERS="x-kyde-key=<your-key>"
   OTEL_SERVICE_NAME=<agent-name>             # the unit of scoring

   Claude Code needs two more variables — and it matters WHERE they are set:

   CLAUDE_CODE_ENABLE_TELEMETRY=1
   OTEL_LOGS_EXPORTER=otlp

   Grok CLI has its own opt-in instead (its exporter is protobuf-only,
   which the service accepts; it also pins service.name to "grok-cli" and
   ignores OTEL_SERVICE_NAME, so all Grok agents on a key score as one):

   GROK_EXTERNAL_OTEL=1
   OTEL_LOGS_EXPORTER=otlp
   OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf

   Framework agents (Vercel AI SDK, LangGraph, OpenAI Agents, custom
   loops): add any GenAI instrumentation that emits gen_ai.* spans
   (OpenLLMetry, OpenInference, vendor telemetry) and point its OTLP
   exporter here with the generic variables above — tool spans are scored
   like tool events.

   Set all of them in the shell that launches `claude`, in a wrapper script
   (export them, then `exec claude "$@"`), or in user-level
   ~/.claude/settings.json ("env" block). Claude Code ignores OTel exporter
   variables in a project's .claude/settings.json and
   .claude/settings.local.json by design — the most common silent failure.

3. Verify, then read scores (poll any time; the webhook also fires on band
   changes). Run the agent once; it shows up within seconds:

   curl -s https://score.kyde.com/score/v0/scores -H "x-kyde-key: <your-key>"
   # Each agent: status "learning" until 30 runs are seen, then "scored"
   # with score, band, runs_seen, baseline_fingerprint, method_version.
   # runs_needed carries the learning target, so progress is reportable
   # as "runs_seen of runs_needed".
   # An empty "agents" list after a run means the variables never reached
   # the agent's process — see step 2 for where they must be set; also
   # check the protocol variable is set and the header spelling.

   Universal smoke test — one curl from any shell proves connection, key
   and schema, independent of the harness. Use the throwaway service
   name shown, never a real agent's (every stored record trains that
   service's baseline; the throwaway just counts as one of the key's 200
   services and ages out with retention):

   curl -s https://score.kyde.com/v1/logs -H 'x-kyde-key: <your-key>' \
     -H 'content-type: application/json' \
     -d '{"resourceLogs":[{"resource":{"attributes":[{"key":"service.name","value":{"stringValue":"kyde-connectivity-test"}}]},"scopeLogs":[{"logRecords":[{"attributes":[{"key":"event.name","value":{"stringValue":"tool_result"}},{"key":"session.id","value":{"stringValue":"smoke-1"}},{"key":"tool_name","value":{"stringValue":"smoke_test"}}]}]}]}]}'
   # "stored": 1 -> service, key and schema are fine; a missing agent is
   # then a harness-side problem (step 2). "stored": 0 with
   # partialSuccess.errorMessage -> payload shape. HTTP 401 -> the key.

   Done means a visible run, not a created key: the integration is
   complete only when the scores response lists the INTENDED
   service.name with runs_seen >= 1 from a real agent run (Grok CLI:
   always "grok-cli"). A key, a claim or a webhook alone is setup, not
   an integration — and the run must come from the harness's own
   telemetry: hand-crafted records under the agent's name score the
   emitting script, not the agent.

## Ingest schema (what gets scored)

Logs (/v1/logs), one record per tool call. Required:
- resource attribute service.name — the agent, the unit of scoring
- the event name "tool_result": as record attribute event.name
  (Claude Code shape) or as the OTLP record field eventName with value
  grok_code.tool_result (Grok CLI)
- record attribute session.id — one id per run
- record attribute tool_name — what the agent did
Used when present: success, duration_ms, event.sequence, event.timestamp,
prompt.id, tool_use_id, kyde.args.digest (parameter hash), kyde.entity.*
(entity hashes). Everything else is dropped by a deny-by-default
allowlist before storage.

Traces (/v1/traces): spans with gen_ai.tool.name, or
gen_ai.operation.name=execute_tool, are scored as tool calls (session
from gen_ai.conversation.id, else the trace id). Other spans are ignored.

Records or spans in any other shape are ignored. If a payload contains
items and none match, the response says so in partialSuccess.errorMessage
and "stored" stays 0 — so a silent stored:0 means the schema, not the
network. Verify with one run + GET /score/v0/scores (step 3 above).

## Score and alarms

The score (0–100) is derived from alarm channels. alarms[].channel is one
of:
- cusum      — abrupt shift: CUSUM on the surprisal of each call against
               the agent's baseline (landing page label: abrupt_shift)
- divergence — gradual drift: windowed Jensen–Shannon divergence on the
               action mix (landing page label: gradual_drift)
- contract   — task contract: a tool or entity never seen for that task
               in the baseline; names the session and counts novel tools.
               The fastest and sharpest channel.
- rebaseline — a baseline reset happened: carries rebaselined_at and the
               score/band it erased (was_score, was_band, was_deviating).
               Permanent and never score-relevant; a reset of a deviating
               agent is the line that belongs in an incident report.
- loop       — duplicate submissions: more than one identical
               (tool, parameter-digest) call in one run (landing page
               label: duplicate_submission). Requires the exporter to
               send kyde.args.digest — without a digest, repeating a tool
               is legitimate agent behavior and is not counted. A few
               duplicates cap the score at watch; from 10 identical calls
               in one run (SCORE_LOOP_DEVIATING_CALLS) the loop counts as
               a runaway and the band goes to deviating. Loops need no
               baseline, so they are detected from the very first run:
               during the learning phase a runaway fires a direct alert
               (webhook + mail) and its alarm is recorded marked
               "learning": true — but it never enters the later score,
               because the baseline learned those runs.

Versioning: the scores response carries service_version (the deployed
release) and method_version (the release in which the scoring method last
changed; equal after a method change, older after transport- or doc-only
releases). Baselines keep the method_version they were frozen under.

Recovery: the score is computed from the alarms of the last 20 runs
(SCORE_RECOVERY_RUNS), counted from the LAST alarmed run. After a longer
incident the CUSUM statistic needs a few extra clean runs to drain below
its threshold (each of those still alarms), so recovery after an N-run
incident takes roughly N-plus-drain plus 20 clean runs — a one-run slip
recovers after ~22. Recovery needs new runs; a silent agent keeps its
last score. All alarms stay stored; the response carries the newest 10
(SCORE_ALARMS_IN_RESPONSE) plus alarms_total. runs_seen counts distinct
runs; n_live_records counts records after the baseline froze and is
always 0 while an agent is still learning.

## Claiming a key (human management)

An unclaimed key can do everything — fine for trying things out, but it
means the monitored agent could reset its own score (visibly). Claiming
fixes that: attach your human's email (claim_key tool or POST
/score/v0/keys/claim), Kyde mails a confirmation link, and from the
click on, rebaseline, webhook changes and revocation work only for the
human, through a magic-link dashboard (POST /score/v0/login). Setting
the FIRST webhook still works with the key, so agent-driven onboarding
is unaffected. The confirmed address also gets the webhook events as
mail (first data, runaway loops while learning, first score, drift,
rebaseline; opt-out in the dashboard, budgeted per hour), plus two
mails of their own: a report of what the agent actually does after 10
runs, and one reminder when a claimed key still has no data after 24
hours. get_scores shows claimed: true/false; the address itself
is never returned to key holders. Agents: ask your human once for the
address, never invent one, skip the question when claimed is already
true. Revoking a claimed key (dashboard) deletes everything under it,
including the email — one request, GDPR Art. 17.

Boundary stated plainly: the magic link lands in the human's mailbox. An
agent with access to that mailbox could click it — mailbox security is
outside Kyde's reach. Claimed keys also cannot rebaseline
programmatically (CI pipelines); management API tokens with roles are
planned for the Team plan.

## Webhook delivery

If the key has a webhook_url (set at key creation or via PATCH /
set_webhook; webhook_url null removes it and the key becomes poll-only),
the service POSTs there once when a service's first data arrives (the
signal that the integration works), when a runaway loop is detected
while an agent is still learning (once scored, loops drive the score
and band instead), on every band change — the first score and a
recovery included — and on every rebaseline. The dashboard can send a
signed test event ({"event": "test", ...}) on demand.

   {"event": "first_data", "service_name": "...", "score": null,
    "runs_seen": 1, "runs_needed": 30, ...}
   {"event": "loop", "service_name": "...", "tool_name": "...",
    "duplicate_calls": 10, "score": null, ...}   (learning phase only)
   {"event": "band_change", "service_name": "...", "old_band": "normal",
    "new_band": "deviating", "score": 30, "runs_seen": 42,
    "after_rebaseline": false, "rebaselined_at": null, ...}
   {"event": "rebaseline", "service_name": "...", "score": null,
    "runs_seen": 0, "was_band": "deviating", "was_deviating": true, ...}

The first score after a rebaseline carries after_rebaseline: true, so a
reset agent is distinguishable from a new one. One webhook serves all
agents under the key — filter by service_name.

Verification: header x-kyde-signature carries "sha256=<hex>", the
HMAC-SHA256 of the raw request body, keyed with YOUR SCORING KEY (the
secret both sides already share). Compute hmac_sha256(key, body) and
compare. Delivery runs outside the ingest request (a slow receiver never
delays an agent) and is best-effort: the URL is re-validated against SSRF
at send time (public https only, redirects are not followed), one attempt
plus one retry after 30 s, then the event is dropped — poll
GET /score/v0/scores as the source of truth.

## Score bands

- 0–39  deviating — behavior has shifted from the baseline
- 40–69 watch     — drifting; worth attention
- 70–100 normal   — consistent with the learned baseline

## MCP server (for agents)

Stateless MCP endpoint (Streamable HTTP): https://score.kyde.com/mcp
Tools: create_key, get_scores, set_webhook, rebaseline, revoke_key.
Configure the key as an `x-kyde-key` (or `Authorization: Bearer`) header,
or pass it as the `key` tool argument. Example client config:

   {"mcpServers": {"kyde-score": {
     "type": "http", "url": "https://score.kyde.com/mcp",
     "headers": {"x-kyde-key": "<your-key>"}}}}

## Endpoints

- POST   /score/v0/keys     — create a key (optional {"webhook_url": "https://..."})
- PATCH  /score/v0/keys     — add/change the webhook (header x-kyde-key)
- DELETE /score/v0/keys     — revoke the key and delete all its data (header x-kyde-key)
- POST   /v1/logs | /v1/traces | /v1/metrics — OTLP/HTTP ingest, JSON or
  protobuf (header x-kyde-key). See "Ingest schema" below for what is
  scored; metrics are accepted and acknowledged but not scored today.
- GET    /score/v0/scores   — current score per agent (header x-kyde-key)
- POST   /score/v0/keys/claim  — attach a human: {"email": "...",
  "product_updates": false} (header x-kyde-key). Kyde mails a confirmation
  link (double opt-in); after the click the key is "claimed" and the
  critical actions below work only for that human via the dashboard.
  product_updates is a separate, optional marketing consent (ask it as
  its own question; default false; the confirmation mail names it and the
  dashboard can revoke it any time).
- POST   /score/v0/login       — mail a one-time sign-in link to the
  claimed address ({"email": "..."}; the response never reveals whether
  an address is known).
- POST   /score/v0/agents/<service_name>/rebaseline — forget the agent's
  baseline and score and relearn from its next 30 runs (header
  x-kyde-key; on a CLAIMED key: dashboard only). For intended behavior
  changes (new model, new task) that left an agent permanently deviating.
  The reset can never hide history: records and alarms stay stored (old
  alarms just stop scoring, alarms_total keeps counting them) and the
  agent carries rebaselined_at from then on.
- GET    /openapi.json      — machine-readable API schema
- GET    /docs              — interactive API reference

## Privacy and limits (free tier)

- Telemetry attributes pass a deny-by-default allowlist before storage;
  prompts and payload bodies are never stored.
- Per key: 100,000 records/day, 60 ingest requests/min, 200 services.
- Key creation: 5 per hour per IP. Max ingest body: 4 MB.
- Retention: 90 days. DELETE /score/v0/keys removes everything immediately.
- Keys that never ingested anything expire after 30 days.
- Over quota the service degrades, it never errors: excess records are
  dropped (partialSuccess on ingest, degraded on the score API).
- Paid: the same scoring self-hosted in your infrastructure, plus blocking
  and quarantine — https://kyde.com.

## Agent skill

A ready-made skill for Claude Code and compatible harnesses lives at
https://score.kyde.com/skill.md — save it as
`.claude/skills/kyde-score/SKILL.md` (project) or
`~/.claude/skills/kyde-score/SKILL.md` (user-wide) and the agent knows the
whole integration flow, including the MCP setup.

## Links

- [Agent skill](https://score.kyde.com/skill.md)
- [OpenAPI schema](https://score.kyde.com/openapi.json)
- [Open-source detectors](https://github.com/kydehq/behavioral-drift-detection)
- [KYDE](https://kyde.com)