Skip to main content
This is a preview/beta feature. Further changes are expected.

​
Overview

An AI coding session is a unit of work an AI agent performed against a repository — the models it used, the tools it invoked, the messages exchanged, and the exact lines of code it produced. Chainloop captures every session automatically through hooks installed by chainloop trace init and turns it into a signed CHAINLOOP_AI_CODING_SESSION attestation. Once those attestations are stored, the platform correlates them with pull requests, evaluates policies against them, and aggregates them on the AI Coding dashboard — giving security and engineering teams a single view of how AI agents contribute to the codebase. Capture, attribution, redaction, attestation, and policy evaluation are open source. Pull request correlation, session lineage, the dashboard, the score, and Chainloop’s curated policies are platform features — see Open Source vs. Platform for the full split.

​
The Problem It Solves

AI agents now write a meaningful share of production code, but the activity is invisible to the rest of the organization. Without a record of what the agent did, security and engineering leaders can’t answer basic questions: which commits were AI-assisted, which models were used, how much did a session cost, did the agent invoke dangerous tools, and how much of the diff was actually human-reviewed. Self-disclosure doesn’t scale. Most agents leave no durable, tamper-evident record of their actions, which means policies (e.g., “only approved models”, “no secrets in transcripts”, “signed commits”) have nothing to evaluate against, dashboards have nothing to aggregate, and reviewers have nothing to anchor on when the diff looks suspicious. Chainloop closes that loop with three complementary pieces:
  • Context — every session is captured as a CHAINLOOP_AI_CODING_SESSION material, paired with the agent’s static configuration via the AI config collector (CHAINLOOP_AI_AGENT_CONFIG). Together they give policies and reviewers something concrete to evaluate, both at runtime and at config time.
  • Policies — built-in checks for signed commits, dangerous commands, secrets, and approved agents, plus custom Rego for organization-specific rules.
  • Tooling — chainloop trace automates capture so developers don’t change how they work, while the dashboard, PR summary, and check run surface the data wherever security and engineering teams already operate.

​
Open Source vs. Platform

chainloop trace, the CHAINLOOP_AI_CODING_SESSION evidence type, and the policy engine that evaluates it are all part of open source Chainloop. Capturing sessions, attesting them, and gating them with your own policies works against a self-hosted Evidence Store with no paid plan involved. In local mode, chainloop trace stores unsigned sessions on disk, with no Chainloop instance. What the platform adds is everything that happens after the evidence lands: correlating sessions with pull requests, rendering them in the UI, aggregating them across projects, and scoring them for reviewers. Without the platform you still get the evidence and the verdict: sessions are signed, stored, retrievable through the CLI and API, and gated by the policies you attach to them.

​
How Sessions Are Captured

While an agent is working, hooks installed in the agent’s runtime record everything that matters:
  • Session metadata — start and end time, duration, slug, agent and version.
  • Model and token usage — primary model, every model used, input/output/cache tokens, estimated cost.
  • Tool invocations — every tool the agent called (Read, Edit, Bash, etc.) and how many times.
  • Conversation summary — counts of user and assistant messages.
  • Per-line code attribution — every time the agent edits a file via Edit, Write, or MultiEdit, the hooks snapshot the file before and after the change. Diffing the snapshots gives the exact line ranges the agent touched.
  • Context — the specs, tasks, plans, skills and images the session worked from. See Session Context.
This data is held locally, in .git/chainloop-trace/ by default, until you push.

​
Session Context

The transcript shows what the agent did. The context shows what it was asked to do. Specs, tasks, plans, skills and screenshots are the inputs of the code, and teams lose them. Someone edits a ticket, a design document moves, and a plan approved in the agent’s plan mode lives only in the chat. chainloop trace vendors those inputs into the session evidence, the way a build vendors its dependencies, so the record keeps the exact instructions the agent read. The sources are signed with the transcript, so editing one after the fact breaks the signature. The context is what turns a session record into something you can grade. The AI Session Score checks the change against the captured spec requirement by requirement, and the session view shows each source next to the transcript. For what is captured, how, and which agents support it, see Capture the Context of a Session.

​
Aggregation on Git Push

A single commit can include changes from multiple sessions, or a mix of AI and human edits. On push, the pre-push hook reconciles all of that into one attestation: For each contributing session, the bundle records git context (repo, branch, commit range), aggregate stats, and a per-file attribution label (ai or human) with the exact line ranges. The bundle is then signed and pushed as a Chainloop attestation. Hooks are non-blocking by default — if anything fails, they log a warning but never block the push (toggle this with the traceRequired setting in .chainloop.yml). Before the evidence leaves the machine, the CLI scans the session for secrets and replaces the ones it finds with placeholders. Only the uploaded copy is redacted — the local session files are left untouched — and redaction fails closed, so a session that can’t be scanned isn’t uploaded at all. What redaction did is recorded in the attestation, so policies can reject sessions that were stored unscanned. After the push, the local session data is cleaned up. The attestation lives in Chainloop, tied to the project the repository is linked to. In local mode, the push runs the same steps but writes the redacted, unsigned evidence to disk instead of signing and uploading it.

​
PR Correlation

Once attestations land in Chainloop, the platform connects them to the pull requests they belong to. When a PR is opened or new commits land, the Chainloop GitHub App receives a webhook event, fetches the PR’s commit list, and matches each commit SHA against the stored AI coding session attestations for the linked project. The matches drive three downstream surfaces — the PR summary comment, the policy check run, and the dashboard’s PR-related metrics. If session data arrives after the PR is opened, the comment is updated automatically once the session lands.
Chainloop AI Session Analysis comment on a pull request showing average score, sessions, failing policies, attribution, files, lines, and total duration

​
Session Lineage

PR correlation shows which pull requests a session fed. The Lineage tab goes further. It follows the code of the session to production, from the pull requests to the artifacts, their deployments and their findings.
The tab reads left to right: When a session has no merged pull request yet, the tab says so and explains why the path is empty. Lineage adds no data of its own. Every link is a commit SHA or a content digest inside a signed attestation. The tab walks evidence that you can verify. It is only as complete as what you attest. The artifacts and deployments columns fill in when your CI attests its builds with the Chainloop CLI and records deployments. You can walk the same graph from the CLI with chainloop discover. For example, chainloop discover -d <merge-sha> starts from a merge commit and finds the sessions that contributed to it.

​
AI Sessions Dashboard Metrics

The org-wide AI Coding dashboard at /u/<your-org>/dashboards/ai aggregates everything captured above. Each card maps to a specific backend metric:
AI Coding dashboard with stat cards, top users, and model breakdown for the selected time window
The PR card uses the same correlation mechanism as the per-PR comment, so the two views are always consistent.

​
Policies and Scoring

Because each session lands as a regular Chainloop material, every policy mechanism that works on materials works on AI coding sessions too.

​
Built-in Policies

Chainloop ships a curated contract of deterministic checks for CHAINLOOP_AI_CODING_SESSION: Custom Rego policies are also supported. The chainloop-trace guide shows examples for restricting models, enforcing a token budget, and capping AI-authored line ratios.

​
PR Check Run

When a session in a PR violates any attached policy, Chainloop publishes a Chainloop AI Policies GitHub check run on the head commit. The check is failure when there are violations, neutral when policy data can’t be evaluated, and success otherwise — making AI policy compliance a first-class merge gate alongside CI.

​
AI Session Score

In addition to deterministic policies, Chainloop computes an AI Session Score — a per-PR confidence signal derived from session data, the diff, and contextual signals like AI code review bot comments. When the session captured a spec, the score grades the change against it, requirement by requirement. Where policies say “did this break a specific rule?”, the score says “does this PR look healthy enough to merge?” and surfaces actionable findings for reviewers.
AI Quality Score breakdown showing per-criterion scores and findings for context-and-planning, solution-quality, user-trust-signal, alignment, scope-discipline, and verification

​
Relationship to AI Config Collection

chainloop trace captures runtime session data. It pairs naturally with the AI config collector, which captures the static configuration an agent runs with (CLAUDE.md, MCP config, settings, rules, skills). Use both together for full coverage: the config collector ensures agents are set up correctly, while trace ensures they behave as expected at runtime.

​
Set It Up

Ready to capture sessions in your repositories? Follow How to record AI coding sessions — it walks through prerequisites, chainloop trace init, recording a single agent run with chainloop trace run, the GitHub App, the dashboard, enforcement options, and troubleshooting.

​
Further Reading