Session data vs git metadata: what each one can actually tell you about AI coding

Every tool that claims to measure AI coding impact is built on one of two data sources, or on a guess that bridges them.

The first is git metadata: commits, pull requests, diffs, review events, merge times. It is easy to get, every vendor has it, and it has one hard property going for it. It only contains work that survived long enough to be committed.

The second is session data: the actual exchange between an engineer and Claude Code, Cursor, Copilot or Codex. Prompts, responses, tool calls, retries, tokens, cost, and the code that got generated whether or not anyone kept it.

Most buying conversations treat these as competing answers to the same question. They are not. They answer different questions, and the interesting problem is what happens when you try to join them.

What git metadata knows, and where vendors start guessing

Git metadata answers "what shipped" cleanly. It answers "was this AI-assisted" badly, because git has no native concept of AI authorship. Every vendor has to infer it, and the inference methods are weaker than the dashboards suggest.

• DX documents that it "uses automatic AI usage attributes when available and falls back to the self-reported AI code assistant usage attribute." When automatic signals are missing, attribution comes from a survey. (DX docs)
• Swarmia counts a pull request as AI-assisted if any commit was authored or co-authored by an AI tool, carries a Made-with: Cursor git trailer, or has a claude-code-assisted label. It also includes a criterion it labels low confidence itself: any commit "made by an author who used an AI tool within the previous 24 hours." (Swarmia docs)
• LinearB relies on PR labeling, either manual or automated through gitStream rules, then compares labeled against unlabeled PRs. (LinearB)

None of these are unreasonable engineering choices. But they mean that when a dashboard tells you "AI-assisted PRs merge 22% faster," the denominator was assembled by a survey answer, a commit trailer, or a 24-hour proximity rule. If you are going to defend a number to a CFO, you should know which.

What session data knows that git will never record

Session data has the opposite shape. It is harder to collect, it requires consent and a clear data agreement, and it is useless for telling you what reached production. What it does hold is everything git deliberately forgets.

Discarded work. An engineer prompts six times, gets code that does not work, abandons the approach and writes it by hand. Git records one human-authored commit. The tokens were still spent, the time was still spent, and by every git-based measure that session never happened. This is the single largest blind spot in metadata-only measurement, and it is invisible by construction.

Retry cost. A task that took eleven prompts and a task that took two can produce identical commits. Retry cost is the line item almost no tooling surfaces, and it is the one that scales fastest as teams move from autocomplete to agents.

How the work was actually done. Whether someone planned before building, whether they were feeding the model the whole repo every turn, whether the same failure mode repeats across a team. This is the difference between an engineer who is expensive because the work is hard and an engineer who is expensive because of how they prompt.

Faros AI, which is broadly aligned with the outcome-over-usage argument, states the boundary plainly: "Claude Code analytics stop at the boundary of the tool. They show what was generated and consumed inside the editor. What happened to that output afterward is not in the data." (Faros AI)

That is exactly right, and it cuts both ways. Session data cannot see past the editor. Git cannot see inside it.

The question neither dataset answers alone

Here is the one that matters to an engineering leader defending a budget: did the AI-assisted work hold up?

Not "was it merged." Merged is not the bar any more. The EASE 2026 study of the AIDev dataset, covering over 932,000 pull requests, found that "61.38% of agent-authored PRs receive no recorded review activity." (Duma et al., arXiv:2605.02273) A merge event is a weaker signal than it was two years ago.

The signal you want is what happened to that code afterwards. Was it rewritten within two weeks. Did it cause a rollback or an incident. Did it come back as a bug against the feature it shipped.

Git metadata can see the rework. It cannot tell you which session caused it. Session data can see the session. It cannot see the rework. The value is not in either dataset. It is in the join.

Google's DORA research has been blunt about why output volume will not substitute: "AI can easily inflate the volume of code generated; therefore, leaders must stop relying on such narrow, output-based metrics as a measure of actual productivity." (DORA) The 2025 DORA report also found that higher AI adoption correlates with increases in both delivery throughput and delivery instability. More output, less stability. That is precisely the trade-off a usage dashboard cannot price.

What the join actually requires

Connecting a session to a commit and then to what happened to that commit is a real engineering problem, not a feature bullet. Four things have to be true:

1. Line-level or near-line-level attribution at write time. Reconstructing it after the fact from commit timestamps is the 24-hour-window approach, and it degrades fast on teams where AI and hand-written work interleave in the same file.
2. Attribution that survives history rewriting. Rebases, squashes and cherry-picks routinely destroy naive attribution.
3. A rework window long enough to be honest. A window that closes at merge catches the cheap, visible rework and misses the expensive kind. Much of what looks like AI leverage is deferred cost that resurfaces two sprints later as churn, fully detached from the session that caused it.
4. A defensible definition of "reworked." This is the hardest part and the one buyers should push on. The most defensible approaches are deliberately mechanical: review rounds before merge, plus churn on those specific lines within a fixed window such as 14 days, both pulled straight from git so nobody argues about the data source.

What to ask before you buy either kind of tool

If you are evaluating measurement tooling, these five questions separate the categories quickly:

1. How do you decide a commit or PR is AI-assisted? Survey, label, trailer, time window, or write-time attribution. Ask for the documentation, not the sales answer.
2. Does your attribution survive a rebase or squash?
3. Can you see work that was generated and then abandoned? If the answer is no, the tool cannot see your wasted spend.
4. How long after merge do you keep watching that code? If the answer is "we stop at merge," you are buying a throughput metric, not a durability metric.
5. How many AI tools can you reconcile at once? Most enterprises run two or three. Single-vendor dashboards, including the native ones from Cursor and GitHub, are structurally unable to give you a cross-tool view.

Question 3 and question 4 are the ones most vendors cannot answer well, including some that market heavily on outcomes.

The honest summary

Session data without git data is a very detailed spend report. Git data without session data is a delivery report that guesses at attribution. Neither one, on its own, answers the question a board is actually asking.

This is the problem Tommbo was built around. We ingest session data from Claude Code and Cursor alongside your git history and reconcile the two, so the question stops being "how much are we using AI" and starts being "what did this session actually produce."

If you are running the evaluation described above and want to see what the join looks like against your own repositories rather than a demo dataset, get in touch and we will walk through it.