There are plenty of ways to measure software quality:
cyclomatic complexity, test coverage, maintainability,
probably seventeen other things
somebody has already put into SonarQube.
Over the last few months,
I’ve accidentally discovered another one:
AI credits.
This is, obviously, a terrible metric.
Unfortunately, it keeps telling me things.
It Started With Copilot
I first noticed it while working on our backend.
A small change would come in:
add a property, adjust some mapping,
change a bit of behavior.
Nothing exciting.
Copilot would find the model,
follow it into the mapper,
update the tests
and usually finish the whole thing
in one conversation.
Maybe a small detour
because it had decided to improve something
nobody asked about,
but still:
fairly uneventful.
Then I’d make an equally small change
in the frontend,
and suddenly we were on an expedition.
Open the component.
Find the state.
Follow the state into another layer.
Discover that the value
isn’t actually stored there.
Find the derived property.
Work out who calculates it.
Realize another component
does something similar,
except slightly differently,
for reasons lost to history.
A few hundred credits later,
the actual implementation was still tiny.
Understanding where it belonged wasn’t.
This Is Not a Metric
To be clear,
AI credits are an objectively terrible
architecture metric.
Different models burn through context differently,
frontend and backend work
aren’t directly comparable,
pricing changes,
tools change,
and tomorrow Copilot might decide
that opening a TypeScript file
requires a small mortgage.
So no,
I am not proposing this
for the architecture dashboard.
Please do not add a KPI
called Credits per Story Point.
Someone will eventually put a target on it,
and then we’ll have achieved
something genuinely terrible.
But It Keeps Revealing Something
Before an AI assistant can change a system,
it has to reconstruct enough of that system
to know what it’s changing.
Where does this state actually live?
Who owns this behavior?
Is this abstraction important,
or merely old?
Why is this value calculated here?
Why is there another version of it over there?
And why does changing either of them
break a modal nobody mentioned?
Those aren’t really coding questions.
They’re architecture questions.
The code change might take thirty seconds.
The rest of the conversation
is often spent figuring out
which thirty seconds matter.
Architecture Has a New Customer
We usually talk about architecture
as something humans have to live with.
Can a new developer understand the system?
Can someone change one part
without accidentally setting fire to another?
Can an experienced engineer
find the right place for a feature
without first consulting
three repositories and Dave?
AI assistants now run into the same structure.
They also benefit from clear boundaries,
predictable ownership
and local reasoning.
They do considerably worse
when a small change requires
understanding half the application
and one very important utility file
called helpers.ts.
In that sense,
architecture suddenly has another customer.
A slightly strange one
that works extremely fast,
never gets tired
and will confidently inspect
the wrong abstraction for fifteen minutes
unless someone stops it.
So, basically, a developer.
Context Has a Cost
The interesting part
isn’t the number of credits.
It’s where they go.
Long AI conversations are rarely long
because writing the code takes ages.
They’re long because the assistant is exploring:
opening another file,
searching for another reference,
tracing another value,
discovering another assumption,
revising its mental model
and repeating the process
until the system finally makes enough sense
to touch.
Humans do exactly the same thing.
We just don’t get a little counter
in the corner
showing how much architecture
we’ve consumed.
The Expensive Part Is Understanding
A simple feature can be cheap to implement
and expensive to understand.
We’ve always known this.
It’s why some codebases feel pleasant
and others make you stare at a ticket
for twenty minutes
before opening the IDE.
AI just makes the process
unusually visible.
If one change needs five prompts
and another needs twenty,
that doesn’t automatically mean anything.
Maybe the model got confused.
Maybe the task was weird.
Maybe Tuesday happened.
But if every tiny change begins
with an extended guided tour
of the application,
there might be a pattern worth noticing.
Not a metric, exactly.
More of a smell.
We already have plenty of those.
One More Consumer of Complexity
I don’t think we’re going to start
evaluating architecture
by counting prompts.
At least I hope not.
But AI assistants are becoming
another thing that has to navigate
the systems we build.
And unlike humans,
they leave a surprisingly visible trail
while doing it:
every file they have to open,
every assumption they have to rediscover,
every prompt that begins
with some variation of
actually, wait.
None of that measures
software quality directly.
But it does expose something
we’ve always paid for:
the amount of system
you have to understand
before you’re allowed
to make a small change.
And sometimes that’s a lot more architecture
than the ticket looked like it contained.