For several years I worked with a CIO who had a habit, when a meeting began to drift, of saying four words: "Measure what you treasure." He didn't say it often, and he never raised his voice to say it. It came the way an experienced card player lays down a card, quietly and after everyone else had finished talking. The phrase has the shape of a corporate aphorism. It rhymes, which is usually a bad sign. It could easily have been filed with "People are our greatest asset" and "What gets measured gets managed," in the large archive of things said at off-sites and forgotten by the time the bus gets back. But it stayed with me, and over the years I came to think it was less a slogan than a small piece of philosophy. What an institution chooses to count tells you what it believes matters. What it doesn't count tends to drop out of sight.
Consider how most organizations bring in a new piece of technology. There is a project, the project has a plan, the plan has a date, and on that date the system "goes live." The servers hum. The licenses are provisioned. Training sessions have been held in windowless rooms with indifferent coffee. Someone senior sends an e-mail thanking the team for its tireless efforts. The project is closed, the budget line is retired, and everyone moves on.
Almost nobody notices that this is the moment the interesting part begins. From its first day of use, a system starts to build up a kind of sediment. It isn't a record of what the system does. It's a record of what people do with it: who opens it, and when; what they were doing before; where they give up; where they keep coming back. It shows which features become indispensable and which stay untouched, monuments to a requirements document that someone spent months writing. This metadata is the exhaust of institutional behavior. Read with care, it can answer a much bigger question than whether the software works. It can tell you whether the organization has changed.
Technology departments have long been fond of a number called adoption. Say a firm rolls out a new platform to a thousand employees, and six months later eight hundred of them have logged in. The slide almost designs itself: a large numeral, eighty per cent, a green arrow pointing confidently toward the upper right. Success.
The trouble is that the number tells you almost nothing. One employee logged in once, found that the new system needed three more clicks than the old one, and never came back. Another uses it forty times a day. A third has become devoted to one obscure feature and ignores the rest. A fourth has quietly rebuilt a two-hour task around it, and the task now takes twenty minutes. These aren't variations on a theme. They are different things that happen to share a word. "Did people use it?" is a question about attendance. The questions that matter come after it: what did using it change, and was the change worth having?
Nowhere are those questions more pressing, or more awkward, than with artificial intelligence.
Picture a lawyer who uses an A.I. system to produce a first draft of something that used to take three hours. It now takes forty-five minutes. To the technology team, this is a triumph: two and a quarter hours of human labor turned into a few minutes of machine work. Then the lawyer fills out a timesheet and records forty-five minutes instead of three hours. To a traditional utilization report, something odd has happened. Productivity has risen sharply, and billable time has fallen. If you measure only hours, the innovation looks a lot like a decline.
This is the paradox now facing professional-services firms. For decades, the hour has done three jobs at once: it has been an accounting unit, a stand-in for effort, and, more loosely, a stand-in for value. A.I. is beginning to pull those three apart, and once they separate, it isn't obvious which one the firm was really keeping track of.
That's why the metadata around A.I. matters so much. Imagine combining usage data with time entries. The point would not be to examine individual lawyers, which would turn an interesting experiment into a surveillance program almost overnight. The point would be to look for patterns across matters, practice groups, and types of work. You might find that heavily assisted matters show a thirty-per-cent drop in time spent on initial document review. You might find that associates spend less time producing first drafts and more time reviewing, negotiating, and advising clients. You might find that some kinds of work don't change at all. Each of these would be worth knowing, because the important question isn't whether people are using A.I. It's whether A.I. is changing the economics and the makeup of the work. A dashboard showing ten thousand prompts tells you that people have found the prompt box. A dashboard showing that assisted matters reach their milestones sooner, and that lawyers' time has moved from repetitive work toward judgment, starts to tell you whether the thing is effective. Effectiveness is what the firm treasures. The prompts are just evidence that someone was in the building.
The same logic applies to far less glamorous technology. Take the service ticket, the humble unit of the corporate help desk. Most organizations count how many tickets are opened and how many are closed, and those are reasonable things to count. But suppose a new support process cuts average resolution time from eighteen hours to six. That's good news, until you see that the number of reopened tickets has doubled, and the story turns. Or suppose resolution time hasn't moved, but tickets now reach the right specialist on the first try instead of passing among four teams. In a conventional report, that improvement would be nearly invisible. Anyone who has explained the same problem to three different help desks in one afternoon knows exactly what it's worth.
The most revealing data sits around the ticket rather than in it: how many times it was transferred, how long it sat waiting compared with how long someone actually worked on it, how often it was escalated, whether the person who filed it was satisfied, and, most telling, whether the same problem keeps coming back. A support operation that becomes extremely good at solving the same problem five thousand times isn't mature. It's a very fast treadmill. The organization that solves the problem once and removes its cause will look worse on the throughput chart and be better in every way that counts. A ticket isn't just a unit of work. It's evidence.
Once you start looking at an institution this way, these traces turn up everywhere.
Documents are usually counted by how many are produced. It might be more useful to know how often each one is revised, how long it waits for review, how often the standard language is rewritten, and which clauses reliably start a negotiation. A contract that takes twelve days to execute may contain four hours of actual work. The other eleven and a half days are waiting. Most organizations have never measured that waiting, so they don't know it's the largest item in the budget.
A firm may announce that its knowledge repository holds a million documents, which sounds impressive until someone asks whether anybody can find the right one. The better measures are less flattering: how often a search succeeds, how often it is repeated, how often it returns nothing useful, how long it takes to get from a question to an answer, and how often precedent is actually reused. Most revealing of all is how often a person gives up on the system and goes down the hall to ask a colleague who has been there twenty years. The failed search often tells you more than the successful one.
Onboarding is usually measured by whether a new hire finished the mandatory training. A more honest measure would be the time between someone's first day and the first piece of meaningful work they complete on their own. Those are two different numbers, and only one of them describes the employee.
Then there are meetings, which organizations count mostly because calendars make them so easy to count. The more interesting questions are how many meetings produce a decision, how many produce an action, how many require a second meeting to settle what the first one left open, and how long decisions stay unresolved afterward. It isn't hard to imagine an organization discovering, with some embarrassment, that its main product is more meetings.
The list goes on. How many security alerts turn out to be real incidents. What share of reports are produced and never opened. Which workflow steps people routinely bypass. How long a matter sits waiting for approval. How often data has to be retyped from one system into another. How many questions are answered again and again because no one ever wrote the answer down where others could find it. Each of these is a small signal. Taken together, they describe the organization more accurately than its org chart does.
There is a catch, and it's an old one. In 1975, the British economist Charles Goodhart observed that any statistical regularity tends to collapse once people put pressure on it for purposes of control. The anthropologist Marilyn Strathern later gave the idea its familiar form: when a measure becomes a target, it ceases to be a good measure. Measurement isn't neutral. It's an intervention, and people respond to it.
Tell a service desk that success means closing tickets quickly and tickets will close quickly, whether or not anything has been fixed. Tell employees that A.I. adoption is a priority and A.I. usage will rise. Some of that use will be transformative. Some will be people dutifully opening the application because management has made clear that someone is watching the dashboard. Measure utilization and you get utilization. Measure speed and you get speed. Measure volume and you get volume. Over time, the metric stops describing reality and starts producing it.
So the search for the one metric, the single number that captures whether something is working, is nearly always a mistake. Measurement works better as a constellation. One star shows activity, another speed, another quality, another outcome, and another whether the behavior lasted once the novelty wore off. No single star is the picture, but together they make a shape you can steer by. For an A.I. system, I'd want to know about frequency, repeat use, the kinds of tasks involved, time saved, how often the output is accepted and how heavily it's revised, whether workflows get finished, and how assisted work relates to the time that is eventually recorded. For a service operation, I'd want volume, resolution time, waiting time, reassignment, recurrence, reopen rates, and satisfaction. For a knowledge system, I'd want searches, successful retrievals, failures, reuse, contributions, time to answer, and some sense of what people did with what they found. None of these tells the whole story. Together, they start to tell one.
The larger point is about curiosity. Organizations are surrounded by data and are often oddly uninterested in it. They install systems to do things: manage documents, record time, track matters, route requests. Without anyone intending it, each of those systems also becomes an instrument of observation, a seismograph recording the small tremors of institutional life. The opportunity isn't to collect more of these readings. Most organizations already have far more than they can use. The opportunity is to connect them around questions worth asking.
Did A.I. reduce the time a particular kind of work takes? Did the knowledge platform reduce duplicate research? Did the workflow system shorten the approval process, or did it just digitize the waiting? Did the new collaboration tool cut down on e-mail, or did it add one more place people have to check? Did automation remove administrative work, or move it somewhere harder to see?
None of these can be answered from inside a single system. They require following the work as it moves across systems. A.I. usage tells you one thing, timesheets another, document activity another, matter milestones another, support data another. On its own, each is a fragment. Put together, they start to suggest cause and effect, and at that point metadata stops being an administrative by-product and becomes something like institutional memory.
There's an obvious temptation, especially now, to collect everything. Storage is cheap, dashboards are everywhere, and A.I. itself has made it possible to sift through quantities of information that would have been absurd a decade ago. But the CIO's phrase has an order to it, and the order matters. It says "measure what you treasure." It doesn't say "treasure what you measure," which is what most organizations end up doing by default, coming to value whatever their systems happen to count.
The treasuring has to come first. If you value speed, measure elapsed time. If you value quality, find credible signs of quality, which is harder and more honest. If you value the reuse of knowledge, check whether knowledge is actually reused. If you value what you deliver to clients, look past your own activity and ask what changed for them. And if you value the promise of artificial intelligence, don't congratulate yourself because your employees generated a million prompts. Ask what happened to the work. Did it get faster, better, or cheaper? Did people spend less time on mechanical tasks and more on judgment? Did the economics change? Did the experience change?
Sometimes the answer will be that nothing changed, and that answer matters too. Organizations often treat measurement as a way to prove that a decision was right. It's much more useful when it's allowed to show that the decision was wrong. The platform no one could imagine doing without turns out to be barely used. The feature no one thought about becomes essential. A.I. transforms one workflow and does almost nothing for the one next to it. The process everyone was proud of automating turns out never to have needed to exist. None of these are failures of measurement. They're what measurement is for. The best measurement doesn't tell management what it wants to hear. It makes the institution a little better at seeing itself.
Years after I first heard those four words, I find myself coming back to them more often, not because everything valuable can be reduced to a number (it can't), but because organizations have remarkably short memories. They implement systems, redesign processes, announce transformations, and move on. Six months later, someone asks whether the transformation worked, and a room full of intelligent people starts trading anecdotes. Measurement offers something better than anecdote: evidence. If the measures are chosen carefully, the metadata collecting around our systems can tell us something surprisingly human: where our time goes, where our attention goes, where friction builds up, how knowledge travels, and where technology actually makes a difference.
I suspect that was what my former colleague was getting at all along. Every organization says it knows what it values. The more revealing question is whether it ever bothered to measure it.