Checking the sign is not checking the room

There is a kind of indicator that stays green for months while the thing it was meant to watch is broken. Not because someone was careless — quite the opposite. Because they measured, and measured the thing that was easy to measure.

Picture a building with a sign pointing to the meeting room. The sign is correct: it points down the right corridor, the number is right, the arrow points the right way. Someone walks past every day and confirms the sign is in place.

And the meeting room does not exist. It was demolished in a refurbishment, and nobody took the sign down.

That is the most common way an indicator lies without lying. It answers, with absolute honesty, a slightly wrong question — and that is exactly why nobody doubts it.

Where this shows up in your company

The AI usage policy is signed by 100% of the team. That is the sign. The room is what any given person answers when you ask, with no warning: what are you allowed to do with AI here? If the answer is hesitant, the signature meant nothing — and it still counts as 100%.

The training had 94% attendance. A sign. The room is someone working differently the following week.

The backup runs every night and the report comes back green. Here the difference is the cruellest of all, because the sign and the room look like the same thing: there is a file, it has a size, it is in the right place. But a backup that has never been restored is not a backup. It is hope with a technical name. The room only shows up when someone actually restores it.

The vendor has a security certification. A sign — and an expensive one, issued by a third party. The room is what happens when you ask them what exactly was certified, and on what date.

In none of these cases is the number wrong. It is right, and it is irrelevant.

Why we fall for it

Because checking the sign costs a minute and checking the room costs an afternoon.

Confirming the policy was signed is reading a filled-in field. Finding out whether anyone understood it takes talking to people, hearing awkward answers, and admitting the document may be badly written — which is a conversation about you, not about them.

Confirming the backup ran is reading a line of log. Restoring a backup means booking an afternoon, finding somewhere to restore it to, and running the risk of discovering that it does not work. Which is precisely the information you most need to have, and the one nobody wants to receive on a Thursday afternoon.

So we measure what is cheap. And in time we forget it was a stand-in — the number goes up, the dashboard goes green, and the original question, is this working?, stops being asked because it looks as though it has already been answered.

An indicator that has been green for a long time becomes scenery. Nobody looks at the scenery.

The question that separates the sign from the room

I know of no general solution. I know one question, and it is uncomfortable on purpose:

If the thing I am measuring were completely broken, would this indicator change?

If the answer is no, you do not have a measure. You have a ritual — something done because it has always been done, producing a number nobody questions.

And the reverse holds too, and it is subtler: if the indicator only changes after everything has gone wrong, it is not measuring either. It is recording the damage, late.

The fixes are tedious, and that is how it is

There is no trick. Checking the room is always more expensive than checking the sign, and the only real decision is choosing which rooms you are going to check for real.

Choose the ones that hurt if they turn out empty. Backup makes the list. The AI policy makes it, if the company depends on it to answer a client. The vendor contract makes it, if they hold your data.

And put it in the calendar, because that is the part that decides it: checking the room has to happen on a calm day, chosen by you — not on the worst possible day, chosen by chance.

What remains

Every organisation has indicators nobody questions any more, because they have been green so long they became part of the scenery. Some of them measure the sign.

Finding out which is quick, and you can do it today: pick three indicators you are proud of, and ask of each whether it would change if the thing it measures broke completely.

One of the three will give you a fright. It is almost always one.

If you got this far, you have probably already thought of a specific indicator. Write and tell me which: contato@inteligenciaanalogica.com. The answer is personal.