Nicole Ginosar
A pilot in clear daylight barely needs the panel. They can see the horizon, judge their altitude by eye, feel whether the plane is level. Instruments look almost decorative. Then the weather closes in, and the same pilot with the same skill flies a perfectly good aircraft into the ground, because without instruments there is no way to tell a gentle climb from a gentle dive. Everything feels fine right up until it isn't.
That's the trap waiting for anyone building an agent that takes real action. The demo is clear daylight. You watch it resolve an exception, update a record, act on a supplier's message, and it looks right, so it feels right. You conclude you don't need much of an instrument panel. And in the demo you don't. The problem is that production isn't the demo, and the moment conditions change you have no way to see that the agent has started flying you into the ground.
The short version
- An agent that only advises fails loudly. An agent that acts fails silently, because a wrong action looks identical to a right one.
- Operational metrics like uptime, latency, and volume can stay green while decision correctness drifts.
- An eval system for an acting agent needs three things: labeled real cases, replay on every change, and a drift monitor on live decisions.
- The eval system, not the agent, is the part that takes years to build and never finishes.
- The more autonomous the action, the more the eval system becomes the entire safety story.
Why acting agents need evals
We've argued before that the machinery around your knowledge matters more than the knowledge itself, and that this matters more now because the system doesn't just advise anymore. It acts. This is what that shift costs you.
For an agent that acts, the eval system is the instrument panel, and it is the part you cannot skip and the part almost everyone underestimates. Not because teams are careless. Because in clear weather the instruments genuinely look optional.
This is the specific danger of an acting agent, and it's easy to miss. When the agent is wrong, the wrong action looks exactly like a right one. It resolves the exception confidently. It updates the field cleanly. There is no error, no exception thrown, no red light, just a decision that happens to be wrong, sitting in your systems looking identical to the thousands of correct ones around it. A human who's unsure hesitates. An agent that's wrong reports success with exactly the confidence it reports a correct one. Nothing in your operational metrics separates them, which means the first real signal is the consequence, surfacing somewhere downstream long after the decision.
Why a green dashboard doesn't mean your agent is correct
It feels fine until it isn't. The reason this is so seductive is that flying blind is smooth. Nothing is rattling. The dashboard is green, the throughput is up, everyone's happy. That calm is not evidence the agent is safe; it's evidence you have no instrument sensitive enough to tell you otherwise. A team building their own tends to instrument the things that are easy to see, uptime, latency, volume, and those all look great while the actual decisions quietly drift out of correctness. The green dashboard isn't reassurance. It's the absence of a warning system, dressed up as one.
What a real eval system requires
Here's what makes this a real buy-vs-build question rather than a lecture. Anyone can build the agent. Building the panel that keeps it safe in real weather is the work that takes years. It needs three things working together:
- Labeled real cases that actually represent your operation, not a synthetic test set.
- Replay on every change, so a fix for one situation doesn't quietly break three others.
- A drift monitor on live decisions, catching correctness sliding before the cost does.
And none of it is build-once. Every new supplier, format, and exception has to make its way into the case set or the panel slowly stops reflecting reality, and a stale instrument is worse than none because you trust it. That maintenance is unintuitive, unglamorous, and never done, which is exactly why in-house versions tend to ship the agent and never finish the panel.
This is also why measuring what you ship belongs in your first week, not your first year. The team that can't say where its product is unreliable, with evidence rather than adjectives, is the team that hasn't built the panel yet.
When building your own evals makes sense
Let me be fair to building, because the case exists. If the agent only proposes actions that a person always approves before anything happens, you're flying in permanent daylight with a human as the instrument, and a light setup is completely reasonable. The weight of this argument scales with how much the agent does on its own. The more autonomous the action, the more the panel stops being optional and starts being the whole safety story. Match the rigor to the stakes, and don't build heavy instrumentation for an agent that never acts unsupervised. And if your way of judging a correct action is itself proprietary, part of your edge, then owning that judgment end to end can be worth the cost of building. That's rarer than people think, but it's real.
How to make the decision
If you're deciding right now, don't compare the agents. Compare the panels. Ask who will own the case set two years from now, after the original builders have moved on and the operation has drifted twice, and whether that panel will still be sensitive enough to catch a confident wrong answer.
If you can't answer that, you're not choosing to build an agent. You're choosing to fly a real one, in real weather, with instruments you never finished.
Nicole Ginosar is Head of Product at Emerix, where she works on how autonomous workflows get built, evaluated, and kept accurate in production. Before Emerix she led data engineering and analytics at At-Bay and spent time at Google and Taboola. She is also the co-founder of Data Queens, Israel's first community for women in data.