I ran the new agent models against my actual workflow
Every few months a new frontier model lands and the discourse follows the same arc: a benchmark chart, a wave of "this changes everything" threads, a wave of "it's overhyped" replies, and very little from people who actually put it to work on a real task with real stakes. So when the current crop showed up, Kimi K2, Claude Fable, the newest tier of agent-grade models, I did the boring thing and pointed them at my own workflow instead of a leaderboard.
My test isn't a benchmark. It's a long-horizon operational task with tools, guardrails, and a cost budget: read telemetry, correlate across sources, propose a diagnosis, and stop at the gate before anything touches production. It rewards the things benchmarks miss. Staying coherent over many steps, knowing when to stop, using tools instead of hallucinating their output, and not confidently inventing a root cause. Here's what actually held up.
The gap that's closing is long-horizon coherence
The single biggest improvement in this generation isn't raw reasoning on a one-shot question. That's been good enough for a while. It's the ability to stay on task across dozens of tool calls without losing the thread, re-deriving facts it already established, or drifting into a different objective halfway through. A year ago, agent runs degraded the longer they went. The newer models hold their shape over a genuinely long task, and that's the difference between an autocomplete and something you can hand a multi-step job.
Kimi K2 impressed me on tool discipline. It reached for the tool and used the result, rather than narrating what it imagined the tool would say. That sounds small; it's enormous. A model that fabricates tool output is worse than useless in operations, because it's confidently wrong about the state of a real system. Claude Fable, the newest of the set, was the most reliable at the part I care about most: knowing when to stop and hand back to a human, rather than plowing through the gate to "finish." For anything touching production, that instinct is worth more than a few points of benchmark score.
What the benchmarks still don't measure
Here's my honest field assessment after running these on real work: the models are now good enough that the model is rarely the bottleneck. The failures I hit were almost never "the model wasn't smart enough." They were retrieval handing it the wrong context, a tool returning ambiguous output, a guardrail that was too loose or too tight, a prompt that under-specified the goal. The intelligence arrived; the engineering around it is where the work moved.
That reframes how I evaluate a new model. I no longer ask "is it smarter." I ask three operational questions: does it use tools honestly, does it stay coherent over a long horizon, and does it know when it's uncertain enough to stop. On those, this generation is a real step up. Not because the demos are flashier, but because the tail behaviors that used to bite you at 2am are meaningfully rarer.
On the Hacker News reaction
The threads on these releases split, as always, into "AGI is here" and "it's a stochastic parrot," and both camps are arguing about the wrong thing. The interesting question was never whether the model is intelligent in some philosophical sense. It's whether you can build a system around it that's reliable enough to trust with real work. And the answer, increasingly, is yes, but only if you do the unglamorous parts: ground it in real data, box its blast radius, gate the irreversible actions, and log everything so you can earn trust incrementally.
The most-upvoted skeptic comment on the last big thread was some version of "I wouldn't let this near production." And they're right. You wouldn't let the model near production. You'd let a carefully constrained system that happens to use the model near production, with a human on every action that can't be undone. That's not a limitation to complain about. It's the design. The people shipping useful AI in operations aren't the ones who trust the model most. They're the ones who engineered the system so they don't have to.