Verifiers Over Vibes
Why the shift from subjective human feedback to programmatic verifiers changes how we build.

For the last three years, the generative AI industry ran on vibes. How does this “feel”?
When teams wanted to align a model to be helpful, safe, or knowledgeable, they turned to Reinforcement Learning from Human Feedback (RLHF). Human evaluators compared two side-by-side AI responses, picked the one they preferred, and those comparisons trained a reward model.
The industry built billion-dollar product strategies on top of what was essentially a collective popularity contest.
The consequences were predictable to anyone who has managed quality assurance in a production software team. The models learned to game the grading system. Anthropic’s researchers documented the pattern directly: across five assistant models, sycophancy showed up in every task they tested, and both humans and preference models sometimes preferred a convincing sycophantic response over a correct one (Sharma et al., October 2023). OpenAI’s scaling work showed the same failure from the other side: optimize hard enough against a proxy reward and the true objective starts to suffer, with policies learning to write very long answers even when short ones were preferable (Gao et al., October 2022). The reward pulled toward sycophancy and verbosity more than correctness.
That era is rapidly closing. DeepSeek-R1 made the alternative recipe public in January 2025: Reinforcement Learning with Verifiable Rewards, or RLVR, where the model trains against checkable rewards instead of taste. OpenAI’s o-series models made the same turn visible in products, though OpenAI hasn’t published its training recipe at the same depth.
Instead of asking a human if an answer feels convincing, RLVR asks a deterministic environment if the answer is actually true.
The Power of a Reward You Can’t Flatter
I spent a significant portion of my career working on software for aviation platforms and financial ledgers. In those environments, you quickly learn that subjective approval is a terrible quality gate.
If an engineer asks their lead whether a navigation algorithm looks sound, the lead’s approval is merely an opinion. Only the test harness can answer the question: does the calculation converge within tolerance across 10,000 simulated flight trajectories? Does the double-entry accounting ledger balance to the penny under concurrent debit requests?
If the automated test fails, no amount of smooth persuasion matters. The code is simply wrong.
RLVR applies that exact discipline to neural networks. In DeepSeek AI’s technical report on DeepSeek-R1 (January 2025), the research team demonstrated what happens when you train a model with pure reinforcement learning using only rule-based, objective rewards. Their R1-Zero variant proved this out before the final model added a supervised cold-start stage on top of the recipe.
The reward signal in RLVR is binary and automated:
- Did the generated Python script pass all unit tests without errors?
- Does the emitted SQL query execute and match the ground-truth relational schema?
- Does the mathematical proof satisfy the formal verification engine?
Because the model cannot flatter a Python interpreter or charm a math solver, it cannot talk its way to a high reward. To maximize its score, the model begins to develop emergent problem-solving behaviors. It generates longer reasoning chains, checks its intermediate logic, revisits failed steps, and tries alternative paths before producing a final answer. DeepSeek’s report credits the RL stage itself for the emergence of self-reflection, verification, and dynamic strategy adaptation. From the outside, those behaviors read as the long pause before an answer. The product side of that pause is the subject of my post on test-time compute.
Why LLM-as-a-Judge Hit a Wall
Many enterprise product teams tried to skip formal verification by using “LLM-as-a-judge” setups. You prompt a large model to grade the output of a smaller model on a numeric scale.
It feels fast, and it fits neatly into an agile sprint. But as demonstrated by LMSYS researchers in their empirical study of LLM evaluators (June 2023), LLM-as-a-judge approaches suffer from position, verbosity, and self-enhancement biases, along with limited capability in grading math and reasoning questions. The scores move with answer position, length, and whether the judge recognizes its own work, and a polished, confident answer carries an edge it didn’t earn.
When you evaluate a product feature with vibes, you ship features that look impressive in a five-minute executive demo yet break in the hands of your first twenty enterprise customers.
| Evaluation Attribute | LLM-as-a-Judge / Vibes | Programmatic Verifiers (RLVR) |
|---|---|---|
| Evaluation Signal | Subjective Likert score (1–5) | Binary deterministic check |
| Vulnerability | Style bias and sycophancy | Executable ground truth |
| Scaling Cost | Linear API cost per review | Local CPU / sandbox execution |
| Failure Mode | Confidently wrong approvals | Concrete syntax / test error |
The Shift from Prompt Craft to Verifier Engineering
For product managers and engineering leaders, the rise of verifiable reasoning demands a major shift in how we spend our engineering hours.
For two years, teams burned endless cycles on prompt engineering. We tweaked system instructions, added role-play personas, and inserted phrases like “think step by step” hoping to coax better answers out of the model. In an RLVR world, verifier engineering replaces prompt engineering.
Your competitive advantage as a product team is your ability to construct deterministic environments that measure real correctness in your business domain, not the cleverness of your prompts.
If you are building an automated code migration tool, your primary investment should not be tuning prompts but building an isolated container sandbox that compiles the migrated code and runs regression suites automatically.
If you are building a financial analytics assistant, your job is to build strict schema validators that verify whether the model’s computed totals match the underlying database tables before the number reaches the client.
One question sits outside the training loop: who decides whether the verifier measures the right thing? A schema validator can be implemented perfectly against the wrong definition of correct. That call is product judgment, and it tends to land on the PM.
When you provide models with strict, immediate programmatic feedback, you transform AI from a probabilistic party trick into reliable software engineering.







