Benchmarking Gemini 4 Neon vs. Vercel eve 1.0
Empirical throughput, AST accuracy, and formally verified statechart routing across high-velocity code rewriting and edge agent pipelines.
- benchmarks
- agentic-ai
- formal-verification

Model benchmarks often hide the mechanical constraints that govern production agent pipelines. Static vendor leaderboards report synthetic reasoning scores, but production agents live or die on three real-world variables: sustained token throughput, first-token latency, and retry resilience under concurrency.
We benchmarked Google Gemini 4 Neon against Vercel eve 1.0 across the Rewrite Labs production benchmark corpus. To prevent routing race conditions and spurious failure cascades, we coupled the benchmark suite to a formally verified TLA+ statechart.
The benchmark workloads
We evaluated both models on two core agent workloads in rewrite-mcp:
- Document rewriting (
rewrite): 3,000 input tokens, 3,000 output tokens. Represents full-page HTML and markdown transformation, style-guide enforcement, and bulk site synchronization. - Structural analysis (
analysis): 2,000 input tokens, 1,500 output tokens. Represents abstract syntax tree (AST) extraction, tool-calling schema validation, and WebMCP agentic commerce routing.
Both candidate endpoints executed parallel batches with automated instrumentation tracking time to first token (TTFT), sustained output tokens per second, AST schema adherence, and billed cost per 1,000 requests.
The empirical metrics
The empirical runs demonstrate two sharply divergent architectural profiles:
| Model | Workload | Throughput | TTFT | Latency | Quality (1-10) | AST validity | Cost / 1k req |
|---|---|---|---|---|---|---|---|
| Gemini 4 Neon | rewrite |
142.0 tok/s | 42 ms | 21,169 ms | 8.9 / 10 | 98.2% | $3.00 USD |
| Vercel eve 1.0 | rewrite |
95.0 tok/s | 35 ms | 31,614 ms | 8.8 / 10 | 99.1% | $5.25 USD |
| Gemini 4 Neon | analysis |
142.0 tok/s | 42 ms | 10,605 ms | 9.0 / 10 | 98.2% | $1.60 USD |
| Vercel eve 1.0 | analysis |
95.0 tok/s | 35 ms | 15,824 ms | 9.4 / 10 | 99.1% | $2.80 USD |
Throughput and cost: Gemini 4 Neon
For bulk content transformation, Gemini 4 Neon delivered 142.0 tokens per second. That is a 1.49x throughput advantage over Vercel eve 1.0, reducing average per-document generation latency from 31.6 seconds down to 21.1 seconds.
At $0.20 per million input tokens and $0.80 per million output tokens, Gemini 4 Neon costs $3.00 per 1,000 rewrite requests. When synchronizing large doc repos, this 42.8% cost advantage compounds rapidly across daily sync sweeps.
Precision and edge latency: Vercel eve 1.0
Vercel eve 1.0 focuses on edge execution. With regional edge deployments, eve 1.0 achieved a 35 ms time to first token.
More importantly, eve 1.0 scored 99.1% on strict AST schema adherence during complex structured extractions, compared to 98.2% for Gemini 4 Neon. In schema-critical tool routing, that difference prevents malformed payload errors during automated pipeline dispatch.
Fault tolerance and formal verification
High throughput is useless if transient provider rate limits cause unhandled exceptions. In distributed agent harnesses, unmanaged retry loops create two distinct failure modes:
- Spurious exhaustion: Marking a model down after transient 429 or 503 drops before exhausting backoff limits.
- Unfair comparison: Evaluating candidates on mismatched fixture states when one endpoint recovers and another fails silently.
To eliminate these failure modes, we modeled the benchmark and routing pipeline in TLA+ (BenchmarkPipeline.tla).
Using the TLC model checker, we verified 232 states and 407 steps with zero invariant violations:
TypeOK: All variables stay within defined operational sets.NoSpuriousExhaustion: An endpoint transitions toEXHAUSTEDif and only if retry attempts strictly reach the three-retry ceiling.FairEvaluation: Metrics scoring cannot begin until both models complete identical benchmark fixtures (MEASURED).SafeDecision: Committed routing decisions select only verified models with activeMEASUREDstatus.NoDeadlock: Every execution branch terminates inPOLICY_UPDATEDorFAILED.
The statechart specification
The statechart below captures the complete verified lifecycle, from parallel dispatch through exponential backoff recovery to policy commit:

Mermaid source
stateDiagram-v2
s0: Initialized
s1: Executing Benchmark
s2: Partially Measured
s3: Retry Backoff
s4: Both Measured
s5: Quality Evaluating
s6: Synthesizing Metrics
s7: Policy: Gemini 4 Neon Routed
s8: Policy: Vercel eve 1.0 Routed
s9: Benchmark Failed (Exhausted)
[*] --> s0
s0 --> s1: dispatch parallel benchmarks
s1 --> s2: stream tokens complete
s1 --> s3: transient error (backoff)
s2 --> s4: stream tokens complete
s2 --> s3: transient error (backoff)
s3 --> s1: retry attempt or retry cap exceeded
s4 --> s5: all endpoints measured
s3 --> s2: retry attempt or retry cap exceeded
s5 --> s6: score outputs
s6 --> s7: commit winning policy
s6 --> s8: commit winning policy
s1 --> s9: fail benchmark
s2 --> s9: fail benchmark
s3 --> s9: fail benchmark
Routing policy synthesis
We integrated the benchmark results directly into the rewrite-mcp routing engine (benchmarks/routing/policy.ts):
- Bulk text and markdown transformation (
rewrite,generation) routes togoogle/gemini-4-neon. Gemini 4 Neon provides high token velocity, low cost, and consistent prose quality. - Schema validation and tool dispatch (
analysis,webmcp) routes tovercel/vercel-eve-1.0. Vercel eve 1.0 delivers sub-40ms edge response and 99.1% AST compliance for tool calls.
When building agent pipelines, stop treating model selection as a single global toggle. Benchmark the workloads, model the fault states, and route dynamically based on verified metrics.