Sixty Frames a Second and Still Unplayable: Benchmarking's Missing Metric

Framerate counts frames. Frametimes measure how evenly they arrive. Neither says whether the world inside those frames is moving at the pace the game intended, which is why two machines can post near-identical numbers and feel nothing alike.

Smoothness and Acceleration Are Different Problems

Percentile lows arrived around 2014 and changed how graphics hardware was judged: before them an average could hide a hitch entirely, afterwards the worst one percent of frames had to answer for themselves. That was a real advance, and it remains the last significant change to how game performance gets published.

The gap those numbers leave is not a measurement error but a different property. Framerate and frametimes describe smoothness: how many images arrive, and whether the intervals are tidy. What no chart described is acceleration — whether the movement inside the frames speeds up and slows down against the pace of the display. Human vision is tuned to notice exactly that, which is why a game can hit a respectable number and still feel wrong in the hands.

Intel called the missing quantity simulation time error before animation error took over, and the concept had been floating around the edges of performance analysis for over a decade without landing in a published chart.

The Flipbook, Defined

Think of a flipbook: the drawings can be perfect, and if you flip the pages at an uneven tempo the animation still stutters. Animation error is that failure, measured — the difference between the pacing of the movement drawn inside the frames and the pacing at which they are shown.

Two timestamps do the work: the moment in game time that a frame depicts, and the moment that frame reaches the display. When consecutive animation timestamps are evenly spaced but the display intervals are not, the mismatch is the error. Positive means too soon, negative too late; at a macro level distance from zero is what counts, in either direction.

The arithmetic is one subtraction per frame: the gap between this frame's animation timestamp and the previous one, minus the interval the previous frame spent on screen. Summing raw values cancels positives against negatives, so totals need absolute values. There is a wrinkle in expressing that as a percentage: because the worst case allows accumulated error to reach twice the total frametime, the denominator has to be doubled, or the chart implies a ceiling that does not exist.

The Demonstrations That Made the Case

The tidy way to prove a new metric is to find a case where the old metrics are blind, and multi-GPU supplied it. A pair of GTX 1080 Tis in SLI against a single card, in games that supported it: both produced averages and percentile lows that looked civilised, and the two-card setup even had the higher average. Nothing in the conventional data suggested a problem.

The animation error plots told a different story. The single card sat close to zero on nearly every frame. The SLI pair carried error in both directions on almost every frame it produced, and rarely approached zero at all. Same class of hardware, similar headline performance, and one was doing something structurally unpleasant the entire time. That is the argument in a single comparison.

The second demonstration is better still because it crosses vendors. In one title, an older card running GPU-accelerated PhysX and a current card running the same effects on the CPU produced averages within a handful of frames of each other, with lows to match. By the animation error ratio one result was an order of magnitude worse than the other. Nothing about the frame count would have told you which machine to avoid. Treat the specific figures as unverified — this is early methodology from a single outlet — but the shape of the result is the point.

Why Frame Generation Breaks the Measurement

Generated frames have no animation time. A frame invented between two real ones depicts a moment that the game never simulated, so there is no reference to compare against the display interval. The metric, as defined, cannot score them. That is not a small caveat in a market where half the performance story is now sold as frames nobody rendered.

Multi-frame generation ships with flip metering, which deliberately redistributes frame timings at the end of the pipeline. Metering early, before the queue, is harmless. Metering late is precisely the mechanism that manufactures animation error, and it is the same mechanism that produced runt frames in the multi-GPU era: fractions of frames shown briefly enough to raise the average while the experience degrades. The suspicion is that flip metering does something similar with a better marketing budget. Proving it needs software access that the vendor has not obviously offered.

Low-latency modes are the other complication. The current proxy for animation time is the moment the CPU starts building a frame, which works until Reflex or XeLL clears the render queue and leaves the CPU waiting for input until the last moment. At that point the proxy and the real animation time diverge badly on nearly every frame. Those modes emit simulation-start events that can be used instead. Anti-Lag 2 does not, so that path cannot be measured accurately yet. And it is worth restating what the metric is not: latency. Delay every frame by the same amount and the animation timing stays perfectly matched, so the error stays at zero while the input lag gets worse.

What I Think

This is the most useful thing to happen to game benchmarking since percentile lows, and I say that knowing it will change almost no conclusions. Most of the time animation error will agree with frametimes, because bad performance is bad in more than one way at once. Its value is in the cases where the numbers disagree — the SLI pair with the better average, the old card with the clean frametime plot and the rubber-banding camera — and those cases are exactly the ones a buyer gets burned by.

My worry is adoption, not validity. A metric that requires PresentMon, careful handling of low-latency modes, and a doubled denominator is a metric most outlets will not implement correctly, and a badly built version is worse than none because it looks authoritative. The vendors could settle this by exposing simulation timestamps properly in every low-latency path. Until they do, treat any animation error chart as research rather than verdict, and keep reading the frametime plot alongside it.