We tested our AI logger against 1,000 USDA foods — three times each
“AI calorie estimates are just guesses.” We built a macro tracker with AI logging features, so we're obviously not neutral — which is exactly why we ran the numbers. Here's every result, misses included.
In one sentence: for the foods people actually log day to day, the AI's calorie estimate typically lands within about 10 calories of the USDA's official value — within 20% for 9 out of 10 foods, and returning the same numbers every time you log it. For context, US nutrition labels are legally allowed to be off by up to 20%.
How we tested it
Built to be checkable — the raw data is linked at the bottom.
1,000 foods, two tiers
100 hand-curated well-known foods with natural phrasing (“1 medium banana”, “1 McDonald's Big Mac”) — plus 900 drawn programmatically from the USDA's FNDDS survey database across ten categories, fed to the AI in the USDA's own words and portions, verbatim. No paraphrasing step that could smuggle in help.
USDA ground truth
FNDDS 2021–2023 via FoodData Central — the federal survey of foods Americans actually eat, with lab-analyzed nutrition per portion. Excluded, in fairness's favor: water, alcohol (invisible to macro-based calorie math — scoring it would inflate our accuracy), and foods under 20 kcal per portion.
The production pipeline, three times per food
Each query went through the exact model, prompt, and response handling that runs when you log a food here — three times. We report the per-food median of three, and measured the run-to-run scatter separately. Nothing was retried for a better answer.
Explicit portions
Every query stated the amount so both sides estimated the same thing. Calories on both sides are derived from macros the same way the app derives them (4/4/9).
The results
All 1,000 foods, per-food median of 3 runs. Calories derived from macros (4·P + 4·C + 9·F) on both sides, exactly how the app computes them.
| Protein | Carbs | Fat | Calories | |
|---|---|---|---|---|
| Median error | 1.1 g | 2.7 g | 1.5 g | 29 kcal |
| Mean absolute error | 2.9 g | 5.9 g | 3.6 g | 57 kcal |
| Average bias (+ = AI high) | +0.9 g | +0.6 g | +0.8 g | +13 kcal |
The tier that matters for daily use
The 100 well-known foods replicate our original study exactly — same 8 kcal median, same ~9-in-10 within 20% — now confirmed across three runs each.
| Food set | Median error | Within ±20% | Within ±10% | Bias |
|---|---|---|---|---|
| 100 well-known foods, natural phrasing | 8 kcal | 88% | 80% | +1 kcal |
| Full 1,000 (incl. the USDA long tail, USDA phrasing) | 29 kcal | 61% | 48% | +13 kcal |
Same food, same numbers
Three runs per food let us measure the “AI is a slot machine” worry directly.
Whatever error exists is mostly a systematic disagreement about a specific food, not randomness. Log the same breakfast tomorrow and you'll get the same numbers.
Accuracy by category
Average calorie error per food across all 1,000. Single-ingredient categories cluster near label variance; named mixed dishes span whole recipe families.
The hits — and the misses
We're publishing the worst results too, because they tell you more than the wins. All values in kcal, per-food median of 3 runs.
| Food | USDA | AI | Δ |
|---|---|---|---|
| 1 oz of salted hard pretzels | 109 | 109 | 0 |
| 1 tablespoon of sunflower oil | 126 | 126 | 0 |
| 1 slice of pumpernickel bread, toasted | 81 | 81 | 0 |
| 3 cups air-popped popcorn, no butter | 97 | 97 | 0 |
| 1 cup of unsweetened almond milk | 35 | 35 | 0 |
| 1 tablespoon of ketchup | 20 | 20 | 0 |
| Food | USDA | AI | Δ |
|---|---|---|---|
| 1 cup of cream, whipped | 139 | 840 | +701 |
| 1 package of Brie (4.5 oz) | 427 | 1744 | +1317 |
| 1 cheese-filled pastry | 77 | 334 | +257 |
| 1 piece of breadfruit, cooked | 47 | 248 | +201 |
| 1 sopaipilla with syrup or honey | 44 | 242 | +198 |
The whipped cream
The USDA's “1 cup of cream, whipped” means already-whipped cream — 40 grams of mostly air. The AI priced a cup of liquid cream. Two different foods hiding in one phrase.
The package of Brie
The USDA's “package” is 4.5 oz. The AI assumed something closer to a whole wheel. Nobody involved disagreed about what Brie is made of.
The breadfruit
The USDA's “1 piece” of cooked breadfruit is a 35-gram chunk. The AI pictured a hefty wedge of a dense, starchy fruit. Same food, very different ideas of a “piece.”
The pattern: when the AI misses big, the words were ambiguous — not the nutrition knowledge. “1 cup of whipped cream” is genuinely two different foods depending on how you read it. A portion in grams or ounces, or a count of a well-defined unit (“2 slices”, “1 tablespoon”), removes the entire failure mode.
Honest limitations
- The 900-food tier uses the USDA's own phrasing — nobody types “1 piece/slice of Carrots, canned, cooked with butter or margarine” into an app. That makes the full-sweep numbers likely understate real-world accuracy on foods you'd actually log.
- Calories are derived from macros (4/4/9), matching how the app works — which is also why alcohol was excluded rather than scored as a free win.
- One model, one point in time, text input with explicit portions. Vague portions (“some rice”) shift the burden to portion guessing — a separate error source we haven't measured yet.
USDA-grade numbers with a fraction of the effort of database searching — and the same numbers every time. Add a portion size and you're inside the error bars of the nutrition label itself.
Questions skeptics should ask
The set was fixed before any testing: 100 hand-picked common foods, plus 900 drawn programmatically from the USDA's database across ten categories. For those 900 we fed the AI the USDA's own description and portion verbatim — no paraphrasing step that could smuggle in help. Every food tested is in the published results.
Three attempts per food, all through the exact production pipeline, and we report the per-food median — plus the run-to-run scatter itself (median spread: 9 kcal; zero on the everyday-foods tier). Nothing was retried for a better answer.
Because scoring them would have inflated our accuracy. The app derives calories from macros (4/4/9), and alcohol calories aren't carried by protein, carbs, or fat — so on beer or wine, the AI and the USDA macro math trivially agree near zero. That's a blind spot shared by every macro tracker, and it shouldn't count as a win. Foods under 20 kcal were excluded for the same reason percentage math on a 3-kcal carrot slice is meaningless.
This test used explicit portions so both sides were estimating the same thing. With a vague description the AI has to guess the portion too — a separate error source we haven't measured yet. The practical takeaway stands: state an amount and you get the accuracy shown here.
Yes — download the full per-food comparison table (CSV): all 1,000 foods, USDA values, the AI's per-food medians, deltas, and the calorie spread across each food's three runs.
Methodology: USDA FNDDS 2021–2023 via FoodData Central · model claude-sonnet-5 · production-identical prompt and response handling · 3 runs per food, per-food median reported · explicit portions · calories derived as 4P + 4C + 9F on both sides · “within” percentages use a ±25 kcal floor so small foods aren't judged on percentage alone · excluded: water, alcohol, foods under 20 kcal per portion · raw data (CSV)