Researchers have introduced WearableQA, a benchmark designed to test whether large language models can reason over the longitudinal wearable records of real users. The work addresses a gap: while wearable sensing now enables continuous monitoring of physiological and behavioral signals, existing benchmarks rarely assess AI reasoning over an actual person's accumulated wearable history.
The benchmark contains 4,084 ten-option multiple-choice questions constructed from the wearable time series, blood biomarkers, and demographics of 200 real users, each with up to roughly 500 days of daily measurements. According to the paper, WearableQA preserves authentic wearable distributions, including device noise, missing days, and inter-individual variability, rather than relying on synthetic or idealized signals.
To probe distinct capabilities, the authors define 16 question types organized along two axes: data versus health reasoning, which separates computation over longitudinal measurements from physiological interpretation, and single- versus cross-signal reasoning, which distinguishes analysis of individual signals from the integration of multiple signals.
Questions were built at scale using a dual-grounding framework that combines literature-grounded physiological findings with statistically validated population-grounded physiological patterns, allowing the benchmark to capture meaningful relationships present in real-world wearable data.
Evaluating 14 proprietary and open-source LLMs, the authors report accuracy ranging from 19.6% to 72.9% against a 10% chance baseline. The benchmark remains far from solved, with most models scoring below 60%, suggesting it offers a realistic and diagnostic test of LLM reasoning over wearable data.