Ai2 introduces StateBench to test long-context state tracking
Ai2 introduced StateBench on 3 November to test whether language models track changing values over long inputs. Its four levels progress from fixed assignments and latest-value retrieval to cumulative updates and dependencies between variables.
SourcesHugging Face Blog
Read development