Estimating LLM Training FLOPs on the Nvidia Jetson Orin Nano

This is a research summary for an ongoing project I am working on as part of the UChicago Existential Risks Laboratory Summer Research Fellowship . I would really appreciate any feedback. Introduction Motivation In want of a quantifiable way to decide what counts as a frontier AI model, compute thresholds have emerged as the standard for AI policy: California’s SB 53 uses 10^26 floating-point operations (FLOPs) in the training run as the threshold for what counts as a frontier model and the EU AI Act applies the same categorization at 10^25. Proposals for international AI agreements ( example 1 , example 2 , example 3 ) extend the use of training FLOPs to determine part or all of the threshold for what counts as a frontier model under the agreement. Current AI laws have no way of actually verifying AI companies’ claims about the number of FLOPs used in training and instead just rely on self-reports, but an international AI agreement can’t assume compliance from each involved party. As such, we’d like to verify the number of FLOPs used in LLM training runs through side-channel GPU readings. This allows AI developers’ code and data to remain hidden from the verifiers of the AI agreement, but allow verification of training FLOPs even under conditions where the model training might be adversarially changed to circumvent them. My work builds a Minimum Viable Product (MVP) for how this verification could work on an Nvidia Jetson Orin Nano. Related Work EpochAI has done work on est…