Liquid AI Open-Sources Pipette: A Reproducible Benchmarking Suite That Measures On-Device Models, Quantization, Runtime and Hardware Together

Liquid AI released Pipette, an open-source Apache 2.0 platform for benchmarking foundation models on edge devices, with Artificial Analysis serving as independent methodology validator. Pipette measures full deployment configurations—model, quantization, runtime, and device—across more than 1,000 configurations spanning 30+ models and llama.cpp builds for macOS, iOS, Windows, and Android. Initial verified results cover a MacBook Pro M5 Max, iPhone 17 Pro, and Galaxy S26 Ultra, including context lengths from 256 to 8,192 tokens.