From Inference Efficiency to Embodied Efficiency: Revisiting Efficiency Metrics for Vision-Language-Action Models

This paper challenges how we measure efficiency in Vision-Language-Action (VLA) models used by robots, showing that traditional metrics like computational cost don't match real-world robot performance. The researchers found that optimizing for standard efficiency measures often makes robots slower, jerkier, or less smooth in practice, even if task success rates stay the same. They propose measuring 'embodied efficiency' instead—focusing on actual robot behaviors like completion time, motion smoothness, and energy use—to get a more accurate picture of how well these AI systems actually perform on real robotic tasks.