Performance and Power Modeling of Embedded AI Computing Platforms

Abstract

Autonomous vehicles and robots rely on embedded machine-learning platforms, which require performance and power estimates early in the design cycle, before the hardware is available; architectural simulation provides these estimates. However, cycle-level graphics processing unit (GPU) simulators and their power models have been developed and validated mainly on discrete high-performance GPUs, and they do not perform satisfactorily on embedded systems, which differ in memory organisation, power management and the hardware instrumentation they expose.

This thesis evaluates the Accel-Sim simulation framework and the AccelWattch power model on the NVIDIA Jetson AGX Xavier, an embedded platform with a Volta GPU, and examines which mechanisms of the model contribute to the remaining error. Our methodology assigns every model parameter a value measured on the device, rather than a value estimated by an automatic tuner or taken from another published configuration. No change is kept or discarded based on its effect on the error. Using measured values initially increases the error on some suites. This is because the previous values kept the error spuriously low by offsetting errors elsewhere in the model, even though the simulator did not represent the device accurately. With measured values, these errors become visible.

Following this methodology, we calibrate the memory system, the execution units and the kernel launch latency, and extend the performance model with three mechanisms: L2 invalidation between kernels, register-bank conflicts, and the access granularity of the main memory (DRAM). For power, we measure the constant and static power of the device and build an independent validation set of application kernels.

Across successive simulator versions, the mean absolute error of simulated cycles decreases on three of the four benchmark suites, by 4.4–12.7 percentage points, while instruction and memory-transaction counts remain within 10% of the hardware on three suites. The remaining error therefore lies in execution timing. With measured activity and execution time, AccelWattch predicts power with a mean absolute error of 16.1%; with simulated time, the error rises to 29.0–56.6% and follows the cycle error. Power prediction on this platform is therefore bounded by the accuracy of the performance model.