Addressing Reproducibility and Energy-efficiency in AI Deployments

Abstract

Deploying AI models on edge devices presents challenges in ensuring reliable and energy-efficient operations. Edge AI processes data directly on devices like IoT sensors and industrial machinery, enabling real-time decision-making and reducing latency, which is crucial for applications such as autonomous driving and robotics. However, deploying these models often involves custom Infrastructure as Code (IaC) scripts, and a lack of reproducibility in these scripts can cause inconsistencies and affect system reliability. Additionally, while advancements in hardware like SoCs, FPGAs, and AI accelerators have improved Edge AI capabilities, these deployments can lead to high energy consumption. Our research addresses these challenges through two main contributions. First, we identify and categorize reproducibility smells in IaC scripts, particularly focusing on an automation platform, Ansible, that allows imperative infrastructure configuration. We developed a tool, Reduse, to detect these reproducibility smells, in the pursuit to ensure that IaC scripts are reliable and consistent. Our empirical study reveals the occurrence of these smells in open-source projects, with significant correlations and co-occurrence patterns among them. For instance, the broken dependency chain smell was found in approximately 71% of Ansible tasks analyzed, highlighting common reproducibility issues. Second, we comprehensively evaluate the selection of AI models on edge devices, including the Raspberry Pi, NVIDIA Jetson Nano, and Intel Neural Compute Stick. By measuring inference power consumption, accuracy, inference time, and memory utilization, we offer insights into the performance and energy efficiency trade-offs of these models. For instance, Jetson Nano provides the best accuracy at the cost of a high energy budget. Thus, our work advances the field of edge AI with the best practices in IaC, contributing to more reliable and effective AI deployments in real- world scenarios.