PTEENet: Post-Trained Early-Exit Neural Networks Augmentation for Inference Cost Optimization

PTEEnet optimizes deep neural networks by introducing early exit shortcuts, extending BranchyNet and EEnet. It uses confidence heads to predict when to exit costly computations, adjusting thresholds for speed and accuracy control. Evaluated on image datasets, PTEEnet reduces computational cost while maintaining model accuracy. The method strategically places early exit branches based on computational cost and distribution methods, enhancing practicality. It calculates cumulative predictions and computational costs through cross-entropy loss, enabling efficient and accurate classification. PTEEnet methodology attaches early exit branches to pre-trained networks, facilitating integration with pruning and compression techniques. It optimizes main networks by enabling early exiting, aiming to enhance accuracy-cost ratio and real-time data propagation control.