HARDWARE-AWARE HYBRID COMPRESSION OF CONVOLUTIONAL NEURAL NETWORKS FOR RESOURCE-CONSTRAINED SYSTEMS

Andrian Kozynets, Lidia Demkiv

Abstract


Background. Deep neural networks face critical computational barriers for deployment on resource-constrained edge devices. While individual compression methods exist, their isolated application causes severe accuracy degradation. The synergistic combination of pruning, quantization, and knowledge distillation presents a solution, yet their interaction and hardware-specific efficiency remain insufficiently explored.

Materials and Methods. This study investigates hybrid compression pipelines applied to linear (VGG) and residual (ResNet) network architectures. We evaluated a Two-Stage Distillation algorithm featuring a preliminary classifier stabilization phase to eliminate gradient shock, combined with structural filter removal and static integer quantization. A cross-platform analysis was conducted on central and graphics processing units, and Pareto efficiency frontiers were constructed to determine the optimal trade-offs.

Results and Discussion. The preliminary stabilization phase successfully mitigated gradient shock, consistently improving student accuracy by up to 2.6%. While isolated structural pruning significantly degraded performance, distillation healing via teacher-generated soft targets enabled a pruned network with nearly 30% of weights removed to outperform its full-sized baseline. On CPUs, the hybrid pipeline reduced memory footprint by 11 times and accelerated inference by 5 times. Conversely, integer quantization demonstrated critical inefficiency on GPUs due to memory latency; for such accelerators, half-precision format was Pareto-optimal for massive linear architectures, whereas standard precision guided by two-stage distillation provided the best efficiency for residual models.

Conclusion. Integrating Two-Stage Distillation with structural pruning and quantization fundamentally outperforms isolated compression techniques. The optimal strategy is strictly hardware- and architecture-dependent: integer-based hybrid pipelines are essential for energy-efficient edge CPUs, while for GPU accelerators, the choice between half-precision and standard precision depends on the network's topological characteristics.

Keywords: Artificial intelligence, Deep learning, Neural network optimization, Resource-constrained systems, Hardware acceleration.


Full Text:

PDF

References


[1] Liu, Z., Mao, H., Wu, C. Y., Feichtenhofer, C., Darrell, T., & Xie, S. (2022). A convnet for the 2020s. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 11976-11986). https://doi.org/10.1109/CVPR52688.2022.01167

[2] Deng, L., Li, G., Han, S., Shi, L., & Xie, Y. (2020). Model compression and hardware acceleration for neural networks: A comprehensive survey. In Proceedings of the IEEE, 108(4), 485-532. https://doi.org/10.1109/JPROC.2020.2976475

[3] Gou, J., Yu, B., Maybank, S. J., & Tao, D. (2021). Knowledge distillation: A survey. International Journal of Computer Vision, 129(6), 1789-1819.
https://doi.org/10.1007/s11263-021-01453-z

[4] Stanton, S., Izmailov, P., Kirichenko, P., Alemi, A. A., & Wilson, A. G. (2021). Does knowledge distillation really work? In Advances in Neural Information Processing Systems (NeurIPS) (Vol. 34, pp. 6906-6919).
https://arxiv.org/abs/2106.05945

[5] Blalock, D., Gonzalez Ortiz, J. J., Frankle, J., & Guttag, J. (2020). What is the state of neural network pruning?. Proceedings of Machine Learning and Systems (MLSys), 2, 129-146. https://arxiv.org/abs/2003.03033

[6] Nagel, M., Fournarakis, M., Amjad, R. A., Bondarenko, Y., Van Baalen, M., & Blankevoort, T. (2021). A white paper on neural network quantization. arXiv preprint arXiv:2106.08295. https://arxiv.org/abs/2106.08295

[7] Gholami, A., Kim, S., Dong, Z., Yao, Z., Mahoney, M. W., & Keutzer, K. (2022). A survey of quantization methods for efficient neural network inference. In G. K. Thiruvathukal, Y.-H. Lu, J. Kim, Y. Chen, & B. Chen (Eds.), Low-power computer vision: Improve the efficiency of artificial intelligence (pp. 291–326). Chapman and Hall/CRC. https://doi.org/10.1201/9781003162810-13

[8] Ngo, T. T., Nguyen, B., Pham, T. B., Truong, S. N., Vo, H. M., Minh, K.-S., & Pham, K. V. (2026). Progressive compression of ResNet: An 8.33× reduction with 90.5% accuracy retention for edge deployment. IEEE Access, 14, 56900–56918.
https://doi.org/10.1109/ACCESS.2026.3682854

[9] Lai, N., Dewi, D. A., Maidin, S. S., Xiao, W., Zhao, S., & Hu, Q. (2026). A comprehensive review of lightweight deep learning models for edge computing with future directions. Discover Computing, 29, 110.
https://doi.org/10.1007/s10791-026-10021-3

[10] Alzubaidi, L., Zhang, J., Humaidi, A. J., Al-Dujaili, A., Duan, Y., Al-Shamma, O., ... & Farhan, L. (2021). Review of deep learning: concepts, CNN architectures, challenges, applications, future directions. Journal of Big Data, 8(1), 1-74.
https://doi.org/10.1186/s40537-021-00444-8

[11] Liebenwein, L., Baykal, C., Lang, H., Feldman, D., & Rus, D. (2020). Provable filter pruning for efficient neural networks. In International Conference on Learning Representations. https://arxiv.org/abs/1911.07412

[12] Wu, H., Judd, P., Zhang, X., Isaev, M., & Micikevicius, P. (2020). Integer quantization for deep learning inference: Principles and empirical evaluation. arXiv preprint arXiv:2004.09602. https://arxiv.org/abs/2004.09602

[13] Yun, J., Kang, B & Fu, Z. (2023). The hidden power of pure 16-bit floating-point neural networks. arXiv preprint arXiv:2301.12809. https://arxiv.org/abs/2301.12809

[14] Gyawali, D. (2023). Comparative analysis of CPU and GPU profiling for deep learning models. arXiv preprint arXiv:2309.02521. https://arxiv.org/abs/2309.02521

[15] Ding, X., Zhang, X., Ma, N., Han, J., Ding, G., & Sun, J. (2021). RepVGG: Making VGG-style ConvNets great again. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 13733-13742)
https://doi.org/10.1109/CVPR46437.2021.01352

[16] Yao, Z., Dong, Z., Zheng, Z., Gholami, A., Yu, J., Tan, E., ... & Keutzer, K. (2021). HAWQV3: Dyadic neural network quantization. In International Conference on Machine Learning (pp. 11875-11886). https://arxiv.org/abs/2011.10680




DOI: http://dx.doi.org/10.30970/eli.35.2

Refbacks

  • There are currently no refbacks.