Dissecting the Neural Code
neuron-by-neuron analysis of a digit recognition network
DOI:
https://doi.org/10.5540/03.2025.011.01.0382Keywords:
neuron-level analysis, neural network pruning, neuron importanceAbstract
This study analyzes the role of individual neurons in neural network performance, focusing on the concept of “rotten” neurons whose removal affects network accuracy. By examining two neural network configurations across the MNIST and SVHN datasets, we demonstrate the diverse impact of neurons, from beneficial to detrimental. Our results reveal that neural network efficiency can be improved by addressing the influence of specific neurons. This research highlights the potential for neuron-level analysis and pruning to enhance neural network optimization.
Downloads
References
Yann LeCun, Corinna Cortes, and Christopher J Burges. The MNIST database of handwritten digits. Online. Accessed on March 8, 2024, http://yann.lecun.com/exdb/mnist/. 1998.
Yuval Netzer and Tao Wang. “Reading digits in natural images with unsupervised feature learning”. In: NIPS workshop on deep learning and unsupervised feature learning. (2011). Vol. 2011, No. 5.
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. “Scikit-learn: Machine Learning in Python”. In: Journal of Machine Learning Research (2011). Vol 12. pp. 2825-2830.
Huan Wang, Can Qin, Yue Bai, and Yun Fu. Why is the State of Neural Network Pruning so Confusing? On the Fairness, Comparison Setup, and Trainability in Network Pruning. 2023. doi: https://doi.org/10.48550/arXiv.2301.05219.
Xin Yu, Thiago Serra, S. Ramalingam, and Shandian Zhe. The Combinatorial Brain Surgeon: Pruning Weights That Cancel One Another in Neural Networks. 2022. doi: 10.48550/arXiv.2203.04466.
