The practical implementation of deep convolutional networks faces challenges such as feature redundancy and diminished learning efficiency. To address these issues, this paper focuses on Deep Convolutional Neural Networks (DCNNs), exploring two key aspects: feature extraction and training efficiency. The aim is to enhance the prediction accuracy, robustness, and generalization of deep convolutional networks. The proposed algorithms are applied to image classification. In response to the problem of feature redundancy in DCNN, an improved DCNN algorithm based on residual Dropout convolution (RD Conv) is proposed. The proposed algorithm includes residual Dropout paths and convolutional paths, which are randomly switched during training, resulting in random changes in network depth and enhancing the diversity of feature extraction. The residual path randomly selects input features, compresses and amplifies them, increasing the diversity of feature inputs in the downstream parameter layer. Only convolutional paths are used in prediction to ensure stable prediction results. To address the issue of decreased saturation training efficiency in DCNN filters caused by the fixed positions of the activation and saturation regions of the activation function, an improved DCNN algorithm based on a random activation function is proposed. By combining the different primary and secondary activation functions in the active and saturated regions, the filters in the saturated region can participate in the weight update process, increasing feature generation. Our RD Conv proposes a new dropout module that simultaneously utilizes the dropout layer, convolution layer and batch norm layer, while eliminating variance offset. Additionally, our random activation mechanism integrates two different activation functions into a new activation function, improving training efficiency. By using two linear functions as sub-activation functions, the random depth network is expanded beyond the constraints of a residual structure. The experimental results on the image classification datasets Cifar-10, Cifar-100 and Caltech-256 demonstrate that the proposed algorithm accelerates the convergence speed of the loss function and effectively improves the network's prediction accuracy. Additionally, it improves the training efficiency, alleviates overfitting, and achieves better generalization and accuracy. In most experiments, the TOP1 accuracy of the model is increased by at least 2%, while the TOP5 accuracy improves by at least 1%. The best result achieved by the proposed algorithm enhances the TOP1 accuracy by 6.24%, and the TOP5 accuracy by 3.62%. On the TGS Salt Identification and THUCNews datasets, the proposed algorithm still improved the performance of the model in image segmentation and text classification tasks, indicating that the proposed algorithm has good generalization ability.
An improved algorithm for deep convolutional neural network structures based on randomness
Karimi, Hamid Reza
2025-01-01
Abstract
The practical implementation of deep convolutional networks faces challenges such as feature redundancy and diminished learning efficiency. To address these issues, this paper focuses on Deep Convolutional Neural Networks (DCNNs), exploring two key aspects: feature extraction and training efficiency. The aim is to enhance the prediction accuracy, robustness, and generalization of deep convolutional networks. The proposed algorithms are applied to image classification. In response to the problem of feature redundancy in DCNN, an improved DCNN algorithm based on residual Dropout convolution (RD Conv) is proposed. The proposed algorithm includes residual Dropout paths and convolutional paths, which are randomly switched during training, resulting in random changes in network depth and enhancing the diversity of feature extraction. The residual path randomly selects input features, compresses and amplifies them, increasing the diversity of feature inputs in the downstream parameter layer. Only convolutional paths are used in prediction to ensure stable prediction results. To address the issue of decreased saturation training efficiency in DCNN filters caused by the fixed positions of the activation and saturation regions of the activation function, an improved DCNN algorithm based on a random activation function is proposed. By combining the different primary and secondary activation functions in the active and saturated regions, the filters in the saturated region can participate in the weight update process, increasing feature generation. Our RD Conv proposes a new dropout module that simultaneously utilizes the dropout layer, convolution layer and batch norm layer, while eliminating variance offset. Additionally, our random activation mechanism integrates two different activation functions into a new activation function, improving training efficiency. By using two linear functions as sub-activation functions, the random depth network is expanded beyond the constraints of a residual structure. The experimental results on the image classification datasets Cifar-10, Cifar-100 and Caltech-256 demonstrate that the proposed algorithm accelerates the convergence speed of the loss function and effectively improves the network's prediction accuracy. Additionally, it improves the training efficiency, alleviates overfitting, and achieves better generalization and accuracy. In most experiments, the TOP1 accuracy of the model is increased by at least 2%, while the TOP5 accuracy improves by at least 1%. The best result achieved by the proposed algorithm enhances the TOP1 accuracy by 6.24%, and the TOP5 accuracy by 3.62%. On the TGS Salt Identification and THUCNews datasets, the proposed algorithm still improved the performance of the model in image segmentation and text classification tasks, indicating that the proposed algorithm has good generalization ability.I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.
