Communication-Efficient Parallelization Strategy for Deep Convolutional Neural Network Training

Keyphrases

• Communication Efficiency 100%
• Parallel Algorithm 100%
• Efficient Parallelisation 100%
• Stochastic Gradient Algorithm 100%
• Master-slave System 100%
• Deep Convolutional Neural Network (deep CNN) 100%
• VGG16 100%
• Convolutional Neural Network Training 100%
• ResNet50 100%
• Design Strategy 50%
• Parallelization 50%
• Popular 50%
• High Cost 50%
• Stochastic Gradient Descent 50%
• Computing Node 50%
• Convergence Rate 50%
• Gradient Averaging 50%
• Overlap Degree 50%
• Averaging Algorithms 50%
• NERSC 50%
• Inter-process Communication 50%
• Staleness 50%
• Allreduce 50%
• Parameter Update 50%
• Convolutional Neural Network Model 50%
• Cori 50%
• Cray XC40 50%
• ImageNet Dataset 50%
• Asynchronous Stochastic Gradient Descent 50%
• Time Complexity Analysis 50%
• Interprocess Synchronization 50%

Computer Science

• Gradient Descent 100%
• Parallelization 100%
• Convolutional Neural Network 100%
• Neural Network Model 66%
• Residual Neural Network 66%
• VGG-16 66%
• Time Complexity 33%
• Design Strategy 33%
• Supercomputer 33%
• Complexity Analysis 33%
• Interprocess Communication 33%
• Convergence Rate 33%