ELIMINATING SHARP MINIMA FROM SGD WITH TRUNCATED HEAVY-TAILED NOISE

Keyphrases

• Local Minima 100%
• Sharp Minima 100%
• Heavy-tailed Noise 100%
• Deep Learning 60%
• Gradient Noise 40%
• Learning Task 20%
• Popular 20%
• Learning Rate 20%
• Partial Solutions 20%
• Fixed Threshold 20%
• Structural Conditions 20%
• Continuous-time Markov Chain 20%
• Truncation Threshold 20%
• Poor Generalization 20%
• Loss Landscape 20%
• Gradient Clipping 20%
• First Exit Time 20%
• Real-data Experiments 20%

Mathematics

• Local Minimum 100%
• Deep Learning Method 60%
• Objective Function 20%
• Continuous Time Markov Chain 20%
• First Exit Time 20%
• Real Data 20%
• Truncation 20%
• Learning Task 20%