Solid waste disposal has been emerging as a challenging issue worldwide following the enhanced economic activities and rapid urbanization. Considerable efforts have been made to use solid waste materials as partial cement replacement in concrete. This work investigates the sulfate resistance of concrete containing different solid waste materials—waste glass powder, coal gangue powder and fly ash, at various substitution levels (10, 20, and 30%). Two water to binder ratios were adopted as 0.41 and 0.54. The sulfate resistance was evaluated by exposing the prepared concrete specimens to a 5% sodium sulfate solution for a total period of 22 months and examining the mass change, compressive strength, splitting tensile strength, ultrasonic pulse velocity, mineralogical and microstructural properties. Results obtained show that the sulfate resistance of concrete can be improved substantially by the addition of solid waste materials. Additionally, waste glass powder is found to be the most effective, followed by gangue coal powder and fly ash. These studies reveal that in addition to helping solid waste management, reusing of these solid waste materials in concrete improves the resistance to sulfate attack.