Here is the danger in allowing yourself to be astonished or scandalized by others, or, put better, allowing yourself to consider others unrighteous:
If you indulge the thought that others are unrighteous, conversely it will include within itself your taste for the idea of yourself as righteous. You then risk making this the motivation for your religion: you pray not because of the love of God, but because you want to be able to think of yourself as a righteous person. Thus, your religion becomes vainglorious at best and an idolatry at worst.
Considering others as less righteous leads one to be even less so himself.