"It could always be worse" is one of those sayings, rather like "it is what it is," that usually only worsens whatever unpleasantness you find yourself mired in.