I think being dairy free is very easy EXCEPT that as a vegetarian, going out to eat and avoiding dairy is an absolute chore. Vegetarian to most of the world = bread + cheese. And I'm gluten free, also.
At home, it's the easiest thing in the world. You can make your own pastries; there are so many alternatives for milks out there or you can even make your own. There's a machine out there called Soybella that makes great nut milks. I borrowed it from a friend but haven't invested in one myself. You put in the water, nuts and whatever sweetener you want - I use dates or stevia liquid. There are a couple lines of coconut ice cream that are delicious. Otherwise, I make my own "ice creams" by running frozen fruit through a juicer or a good blender. My favorite is a frozen banana, two regular bananas, dates, the juice of one or two oranges, and a package of frozen raspberries. Extraordinary!!
As far as yogurts and cheeses go, there aren't great alternatives. I don't eat soy, either
. You just have to get over it, I guess. Most cheeses do nothing for me, so I never care much. There are select cheeses out there, however, that are hard for me to resist if they're right in front of me. I indulge here and there. I never go out of my way for it, though.