I think it depends on your height and how much you have to lose. I've lost 10 pounds and feel like a new person, but I don't think anyone's noticed. If I lose 8 more, I'll be on the low end of my BMI and will hopefully fit into some smaller (and cuter!) clothes. But I'm 5'11" and while people don't notice as much when I gain weight, the same goes when I lose it.
I've heard that if you lose 10% of your body weight, there are lots of benefits healthwise.