First thing, as others have suggested, is to determine what you want to use it for. Are you planning to ride the local bike trails? Do you want to train for a long ride (perhaps Team in Training and a 100 mile ride)? Would you like to get out there and do the dirt? Once you figure out the main focus of where you want to ride, find 2 or 3 Local Bike Shops (LBS) and just ask them questions about what they think would be a good bike for you (avoid the bike chains - they won't help you like an LBS will).
Ride as many as you can. You'll want something that fits you well. Many of the top brands now have Women Specific Designs (WSD) that have a shorter top tube to account for the fact that we tend to have longer legs and shorter torsos than men. Try some of these and some of the standard bikes. I find I do best with a slightly smaller men's frame with the seat elevated and the handle bars raised slightly. The WSD frames don't fit me very well. Everyone is different. If the bike does not feel good under you, it's not the bike for you, no matter how pretty.
I also agree that a saddle is not somewhere you want to go cheap. Bigger and puffier is not necessarily better. I ride with a Brooks saddle which is leather with no padding - and I can ride 100 miles on it without soreness. I LOVE my Brooks saddle. Best $125 I've ever spent. Others swear by Terry. There are many great brands out there - the only way to know is to try them. There's where the bigger stores can help. REI will take any saddle back after you've ridden on it.
Depending on the bike, be prepared to spend a good amount of money. For a hybrid (riding the local trails, riding to the grocery store, etc) you'll spend $400-$800. For a decent road or mountain bike - $1000+ My Cannondale Road was $1400 and it's an aluminum frame. Spend as much money as you can afford so you don't out grow the bike as you ride more (believe me, I've learned this from experience - I currently have 5 bikes in my stable).
As for your knees - a good LBS will put the bike on a trainer, have you hop on and fit the bike to you. If the bike is fit correctly, you should have no knee pain. If you experience it, adjust the seat up or down by just a little. The slightest adjustment can make the biggest difference.