Generally, I think the books are better than the movies--there's more detail in them and you get to know the characters and their motivations. I like the movies, though--I really enjoy seeing the books brought to life. Overall the whole series is for kids, but as an adult I read the books anyhow because I think Rowling does a great job of portraying the fictional world that she developed and because I care about the characters she has created. That said, I also find that I get frustrated by the simplistic, predictable and really annoying plot lines and writing devices that Rowling uses, especially in many of the more recent books. I just care enough about the fictional world in the books at this point that I keep reading.
My suggestion--get the first book out of the library and see if you enjoy reading it; if not, no harm. Overall, I think it's worth it--it's not great literature, but it's a lot of fun.