I agree what others say about math.
learning calculus is actually different than learning 'math' to my mind, as long as you have a grasp of the algebra involved. The theory is just fascinating to me...and if you understand the concepts (the fun part), and have the algebra skills...it's nothing but fun and games, and is actually very easy.
As I recall, from back in the day, all that was required for a comp sci major at Rutgers, where I went, was 2 semesters of calculus and 2 semesters of discrete structures...which is, essentially, a logic course. Look into the programs you are considering. IT is actually suprisingly easy to get through 2 semesters of calculus; I have seen lots of people short on math skills do it.
As far as taking the plunge into IT...working in the industry, and having family members who work/have worked in several aspects of it...ask yourself if you are willing to put in the long hours and the off-hours work necessary, ask yourself if think being on-call and tied to a pager is something you can work with. As yourself, depending on the field you get into, if 24 hour work days are ok with you. I'm on the support side, as a sys admin, so I get to fix broken things and work a lot of off-hours, late into the night, overnight, and on weekends when we have scheduled maintenance. Developers also have to work long hours to meet deadlines, then have to implement on weekends. On an app team? You also get to live with the off-hours maintenance windows.
IT by itself can be lots of fun and fascinating, but the reality of working in it...can be a very different story. Most people I work with, when questioned, say although they enjoy aspects of the work, they would choose another path if they could go back and do it again.
I don't mean to discourage you... just talk to as many people on the IT side and get their perspectives, and be cognizant of what you are getting into.
Good luck with whatever you decide.