Leslie:
First, YES (a resounding yes), you should not talk to people who try to bring you down. If they can't be supportive and provide encouragement, then do not subject yourself to their negative banter.
Second, I just want to say that it is the hard things in life that build the most character in us and bring out who we really are. Challenging tasks, especially ones that scare us, break us out of the safe little mold we've built up around ourselves and stretch our souls toward success and happiness. So, when something scares you, but makes you excited at the same time, you should do it. That which doesn't kill us makes us stronger. And that is soooo true. Good luck, girl! You can do it! Look at all you've faced in life so far. Getting your GED is nothing compared to the hardships you've faced in the past. You've raised a kid, for crying out loud! If you can do that, then getting your GED will be a snap!
One last thing: Just remember, "baby steps". When you start feeling too scared, or overwhelmed, take a step back and proceed in little baby steps. One class at a time. One homework assignment at a time. One step at a time. Sometimes, trying to see the whole big picture is overwhelming and scary, but if you just focus on one piece at a time of the big picture, it's seems much more attainable. Make sure you give yourself time to rest during your new journey. I am taking classes right now, too, on my way to earning my Bachelor's. I make sure to take a session off every-so-often so I don't get so overwhelmed. I'm able to do this because I'm in what's called an Advanced Learning Program, where each class is only 5 weeks long. So, when I take a session off, it gives me about 6-7 weeks off. It allows me to recharge my battery.