Hi Tia, I'm sure I broke my pinkie toe once. I never went to the doctor, but I'm pretty sure. It was excruiciating pain with immediate swelling and bruising. Nothing like when I just hit it on something. I actually did keep exercising, but it took several MONTHS for it to heal. It shouldn't take that long to heal, but since I didn't let it heal properly it took forever. It hurt to put my exercise shoes on, but once I got them on the toe did not hurt anymore because it was held in a stable position. I could do step, hi/lo, etc without any pain whatsoever. It actually hurt more without shoes because I didn't buddy tape to the toe next it. You really should do that as it helps with healing. My little toe even has a funny little shape now also. I really didn't have much of a choice on the doctor thing as I was in between jobs at the time and had no insurance. I really think they would have only buddy taped the toes together, however.
So, to make a long story short. You can exercise if you want the toe to take months to heal or you can lay off the cardio for a while and let it heal correctly. To be honest, I probably would do the same thing again and keep exercising. Not the right thing to do, but that's me.