I've read that you can start losing strength after about 5 days. But that's with no work at all for the muscle. Remember that if you work your chest, you're also working the front shoulder and triceps, if you work the shoulders, then you're working the triceps, if you work the back, you also work the biceps and rear shoulders (unless you are doing isolation work for the larger muscle group, like flyes for the chest rather than presses), so just because you don't do a targeted shoulder workout (for example) within that time, doesn't mean you're not working the shoulders. That's a fact.
Now, an opinion: I think as well that even though they are opposing muscle groups and not synergists (helpers), you are working back (for example) somewhat just by doing an upper body workout, especially a chest workout. That's why split routines that may have you working each muscle part directly only once a week (like doing the Gym styles each once a week, which is how I used it) work. (And the legs may get only one direct resistance workout, but if you do workouts like any of the IMAXes or Low Max, you will work the legs as well, though in a bit different way.)
I've also read in several places that if it takes you x amount of time (for example) to lose muscle, you can gain it back in less time because of the "muscle memory" you've developed.
ETA: Like Kim, I find that I have better results (strength and size gains) if I lift heavy and leave more time between workouts (2-4 days between same body part rather than just one day). In fact, when I was doing a GS rotation, I did one upper body workout on Monday (or "day 1") lower body on Wednesday (or "day 3"), and the other upper body on Friday (or "day 5") to put more time between upper body workouts and allow for recovery.