Just adding my two cents here...we did the same thing for my 8 year old son. It was actually cheaper to hand down my wife's 4 to him than to buy a new iPod Touch.
The phone will function exactly like the iPod touch (no data service or phone service like previously mentioned), but with one caution: be aware that you can still make an emergency call (911, 999, whatever applies to your geographic area) with the phone. Since a SIM card or active plan is not necessary to do this, just be sure to instruct your youngster not to hit the emergency call button because it WILL work!