Maybe because he feels you are not serious!!!??? I don't know, I am not sure if I am missing something. But what is the problem, if you are absolutely serious about ending that relationship and he says, "Are you sure that's what you want?" Yeah, that's what I want, and that's the end of the discussion. He gets his stuff, you get your stuff, DONE!
I just get the impression that you are not quite sure that you want to break it off either and you maybe just looking to trigger a reaction from him. If that's what it is, my advise would be, DON'T DO IT!!!
I think you will need to do serious soul searching as to what YOU want and if this is the guy that you want to be with. No relationship is easy, most of them require a lot of work on both parts, nothing is ever perfect. I am not, you are not, he is not!!
If you think he is worth it, stick it out, work with him, if he is open to it. If he is not, kick him to the curb. But this half-hearted back and forth is very unproductive and will take a toll on your well-being.