OK Wendy, I tested this on my husband this morning. Now, he's already told me that I look awesome, so I know he's noticed the improvement in my body. So this morning I asked him if it was obvious that I work out. He tends to be a little more careful in answering questions, so he asked me what I meant. I asked if you could tell by looking at me that I work out. He looks carefully at my arms for a little bit and says, "it's hard to say..." Knowing how hard it is to get him to answer a question, I put it another way..."If you didn't KNOW that I work out and didn't KNOW what I looked like before I started working out, would you be able to tell that I work out now?" And he said, "no".
So there you go! It wouldn't be obvious to a stranger that you work out unless you look like Linda Hamilton in T2! Don't judge your man too harshly. He honestly didn't understand the question the way you meant it, he was being literal.