While I really hesitated answering here (I am in no way qualified to give legal advice), but I thought I would throw something out there that may help you. With non-compete clauses, the trouble comes when YOU actively go and solicit former clients. If your clients are contacting you, without YOUR PURPOSEFUL solicitation, I don't think there are any legalities that you are breaking. Plus, since you never signed any non-compete contract, you also have a bit of an opening.
AGAIN...these are just ideas..NOT in any way to be substituted for any REAL legal advice.
Ethically...that is a whole different issue. You have to decide if you yourself would be comfortable in taking on these clients.
Also, another thing to keep in mind. It is ALWAYS the clients decision as to who they want to work with. Always. If they have a personal relationship with these clients and they feel comfortable with you, that is another area to consider as well.