Ditto on Donna's post. And tell him you're still sick and don't want to give him or his nieces anything contagious - especially in his weakened state of health!

And then maybe gradually ease out of your association with him if he makes you uncomfortable.

He might be harmless, but you never know. Go with your instincts!