You could always try, "My surgery is a private matter now. It's between me and my doctor. You don't have to concern yourself with it."

(The last line meaning NONEOFYOUREFFINBUSINESS! )