I am glad you stood up for yourself, but would never advocate hitting someone in that situation. You are very, very, very fortunate the situation did not get worse, or escalate into his friends jumping in, or charges being brought.

You needn't take the initial bait when someone says something rude, as you now know, it doesn't help.