I think you should talk to him seriously about this. Don't allow any joking or kidding around, but no anger either. I think if you don't say something, you'll never be content. If do you speak to him about it and give him fair warning that it should stop and it doesn't you can have a clear conscious about reporting it to your boss. However, if you even remotely think that he could turn this against you, in any way, just go straight to your boss. Don't mess around and get your butt in a sling, just because he's stupid.