Harrasment or not, it is completely inappropriate for anyone to make the sort of comments you are saying he is making.