That's creepy! I wouldn't like that at all. I don't want my boss to know anything about my time off work. Maybe they were just in the neighborhood. But I don't think he's trying to be scary - if he was he probably wouldn't have told you!! What a weirdo.