Maybe he feels like you like him seriously and he isn't ready for that...just be more laid back about him and he will probably come around.
Good point, Caseysmom!! The more I know men, the more I like my cats!!