Hmmmm.... if I am following this as clearly as I think I am, this guy must still feel guilty of cheating on his wife, despite the fact that she's been gone for a while now. It could mean two things, A) that he's a sweet guy who takes commitment seriously and is battling himself with knowing that he has to move on in order to live the life he has. Or B) he's just incapable of having a strong emotional connection, which denying access to his private room would demonstrate.

Of course I do not know the entire story, nor should I know the entire story. The thing is, if you feel annoyed, then its an issue. I know you're trying to get a different perspective (I often do this myself to gauge whether I'm over reacting emotionally when there's really nothing to react to) so this is a good step.... see if there's a reason to be annoyed. If not, then still speak with him and let him know you need a little something to soothe your wounded womanhood