Since humour and body language don't always translate through in text, I'd ask him to be sure.

But personallyl if that was me and my hubby, I'd think it meant that he knew I'd be asking him to spend some time together when we could and he was being jovial about it.