It does sound like stress. It hasn't been that long that he was rescued by you. He's probably still a little insecure after all he's been through. Hopefully, with time, he'll see these changes do not mean he's going somewhere else, or being punished!

We have some cats that have nothing to do with our bed. They've never been punished for being up there, they just don't like to be. Then there's those we can't keep off of it! Maybe he just doesn't like being on the bed.