Its not that they don't remember, it is that they have a 3-5 second association time. If you come in yelling at them for something they did hours ago, it is just confusing and scary for them.

(Feeling really sick right now or I'd elaborate more)