If I am replying to a PM someone sent me, I reply underneath - to me it is the same as quoting someone's post in a thread. If the PMs have gone back and forth in conversation, I only keep the very latest reply quoted and delete the rest.

In e-mails, I'll reply above the person's message to me. Again, if it has been going back and forth in conversation, I'll only keep the very latest e-mail response and delete everything below - my old messages included.