I always say thank you - and it drives me absolutely bonkers when I hold the door for someone else and they don't!
When this happens I make a point to say "You're welcome" in a very polite voice. I don't mind pointing out that they were rude.