Pointing at him and telling him he's naughty only prooved to him one thing: that going potty is something to be ashamed of and something he shouldn't do, because if he does do it, you will yell at him. Or at least thats how he sees it.

Secondly, a lot of dogs who opt to potty as soon as they come inside do it because they think going potty automatically means a trip back inside. So they hold it just to remain outside longer. Then as soon as they are brought back inside they go potty because there's no reason to hold it any longer -- they're inside already anyway. If he fits this category, the trick is to keep him outside for a while after he potties so he doesn't think potty=end of time outside.