I think the size of horse is more dependent on what you are comfortable with. I know some who is 5 feet tall and rides a 17+ hand draft tb cross ("American warmblood") stallion. She is doing extremely well showing PSG.
But, I don't think really tall or big people should ride ponies and I don't think that really short people should be riding 17 hand horses. But there is always exceptions.