The sooner you go, the better, both in case you broke something (so it can set properly and begin healing) and if you didn't (so you can stop worrying)!