Yes, definetly get rid of the old food. Sometimes when you put a new fish in a tank it'll take a few days before they want to eat. If he still doesn't eat it then you should try a different kind of food. (I have had some picky bettas myself )
Have any pics of your betta???