Kim, I have very much your same problem, except I have a butt too to worry about! When I get things that give my belly and behind room, then I have the saggy crotch sydrome! HATE THAT!!!! I would go the smaller size, if it is not too tight and uncomfortable and just wear longer shirts and sweaters to that conceal your belly a bit. I tend to buy smaller pants to feel comfortable and not sag or risk falling down, and bigger shirts to fit more loosely!

Hope this helps!
Kim