No, you just say, "Thanks, but I'm not looking to hang out one on one right now. Too much going on!"