You are assuming he assumed you were racist. SO assumptions aside, he probably just didn't know how to initiate conversation, and perhaps he has been treated by others to the point where he feels most people are scared of him and in all reality was probably not directed towards saying you are racist in any way. I think you are reading too much into it, but it's too bad that you drove off and left and didn't help him based on an assumption=(