I think of a cap as something more casual than a hat.

Baseball caps are something I think should be worn with the eyeshade shading the eyes instead of the nap of the neck...but that is a whole different discussion.