Yep, icecubes are fine.

When we thought Simba had a heat stroke, the Vet's office told us to put and icecube in his mouth and hold it shut for a minute or so.