Its fine.
Half of my tanks have gravel and half of them don't. The ones without gravel seem to keep a bit cleaner longer.