When Jake used to eat our drywall (he chewed up all the corners on our walls!) somebody said it was probably from a deficiency of calcium or another mineral found in the drywall. We gave him vitamins and eventually either the vitamins helped or he just grew out of it.

Maybe you could find out what sort of minerals are in the soap and try supplementing to see if he'll stop. Are you feeding high-quality food?