If it's like peach, cherry, plum (stone-fruit) leaves, they are only dangerous in the wilted stage...both fresh/green and totally dry are harmless...but wilted can be very dangerous...I don't think ficus are related to these others, so can't tell you whether this might be true for them too, though.

This link was posted in a previous thread and goes into toxicities specific to cats...I don't remember who put me on to it, or I would give them credit for bringing it to our attention.

http://mainecoonrescue.com/poison.html