I agree with Donna...the museum gift shops are wonder. MOMA has one at Rockefeller Plaza. It is terrific. There are often craft shows taking place somewhere.

As for discount clothes...I can't think of the name of it...but right across the street from Macy's.

And for fun shopping...go to Battery Park (you'll want to catch the ferry to Ellis Island from there. Ellis Island also has a great gift shop....) There are artist and artist wanabees and just plain people making a living selling all kinds of things. I bought a painting from a Haitian guy there once. He assured me one day he'll be famous and I will own one of his paintings!