Other helpful info from a cavie site:
Orphaned cavies
If babies are rejected, or the mother dies during or shortly after labor, you will need to make arrangements to raise the babies. If you are fortunate enough to have another sow that has just given birth within the last one or two days, you can opt to try and raise the guinea pigs with her as a foster mother. To do this, remove all but one of the mother's litter, and rub your orphaned litter against these babies so that they will pick up their scent. Then introduce the orphans to their foster mother; she will sniff them, and then accept or reject them. If they are rejected, you'll have to hand-raise them.
If you have to hand-raise the babies, they will need a milk substitute formula every two hours during the daytime. Some formulas and feeding tips can be found here. If you feed them late at night before going to bed, and keep them very warm, they should be fine until the next morning. After each feeding, be sure to wash their mouths to remove any spilled milk, as guinea pigs are miserable if their coat gets sticky. You will need to feed them this way for the first four days.

You will also need to wipe the anus and genital organs of each cavy with a clean, damp cloth to induce urination and the passing of feces. If the father is present with the babies, he may actually perform this duty for you by licking them, but you should watch closely to make sure he is doing so.

After the fourth day, they will need to learn how to eat solid foods. Again, if the father is present, they may learn from him through observation, and may in fact have already picked it up. If not, any young cavy placed with the babies can teach them this activity; the babies will learn from imitation how to pick up pellets and fresh greens and put them into their mouths.

And as lut said...make sure to have hay available at all times.
Good Luck!