Guinea pigs are among the easiest small animals to care for and rate high on the cuddliness scale. As pets, they are ideal for (responsible, gentle) children because they tend to be sweet-tempered, pettable, and relatively easy to catch if they escape from your child's hands -- mice, hamsters, and gerbils, by contrast, are able to hide for weeks or more if they escape.
Guinea pigs are larger than most rodents (about the size and shape of a large tennis shoe when grown), which makes them easy to find and to handle. If you are looking for a highly intelligent and sociable pet, you may be looking for a rat (seriously). If, however, you want a sweet, lovable furball who will sit on your lap to be petted for hours (well, minutes, anyway), a guinea pig may be the pet for you.
There are basically four places to get guinea pigs -- from a breeder, from a pet store, from an ordinary guinea-pig owner who has had a litter of small guinea pigs, or from an animal shelter. There are advantages and disadvantages to each, but a detailed discussion is not in the scope of this page. To be brief -- reputable breeders often sell high-quality pets but they cost slightly more. Try to get recommendations from other guinea pig owners, or by asking on the net, before choosing a breeder.
Pet stores are somewhat cheaper, but depending on the pet store, the guinea pigs are more likely to have caught a disease and may have been improperly cared for. Most people do not recommend purchasing animals from pet stores.
If you can find an acquaintance giving away a litter, or selling them at a reasonable price, by all means go ahead -- this way you can be fairly sure of getting a healthy, well-treated baby, while probably not paying too much.
Do check out your local animal shelter to see if they have guinea pigs -- you may be able to find a lovable pet and save a guinea pig at the same time.
Whichever route you choose to go, make sure you choose a healthy- looking, energetic guinea pig with no signs of disease. If you go to pick him (or her...) up and he shows very little interest in the procedure, there's something wrong-- a healthy guinea pig will usually either run away from your hand or investigate it. Spend a few minutes with him before taking him home, to make sure the two of you get along and aren't allergic to one another.