You can get candy through a few different means:
Everytime you catch a pokemon, you get 3 candy of their type. This is why i recommend catching every pokemon you see - for the candy and dust.
You can "transfer" a pokemon to the professor and he will give you 1 candy of the type of pokemon that you transfer - but you lost the pokemon permanently obviously.
When you hatch an egg, you get quite a lot of candy of the type of pokemon that hatched. For intance, i got 27 candy from 1 egg hatch. Note that the longer the distance required to hatch the egg, the more candy it rewarded.
The main purpose of candy is to power up and evolve your pokemon.
*Evolving your pokemon only requires candy of the same type of pokemon - typically it costs around 50 candy (each pokemon is different).
There are a few ways to get stardust - the most common will be to catch Pokemon.