-Oats. They're super expensive in the store, so buy them in bulk. I get a 50 pound bag of quick cooking oats for $50 at allbullfoods.com after shipping. You can also try Honeywell which have standard $5 shipping anywhere in the continental US.
dry pasta $1 or less per pound
rice. Buy the biggest bag if you have storage space. White and brown are the cheapest. Go for white since it's less filling and goes with everything.
beans. Lentils are the most notorious frugal food, even becoming a satirical form of currency at /r/frugaljerk. Just play around with different beans.
canned tuna. Don't pay more than $1 per 5oz container. Also, don't eat more than a couple cans a week because of mercury.
eggs use to be reliable cheap protein sources, but with the avian flu destroying live stock in the west, the price of eggs here have more than doubled. I still eat 3 eggs a day, but my cheap breakfast isn't so cheap anymore. $3-4 per dozen. Buy in large quantities. I'll usually buy 90 eggs at a time. Costco has a good deal, but Target also sells a 30 pack for a bit cheaper than the standard dozen.
boneless chicken breasts. Don't waste your time with whole birds or other cuts since most of the weight will be from bone and fat. I use to get it for $1.99/lb, but now it's $2.49 lb.
cheap bread. White and whole wheat bread. I'll buy like 10 loafs at a time and go through it in like 2 weeks. $1- $1.40 per loaf
Italian sausage. $2.49-2.99 a lb. Lots of calories, protein and makes pasta dishes awesome.
frozen veggies, broccoli is king.
extra virgin olive oil. Make sure you're buying real stuff. There's some fraudulent items on the shelves claiming to be extra virgin when in reality, they're a mix with other oils. 150 calories per teaspoon. (If it's made in California, it's probably real. Italian products tend to be the real culprits)
-peanut butter. Buy generic. $4.39 for a 40 oz container. Pricy, but now is a cheaper option for protein than eggs.
Dirty bulking: don't rely on these since they're not very healthy.
canned chili. At nearly 600 calories, 27 g of protein and all for $1.25, it's a good buy. Mix in a mountain of rice and you got yourself a feast.
hot dogs. I get these generic chicken/pork franks. 32 for $4 from Stop and Shop. I call them "Cancer dogs." They taste kinda crappy, but 200 calories and 6 g of protein per frank. Easy 1 minute meal if you microwave them.
Overall, stick to basics. If you're a picky eater or have a stick up your ass about only eating the finest of foods, then you won't be able to bulk on a budget.
Condiments are cheap and your taste buds will adapt. That's coming from someone who use to eat out every meal.
I've been eating like this for well over a year. I spend about $100 a month on food, not counting the bag of oats that takes me 2-3 months to finish and a container of ON gold standard Whey I use as a post workout.
Find a balance between cheap and healthy. A multivitamin won't hurt.