One immediate way to cut your organic food bill is to skip the usual trip to your local supermarket and find some other sources that sell organic foods. Supermarkets require higher profit margins, plus they ship in foods from all over the place, so they have to mark up the price to cover these additional costs.
Checking out your local farmers' markets is a no-brainer. Not only are you likely to get better prices than at the grocery store, but you'll also be supporting a farm that's likely within a reasonable driving distance of your home. Co-ops and local health food stores often support local farmers, too.
You can also join a CSA (community supported agriculture) program, which delivers fresh vegetables from a local farm each week. This means you'll get fresh produce that's in season for a good price. The only disadvantage of a CSA is that you don't get to pick what you receive, so you'll need to get creative with your recipes based on your weekly delivery.
You should also take a little time to shop around; you'll find that your organic staples are available for less in certain places. The market is driven by supply and demand, so the store that has the most organic asparagus is likely to have the lowest prices. Packaged organic foods are in greater abundance these days, so be sure to clip coupons if you buy these items. A combination of all of these efforts is generally the best prescription for eating organic on a budget. And of course, if you really want to save, you can grow your own.