To add to what others have said, another way to build (and save some money) is to gut a retail PC and use its case and power supply. I don't get excited by most of the aftermarket computer cases; most are too gaudy and "showy" for me. I like the "blend in" nature of most retail PC cases. Fortunately, you can swap aftermarket stuff into them pretty easily. You do have to watch out for a few things, like make sure the case is the right architecture (micro/mini ATX is common, but there is some BTX stuff out there, for which no aftermarket really exists), and every now and again, you'll run into a non-standard power supply. But it's usually straight-forward.
You can often find a "dead" computer for cheap/free on Craigslist or at your local electronic recycling facility. I have an eMachines desktop in our garage into which I've swapped in a much faster dual-core Athlon 64 than what it came with; the new processor was a $20 Buy-It-Now on eBay with free shipping. Runs fantastic at this point. Next step, if I upgrade it further, is swapping out the motherboard/processor to something like the budget Haswell build seen above. The only things I'll need to buy, though, are the motherboard, the processor, and the RAM. I've already got the HDD, the optical drive, the multi card reader, the power supply, the case...etc.
Starting fresh is great. But refurbing something else is sometimes fun, too.