Depends on needs. I have a lot of high end audio stuff (and portable audio stuff)... you can pick up a Sansa Fuze for cheap. Great battery life, decent SQ, very very small, and easy to transfer music to. If you want a touch screen interface, 4G iPod Touches can be had cheap (I hate iTunes, but iPod Touches are fantastic to use). Samsung and Sony both make decent portable music players, as well.
So it all depends...if you want the best technology, SONY, Apple, and Samsung all have a nice selection running over $100. For cheap, small, and good battery life...think Sansa Fuze (with a memory card which can be picked up cheaply).
There gets to a point where those players are the same price as a small tablet. So how much is portability worth to you?