Yeah, this. Measure the bearing in millimeters and get a new one on ebay or through amazon. A 6203 will be 17 x 40 x 12mm. Existing bearing may even have numbers on it somewhere.Officially the bearing is only sold complete with pulley, which might cost about $40 if still available, but it is almost certainly a generic 6200 series ball bearing that only costs a few dollars. Putting a new bearing in an old pulley would involve a few minutes with a press.
Put the idler on two blocks of wood, line up a 20-something mm socket on the bearing part, and knock it out with a hammer. Put the new bearing in the freezer and it should slide in pretty easily.