Have you ever tried CCleaner? It is a free program that cleans out old temp files, Internet browser history files, and registry errors. I have used CCleaner for years with XP, Vista and even the Windows 7 Beta with excellent results. CCleaner allows you to control which programs start automatically whenever the computer is rebooted, which helps to cut down on the "clutter".
I also like a lean/mean system, so I never upgrade an OS and opt instead to perform a clean OS installation. I reformat the HDD and perform clean OS installations on every new OEM desktop and notebook computer I buy to get rid of the junk and trial-ware that Dell, Acer, Compaq and many other OEM's install on new machines. I also disable unnecessary services, such as the Tablet PC service when the computer is a desktop or notebook.
Long story short, there are many things that you can do to have a clean, clutter free computer that do not require 3rd party software. Please keep in mind that using each item I have mentioned above does entail some risk though.