How long to age a wine depends on the grape, climate, wine making method, storage conditions, and your preferences. Wine changes with age due to internal chemical reactions, including some oxidation. Tannin is a natural anti-oxidant/preservative that comes from the skins of red grapes and the oak aging barrels. How much tannin is in the wine depends on how long the skins are kept in the juice during fermentation and how long the wine is aged in wood. Highly tannic red wines will age longer than lighter reds or whites in general. Tannin is very astringent, i.e. the component that makes your mouth pucker (and some describe incorrectly as bitter).
As a wine ages the tannin reacts with other components and the astringency lessens over time. At the same time other reactions change the flavor and body of the wine, such as a conversion of the fresh fruit taste into a more mature and complex flavor. At some point during the aging the the wine may reach a balance that meets your taste buds and will then decline. If the tannin is too high it will mask the flavor peak, and by the time the tannin softens enough to drink the fruit is gone.
Generally speaking big red wines will benefit from aging if they are made for that purpose. Some red wines such a Beaujolais are designed to be drunk young and will peak early. How long to age your wine is difficult to predict, although true experts can get a decent estimate based on the depth and nature of the fruit and the amount of tannin. Even they, however, often change their estimates as the wine ages.
Most wine lovers buy several bottles or cases of a wine and taste it at various ages, looking for the balance that most pleases them, and then finishing the rest. I recently tasted some 1970 classified growth Bordeaux reds from my cellar that were past their peak but still quite pleasing. In the same tasting these Bordeaux were clearly outdone by a simple regional California Cabernet, also from 1970. I would have expected the opposite.
Also note that during aging, the tannin reactions sometimes create sediment that settles to the side or bottom of the bottle, and the color turns more of a brownish brick color. These are signs of age. The sediment is harmless but not pleasing on the palate, so such a wine should be decanted to leave the sediment behind in the bottle.
Generally speaking white wines have very little tannin and do not benefit from aging, unless they have spent some time in oak barrels such as some Chardonnays.