Added 1/20/2012: If you are using historical postage rates in classroom instruction, you might also appreciate this handy tool to adjust prices for inflation.
Added 9/5/2015: The current postage rates are listed on the USPS web site. Beginning Sunday [4/10/2016], the price of a first-class stamp drops two cents, to 47 cents.
Regulators approve hike in cost of first-class stamps.
Postal regulators on Tuesday [12/24/2013] approved a price hike of 3 cents for a first-class stamp, bringing the charge to 49 cents a letter. The higher rate will last no more than two years, allowing the Postal Service to recoup $2.8 billion in losses.
Domestic Mail Rate History: This document shows that ZIP 4 does hold down the cost of postage... In other words, it makes unsolicited junk mail easier to send, but it doesn't reduce the cost of a first-class letter at all.
Along the way I also found a complete Glossary of postal terms [PDF file] and a Postage Rate Calculator.
Timeline of Significant Dates in Post Office History.