(1) When temperatures are colder, the organisms incorporate more of the heavier isotope of oxygen into their skeletons.  Measuring the ratio of the heavier to the lighter oxygen isotope in the skeletons therefore indicates how warm the water was when the organism was alive.
(2) The ratio of the heavier to the lighter oxygen isotope also varies in the ambient seawater because of the variations in total ice volume.  The water molecules containing the heavier isotope are less likely evaporate and be incorporated into the ice sheet.  So as total ice volume increases, the ocean becomes increasingly enriched in the heavier isotope of oxygen. 
So when ocean temperatures are cold and ice volume is large, the skeletons should be enriched in the heavier isotope.  Presumably temperature and ice volume are tightly correlated, so an examination of the oxygen isotope record gives us a clear picture of when the ice ages occurred.