Here is a nice record of oxygen isotopes obtained for the last 800,000 years:

Examining the above picture, you can see the complete record of of all the glaciations for the last 800 years. In fact, there are nine glacial intervals, many more than the four detected from the land-based record. Actually, there are many more than that, going back some two and a half million years. Prior to that, there was no permanent ice in the norther hemisphere.

You might also notice that the pattern of glacials and interglacials is quite rythmic. Careful analysis of the record reveals that the curve is really the sum of three curves, pulsing at 100, 40 and 20 thousand year intervals. What drives these curves?

It turns out that the pulses of glaciation correspond to variations in the Earth's orbit around the sun, matching the amount of heat insolation reaching northern latitudes in the summer. The Earth's orbit is not a perfect circle, nor is the tilt with respect to the sun constant.

Like a top, the axis of rotation (the spin axis) precesses around in a circle. It takes about 26,000 years to complete one rotation. This results in the position of the equinoxes to change with a period of about 23,000 years. Also, the tilt of the spin axis (obliquity) with respect to the ecliptic (the plane in which the earth rotates about the sun) changes. It varies between about 24.5 and 21.5 degrees. (Now it is some 23.5 degrees). This variation in tilt has a period of about 41,000 years. Finally, the degree of ovalness (eccentricity) of the Earth's orbit changes from more circular to more oval (elliptical). This variation has a period of about 100,000 and 400,000 years.