I bet this happens because when you go back, there's some new posts at the top, so e.g. "page 2" is now a bit shifted compared to where it was before.

-- Frost