Watch the movie named The Time Traveller's Wife. It is absolutely superb and based on sort of that idea, but not quite.