Time Travel Impossible?

Some scientists say that you cannot travel back in time.
It is argued that because the fabric of space-time is 4 dimensional instead of 3 dimensional, mathematically you cannot go backwards.

Is HG Wells rolling in his grave?
Discuss