wget: Checking for file timestamp and overwriting

Question

Closed. This question is off-topic. It is not currently accepting answers.

---

Want to improve this question? Update the question so it's on-topic for Stack Overflow.

Closed 8 years ago.

Improve this question

How can I get wget to do this:

Download a file from a location say x, only if the local copy of the file has an older time stamp than that of the file's time stamp on x. That means, it should download a file from a given location, only if there is a newer version of the file.

In case there is a newer version of the file, wget should overwrite the file.

Is it possible to do this?

Answer 1

Sounds like you're looking for the TimeStamping functionality of wget: http://www.gnu.org/software/wget/manual/wget.html#Time_002dStamping

Say you would like to download a file so that it keeps its date of modification.

wget -S http://www.gnu.ai.mit.edu/ 

A simple ls -l shows that the time stamp on the local file equals the state of the Last-Modified header, as returned by the server. As you can see, the time-stamping info is preserved locally, even without ‘-N’ (at least for http).

Several days later, you would like Wget to check if the remote file has changed, and download it if it has.

wget -N http://www.gnu.ai.mit.edu/