If-Modified-Since not considered while using httr

Question

My requirement/objective:

I'm retrieving Facebook posts for the past 3 months from a public page, and I would like to select only those posts which have been modified in the recent past (in the last 2 days).

I'm using httr R package, and using the GET function to accomplish the above.

I tried the following :

1. url.data <- GET(url, config(token=token), config(add_headers("If-Modified-Since" = "2016-09-08 11:45")))

2. url.data <- GET(url, config(token=token, add_headers("If-Modified-Since" = "2016-09-08 11:45"))

In the above 'url' has the 'since', 'until' fields specifying the time period (3 months) for which the posts need be retrieved. 'token' is the OAuth token for authentication.

In both of the above methods, all the posts are retrieved, rather than just the posts that are modified in the past 2 days.

Is there another way of passing the If-Modified-Since to GET which will result in only the desired posts to be retrieved?

Note: I have already checked these posts related to If-Modified-Since, and they do not answer my question :

1. "If-Modified-Since" header
2. "If-Modified-Since" Header?
3. If modified since - HTTP protocol
4. If-Modified-Since Date Format

and a few more questions which are not exactly related to my scenario.

Any help would be appreciated.

Answer 1

I think the core of your issue here is a confusion between the purpose of the "If-Modified-Since" header, and the filtering of results by Facebook.

"If-Modified-Since" asks the server "Did the contents to be returned when I call this URL/make this query change since this date/time?"

The "since" and "until" parameters sent to Facebook filter the request to only show content which was posted between those two dates.

Facebook will not merge the two to return posts which satisfy both questions - which fall between the two date/times set in the parameters and have been modified since the date/time set with "If-Modified-Since".

This is because Facebook performs the search/filter, looks at the entirety of the response and answers that question "Did the contents to be returned when I call this URL/make this query change since this date/time?" based on the whole response. If anything has changed in the entire response, then it will send the entire response to you.

If you want to identify individual posts which have changed, you will have to loop through the array and check each element individually.