Cache using PHP cURL

Question

I'm using PHP cURL to fetch information from another website and insert it into my page. I was wondering if it was possible to have the fetched information cached on my server? For example, when a visitor requests a page, the information is fetched and cached on my server for 24 hours. The page is then entirely served locally for 24 hours. When the 24 hours expire, the information is again fetched and cached when another visitor requests it, in the same way.

The code I am currently using to fetch the information is as follows:

$url = $fullURL;
$ch = curl_init();    
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); 
curl_setopt($ch, CURLOPT_URL, $url); 
$result = curl_exec($ch); 
curl_close($ch); 
echo $result;

Is this possible? Thanks.

Answer 1

Honza's suggestion to use Nette cache worked great for me, and here's the code I wrote to use it. My function returns the HTTP result if it worked, false if not. You'll have to change some path strings.

use Nette\Caching\Cache;
use Nette\Caching\Storages\FileStorage;
Require("/Nette/loader.php");

function cached_httpGet($url) {
    $storage = new FileStorage("/nette-cache");
    $cache = new Cache($storage);
    $result = $cache->load($url);
    if ($result) {
        echo "Cached: $url";
    }
    else {
        echo "Fetching: $url";
        $ch = curl_init();    
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); 
        curl_setopt($ch, CURLOPT_URL, $url); 
        $result = curl_exec($ch); 
        if (curl_errno($ch)) {
            echo "ERROR " .  curl_error($ch) . " loading: $url";
            return false;
        } else
            $cache->save($url, $result, array(Cache::EXPIRE => '1 day'));
        curl_close($ch); 
    }
    return $result;
}

Answer 2

If you've got nothing against file system access, you could just store it in a file. Then maybe use a script on the server that checks the file's timestamp against the current time and deletes it if it's too old.

If you don't have access to all aspects of the server you could just use the above idea and store a timestamp with the info. Every time the page is requested check against the timestamp.

And if you're having problems with the fs bottlenecking, you could use a MySQL database stored entirely in RAM.

Answer 3

Use Nette Cache. All you need solution, simple to use and of course - thread-safe.

Answer 4

You can cache it using memcache ( a session ) you can cache it using files on your server and you can cache it using a database, like mySQL.

file_put_contents("cache/cachedata.txt",$data);

You will need to set the permissions of the folder you want to write the files to, otherwise you might get some errors.

Then if you want to read from the cache:

if( file_exists("cache/cachedata.txt") )
{ $data = file_get_contents("cache/cachedate.txt"); }
else
{ // curl here, we have no cache
 }

Answer 5

You need to write or download a php caching library (like extensible php caching library or such) and adjust your current code to first take a look at cache.

Let's say your cache library has 2 functions called:

save_cache($result, $cache_key, $timestamp)

and

get_cache($cache_key, $timestamp)

With save_cache() you will save the $result into the cache and with get_cache() you will retrieve the data.

$cache_key would be md5($fullURL), a unique identifier for the caching library to know what you want to retrieve.

$timestamp is the amount of minutes/hours you want the cache to be valid, depending on what your caching library accepts.

Now on your code you can have a logic like:

$cache_key = md5($fullURL);
$timestamp = 24 // assuming your caching library accept hours as timestamp

$result = get_cache($cache_key, $timestamp); 
if(!result){
   echo "This url is NOT cached, let's get it and cache it";
  // do the curl and get $result
  // save the cache:
  save_cache($result, $cache_key, $timestamp);
}
else {
  echo "This url is cached";
}
echo $result;

Answer 6

The best way to avoid caching is applying the time or any other random element to the url, like this:
$url .= '?ts=' . time();

so for example instead of having
http://example.com/content.php
you would have
http://example.com/content.php?ts=1212434353