Delete firebase data older than 2 hours

Question

I would like to delete any data that is older that two hours. Currently, on the client side, I loop through all the data, and run a delete on any that is older. When I do this, the db.on('value') function is invoked every time something is deleted. Also, things will only be deleted when a client connects, and what might happen if two clients connect at once?

Where can I set up something that deletes old data? I have a time stamp inside each object created by a JavaScript Date.now().

Answer 1

I have a http triggered cloud function that deletes nodes, depending on when they were created and their expiration date.

When I add a node to the database, it needs two fields: timestamp to know when it was created, and duration to know when the offer must expire.

Then, I have this http triggered cloud function:

const functions = require('firebase-functions');
const admin = require('firebase-admin');
admin.initializeApp();

/**
 * @function HTTP trigger that, when triggered by a request, checks every message of the database to delete the expired ones.
 * @type {HttpsFunction}
 */
exports.removeOldMessages = functions.https.onRequest((req, res) => {
    const timeNow = Date.now();
    const messagesRef = admin.database().ref('/messages');
    messagesRef.once('value', (snapshot) => {
        snapshot.forEach((child) => {
            if ((Number(child.val()['timestamp']) + Number(child.val()['duration'])) <= timeNow) {
                child.ref.set(null);
            }
        });
    });
    return res.status(200).end();
});

You can create a cron job that every X minutes makes a request to the URL of that function: https://cron-job.org/en/

But I prefer to run my own script, that makes a request every 10 seconds:

watch -n10 curl -X GET https://(your-zone)-(your-project-id).cloudfunctions.net/removeOldMessages

Answer 2

In the latest version of Firebase API, ref() is changed to ref

var ref = new Firebase('https://yours.firebaseio.com/path/to/items/');
var now = Date.now();
var cutoff = now - 2 * 60 * 60 * 1000;
var old = ref.orderByChild('timestamp').endAt(cutoff).limitToLast(1);
var listener = old.on('child_added', function(snapshot) {
    snapshot.ref.remove();
});

Answer 3

You could look into Scheduling Firebase Functions with Cron Jobs. That link shows you how to schedule a Firebase Cloud Function to run at a fixed rate. In the scheduled Firebase Function you could use the other answers in this thread to query for old data and remove it.

Answer 4

Firebase does not support queries with a dynamic parameter, such as "two hours ago". It can however execute a query for a specific value, such as "after August 14 2015, 7:27:32 AM".

That means that you can run a snippet of code periodically to clean up items that are older than 2 hours at that time:

var ref = firebase.database().ref('/path/to/items/');
var now = Date.now();
var cutoff = now - 2 * 60 * 60 * 1000;
var old = ref.orderByChild('timestamp').endAt(cutoff).limitToLast(1);
var listener = old.on('child_added', function(snapshot) {
    snapshot.ref.remove();
});

As you'll note I use child_added instead of value, and I limitToLast(1). As I delete each child, Firebase will fire a child_added for the new "last" item until there are no more items after the cutoff point.

Update: if you want to run this code in Cloud Functions for Firebase:

exports.deleteOldItems = functions.database.ref('/path/to/items/{pushId}')
.onWrite((change, context) => {
  var ref = change.after.ref.parent; // reference to the items
  var now = Date.now();
  var cutoff = now - 2 * 60 * 60 * 1000;
  var oldItemsQuery = ref.orderByChild('timestamp').endAt(cutoff);
  return oldItemsQuery.once('value', function(snapshot) {
    // create a map with all children that need to be removed
    var updates = {};
    snapshot.forEach(function(child) {
      updates[child.key] = null
    });
    // execute all updates in one go and return the result to end the function
    return ref.update(updates);
  });
});

This function triggers whenever data is written under /path/to/items, so child nodes will only be deleted when data is being modified.

This code is now also available in the functions-samples repo.