Denormalizing data structures for private data acc

Question

I would like to create data that scales (to track private data of a user).

The Firebase documentation recommends to nest the child objects under the parent like this:

{
  "users": {
    "google:1234567890": {
      "displayName" : "Username A",
      "provider" : "google",
      "provider_id" : "1234567890",
      "todos": {
          "rec_id1": "Walk the dog",
          "rec_id2": "Buy milk",
          "rec_id3": "Win a gold medal in the Olympics",
          ...
      }
    }, ...
  }
}

(Where rec_id is a unique key generated by Firebase.push().)

But as mentioned in Denormalizing Your Data is Normal I think it would be better to structure it this way:

{
  "users" : {
    "google:1234567890" : {
      "displayName" : "Username A",
      "provider" : "google",
      "provider_id" : "1234567890"
    }, ...
  },
  "todos" : {
    "google:1234567890" : {
      "rec_id1" : {
        "todo" : "Walk the dog"
      },
      "rec_id2" : {
        "todo" : "Buy milk"
      },
      "rec_id3" : {
        "todo" : "Win a gold medal in the Olympics"
      }, ...
    }, ...
  }
}

And then to only allow the user to write/read it's own data apply the following security rules:

{
  "rules": {
    "users": {
      "$uid": {
        // grants write and read access to the owner of this user account whose uid must exactly match the key ($uid)
        ".write": "auth !== null && auth.uid === $uid",
        ".read": "auth !== null && auth.uid === $uid"
      }
    },
    "todos": {
      "$uid": {
        // grants write and read access to the owner of this user account whose uid must exactly match the key ($uid)
        ".write": "auth !== null && auth.uid === $uid",
        ".read": "auth !== null && auth.uid === $uid"
      }
    }
  }
}

As I'm new to this kind of databases I wonder if there are any downsides the way I'd like to structure it.

Would it be better to nest all todos directly under the user as recommended in the first example?

Answer 1

First, a couple resources if you haven't run into them yet:

• "Structuring Your Data" portion of the Getting Started Guide in the Firebase Docs?
• "Creating Data That Scales" section of that guide is the important part, as you've said.
• Also, check out this answer to "Clarification on Firebase denormalization blog post"

EDIT: You have obviously ran into these resources since it's linked in your question, but I would suggest re-reading over the Structuring Your Data guide a few more times.

• EDIT 2: See this answer @Frank van Puffelen posted for In firebase, is modeling many-many relationships using a separate endpoint a good idea?
  • And this
• I'd also suggest reading up on normalization [1][2] and denormalization [3][4], I think there might be some confusion on which of the two exactly you're referring to.

---

That brings us to your scenario...

The two ways you have your data set up actually accomplish almost the same thing!

You'll notice though, that the first example is actually listed as an anti-pattern in the "Structuring Your Data" guide.

• Doing it your second way would be useful if you want to load a users data, and then that user's todos at a different time.
• The way you have it set up is fine if you only have one user accessing each todo list.
  • For example, I do this in an app where I know that each user has one single location in a history list, and I only want to load the history of a user in certain scenarios.
  • /users/$userUid gives me the user data and /history/$userUid gives me the user's history.
  • It makes it easy to segment the loading.
• However, this structure doesn't give any benefits if the todo lists are shared between different users and have to be updated from multiple sources.
• If you want shared access, you're on the right track and just need to make use of keys as references.

---

The different approach is:

• Instead of explicitly setting todo objects under /todos/$uid, you can push a new todo object to /todos so that it gets a new unique ID (called key).
• Then, you add that key to the correct user object's todos child.
• This will allow you to load the user's data first and get only the indices (keys) of the todos to which the user belongs, and
• You can then load all those todos the user belongs to independently.

Doing it this way would:

• Prevent user objects from getting huge
• Allow multiple users to update a single todo, without having to update it's child parameters across multiple location.
• Have scalable data by splitting it into separate paths.

Here is the last data sample from the "Creating Data That Scales" section of the guide: (with some comments I added)

  // An index to track Mary's memberships
  {
    "users": {
      "mchen": {
        "name": "Mary Chen",
        // index Mary's groups in her profile
        "groups": {
           // the value here doesn't matter, just that the key exists
           // these keys are used to figure out which groups should 
           // be loaded (at whatever appropriate time) for Mary,
           // without having to load all the group's data initially (just the keys). 
           "alpha": true,
           "charlie": true
        }
      },
      ...
    },
    // Here is /groups. In here, there would be a group with the key
    // 'alpha' and another one with the key 'charlie'. Once Mary's
    // data is loaded on the client, you would then proceed to load 
    // the groups from this list, since you know what keys to look for.
    "groups": { ... }
  }

This achieves a flatter structure.

As the documentation says,

Yes. This is a necessary redundancy for two-way relationships. It allows us to quickly and efficiently fetch Mary's memberships, even when the list of users or groups scales into the millions, or when Security and Firebase Rules would prevent access to some of the records.

---

So the question is, how could your data look in order to allow a user to have multiple todo lists which can be shared with other users as well?

Here is an example:

{
  "users" : {
    "google:1234567890" : {
      "displayName" : "Username A",
      "provider" : "google",
      "provider_id" : "1234567890",
      "todoLists" : {
        "todoList1": true,
        "todoList2": true
      }
    },
    "google:0987654321" : {
      "displayName" : "Username B",
      "provider" : "google",
      "provider_id" : "0987654321",
      "todoLists" : {
        "todoList2": true
      }
    }
  },
  "todoLists" : {
    "todoList1" : {
      // 'members' user for rules
      "members" : {
        "google:1234567890" : true
      },
      "records" : {
        "rec_id1" : {
          "todo" : "Walk the dog",
          "createdAt" : "1426240376047"
        },
        "rec_id2" : {
          "todo" : "Buy milk",
          "createdAt" : "1426240376301"
        },
        "rec_id3" : {
          "todo" : "Win a gold medal in the Olympics",
          "createdAt" : "1426240376301"
        }
      }
    },
    "todoList2" : {
      "members" : {
        "google:1234567890" : true,
        "google:0987654321" : true
      },
      "records" : {
        "rec_id4" : {
          "todo" : "Get present",
          "createdAt" : "1426240388047"
        },
        "rec_id5" : {
          "todo" : "Run a mile",
          "createdAt" : "1426240399301"
        },
        "rec_id6" : {
          "todo" : "Pet a cat",
          "createdAt" : "1426240400301"
        }
      }
    }
  }
}

• In this case, user A would load both lists, but user B would only load the second list. If you set up your rules correctly, all would be well.
• In operation, you would first load the user's data, then load each todo list in a users todoLists, from /todoList

---

• But really, all of this is totally unnecessary if you're making an app where one user has one todo list with todo items that just have a single content.
• Side note, these "ids" should probably be a unique key, which can be accomplished using Firebase.push().
  • See this answer for using push to generate a new ID.

---

In conclusion, it all really comes down to how and when your app needs your data, how often data is updated and by whom, and also to minimizing unnecessary reads and watchers. Space is usually cheap, operations (and watchers) usually aren't.

Last but not least, rules and security are another extremely important consideration. The last part of the guide says:

"Thus, the index is faster and a good deal more efficient. Later, when we talk about securing data, this structure will also be very important. Since Security and Firebase Rules cannot do any sort of "contains" on a list of child nodes, we'll rely on using keys like this extensively."

---

It's early and I hope I'm not blabbering, but I ran into these same questions when I first went from only knowing MySql to using unstructured, so I hope that helps!