How to deal with relational data in Redux?

Question

The app I'm creating has a lot of entities and relationships (database is relational). To get an idea, there're 25+ entities, with any type of relations between them (one-to-many, many-to-many).

The app is React + Redux based. For getting data from the Store, we're using Reselect library.

The problem I'm facing is when I try to get an entity with its relations from the Store.

In order to explain the problem better, I've created a simple demo app, that has similar architecture. I'll highlight the most important code base. In the end I'll include a snippet (fiddle) in order to play with it.

Demo app

Business logic

We have Books and Authors. One Book has one Author. One Author has many Books. As simple as possible.

const authors = [{
  id: 1,
  name: 'Jordan Enev',
  books: [1]
}];

const books = [{
  id: 1,
  name: 'Book 1',
  category: 'Programming',
  authorId: 1
}];

Redux Store

Store is organized in flat structure, compliant with Redux best practices - Normalizing State Shape.

Here is the initial state for both Books and Authors Stores:

const initialState = {
  // Keep entities, by id:
  // { 1: { name: '' } }
  byIds: {},
  // Keep entities ids
  allIds:[]
};

Components

The components are organized as Containers and Presentations.

<App /> component act as Container (gets all needed data):

const mapStateToProps = state => ({
  books: getBooksSelector(state),
  authors: getAuthorsSelector(state),
  healthAuthors: getHealthAuthorsSelector(state),
  healthAuthorsWithBooks: getHealthAuthorsWithBooksSelector(state)
});

const mapDispatchToProps = {
  addBooks, addAuthors
}

const App = connect(mapStateToProps, mapDispatchToProps)(View);

<View /> component is just for the demo. It pushes dummy data to the Store and renders all Presentation components as <Author />, <Book />.

Selectors

For the simple selectors, it looks straightforward:

/**
 * Get Books Store entity
 */
const getBooks = ({books}) => books;

/**
 * Get all Books
 */
const getBooksSelector = createSelector(getBooks,
    (books => books.allIds.map(id => books.byIds[id]) ));


/**
 * Get Authors Store entity
 */
const getAuthors = ({authors}) => authors;

/**
 * Get all Authors
 */
const getAuthorsSelector = createSelector(getAuthors,
    (authors => authors.allIds.map(id => authors.byIds[id]) ));

It gets messy, when you have a selector, that computes / queries relational data. The demo app includes the following examples:

1. Getting all Authors, which have at least one Book in specific category.
2. Getting the same Authors, but together with their Books.

Here are the nasty selectors:

/**
 * Get array of Authors ids,
 * which have books in 'Health' category
 */  
const getHealthAuthorsIdsSelector = createSelector([getAuthors, getBooks],
    (authors, books) => (
    authors.allIds.filter(id => {
      const author = authors.byIds[id];
      const filteredBooks = author.books.filter(id => (
        books.byIds[id].category === 'Health'
      ));

      return filteredBooks.length;
    })
)); 

/**
 * Get array of Authors,
 * which have books in 'Health' category
 */   
const getHealthAuthorsSelector = createSelector([getHealthAuthorsIdsSelector, getAuthors],
    (filteredIds, authors) => (
    filteredIds.map(id => authors.byIds[id])
)); 

/**
 * Get array of Authors, together with their Books,
 * which have books in 'Health' category
 */    
const getHealthAuthorsWithBooksSelector = createSelector([getHealthAuthorsIdsSelector, getAuthors, getBooks],
    (filteredIds, authors, books) => (
    filteredIds.map(id => ({
        ...authors.byIds[id],
      books: authors.byIds[id].books.map(id => books.byIds[id])
    }))
));

---

Summing up

1. As you can see, computing / querying relational data in selectors gets too complicated.
   1. Loading child relations (Author->Books).
   2. Filtering by child entities (getHealthAuthorsWithBooksSelector()).
2. There will be too many selector parameters, if an entity has a lot of child relations. Checkout getHealthAuthorsWithBooksSelector() and imagine if the Author has a lot of more relations.

So how do you deal with relations in Redux?

It looks like a common use case, but surprisingly there aren't any good practices round.

*I checked redux-orm library and it looks promising, but its API is still unstable and I'm not sure is it production ready.

const { Component } = React
const { combineReducers, createStore } = Redux
const { connect, Provider } = ReactRedux
const { createSelector } = Reselect

/**
 * Initial state for Books and Authors stores
 */
const initialState = {
  byIds: {},
  allIds:[]
}

/**
 * Book Action creator and Reducer
 */

const addBooks = payload => ({
  type: 'ADD_BOOKS',
  payload
})

const booksReducer = (state = initialState, action) => {
  switch (action.type) {
  case 'ADD_BOOKS':
    let byIds = {}
    let allIds = []

    action.payload.map(entity => {
      byIds[entity.id] = entity
      allIds.push(entity.id)
    })

    return { byIds, allIds }
  default:
    return state
  }
}

/**
 * Author Action creator and Reducer
 */

const addAuthors = payload => ({
  type: 'ADD_AUTHORS',
  payload
})

const authorsReducer = (state = initialState, action) => {
  switch (action.type) {
  case 'ADD_AUTHORS':
    let byIds = {}
    let allIds = []

    action.payload.map(entity => {
      byIds[entity.id] = entity
      allIds.push(entity.id)
    })

    return { byIds, allIds }
  default:
    return state
  }
}

/**
 * Presentational components
 */
const Book = ({ book }) => <div>{`Name: ${book.name}`}</div>
const Author = ({ author }) => <div>{`Name: ${author.name}`}</div>

/**
 * Container components
 */

class View extends Component {
  componentWillMount () {
    this.addBooks()
    this.addAuthors()
  }

  /**
   * Add dummy Books to the Store
   */
  addBooks () {
    const books = [{
      id: 1,
      name: 'Programming book',
      category: 'Programming',
      authorId: 1
    }, {
      id: 2,
      name: 'Healthy book',
      category: 'Health',
      authorId: 2
    }]

    this.props.addBooks(books)
  }

  /**
   * Add dummy Authors to the Store
   */
  addAuthors () {
    const authors = [{
      id: 1,
      name: 'Jordan Enev',
      books: [1]
    }, {
      id: 2,
      name: 'Nadezhda Serafimova',
      books: [2]
    }]

    this.props.addAuthors(authors)
  }

  renderBooks () {
    const { books } = this.props

    return books.map(book => <div key={book.id}>
      {`Name: ${book.name}`}
    </div>)
  }

  renderAuthors () {
    const { authors } = this.props

    return authors.map(author => <Author author={author} key={author.id} />)
  }

  renderHealthAuthors () {
    const { healthAuthors } = this.props

    return healthAuthors.map(author => <Author author={author} key={author.id} />)
  }

  renderHealthAuthorsWithBooks () {
    const { healthAuthorsWithBooks } = this.props

    return healthAuthorsWithBooks.map(author => <div key={author.id}>
      <Author author={author} />
      Books:
      {author.books.map(book => <Book book={book} key={book.id} />)}
    </div>)
  }

  render () {
    return <div>
      <h1>Books:</h1> {this.renderBooks()}
      <hr />
      <h1>Authors:</h1> {this.renderAuthors()}
      <hr />
      <h2>Health Authors:</h2> {this.renderHealthAuthors()}
      <hr />
      <h2>Health Authors with loaded Books:</h2> {this.renderHealthAuthorsWithBooks()}
    </div>
  }
};

const mapStateToProps = state => ({
  books: getBooksSelector(state),
  authors: getAuthorsSelector(state),
  healthAuthors: getHealthAuthorsSelector(state),
  healthAuthorsWithBooks: getHealthAuthorsWithBooksSelector(state)
})

const mapDispatchToProps = {
  addBooks, addAuthors
}

const App = connect(mapStateToProps, mapDispatchToProps)(View)

/**
 * Books selectors
 */

/**
 * Get Books Store entity
 */
const getBooks = ({ books }) => books

/**
 * Get all Books
 */
const getBooksSelector = createSelector(getBooks,
  books => books.allIds.map(id => books.byIds[id]))

/**
 * Authors selectors
 */

/**
 * Get Authors Store entity
 */
const getAuthors = ({ authors }) => authors

/**
 * Get all Authors
 */
const getAuthorsSelector = createSelector(getAuthors,
  authors => authors.allIds.map(id => authors.byIds[id]))

/**
 * Get array of Authors ids,
 * which have books in 'Health' category
 */
const getHealthAuthorsIdsSelector = createSelector([getAuthors, getBooks],
  (authors, books) => (
    authors.allIds.filter(id => {
      const author = authors.byIds[id]
      const filteredBooks = author.books.filter(id => (
        books.byIds[id].category === 'Health'
      ))

      return filteredBooks.length
    })
  ))

/**
 * Get array of Authors,
 * which have books in 'Health' category
 */
const getHealthAuthorsSelector = createSelector([getHealthAuthorsIdsSelector, getAuthors],
  (filteredIds, authors) => (
    filteredIds.map(id => authors.byIds[id])
  ))

/**
 * Get array of Authors, together with their Books,
 * which have books in 'Health' category
 */
const getHealthAuthorsWithBooksSelector = createSelector([getHealthAuthorsIdsSelector, getAuthors, getBooks],
  (filteredIds, authors, books) => (
    filteredIds.map(id => ({
      ...authors.byIds[id],
      books: authors.byIds[id].books.map(id => books.byIds[id])
    }))
  ))

// Combined Reducer
const reducers = combineReducers({
  books: booksReducer,
  authors: authorsReducer
})

// Store
const store = createStore(reducers)

const render = () => {
  ReactDOM.render(<Provider store={store}>
    <App />
  </Provider>, document.getElementById('root'))
}

render()

<div id="root"></div>

<script src="https://cdnjs.cloudflare.com/ajax/libs/react/15.1.0/react.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/react/15.1.0/react-dom.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/babel-core/5.8.24/browser.js"></script>
<script src="https://npmcdn.com/reselect@3.0.1/dist/reselect.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/redux/3.3.1/redux.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/react-redux/4.4.6/react-redux.min.js"></script>

JSFiddle.

Answer 1

Author of the question's here!

One year later, now I'm going to summarize my experience and thoughts here.

I was looking into two possible approaches for handling the relational data:

1. Indexing

aaronofleonard, already gave us a great and very detailed answer here, where his main concept is as follows:

We're not looking to recreate copies of the state with normalized data. We're trying to *create indexed representations (read: references) of that state which are easily digested by components.

It perfectly fits to the examples, he mentions. But it's important to highlight that his examples create indexes only for one-to-many relations (one Book has many Authors). So I started to think about how this approach will fit to all my possible requirements:

1. Handing many-to-many cases. Example: One Book has many Authors, through BookStore.
2. Handling Deep filtration. Example: Get all the Books from the Healthy Category, where at least on Author is from a specific Country. Now just imagine if we have many more nested levels of entities.

Of course it's doable, but as you can see the things can get serious very soon.

If you feel comfortable with managing such complexity with Indexing, then make sure you have enough design time for creating your selectors and composing indexing utilities.

I continued searching for a solution, because creating such an Indexing utility looks totally out-of-scope for the project. It's more like creating a third-party library.

So I decided to give a try to Redux-ORM library.

2. Redux-ORM

A small, simple and immutable ORM to manage relational data in your Redux store.

Without being verbose, here's how I managed all the requirements, just using the library:

// Handing many-to-many case.
const getBooks = createSelector({ Book } => {
  return Books.all().toModelArray()
   .map( book => ({ 
     book: book.ref,
     authors: book.authors.toRefArray()
   })
})

// Handling Deep filtration.
// Keep in mind here you can pass parameters, instead of hardcoding the filtration criteria.
const getFilteredBooks = createSelector({ Book } => {
  return Books.all().toModelArray()
   .filter( book => {
     const authors = book.authors.toModelArray()
     const hasAuthorInCountry = authors.filter(a => a.country.name === 'Bulgaria').length

     return book.category.type === 'Health' && hasAuthorInCountry
   })
   .map( book => ({ 
     book: book.ref,
     authors: book.authors.toRefArray()
   })
})

As you can see - the library handles all the relations for us and we can easily access all the child entities and perform complex computation.

Also using .ref we return the entity Store's reference, instead of creating a new object copy (you're worried about the memory).

So having this type of selectors my flow is as follows:

1. Container components fetches the data via API.
2. Selectors get only the needed slice of data.
3. Render the Presentation components.

However, nothing is perfect as it sounds as. Redux-ORM deals with relational operations as querying, filtering, etc. in a very easy of use way. Cool!

But when we talk about selectors reusability, composition, extending and so on - it's kind of tricky and awkward task. It's not a Redux-ORM problem, than to the reselect library itself and the way it works. Here we discussed the topic.

Conclusion (personal)

For simpler relational projects I would give a try to the Indexing approach.

Otherwise, I would stick with Redux-ORM, as I used it in the App, for which one I asked the question. There I have 70+ entities and still counting!

Answer 2

This reminds me of how I started one of my projects where the data was highly relational. You think too much still about the backend way of doing things, but you gotta start thinking of more of the JS way of doing things (a scary thought for some, to be sure).

1) Normalized Data in State

You've done a good job of normalizing your data, but really, it's only somewhat normalized. Why do I say that?

...
books: [1]
...
...
authorId: 1
...

You have the same conceptual data stored in two places. This can easily become out of sync. For example, let's say you receive new books from the server. If they all have authorId of 1, you also have to modify the book itself and add those ids to it! That's a lot of extra work that doesn't need to be done. And if it isn't done, the data will be out of sync.

One general rule of thumb with a redux style architecture is never store (in the state) what you can compute. That includes this relation, it is easily computed by authorId.

2) Denormalized Data in Selectors

We mentioned having normalized data in the state was not good. But denormalizing it in selectors is ok right? Well, it is. But the question is, is it needed? I did the same thing you are doing now, getting the selector to basically act like a backend ORM. "I just want to be able to call author.books and get all the books!" you may be thinking. It would be so easy to just be able to loop through author.books in your React component, and render each book, right?

But, do you really want to normalize every piece of data in your state? React doesn't need that. In fact, it will also increase your memory usage. Why is that?

Because now you will have two copies of the same author, for instance:

const authors = [{
  id: 1,
  name: 'Jordan Enev',
  books: [1]
}];

and

const authors = [{
  id: 1,
  name: 'Jordan Enev',
  books: [{
      id: 1,
      name: 'Book 1',
      category: 'Programming',
      authorId: 1
  }]
}];

So getHealthAuthorsWithBooksSelector now creates a new object for each author, which will not be === to the one in the state.

This is not bad. But I would say it's not ideal. On top of the redundant (<- keyword) memory usage, it's better to have one single authoritative reference to each entity in your store. Right now, there are two entities for each author that are the same conceptually, but your program views them as totally different objects.

So now when we look at your mapStateToProps:

const mapStateToProps = state => ({
  books: getBooksSelector(state),
  authors: getAuthorsSelector(state),
  healthAuthors: getHealthAuthorsSelector(state),
  healthAuthorsWithBooks: getHealthAuthorsWithBooksSelector(state)
});

You are basically providing the component with 3-4 different copies of all the same data.

Thinking About Solutions

First, before we get to making new selectors and make it all fast and fancy, let's just make up a naive solution.

const mapStateToProps = state => ({
  books: getBooksSelector(state),
  authors: getAuthors(state),
});

Ahh, the only data this component really needs! The books, and the authors. Using the data therein, it can compute anything it needs.

Notice that I changed it from getAuthorsSelector to just getAuthors? This is because all the data we need for computing is in the books array, and we can just pull the authors by id one we have them!

Remember, we're not worrying about using selectors yet, let's just think about the problem in simple terms. So, inside the component, let's build an "index" of books by their author.

const { books, authors } = this.props;

const healthBooksByAuthor = books.reduce((indexedBooks, book) => {
   if (book.category === 'Health') {
      if (!(book.authorId in indexedBooks)) {
         indexedBooks[book.authorId] = [];
      }
      indexedBooks[book.authorId].push(book);
   }
   return indexedBooks;
}, {});

And how do we use it?

const healthyAuthorIds = Object.keys(healthBooksByAuthor);

...
healthyAuthorIds.map(authorId => {
    const author = authors.byIds[authorId];

    return (<li>{ author.name }
       <ul>
         { healthBooksByAuthor[authorId].map(book => <li>{ book.name }</li> }
       </ul>
    </li>);
})
...

Etc etc.

But but but you mentioned memory earlier, that's why we didn't denormalize stuff with getHealthAuthorsWithBooksSelector, right? Correct! But in this case we aren't taking up memory with redundant information. In fact, every single entity, the books and the authors, are just reference to the original objects in the store! This means that the only new memory being taken up is by the container arrays/objects themselves, not by the actual items in them.

I've found this kind of solution ideal for many use cases. Of course, I don't keep it in the component like above, I extract it into a reusable function which creates selectors based on certain criteria. Although, I'll admit I haven't had a problem with the same complexity as yours, in that you have to filter a specific entity, through another entity. Yikes! But still doable.

Let's extract our indexer function into a reusable function:

const indexList = fieldsBy => list => {
 // so we don't have to create property keys inside the loop
  const indexedBase = fieldsBy.reduce((obj, field) => {
    obj[field] = {};
    return obj;
  }, {});

  return list.reduce(
    (indexedData, item) => {
      fieldsBy.forEach((field) => {
        const value = item[field];

        if (!(value in indexedData[field])) {
          indexedData[field][value] = [];
        }

        indexedData[field][value].push(item);
      });

      return indexedData;
    },
    indexedBase,
  );
};

Now this looks like kind of a monstrosity. But we must make certain parts of our code complex, so we can make many more parts clean. Clean how?

const getBooksIndexed = createSelector([getBooksSelector], indexList(['category', 'authorId']));
const getBooksIndexedInCategory = category => createSelector([getBooksIndexed],
    booksIndexedBy => {
        return indexList(['authorId'])(booksIndexedBy.category[category])
    });
    // you can actually abstract this even more!

...
later that day
...

const mapStateToProps = state => ({
  booksIndexedBy: getBooksIndexedInCategory('Health')(state),
  authors: getAuthors(state),
});

...
const { booksIndexedBy, authors } = this.props;
const healthyAuthorIds = Object.keys(booksIndexedBy.authorId);

healthyAuthorIds.map(authorId => {
    const author = authors.byIds[authorId];

    return (<li>{ author.name }
       <ul>
         { healthBooksByAuthor[authorId].map(book => <li>{ book.name }</li> }
       </ul>
    </li>);
})
...

This is not as easy to understand of course, because it relies primarily on composing these functions and selectors to build representations of data, instead of renormalizing it.

The point is: We're not looking to recreate copies of the state with normalized data. We're trying to *create indexed representations (read: references) of that state which are easily digested by components.

The indexing I've presented here is very reusable, but not without certain problems (I'll let everyone else figure those out). I don't expect you to use it, but I do expect you to learn this from it: rather than trying to coerce your selectors to give you backend-like, ORM-like nested versions of your data, use the inherent ability to link your data using the tools you already have: ids and object references.

These principles can even be applied to your current selectors. Rather than create a bunch of highly specialized selectors for every conceivable combination of data... 1) Create functions that create selectors for you based on certain parameters 2) Create functions that can be used as the resultFunc of many different selectors

Indexing isn't for everyone, I'll let others suggest other methods.

Answer 3

When you start "overloading" your selectors (like getHealthAuthorsSelector) with other named selectors (like getHealthAuthorsWithBooksSelector, ...) you might end up with something like getHealthAuthorsWithBooksWithRelatedBooksSelector etc etc.

That is not sustainable. I suggest you stick to the high level ones (ie getHealthAuthorsSelector) and use a mechanism so that their books and the related books of those books etc are always available.

You can use TypeScript and turn the author.books into a getter, or just work with covenience functions to get the books from the store whenever they are needed. With an action you can combine a get from store with a fetch from db to display (possibly) stale data directly and have Redux/React take care of the visual update once the data is retrieved from the database.

I hadn't heard of this Reselect but it seems like it might be a good way to have all sorts of filters in one place to avoid duplicating code in components.
Simple as they are, they are also easily testable. Business/Domain logic testing is usually a (very?) good idea, especially when you are not a domain expert yourself.

Also keep in mind that a joining of multiple entities into something new is useful from time to time, for example flattening entities so they can be bound easily to a grid control.