crossfilter - calculating percent of all records w

Question

here is my problem:

i am using a python flask server that fetches json data from mongo db, and in there i specify what fields to import.This data is in json format and is fetched like that only. Is it possible to do transformations to these fields once passed through crossfilter in graphs.js? e.g. i have a status attribute which can take values "Pass","In Progress","on Hold" or "Fail". I basically want to do a metric which tells me percent failure. So ideally i have to do some calculations on the data. Please advise on this.

Sample data (in tabular form for clarity) looks like:
TrialLocation     | Subject Status
Site A            | In progress
Site A            | Pass
Site B            | In progress
Site A            | In progress
Site B            | On Hold
Site A            | Screen Failure

In this case i should get a bar chart with site name on x axis and on the y axis, i should get the metric calculating the failure percentage. which in this case would be 25% for Site A and 0% for Site B.

So i created chart in the first place which gave me the count of subjects per site.

var siteName = ndx.dimension(function(d) { return d["TrialLocation"];});
var numSubjectsBySite = siteName.group();
var siteLevelChart = dc.barChart("#site-level-count", "subjectView");

and finally the chart:

siteLevelChart
 .width(2000)
 .height(200)
 .transitionDuration(1000)
 .dimension(siteName)
 .group(numSubjectsBySite)
 .ordering(function(d){return d.value;})

So i thought, i would calculate the count of rows with SubjectStatus = "Screen Failure" and divide that by the total number of rows which in this case would be "numSubjectsBySite" variable Then when i introduced this code:

var countScreenFailures = ndx.dimension(function(d){ return d["SubjectStatus"];});
 countScreenFailures.filter("Off Study");

My bar chart only shows the rows where Subject Status ="ScreenFailure".

How can i calculate the screen failure rate and then use it ? Please help me out?

thank you so much. Anmol

Answer 1

You can use a custom groupAll for this. Here is a straight crossfilter solution, based on the jsfiddle you provided in a later question.

(It's a lot easier to answer with a fiddle to work with!)

var all = ndx.groupAll();
var failurePercentGroup = all.reduce(
    function(p, v) {
        ++p.count;
        p.failures += (v.Status === 'Screen Failure' ? 1 : 0);
        p.failPercent = p.count ? p.failures/p.count : 0;
        return p;
    },
    function(p, v) {
        --p.count;
        p.failures -= (v.Status === 'Screen Failure' ? 1 : 0);
        p.failPercent = p.count ? p.failures/p.count : 0;
        return p;
    },
    function() {
        return {
            count: 0,
            failures: 0,
            failPercent: 0
        };
    }
);

failurePercent.valueAccessor(function (x) {
    return x.failPercent;
})
    .group(failurePercentGroup);

@Ethan's answer looks like it should work, but you remarked elsewhere that you couldn't get it to work.

Updated fiddle here: http://jsfiddle.net/gordonwoodhull/vct0dzou/8/

I didn't deal with formatting it as a percent, so it just shows a ratio, but you should be able to figure that part out.

Answer 2

You'll need to build custom grouping/reduce functions to track the count of each status as well as the total count. Then you can just divide in the chart to calculate your percentage. If you are interested in using Reductio, you can probably do the following:

var reducer = reductio().count(true);

// Do this as many times as you need for different status counts. Each
// call of reducer.value will add a new property to your groups where
// you can store the count for that status.
reducer.value("ScreenFailure").sum(
  function(d) {
    // This counts records with SubjectStatus = "Screen Failure"
    return d["SubjectStatus"] === "Screen Failure" ? 1 : 0;
  });

// Build the group with the Reductio reducers.
var numSubjectsBySite = reducer(siteName.group());

// In your dc.js chart, calculate the % using a value accessor.
siteLevelChart
 .width(2000)
 .height(200)
 .transitionDuration(1000)
 .dimension(siteName)
 .group(numSubjectsBySite)
 .valueAccessor(function(p) { return p.value.ScreenFailure.sum / p.value.count; })
 .ordering(function(d){return d.value;})