Resample Pandas Dataframe with “bin size”/“frequen

Question

9I have a multi-indexed dataframe which I would like to resample to reduce the frequency of datapoints by a factor of 3 (meaning that every 3 rows become one).

This:

                    time  value
   ID  measurement  
ET001            0     0      2
                 1  0.15      3
                 2   0.3      4
                 3  0.45      3
                 4   0.6      3
                 5  0.75      2
                 6   0.9      3
ET002            0     0      2
                 1  0.16      5
                 2  0.32      4
                 3  0.45      3
                 4   0.6      3
                 5  0.75      2

I want to turn into this:

                    time  value
   ID  measurement  
ET001            0  0.15      3 
                 1   0.6    2.7
                 2   0.9      3
ET002            0  0.16    3.7 
                 1   0.6    2.7

I tried to turn my time column into a pandas datetime index like so, and then use resample:

df = df.set_index(pd.DatetimeIndex(timecourse_normed['Time']))
df = df.groupby(level=0).resample(rule='0.1S', how=np.mean) 

But the first line of that gives me actual dates (1970-something) which is quite unhelpful for the second line. Browsing arund stack overflow I found some similar quiestios which all had solutions NOT based on panda's resample - and also, sadly, not viable for my use case.

Could you give me a hand?

Answer 1

I think the idea for you could be - divide records inside each ID into bins by 3 records each (like ntile(3) in SQL) group by it and calculate mean. To create this numbers we can use the fact that you already have sequential numbers for each row - measurement level of index. So we can just divide this number by 3 to get numbers we need:

>>> df
                   time  value  ntile
ID    measurement                  
ET001 0            0.00      2      0
      1            0.15      3      0
      2            0.30      4      0
      3            0.45      3      1
      4            0.60      3      1
      5            0.75      2      1
      6            0.90      3      2
ET002 0            0.00      2      0
      1            0.16      5      0
      2            0.32      4      0
      3            0.45      3      1
      4            0.60      3      1
      5            0.75      2      1

So we can use helper function like this and apply it to each group to get desired results.

>>> def helper(x):
...     x = x.reset_index()
...     x = x.groupby(x['measurement'].div(3)).mean()
...     del x['measurement']
...     return x
... 
>>> df.groupby(level=0).apply(helper)
                   time     value
ID    measurement                
ET001 0            0.15  3.000000
      1            0.60  2.666667
      2            0.90  3.000000
ET002 0            0.16  3.666667
      1            0.60  2.666667

Hope it helps.