I have a pandas time series which contains cumulative monthly values.
If in a month on a certain date, the value becomes a certain number, I need to set rest of the days to 1000.
E.g.
df:
Date cummulative_value
1/8/2017 -3
1/9/2017 -6
1/10/2017 -72
1/11/2017 500
1/26/2017 575
2/7/2017 -5
2/14/2017 -6
2/21/2017 -6
My cutoff value is -71 so in above example I need to achieve the following:
Date cummulative_value
1/8/2017 -3
1/9/2017 -6
1/10/2017 1000
1/11/2017 1000
1/26/2017 1000
2/7/2017 -5
2/14/2017 -6
2/21/2017 -6
I am trying to leverage groupby in pandas but I am not sure how to go about it. Any other more efficient way will help also.
Use groupby and cumprod:
df['cummulative_value'] = (df.groupby(df['Date'].dt.strftime('%Y%m'))['cummulative_value']
.transform(lambda x: np.where(x.ge(-71).cumprod(),x,1000)))
print(df)
Output:
Date cummulative_value
0 2017-01-08 -3
1 2017-01-09 -6
2 2017-01-10 1000
3 2017-01-11 1000
4 2017-01-26 1000
5 2017-02-07 -5
6 2017-02-14 -6
7 2017-02-21 -6
Here's one approach that involves creating a mask:
df.set_index(pd.to_datetime(df['Date'], format="%m/%d/%Y"), inplace=True)
mask = df['cummulative_value'].lt(-71).groupby(df.index.month).cumsum()
# Date
# 2017-01-08 False
# 2017-01-09 False
# 2017-01-10 True
# 2017-01-11 True
# 2017-01-26 True
# 2017-02-07 False
# 2017-02-14 False
# 2017-02-21 False
df.loc[mask, 'cummulative_value'] = 1000
df.reset_index(drop=True)
# Date cummulative_value
# 0 1/8/2017 -3
# 1 1/9/2017 -6
# 2 1/10/2017 1000
# 3 1/11/2017 1000
# 4 1/26/2017 1000
# 5 2/7/2017 -5
# 6 2/14/2017 -6
# 7 2/21/2017 -6
