Fill in missing values in pandas dataframe using m

Question

datetime
2012-01-01    125.5010
2012-01-02    NaN
2012-01-03    125.5010
2013-01-04    NaN
2013-01-05    125.5010
2013-02-28    125.5010
2014-02-28    125.5010
2016-01-02    125.5010
2016-01-04    125.5010
2016-02-28    NaN

I would like to fill in the missig values in this dataframe by using a climatology computed from the dataset i.e fill in missing 28th feb 2016 value by averaging values of 28th feb from other years. How do i do this?

Answer 1

You can use groupby by month and day and transform with fillna mean:

print df.groupby([df.index.month, df.index.day]).transform(lambda x: x.fillna(x.mean()))
datetime           
2012-01-01  125.501
2012-01-02  125.501
2012-01-03  125.501
2013-01-04  125.501
2013-01-05  125.501
2013-02-28  125.501
2014-02-28  125.501
2016-01-02  125.501
2016-01-04  125.501
2016-02-28  125.501