I have data in files such as "file.csv". I would like to read them with np.genfromtxt and do some statistics like average, variance etc. on some columns (X, Y, Z). However I want to make the statistics on for X > 1, Y > 3 Z > 2 etc. This is a simple example here.
This code produces almost correct results but it includes ALL Xs, Ys and Zs, I want to do the same but with the X,Y,Z conditions i specified above.
#file.csv
X,Y,Z
1,2,3
4,2,5
15,9,1
#
data = np.genfromtxt(file.csv, delimiter=',', dtype=float, unpack=True, skiprows = 0)
X=data[0];Y=data[1];Z=data[2]
Mean = np.average(X)
--> Doing a great job getting the average. However, I want i to get average ONLY IF X > 1 (for example)... How do I make it do so?
In order to average over only some fields, you break down your averaging as follows:
- Find the indexes (ind) of those elements that meet a certain
criteria
- Find the mean of the array indexed only with the the values in ind
The following code does exactly this:
indexes = np.where(X>1)[0] # We index with '0' here to get to the 1st element of the returned tuple
Mean = np.mean(X[indexes])
You could use so-called "fancy-indexing", X[X>1], to select the part of the array you want:
import numpy as np
X,Y,Z = np.genfromtxt('file.csv', delimiter=',', dtype=float, unpack=True, skiprows = 0)
print(X)
# [ nan 1. 4. 15.]
print(X[X>1])
# [ 4. 15.]
print(np.average(X[X>1]))
# 9.5
To combine two masks (boolean arrays) with bit-wise logical-and, use the & operator:
print(np.average(X[(X>1)&(X<10)]))
# 4.0
