using os.walk cannot open the file from the list

Question

My problem is to read '.csv' files in catalogs and do some calculations on them. I have calculations working but my for loop seem not to work as I want to.

d = 'F:\MArcin\Experiments\csvCollection\'
for dirname, dirs, files in os.walk(d):

    for i in files:
        if i.endswith('.csv'):
            data1 = pd.read_csv(i, sep=",")
            data = data1['x'][:, np.newaxis]
            target = data1['y']

The error Iam getting is: IOError: File 1.csv does not exist

files is list of all '.csv' files inside dirname

i is str of size 1 and contains 1.csv (that is first of the files in catalog)

Any ideas why this is not working?

Thanks for any help.

Answer 1

Because 1.csv is somewhere on the filesystem and when you call read_csv() it opens file relative to current directory.

Just open it using absolute path:

data1 = pd.read_csv(os.path.join(dirname, i), sep=",")

dirname in os.walk represents actual directory where file 1.csv is located.