Postgres/GCS/Python: Insert data stored in variabl

2019-08-26 09:40发布

问题:

I am trying to read a file from GCS and store it in variable and create a postgresql table from it. I can connect to GCs from my code and be able to store data in variabe with this:

result = blob.download_as_string()
result = result.decode('utf8').strip()

which the printed result are in the correct format. Then, I tried to insert data from this variable into the table which I did:

sql = "COPY tablename FROM STDIN WITH DELIMITER ',' NULL AS '\\N' CSV HEADER;"
cursor.copy_expert(sql, result)

and I got this error:

file must be a readable file-like object for COPY FROM; a writable file-like object for COPY TO

I also tried with another function:

cursor.copy_from(result, 'table' ,sep=',')

but I got this result:

argument 1 must have a .read() method

So, according to my question how can I put the data in the variable to the table I created. Or do I have to download it to my local and use this:

sql = "COPY tablename from STDIN WITH DELIMITER ',' CSV HEADER"
with open('/path/to/csv' , 'r+') as file:
     cursor.copy_expert(sql, file)

This one is worked but I don't want to download it to my local. I just want to read it and insert it into my table

Thank you