Reading large JSON files from S3 in RStudio EC2 in

2019-05-16 11:50发布

I am going through a similar issue as this question here:

I have a big JSON file on AWS S3 and am trying to access it via RStudio (EC2 instance from Louis Aslett's AMI). I have even tried moving from t2 to r4.xlarge with 30GB of memory but to no avail: I receive errors:

Error in writeBin(httr::content(r, as = "raw"), con = file) : long vectors not supported yet: connections.c:4147

If I use the free tier instance then it gives me the error:

Error in curl::curl_fetch_memory(url, handle = handle) : Failed writing body (0 != 16360)

It seems that the question that I mentioned in the reference has figured out a way to do it, though I am not able to follow it completely. Can someone please explain a little when they say that the directory needs to be something else than "home". How do you implement it? Because there is no permission to do that in Louis AMI. The question may be very basic but I am getting out of my wits here.

Cheers! A

0条回答
登录 后发表回答