How to save a file from an https link to Google Cl

2019-08-14 07:15发布

I would like to save a large file (approximately 50 GB) directly on Google Cloud storage. I tried gsutil cp https://archive.org/download/archiveteam-twitter-stream-2015-08/archiveteam-twitter-stream-2015-08.tar gs://my/folder, but that didn't work (InvalidUrlError: Unrecognized scheme "https").

Is there a way of doing that, without having to first download the file to my local storage?

Thanks!

1条回答
地球回转人心会变
2楼-- · 2019-08-14 07:37

You can use curl to fetch the URL and pipe it to gsutil. For example:

curl -L https://archive.org/download/archiveteam-twitter-stream-2015-08/archiveteam-twitter-stream-2015-08.tar | gsutil cp - gs://your/folder/archiveteam-twitter-stream-2015-08.tar
查看更多
登录 后发表回答