Google GSutil create folder

Question

How can u create a new folder inside a bucket in google cloud storage using the gsutil command?

I tried using the same command in creating bucket but still got an error

gsutil mb -l us-east1 gs://my-awesome-bucket/new_folder/

Thanks!

Answer 1

You cannot create folders with gsutils as gsutils does not support it (workaround see below).

However, it is supported via:

• UI in browser
• write your own GCS client (we have written our own custom client which can create folders)

So even if google has a flat name space structure as the other answer correctly points out, it still has the possibility to create single folders as individual objects. Unfortunately gsutils does not exposed this.

(Ugly) workaround with gsutils: Add a dummy file into a folder and upload this dummy file - but the folder will be gone once you delete this file, unless other files in that folder are present.

Answer 2

The concept of directory is abstract in Google Cloud Storage. From the docs (How Subdirectories Work) :

gsutil provides the illusion of a hierarchical file tree atop the "flat" name space supported by the Google Cloud Storage service. To the service, the object gs://your-bucket/abc/def.txt is just an object that happens to have "/" characters in its name. There is no "abc" directory; just a single object with the given name.

So you cannot "create" a directory like in a traditional File System.

Answer 3

If you're clear about what folders and objects already exist in the bucket, then you can create a new 'folder' with gsutil by copying an object into the folder.

>mkdir test
>touch test/file1
>gsutil cp -r test gs://my-bucket
Copying file://test\file1 [Content- 
Type=application/octet-stream]...
/ [1 files][    0.0 B/    0.0 B]
Operation completed over 1 objects.

>gsutil ls gs://my-bucket
gs://my-bucket/test/

>gsutil ls gs://my-bucket/test
gs://my-bucket/test/file1

It won't work if the local directory is empty.

More simply:

>touch file2
>gsutil cp file2 gs://my-bucket/new-folder/
Copying file://test\file2 [Content- ...

>gsutil ls gs://my-bucket/new-folder
gs://my-bucket/new-folder/file2

Be aware of the potential for Surprising Destination Subdirectory Naming. E.g. if the target directory already exists as an object. For an automated process, a more robust approach would be to use rsync.

I don't know if its possible to create an empty folder with gsutil. For that, use the console's Create Folder button.