Finding the layers and layer sizes for each Docker

Question

For research purposes I'm trying to crawl the public Docker registry ( https://registry.hub.docker.com/ ) and find out 1) how many layers an average image has and 2) the sizes of these layers to get an idea of the distribution.

However I studied the API and public libraries as well as the details on the github but I cant find any method to:

• retrieve all the public repositories/images (even if those are thousands I still need a starting list to iterate through)
• find all the layers of an image
• find the size for a layer (so not an image but for the individual layer).

Can anyone help me find a way to retrieve this information?

Thank you!

EDIT: is anyone able to verify that searching for '*' in Docker registry is returning all the repositories and not just anything that mentions '*' anywhere? https://registry.hub.docker.com/search?q=*

Answer 1

Here is a good article about Show Layers of Docker Image

You can first find the image ID:

$ docker images -a

Then find the its layers and their sizes:

$ docker history --no-trunc <Image ID>

Note: I'm using Docker version 1.13.1

$ docker -v
Docker version 1.13.1, build 092cba3

Answer 2

In my opinion, docker history <image> is sufficient. This returns the size of each layer.

$ docker history jenkinsci-jnlp-slave:2019-1-9c
IMAGE               CREATED             CREATED BY                                      SIZE                COMMENT
93f48953d298        42 minutes ago      /bin/sh -c #(nop)  USER jenkins                 0B
6305b07d4650        42 minutes ago      /bin/sh -c chown jenkins:jenkins -R /home/je…   1.45GB

What suprised me is that just changing the owner created a huge blob.

Answer 3

Can check out dive written in golang.

Awesome tool.

You can adjust the source code so that it exports all the info it shows into a json file.

Answer 4

They have a very good answer here: https://stackoverflow.com/a/32455275/165865

Just run below images:

docker run --rm -v /var/run/docker.sock:/var/run/docker.sock nate/dockviz images -t

Answer 5

I've solved this problem by using the search function on Docker's website where '*' is a valid search that returns 200k repositories and then I crawled each invididual page. HTML parsing allows me to extract all the image names on each page.

Answer 6

This will inspect the docker image and print the layers:

$ docker image inspect nginx -f '{{.RootFS.Layers}}'
[sha256:d626a8ad97a1f9c1f2c4db3814751ada64f60aed927764a3f994fcd88363b659 sha256:82b81d779f8352b20e52295afc6d0eab7e61c0ec7af96d85b8cda7800285d97d sha256:7ab428981537aa7d0c79bc1acbf208c71e57d9678f7deca4267cc03fba26b9c8]