I am familiar with artifact repositories such as Artifactory, Maven & Nexus.
What do Berkshelf-API / Chef Supermarket do differently than the above Artifact Repositories?
On first glance it appears they are duplicating features in already existing artifact repositories (immutability, transitive dependency resolution, etc).
Clarification: This is not an opinion-based question, I am seeking the technical reasons why Berkshelf-API / Chef Supermarket are used for Chef Cookbooks instead of other typical artifact repositories.
tl;dr - Berkshelf-API & Chef Supermarket are what Berkshelf uses for downloading cookbook dependencies. They supply Berkshelf with a specially formatted .JSON document for determining dependencies of cookbooks. I have found no reason why other artifact repositories could not be modified to return a similarly formattied .JSON document, but to my knowledge, none presently do.
Berkshelf-API: "A server which indexes cookbooks from various sources and hosts it over a REST API"
In fact, Berkshelf-API is used by Chef Supermarket.
You can manually pull the list of cookbooks from the Supermarket by appending "/universe" to the url (https://supermarket.getchef.com/universe).
When Berkshelf is used to resolve dependencies (using 'berks install' or 'berks update'), it looks for the source entry (typically 'source "https://supermarket.getchef.com"' at the top of the cookbook's Berksfile).
The Berksfile also has 'metadata' below the source entry. This loads the cookbook dependencies listed in the cookbooks metadata.rb file. These can be overridden by cookbook dependencies listed below the 'metadata' entry in the Berksfile.
Berkshelf then starts downloading cookbook dependencies from the source entry (in this case Supermarket).
However, Berkshelf does not recursively load the metadata.rb or Berkshelf files of dependencies.
ex: if Foo depends on Bar which depends on Baz
Foo => Bar => Baz
Foo will indicate the Bar dependency in either metadata.rb or Berksfile.
Bar will indicate the Baz dependency in either metadata.rb or Berksfile.
However Berkshelf will not determine the dependencies for Bar (in this case Baz) by reading Bar's metadata.rb or Berksfile. It instead determines Bar's dependencies by using a cached list of versioned cookbooks from Berkshelf-API that is returned from the call to the source entry.
A private Berkshelf-API server can be set up with multiple endpoints pointing to different cookbook repositories (either another Berkshelf-API server or a Chef Server).
So a common usage is a Private Berkshelf-API server pointing to both a Private Chef Server (storing non-public cookbooks) and the Chef Supermarket server (storing public cookbooks).