Why does one call `git read-tree` after a sparse c

According to Subdirectory Checkouts with git sparse-checkout one calls git read-tree -mu HEAD after configuring a sparse checkout in the case of an already existing repository, i.e.:

# Enable sparse-checkout:
git config core.sparsecheckout true

# Configure sparse-checkout 
echo some/dir/ >> .git/info/sparse-checkout
echo another/sub/tree >> .git/info/sparse-checkout

# Update your working tree:
git read-tree -mu HEAD

Can you please explain the read-tree step in more detail?
How does it work?
What is going on?
Why does one use read-tree and not, let us say, checkout?
Why does one use -mu (why is this a merge, and what is merged)?

-m

    Perform a merge, not just a read. The command will refuse to run if
    your index file has unmerged entries, indicating that you have not
    finished previous merge you started.

-u

    After a successful merge, update the files in the work tree with the
    result of the merge.

With Git 2.25 (Q1 2020), Management of sparsely checked-out working tree has gained a dedicated "sparse-checkout" command.
It introduces a cone mode (that I detail in "Git sparse checkout with exclusion"), which will make a sparse-checkout must faster.

But it also indirectly describes why git read-tree -mu HEAD is used (or, with the new "cone" mode, was used).

See commit e6152e3 (21 Nov 2019) by Jeff Hostetler (Jeff-Hostetler).
See commit 761e3d2 (20 Dec 2019) by Ed Maste (emaste).
See commit 190a65f (13 Dec 2019), and commit cff4e91, commit 416adc8, commit f75a69f, commit fb10ca5, commit 99dfa6f, commit e091228, commit e9de487, commit 4dcd4de, commit eb42fec, commit af09ce2, commit 96cc8ab, commit 879321e, commit 72918c1, commit 7bffca9, commit f6039a9, commit d89f09c, commit bab3c35, commit 94c0956 (21 Nov 2019) by Derrick Stolee (derrickstolee).
^{(Merged by Junio C Hamano -- gitster -- in commit bd72a08, 25 Dec 2019)}

sparse-checkout: update working directory in-process

^{Signed-off-by: Derrick Stolee}

The sparse-checkout builtin used 'git read-tree -mu HEAD' to update the skip-worktree bits in the index and to update the working directory.
This extra process is overly complex, and prone to failure. It also requires that we write our changes to the sparse-checkout file before trying to update the index.

Remove this extra process call by creating a direct call to unpack_trees() in the same way 'git read-tree -mu HEAD' does.
In addition, provide an in-memory list of patterns so we can avoid reading from the sparse-checkout file. This allows us to test a proposed change to the file before writing to it.

An earlier version of this patch included a bug when the 'set' command failed due to the "Sparse checkout leaves no entry on working directory" error.
It would not rollback the index.lock file, so the replay of the old sparse-checkout specification would fail. A test in t1091 now covers that scenario.

And, with Git 2.27 (Q2 2020), "sparse-checkout" knows how to reapply itself:

See commit 5644ca2, commit 681c637, commit ebb568b, commit 22ab0b3, commit 6271d77, commit 1ac83f4, commit cd002c1, commit 4ee5d50, commit f56f31a, commit 7af7a25, commit 30e89c1, commit 3cc7c50, commit b0a5a12, commit 72064ee, commit fa0bde4, commit d61633a, commit d7dc1e1, commit 031ba55 (27 Mar 2020) by Elijah Newren (newren).
^{(Merged by Junio C Hamano -- gitster -- in commit 48eee46, 29 Apr 2020)}

sparse-checkout: provide a new reapply subcommand

^{Reviewed-by: Derrick Stolee}
^{Signed-off-by: Elijah Newren}

If commands like merge or rebase materialize files as part of their work, or a previous sparse-checkout command failed to update individual files due to dirty changes, users may want a command to simply 'reapply' the sparsity rules.

Provide one.

The updated git sparse-checkout man page now includes:

reapply:

Reapply the sparsity pattern rules to paths in the working tree.

Commands like merge or rebase can materialize paths to do their work (e.g. in order to show you a conflict), and other sparse-checkout commands might fail to sparsify an individual file (e.g. because it has unstaged changes or conflicts).

In such cases, it can make sense to run git sparse-checkout reapply later after cleaning up affected paths (e.g. resolving conflicts, undoing or committing changes, etc.).

But, with Git 2.27, it won't reapply/update itself using git read-tree anymore:

unpack-trees: add a new update_sparsity() function

^{Reviewed-by: Derrick Stolee}
^{Signed-off-by: Elijah Newren}

Previously, the only way to update the SKIP_WORKTREE bits for various paths was invoking git read-tree -mu HEAD or calling the same code that this codepath invoked.

This however had a number of problems if the index or working directory were not clean.

First, let's consider the case:
Flipping SKIP_WORKTREE -> !SKIP_WORKTREE (materializing files)
If the working tree was clean this was fine, but if there were files or directories or symlinks or whatever already present at the given path then the operation would abort with an error.

Let's label this case for later discussion:

A) There is an untracked path in the way

Now let's consider the opposite case:
Flipping !SKIP_WORKTREE -> SKIP_WORKTREE (removing files)
If the index and working tree was clean this was fine, but if there were any unclean paths we would run into problems.

There are three different cases to consider:

B) The path is unmerged

C) The path has unstaged changes

D) The path has staged changes (differs from HEAD)

If any path fell into case B or C, then the whole operation would be aborted with an error.

With sparse-checkout, the whole operation would be aborted for case D as well, but for its predecessor of using git read-tree -mu HEAD directly, any paths that fell into case D would be removed from the working copy and the index entry for that path would be reset to match HEAD -- which looks and feels like data loss to users (only a few are even aware to ask whether it can be recovered, and even then it requires walking through loose objects trying to match up the right ones).

Refusing to remove files that have unsaved user changes is good, but refusing to work on any other paths is very problematic for users.

If the user is in the middle of a rebase or has made modifications to files that bring in more dependencies, then for their build to work they need to update the sparse paths.

This logic has been preventing them from doing so.

Sometimes in response, the user will stage the files and re-try, to no avail with sparse-checkout or to the horror of losing their changes if they are using its predecessor of git read-tree -mu HEAD.

Add a new update_sparsity() function which will not error out in any of these cases but behaves as follows for the special cases:

A) Leave the file in the working copy alone, clear the SKIP_WORKTREE bit, and print a warning (thus leaving the path in a state where status will report the file as modified, which seems logical).

B) Do NOT mark this path as SKIP_WORKTREE, and leave it as unmerged.

C) Do NOT mark this path as SKIP_WORKTREE and print a warning about the dirty path.

D) Mark the path as SKIP_WORKTREE, but do not revert the version stored in the index to match HEAD; leave the contents alone.

I tried a different behavior for A (leave the SKIP_WORKTREE bit set), but found it very surprising and counter-intuitive (e.g. the user sees it is present along with all the other files in that directory, tries to stage it, but git add ignores it since the SKIP_WORKTREE bit is set).

A & C seem like optimal behavior to me.

B may be as well, though I wonder if printing a warning would be an improvement.

Some might be slightly surprised by D at first, but given that it does the right thing with git commit and even git commit -a (git add ignores entries that are marked SKIP_WORKTREE and thus doesn't delete them, and commit -a is similar), it seems logical to me.

And, still with GIt 2.27 (Q2 2020):

See commit 6c34239 (14 May 2020) by Elijah Newren (newren).
^{(Merged by Junio C Hamano -- gitster -- in commit fde4622, 20 May 2020)}

unpack-trees: also allow get_progress() to work on a different index

^{Noticed-by: Jeff Hostetler}
^{Signed-off-by: Elijah Newren}

commit b0a5a12a60 ("unpack-trees: allow check_updates() to work on a different index", 2020-03-27, Git v2.27.0-rc0 -- merge listed in batch #5) allowed check_updates() to work on a different index, but it called get_progress() which was hardcoded to work on o->result much like check_updates() had been.

Update it to also accept an index parameter and have check_updates() pass that parameter along so that both are working on the same index.