How to prevent recursion through node_modules for

Question

I found this fantastic powershell script

gci -include bin,obj -recurse | remove-item -force -recurse

in a comment by Chris J on this answer

I use this multiple times a day.

How do you prevent this from running through any node_modules folders if they exist or their children?

I've tried various versions of

gci -exclude node_modules -include bin,obj -recurse | remove-item -force -recurse

without success.

UPDATE: After reading @HAL9256's informative answer i realized i failed to include an important requirement. I run this script at the root of my git repo. Therefore, it traverses multiple C# project folders removing the bin and obj folders for all of them. This is only necessary because Visual Studio 2017 & 2019 solution clean doesnt remove everything and as we are in the process of converting from framework to standard/core there are artifacts that get left in the bin and/or obj folder after cleaning that causes things to break when transitioning (probably from core back to framework) a given project via a git branch change.

Answer 1

The root of this issue is the combination of a misuse of recursion combined with filtering. I will go through some examples to illustrate. First we should set up an example directory structure:

mkdir Temp
mkdir Temp\bin
mkdir Temp\obj
mkdir Temp\node_modules
mkdir Temp\node_modules\bin
mkdir Temp\node_modules\obj

Out-File .\Temp\Readme.md
Out-File .\Temp\bin\Readme.md
Out-File .\Temp\obj\Readme.md
Out-File .\Temp\node_modules\Readme.md
Out-File .\Temp\node_modules\bin\Readme.md
Out-File .\Temp\node_modules\obj\Readme.md

cd .\Temp\

Let's try some examples. I will pipe the contents of Get-ChildItem into a Select-Object to better illustrate what we are retrieving. First, a vanilla Get-ChildItem:

PS C:\Temp> Get-ChildItem | Select-Object FullName

FullName
--------
C:\Temp\bin
C:\Temp\node_modules
C:\Temp\obj
C:\Temp\Readme.md

This returns what we expect, equivalent to a dir. Now, let's try a -Include to only "Include" the bin folder.

PS C:\Temp> Get-ChildItem -Include bin | Select-Object FullName
PS C:\Temp>

Hmmm. It returned nothing... We thought that it would take the list of everything, and "only" include the bin folder. What it instead did is interpret the command as, Get-ChildItem of, "nothing", (because we didn't specify any files to get), and "Include" of the "nothing", the pattern bin. This makes sense, and is listed in the Docs that "Include needs the Path parameter". Ok. Let's change it to actually get everything * then do an -Include bin, then we should get what we want... Right?

PS C:\Temp> Get-ChildItem * -Include bin | Select-Object FullName
PS C:\Temp>

Hu? I thought with the wildcard, it would include the bin folder right? Isn't that how it works?.... well not exactly. Let's change it to -Include a file instead of a directory:

PS C:\Temp> Get-ChildItem * -Include Readme.md | Select-Object FullName

FullName
--------
C:\Temp\Readme.md

So it looks like the explicit filtering, with a wildcard is for files and not directories. Then how does the original example, where we included directories work?... Well it's because -Recurse works differently. -Recurse changes what we get, it returns everything so the Include works:

PS C:\Temp> Get-ChildItem -Include bin -Recurse | Select-Object FullName

FullName
--------
C:\Temp\bin
C:\Temp\node_modules\bin

In this example, when we don't have to specify the wildcard * filter the -Recurse implicitly gets everything from the directory. Notice: it included directories that matched the name bin. Also, notice, it did not include files inside the directories.

Why example 1:

gci -include bin,obj -recurse | remove-item -force -recurse

Works is because the Get-ChildItem returns the directories, and the Remove-Item with the -Force and -Recurse takes care of removing the files.

To prove that nothing funky goes on, let's do an -Exclude node_modules:

PS C:\Temp> Get-ChildItem -Exclude node_modules -Recurse | Select-Object FullName

FullName
--------
C:\Temp\bin
C:\Temp\bin\Readme.md
C:\Temp\node_modules\bin
C:\Temp\node_modules\bin\Readme.md
C:\Temp\node_modules\obj
C:\Temp\node_modules\obj\Readme.md
C:\Temp\node_modules\Readme.md
C:\Temp\obj
C:\Temp\obj\Readme.md
C:\Temp\Readme.md

Well... that returned more than expected... but not. Notice that it did not return C:\Temp\node_modules. This is because -Exclude node_modules was correct. It excluded the folder specifically named: node_modules, and included everything else.

This is why the second example does not work:

gci -exclude node_modules -include bin,obj -recurse | remove-item -force -recurse

The exclusion filter will exclude the specific folder named node_modules, but, due to -Recurse will -Include any sub folder that matches bin, or obj. Which of course, node_modules has a ton of, and hence, doesn't work as we want.

So, if -Include doesn't work with directories, and -Recurse goes too deep, what will solve @cResults question? Well, instead of messing around with filters, why don't we just simply ask for the folders that we want like normal?

PS C:\Temp> Get-ChildItem bin,obj | Select-Object FullName

FullName
--------
C:\Temp\bin\Readme.md
C:\Temp\obj\Readme.md

Well this is ok, it returned the contents of the folders, just like the earlier example, the child items and not directories... Well what we really wanted was the folder item so instead of Get-ChildItem we use Get-Item:

PS C:\Temp> Get-Item bin,obj | Select-Object FullName

FullName
--------
C:\Temp\bin
C:\Temp\obj

Ah now, with the switch to Get-Item we get the folder, and not the child items inside the folder, which we can now use with Remove-Item which takes care of the rest:

Get-Item bin,obj | Remove-Item -Force -Recurse