How do I download a protected file using PowerShel

Question

I am trying to download a file using PowerShell 3.0 from my TeamCity build server. I have configured TeamCity to use NTLM authentication but I cannot download the file directly and get redirected to login.

I am trying to use the following PowerShell code to download the file.

$artifacts = "http://teamcity/repository/download/bt1/.lastSuccessful/%7Bbuild.number%7D.zip"
Invoke-WebRequest -Uri $artifacts -UseDefaultCredentials

My response from the request is a redirection to the login page.

Answer 1

Here is the code for the final solution.

---

$artifacts = "http://teamcity/repository/download/bt1/.lastSuccessful/%7Bbuild.number%7D.zip"
$login = "http://teamcity/ntlmLogin.html"
$dest = "Artifacts.zip"

$TeamCitySession = New-Object Microsoft.PowerShell.Commands.WebRequestSession
Invoke-WebRequest -Uri $login -WebSession $TeamCitySession -UseDefaultCredentials -UseBasicParsing
Invoke-WebRequest -Uri $artifacts -WebSession $TeamCitySession -UseBasicParsing -OutFile $dest

---

In order to figure out what was happening I needed to use Fiddler to trace what a successful request looks like and also trace what was happening in PowerShell. In order to do that I had to make my PowerShell request use it. The following is how I turned on Fiddler tracing from within PowerShell.

Invoke-WebRequest -Uri $artifacts -UseDefaultCredentials -Proxy http://localhost:8888/

By adding the -Proxy argument to the command it told he command to use Fiddler as a proxy server.

From here I saw that TeamCity was redirecting me to the login page. Since I have NTLM authentication turned on there is a special page that you browse to in order to login. So what I wanted to do from here was to visit this login page and then download the file using the cookies that I get back as TeamCity uses a cookie to track authentication status.

It also turns out that the Invoke-WebRequest cmdlets also allow you to connect them using a web session. There are two ways of accomplishing this using either the -WebSession or the -SessionVariable parameter. After some trial and error it turns out that if you use the -SessionVariable parameter it will overwrite the session variable after each request, so that it doesn't actually share the state. Clearly this is not the behavior I am looking for. Instead I had to use the -WebSession parameter and then I could chain together the login and the download of the file. Once I did this then everything started working.

Answer 2

The reason using -SessionVariable was muting (changing) the state is that -SessionVariable is meant to output the results of that particular Invoke-WebReqeust websession as a variable to be used further down the line.

If we look at the Get-Help for that particular param:

PS C:\windows\system32> get-help Invoke-WebRequest -Parameter SessionVariable

-SessionVariable <String>
    Creates a web request session and saves it in the value of the specified variable. Enter a variable name without the dollar sign ($) symbol.

When you specify a session variable, Invoke-WebRequest creates a web request session object and assigns it to a variable with the specified name in your
Windows PowerShell session. You can use the variable in your session as soon as the command completes.

Unlike a remote session, the web request session is not a persistent connection. It is an object that contains information about the connection and the
request, including cookies, credentials, the maximum redirection value, and the user agent string. You can use it to share state and data among web
requests.

To use the web request session in subsequent web requests, specify the session variable in the value of the WebSession parameter. Windows PowerShell uses
the data in the web request session object when establishing the new connection. To override a value in the web request session, use a cmdlet parameter,
such as UserAgent or Credential. Parameter values take precedence over values in the web request session.

You cannot use the SessionVariable and WebSession parameters in the same command.

Required?                    false
Position?                    named
Default value
Accept pipeline input?       false
Accept wildcard characters?  false

This can be viewed as a case of a somewhat confusing variable name. Thanks for posting your completed code here!