Redirection of standard and error output appending

Question

I need to collect the standard output and error log from several processes into one single log file.

So every output must append to this log file.

I want to call all the jobs with lines like this:

$p=start-process myjob.bat -redirectstandardoutput $logfile -redirecterroroutput $logfile -wait

Where do I have to put the information to append?

Answer 1

Andy gave me some good pointers, but I wanted to do it in an even cleaner way. Not to mention that with the 2>&1 >> method PowerShell complained to me about the log file being accessed by another process, i.e. both stderr and stdout trying to lock the file for access, I guess. So here's how I worked it around.

First let's generate a nice filename, but that's really just for being pedantic:

$name = "sync_common"
$currdate = get-date -f yyyy-MM-dd
$logfile = "c:\scripts\$name\log\$name-$currdate.txt"

And here's where the trick begins:

start-transcript -append -path $logfile

write-output "starting sync"
robocopy /mir /copyall S:\common \\10.0.0.2\common 2>&1 | Write-Output
some_other.exe /exeparams 2>&1 | Write-Output
...
write-output "ending sync"

stop-transcript

With start-transcript and stop-transcript you can redirect ALL output of PowerShell commands to a single file, but it doesn't work correctly with external commands. So let's just redirect all the output of those to the stdout of PS and let transcript do the rest.

In fact, I have no idea why the MS engineers say they haven't fixed this yet "due to the high cost and technical complexities involved" when it can be worked around in such a simple way.

Either way, running every single command with start-process is a huge clutter IMHO, but with this method, all you gotta do is append the 2>&1 | Write-Output code to each line which runs external commands.

Answer 2

In order to append to a file you'll need to use a slightly different approach. You can still redirect an individual process' standard error and standard output to a file, but in order to append it to a file you'll need to do one of these things:

1. Read the stdout/stderr file contents created by Start-Process
2. Not use Start-Process and use the call operator, &
3. Not use Start-Process and start the process with .NET objects

The first way would look like this:

$myLog = "C:\File.log"
$stdErrLog = "C:\stderr.log"
$stdOutLog = "C:\stdout.log"
Start-Process -File myjob.bat -RedirectStandardOutput $stdOutLog -RedirectStandardError $stdErrLog -wait
Get-Content $stdErrLog, $stdOutLog | Out-File $myLog -Append

The second way would look like this:

& myjob.bat 2>&1 >> C:\MyLog.txt

Or this:

& myjob.bat 2>&1 | Out-File C:\MyLog.txt -Append

The third way:

$pinfo = New-Object System.Diagnostics.ProcessStartInfo
$pinfo.FileName = "myjob.bat"
$pinfo.RedirectStandardError = $true
$pinfo.RedirectStandardOutput = $true
$pinfo.UseShellExecute = $false
$pinfo.Arguments = ""
$p = New-Object System.Diagnostics.Process
$p.StartInfo = $pinfo
$p.Start() | Out-Null
$p.WaitForExit()
$output = $p.StandardOutput.ReadToEnd()
$output += $p.StandardError.ReadToEnd()
$output | Out-File $myLog -Append

Answer 3

Maybe it is not quite as elegant, but the following might also work. I suspect asynchronously this would not be a good solution.

$p = Start-Process myjob.bat -redirectstandardoutput $logtempfile -redirecterroroutput $logtempfile -wait
add-content $logfile (get-content $logtempfile)

Answer 4

Like Unix shells, PowerShell supports > redirects with most of the variations known from Unix, including 2>&1 (though weirdly, order doesn't matter - 2>&1 > file works just like the normal > file 2>&1).

Like most modern Unix shells, PowerShell also has a shortcut for redirecting both standard error and standard output to the same device, though unlike other redirection shortcuts that follow pretty much the Unix convention, the capture all shortcut uses a new sigil and is written like so: *>.

So your implementation might be:

& myjob.bat *>> $logfile