Powershell - Process Redirect - Polling

Hi

I found a number of articles on running processes and BAT files and capturing the output and errors, e.g.

http://www.chinabtp.com/captured-output-of-command-run-by-powershell-is-sometimes-incomplete/
http://stackoverflow.com/questions/24370814/how-to-capture-process-output-asynchronously-in-powershell

So I created a test function below and it "works"
However I wanted to have control over the display of the output, i.e. close to real time. According to the doco the event gets fired for each newline, my tests indicate otherwise.
i.e. the script may run for a long time and have 1000s of lines of output..

I did a test where I told the BAT file to sleep. However the results where not as expected.
There was a delay until the first output occurred, then a further delay which could be accounted for due to the sleep.

https://msdn.microsoft.com/en-us/library/microsoft.powershell.utility.activities.registerobjectevent_methods(v=vs.85).aspx
https://msdn.microsoft.com/en-us/library/microsoft.powershell.utility.activities.registerobjectevent_members(v=vs.85).aspx

Don't help because there are poorly documented, e.g. MaxtriggerCount, provides access to maxtriggercount, none of the items are actually described by function.

The C# process has ReadToEnd() but this does not apply to Powershell

So the question :

>> Is there a way for me fetch the output event data, say in the 1s while loop ?

SBS_Logger justs adds the timestamp

Function SBS_RunAProcess
{
[CmdletBinding()]
 param(
 		[Parameter(Mandatory = $True)] 
       [String]$Label )
	   
		# Setup stdin\stdout redirection
		$StartInfo = New-Object System.Diagnostics.ProcessStartInfo -Property @{
			FileName = "F:\ScriptWork\helloworld.bat"
			Arguments = $Label
			UseShellExecute = $false
			RedirectStandardOutput = $true
			RedirectStandardError = $true
		}
		# Create new process
		$Process = New-Object System.Diagnostics.Process
		# Assign previously created StartInfo properties
		$Process.StartInfo = $StartInfo
		# Register Object Events for stdin\stdout reading
		$OutEvent = Register-ObjectEvent -InputObject $Process -EventName OutputDataReceived -Action {

		SBS_Logger $Event.SourceEventArgs.Data
		}
		$ErrEvent = Register-ObjectEvent -InputObject $Process -EventName ErrorDataReceived -Action {

		SBS_Logger $Event.SourceEventArgs.Data
		}
		# Start process
		[void]$Process.Start()
		# Begin reading stdin\stdout
		$Process.BeginOutputReadLine()
		$Process.BeginErrorReadLine()
		# Do something else while events are firing
		do
		{
			#Write-Host 'Still alive!' -ForegroundColor Green
			Start-Sleep -Seconds 1
		}
		while (!$Process.HasExited)
		# Unregister events
		$OutEvent.Name, $ErrEvent.Name |
		ForEach-Object {Unregister-Event -SourceIdentifier $_}
 }
BAT Content

@echo off
echo "hello world from a BAT file"
echo "args where %1%"
rem timeout /t 10 /nobreak > NUL
SET

===

2015-09-03 14:04:00 Task test_bat running
2015-09-03 14:04:01 "hello world from a BAT file"
2015-09-03 14:04:01 "args where greg"
2015-09-03 14:04:01 ALLUSERSPROFILE=C:\ProgramData
2015-09-03 14:04:01 APPDATA=C:\Users\build\AppData\Roaming


=====================
BAT Content

echo off
echo "hello world from a BAT file"
echo "args where %1%"
timeout /t 10 /nobreak > NUL
SET

===

2015-09-03 14:08:43 Task test_bat running
2015-09-03 14:08:54 "hello world from a BAT file"  <= 9s delay
2015-09-03 14:08:54 "args where greg"
2015-09-03 14:09:03 ALLUSERSPROFILE=C:\ProgramData <= 9s delay



September 3rd, 2015 1:11am

The event data is not "fetched" it only exists when the event is triggered.

It is very difficult to understand what you are trying to do .  Can you ask a specific question without so much discussion. Just state what it is you need to do.

If you output the data it will list at the console. It appears you are using Write-Host. That will throw assay all data after it is displayed.

Free Windows Admin Tool Kit Click here and download it now
September 3rd, 2015 1:34am

phew

I really tried, it I did too little then I would be asked for more context and examples. We are on different wavelengths

Existing examples show capturing process output

I did this and found that quirks, i.e. for each line of BAT file output, and event was not triggered

Rather it happened later that expected.

>> Is this approach wrong, and I need some ReadtoEnd() approach from the output ?

or is the issue that the cmd.exe itself is only firing output events when it thinks it should, not every line ?

Thanks

September 3rd, 2015 1:40am

There is nothing to read.  The text is being sent to you in the event data.  The events are fired when the output of a program is available.

Change this as it will be more reliable:

		$StartInfo = New-Object System.Diagnostics.ProcessStartInfo -Property @{
			FileName = $env:comspec
			Arguments = @('F:\ScriptWork\helloworld.bat', $Label)
			UseShellExecute = $false
			RedirectStandardOutput = $true
			RedirectStandardError = $true
		}

Free Windows Admin Tool Kit Click here and download it now
September 3rd, 2015 2:40am

Thanks for this tip,

I modified my test bat and its contents are below

The conclusion is although the BAT "timeout" does not affect output when run manually, it does when run from powershell.  This would suggest there isn't much we can do about this and the code is not at fault.

=====

- Running it interactively the first 3 lines are printed straight away, then 10s later the env. variables are printed, then 10s later the end

- When running the same BAT via powershell

2015-09-03 16:54:15 Task test_bat running (from powershell)
2015-09-03 16:54:36 "hello world from a BAT file"
2015-09-03 16:54:36 "args where greg"
2015-09-03 16:54:36 "I am going to sleep for 10s"
2015-09-03 16:54:46 ALLUSERSPROFILE=C:\ProgramData
....
2015-09-03 17:02:23 SystemRoot=C:\WINDOWS
2015-09-03 17:02:23 "Another 10s"
2015-09-03 17:02:23 "Done"

- Now running the same script via powershell with the pause removed

2015-09-03 17:06:02 Task test_bat running
2015-09-03 17:06:03 "hello world from a BAT file"
2015-09-03 17:06:03 "args where greg"
2015-09-03 17:06:03 ALLUSERSPROFILE=C:\ProgramData

======================

@echo off
echo "hello world from a BAT file"
echo "args where %1%"
echo "I am going to sleep for 10s"
timeout /t 10 /nobreak > NUL
SET
echo "Another 10s"
timeout /t 10 /nobreak > NUL
echo "Done"

September 3rd, 2015 3:16am

Thanks for this tip,

I modified my test BAT and its contents are below
While one lot of doc said the call back occues after each line, my tests indicate this is not always the way.

The conclusion is although the BAT "timeout" does not affect output when run manually, it does when run from powershell.  This would suggest there isn't much we can do about this and the code is not at fault.

=====

- Double clicking on the BAT file interactively the first 3 lines are printed straight away, then 10s later the env. variables are printed, then 10s later the end, e.g.

 "hello world from a BAT file"
 "args where greg"
(10s delay before SET info comes out)
 ALLUSERSPROFILE=C:\ProgramData

- When running the same BAT via powershell capturing the output

2015-09-03 16:54:15 Task test_bat running (from powershell)(delay)
2015-09-03 16:54:36 "hello world from a BAT file"
2015-09-03 16:54:36 "args where greg"
2015-09-03 16:54:36 "I am going to sleep for 10s"
(delay expected)
2015-09-03 16:54:46 ALLUSERSPROFILE=C:\ProgramData
....
2015-09-03 17:02:23 SystemRoot=C:\WINDOWS
2015-09-03 17:02:23 "Another 10s"
(delay expected but not in results)
2015-09-03 17:02:23 "Done"

- Now running the same script via powershell with the "timeout" removed (ran time)

2015-09-03 17:06:02 Task test_bat running
2015-09-03 17:06:03 "hello world from a BAT file"
2015-09-03 17:06:03 "args where greg"
2015-09-03 17:06:03 ALLUSERSPROFILE=C:\ProgramData

======================

This is a BAT file contents

@echo off
echo "hello world from a BAT file"
echo "args where %1%"
echo "I am going to sleep for 10s"
timeout /t 10 /nobreak > NUL  <= this is where a delay can be added
SET
echo "Another 10s"
timeout /t 10 /nobreak > NUL
echo "Done"


Free Windows Admin Tool Kit Click here and download it now
September 3rd, 2015 7:10am

Thanks for this tip,

I modified my test BAT and its contents are below
While one lot of doc said the call back occues after each line, my tests indicate this is not always the way.

The conclusion is although the BAT "timeout" does not affect output when run manually, it does when run from powershell.  This would suggest there isn't much we can do about this and the code is not at fault.

=====

- Double clicking on the BAT file interactively the first 3 lines are printed straight away, then 10s later the env. variables are printed, then 10s later the end, e.g.

 "hello world from a BAT file"
 "args where greg"
(10s delay before SET info comes out)
 ALLUSERSPROFILE=C:\ProgramData

- When running the same BAT via powershell capturing the output

2015-09-03 16:54:15 Task test_bat running (from powershell)(delay)
2015-09-03 16:54:36 "hello world from a BAT file"
2015-09-03 16:54:36 "args where greg"
2015-09-03 16:54:36 "I am going to sleep for 10s"
(delay expected)
2015-09-03 16:54:46 ALLUSERSPROFILE=C:\ProgramData
....
2015-09-03 17:02:23 SystemRoot=C:\WINDOWS
2015-09-03 17:02:23 "Another 10s"
(delay expected but not in results)
2015-09-03 17:02:23 "Done"

- Now running the same script via powershell with the "timeout" removed (ran time)

2015-09-03 17:06:02 Task test_bat running
2015-09-03 17:06:03 "hello world from a BAT file"
2015-09-03 17:06:03 "args where greg"
2015-09-03 17:06:03 ALLUSERSPROFILE=C:\ProgramData

======================

This is a BAT file contents

@echo off
echo "hello world from a BAT file"
echo "args where %1%"
echo "I am going to sleep for 10s"
timeout /t 10 /nobreak > NUL  <= this is where a delay can be added
SET
echo "Another 10s"
timeout /t 10 /nobreak > NUL
echo "Done"


September 3rd, 2015 7:10am

I am having difficulty understanding your specific question.

What, exactly and specifically, do you want to do in PowerShell? What is the purpose/goal?

Also post a short example script that "doesn't work" and explain specifically how it doesn't work. Remember, we don't know what you are trying to do, and we can't see your screen.

Free Windows Admin Tool Kit Click here and download it now
September 3rd, 2015 11:13am

I updated by previous comment to try to make it clearer.

The script "does work" i.e. all output is captured

What doesn't work is this capture is no where near realtime.

I'll place some sample files in onedrive soon.

In onedrive http://1drv.ms/1EDJPQf

I added some test scripts

I expanded the while check to be

}

while(!$Process.HasExited -or$OutEvent.HasMoreData -or$ErrEvent.HasMoreData)

What I found what that a short time after the process ends, the events events stop even though they still have events in them

I have not found how to flush the events

Thanks

September 3rd, 2015 8:37pm

You have a sleep. It will delay the output.

What is the purpose of this?

Free Windows Admin Tool Kit Click here and download it now
September 3rd, 2015 11:31pm

I found this in several scripts of the web as an example of this approach

On removing the sleep more lines get truncated from the results of the BAT

Removing this completely and using

$Process.WaitForExit()

gets a few more lines but some end lines are missing

Thanks



September 3rd, 2015 11:39pm

I updated by previous comment to try to make it clearer.

The script "does work" i.e. all output is captured

What doesn't work is this capture is no where near realtime.

I'll place some sample files in onedrive soon.

In onedrive http://1drv.ms/1EDJPQf

I added some test scripts

I expanded the while check to be

}

while(!$Process.HasExited -or$OutEvent.HasMoreData -or$ErrEvent.HasMoreData)

What I found what that a short time after the process ends, the events events stop even though they still have events in them

I have not found how to flush the events

Thanks

Free Windows Admin Tool Kit Click here and download it now
September 4th, 2015 12:33am

I found this in several scripts of the web as an example of this approach

On removing the sleep more lines get truncated from the results of the BAT

Removing this completely and using

$Process.WaitForExit()

gets a few more lines but some end lines are missing

Thanks



September 4th, 2015 3:35am

Are you saying you want to output to the console and write to a file at the same time?

You mean like Tee-Object?

Free Windows Admin Tool Kit Click here and download it now
September 4th, 2015 2:23pm

I want to run a process and capture its output

In this example the process is a BAT file.

This allows me to add time stamps and do some adjustments to the output as needed.

NB: I have found in the last few hours that if an output line is empty this upsets the event call back

  $OutEvent = Register-ObjectEvent -InputObject $Process -EventName OutputDataReceived -Action {
  SBS_Logger $Event
.SourceEventArgs.Data
}

and a try - catch resolves this issue around the Logger

thanks

September 4th, 2015 2:32pm

What does the shell script (batch file) do? Why not do everything you need in PowerShell?

Why do you think you need to register a .NET object event?

what is the overall purpose/design goal of this script? (Tell what you want to do, not how you think you need to do it.)

Free Windows Admin Tool Kit Click here and download it now
September 4th, 2015 2:35pm

Hi Bill

Background:

I am developing a framework for running our extensive build system under TFS Build. This extensive system comprises many BAT files driving msbuild and mstest plus doing BAT operations.

Over time we will move most of the build content into powershell.

Now there are several ways to attack this, but my probe at the moment was running the BAT files from powershell. This has the advantage that if output is captured then some enhancements can be done to the output, e.g. time stamps. As well as issue tracking reporting.

So I googled and came across numerous references to ioredirect and Register-ObjectEvent Then I found quirks in this process.

My status now is I have found that after the process has exited there still are items in the event queue. I have found that Unregister-Event in fact is the item that will force the last of the traces out. The While loop is not needed and a $Process.WaitForExit() will do the job.

So the intent of this post was to work through how to capture process  redirect successfully. NB: If the BAT files has pauses in it then the capture system becomes undeterministic (see top of post). Possibly related to the BAT input (? peeks in "timeout" to see if a key was pressed)

regards

September 4th, 2015 10:33pm

This is a shortcoming of running an unattended process.    The program (bat file) is terminating before you can grab the buffer.  There is no way to prevent this.  You need to output the information to a file and then monitor the file.

.\MyBatch.bat > mybuild.log
Get-Content mybuild.log -wait -tail 1

You can modify the output and write to a new file if needed.

Free Windows Admin Tool Kit Click here and download it now
September 4th, 2015 11:05pm

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics