{var%20f='http://v.t.sina.com.cn/share/share.php?appkey=1515056452',u=z||d.location,p=['&url=',e(u),'&title=',e(t||d.title),'&source=',e(r),'&sourceUrl=',e(l),'&content=',c||'gb2312','&pic=',e(p||'')].join('');function%20a(){if(!window.open([f,p].join(''),'mb',['toolbar=0,status=0,resizable=1,width=440,height=430,left=',(s.width-440)/2,',top=',(s.height-430)/2].join('')))u.href=[f,p].join('');};if(/Firefox/.test(navigator.userAgent))setTimeout(a,0);else%20a();})(screen,document,encodeURIComponent,'','','https://www.manongdao.com/data/attach/logo/logo.png', '推荐 无色无味的生活 的问题《Parallel execution of shell processes》','https://www.manongdao.com/q-9379.html','页面编码gb2312|utf-8默认gb2312'));){kind=link}
Is there a tool available to execute several process in parallel in a Windows batch file? I have found some interesting tools for Linux (parallel and PPSS), however, I would need a tool for Windows platforms.
Bonus: It would be great if the tool also allowed to distribute processes in an easy way among several machines, running the processes remotely a la PsExec.
Example: I would like that in the following for loop
for %F in (*.*) do processFile.exe %F
a limited amount of instances of processFile.exe are running in parallel to take advantage of multi-core CPUs.
Sounds more like you want to use Powershell 2. However, you can spawn new
cmdwindows (or other processes) by usingstart, see also this answer. Although you probably have to use some other tools and a little trickery to create something like a "process pool" (to have only a maximum of n instances running at a time). You could achieve the latter by usingtasklist /imand counting how many are already there (forloop orwc, if applicable) and simply wait (ping -n 2 ::1 >nul 2>&1) and re-check again whether you can spawn a new process.I have cobbled together a little test batch for this:
It spawns a maximum of four new processes that execute in parallel and minimized. Wait time needs to be adjusted probably, depending on how much each process does and how long it is running. You probably also need to adjust the process name for which tasklist is looking if you're doing something else.
There is no way to properly count the processes that are spawned by this batch, though. One way would be to create a random number at the start of the batch (
%RANDOM%) and create a helper batch that does the processing (or spawns the processing program) but which can set its window title to a parameter:This would be a simple batch that sets its title to the first parameter and then runs the second parameter with the third as argument. You can then filter in tasklist by selecting only processes with the specified window title (
tasklist /fi "windowtitle eq ..."). This should work fairly reliable and prevents too many false positives. Searching forcmd.exewould be a bad idea if you still have some instances running, as that limits your pool of worker processes.You can use
%NUMBER_OF_PROCESSORS%to create a sensible default of how many instances to spawn.You can also easily adapt this to use
psexecto spawn the processes remotely (but wouldn't be very viable as you have to have admin privileges on the other machine as well as provide the password in the batch). You would have to use process names for filtering then, though.Edit - I modified the script to optionally display the output of each process
Here is a native batch solution that reliably runs a list of commands in parallel, never launching more than n processes at a time.
It even has a mechanism built in to distribute the processes to specific CPUs or remote machines via PSEXEC, but I haven't tested that feature.
The trick to make this work is to START each command through a CMD process that redirects either stdout or an undefined handle to a lock file. The process will maintain an exclusive lock on the file until it terminates. It doesn't matter how the process terminates (normal exit, crash, killed process), the lock will be released as soon as it does.
The master script can test if the process is still active by attempting to redirect to the same lock file. The redirection will fail if the process is still active, succeed if it has terminated.
By default, the script ignores the output of each process. If started with the
/Ooption as the 1st parameter, then it displays the output of each process, without interleaving.My demo sets the process limit to 4, and simply runs a series of PING commands of varying length.
I've tested this on XP, Vista, and Windows 7.
Here is output from a sample run that ignores process output
Here is the output if run with the
/Ooption showing process outputThere is a basic Windows xargs-like-clone which does support the -P parallel processing option at http://www.pirosa.co.uk/demo/wxargs/wxargs.html
Try
start:It opens a new window with the given title and executes the BAT, CMD or EXE file. You can also set the priority, set the same environment etc.
Files being not executeable are opened with the associated program.
Further reading: Start -> Run
Start is available at least since WinME.
Good luck!
GNU xargs under Linux has a "-P n" switch to launch "n" processes in parallel.
Maybe cygwin/mingw build of xargs also supports this?
Then you can use:
No fancy multi-node process spawning, though.