Using log parser to parse lot of logs in different

Question

I recently started to use Log Parser with visual interface.

The logs that I want to parse come from IIS, and they are related to SharePoint. For example, I want to know how many people were visiting particular web pages, etc.

And it seems that IIS creates logs in different folders (I don't know why) and every day there is a new log file in a different folder.

So my question is, is it possible to approach all those files in different folders?

I know you can use From-clause, put different folders, but it is too difficult especially if in the future new folders are added. The goal is to create one script which would be executed.

So for example in a folder log named LogFIles, I have folders folder1, folder2, folder3, folder4, etc. and in each folder there are log files log1, log2, log 3, logN, etc.

So my query should be like this: Select * FROM path/LogFiles/*/*.log but the log parser doesn't accept it, so how to realize it?

Answer 1

You can use the -recurse option when calling logparser.

For example:

logparser file:"query.sql" -i:IISW3C -o:CSV -recurse

where query.sql contains:

select *
from .\Logs\*.log

and in my current directory, there is a directory called "Logs" that contains multiple sub-directories, each containing log files. Such as:

\Logs\server1\W3SVC1
\Logs\server1\W3SVC2 
\Logs\server2\W3SVC1 
\Logs\server2\W3SVC2 
etc.

Answer 2

LogParser's help does not list the -recurse option so I'm not sure it it's still supported. However, this is what I did to get around it:

Let's say you use the following command to execute logparser -

logparser "SELECT * INTO test.csv FROM 'D:\samplelog\test.log'" -i:COM -iProgID:Sample.LogParser.Scriptlet -o:CSV

Then simply create a batch script to "recurse" through the folder structure and parse all files in it. The batch script that does this looks like this -

echo off
for /r %%a in (*) do (
    for %%F in ("%%a") do (
        logparser "SELECT * INTO '%%~nxF'.csv FROM '%%a'" -i:COM -iProgID:Sample.LogParser.Scriptlet
        REM echo %%~nxF
    )
)

Execute it from the path where the log files that need to be parsed are located. This can further be customized to spit all parsed logs in one file using append (>>) operator.

Hope this helps.

Answer 3

You can merge the logs then query the merged log

what i have to do is

LogParser.exe -i:w3c "select * into E:\logs\merged.log from E:\logs\WEB35\*, E:\logs\WEB36\*, E:\logs\WEB37\*" -o:w3c

Answer 4

I prefer powershell like this:

Select-String C:\Logs\diag\*.log -pattern "/sites/Very" | ?{$_.Line -match "Important"}

Answer 5

Check this out: https://stackoverflow.com/a/31024196/4502867 by using powershell to recursively get file items in sub directories and parse them.