Converting Python win32evtlog objects to xml

Question

I have a app that uses win32evtlog to get and display different events and I would like to limit the display to events of a specific level but win32evtlog doesn't return this. It seems that you can convert an event to XML and then pull this info but I can't work out how you get the event from a loop to XML.

I can get up to the following and use it to display data the LogObject has such as LogObject.TimeGenerated

Log = win32evtlog.OpenEventLog('localhost', 'Application')
while 1:
    LogObjects = winev32tlog.ReadEventLog(Log, win32evtlog.EVENTLOG_BACKWARDS_READ|wine32vtlog.EVENTLOG_SEQUENTIAL_READ, 0)
    if not LogObjects:
        break
    for LogObject in LogObjects:

I tried the convert using

LogObjectXML = win32evtlog.EvtRender(LogObject, 1)

This unfortunately returns

TypeError: The object is not a PyHANDLE object

So I know I need to get some sort of handle object that I can use to point the EvtRender at the correct event but can't work out how I do that.

This question is quite similar to How retrieve from Python win32evtlog rest of info? but the solution there didn't answer the critical bit of how we convert the object to XML.

--== Edited with information about the XML for CristiFati ==--

Below is an example of an Application event where the event message reads:-

Updated Windows Defender status successfully to SECURITY_PRODUCT_STATE_ON.

The XML as per event viewer is as below

- <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
- <System>
  <Provider Name="SecurityCenter" /> 
  <EventID Qualifiers="0">15</EventID> 
  <Level>4</Level> 
  <Task>0</Task> 
  <Keywords>0x80000000000000</Keywords> 
  <TimeCreated SystemTime="2017-05-23T07:36:27.627108000Z" /> 
  <EventRecordID>49419</EventRecordID> 
  <Channel>Application</Channel> 
  <Computer>Name.domain.here</Computer> 
  <Security /> 
  </System>
- <EventData>
  <Data>Windows Defender</Data> 
  <Data>SECURITY_PRODUCT_STATE_ON</Data> 
  </EventData>
  </Event>

Answer 1

ReadEventLog returns PyEventLogRecords (wrapper over [MS.Docs]: _EVENTLOGRECORD structure), while EvtRender expects (you need to work with) PyHANDLEs (PyEVT_HANDLEs (wrapper over EVT_HANDLE ([MS.Docs]: Windows Event Log Data Types) to be more precise)).
So, for getting XML data, you need to use the functions family that works with this type: e.g. EvtQuery, EvtNext.

code.py:

#!/usr/bin/env python3

import sys
import pywintypes
import win32evtlog

INFINITE = 0xFFFFFFFF
EVTLOG_READ_BUF_LEN_MAX = 0x7FFFF


def get_record_data(eventlog_record):
    ret = dict()
    for key in dir(eventlog_record):
        if 'A' < key[0] < 'Z':
            ret[key] = getattr(eventlog_record, key)
    return ret


def get_eventlogs(source_name="Application", buf_size=EVTLOG_READ_BUF_LEN_MAX, backwards=True):
    ret = list()
    evt_log = win32evtlog.OpenEventLog(None, source_name)
    read_flags = win32evtlog.EVENTLOG_SEQUENTIAL_READ
    if backwards:
        read_flags |= win32evtlog.EVENTLOG_BACKWARDS_READ
    else:
        read_flags |= win32evtlog.EVENTLOG_FORWARDS_READ
    offset = 0
    eventlog_records = win32evtlog.ReadEventLog(evt_log, read_flags, offset, buf_size)
    while eventlog_records:
        ret.extend(eventlog_records)
        offset += len(eventlog_records)
        eventlog_records = win32evtlog.ReadEventLog(evt_log, read_flags, offset, buf_size)
    win32evtlog.CloseEventLog(evt_log)
    return ret


def get_events_xmls(channel_name="Application", events_batch_num=100, backwards=True):
    ret = list()
    flags = win32evtlog.EvtQueryChannelPath
    if backwards:
        flags |= win32evtlog.EvtQueryReverseDirection
    try:
        query_results = win32evtlog.EvtQuery(channel_name, flags, None, None)
    except pywintypes.error as e:
        print(e)
        return ret
    events = win32evtlog.EvtNext(query_results, events_batch_num, INFINITE, 0)
    while events:
        for event in events:
            ret.append(win32evtlog.EvtRender(event, win32evtlog.EvtRenderEventXml))
        events = win32evtlog.EvtNext(query_results, events_batch_num, INFINITE, 0)
    return ret


def main():
    import sys, os
    from collections import OrderedDict
    standard_log_names = ["Application", "System", "Security"]
    source_channel_dict = OrderedDict()

    for item in standard_log_names:
        source_channel_dict[item] = item

    for item in ["Windows Powershell"]: # !!! This works on my machine (96 events)
        source_channel_dict[item] = item

    for source, channel in source_channel_dict.items():
        print(source, channel)
        logs = get_eventlogs(source_name=source)
        xmls = get_events_xmls(channel_name=channel)
        #print("\n", get_record_data(logs[0]))
        #print(xmls[0])
        #print("\n", get_record_data(logs[-1]))
        #print(xmls[-1])
        print(len(logs))
        print(len(xmls))

if __name__ == "__main__":
    print("Python {:s} on {:s}\n".format(sys.version, sys.platform))
    main()

Notes:

• The 2 lists should have the same length. The n^th entry in each of them should reference the same event (as long as both functions are called with same value for backwards argument (read below))
• get_events_xmls:
  • Returns a list of XML blobs associated to the events
  • The error handling is not the best, you could wrap all API calls in try / except clauses (I didn't run into errors, so I'm not sure what are the situations where exception could be raised)
  • You can play a little bit with [MS.Docs]: EvtNext function's arguments (Timeout and EventsSize for performance fine tuning; for me, ~20k events were processed in a matter of <10 seconds - out of which text printing and conversions took the most)
  • In Python 3, the XMLs are bytes ([Python 3.Docs]: Built-in Types - class bytes([source[, encoding[, errors]]])) rather than normal strings (I had to encode them because some contain some non-ASCII chars, and attempting to print them would raise UnicodeEncodeError)
  • Event filtering is possible, check [MS.Docs]: EvtQuery function's args (Flags and Query)
  • Note the backwards argument which allows traversing the events in reversed (chronological) order (default set to True).
• get_record_data:
  • It's just a convenience function, it converts a PyEventLogRecord object into a Python dictionary
  • The conversion is based on the fact that fields that we care about start with a capital letter (EventID, ComputerName, TimeGenerated, ...), that's why it shouldn't be used in production
  • It doesn't convert the actual values (TimeGenerated's value is pywintypes.datetime(2017, 3, 11, 3, 46, 47))
• get_eventlogs:
  • Returns a list of PyEventLogRecords
  • As in get_events_xmls's case note the backwards argument
  • I must insist on buf_size. As [MS.Docs]: ReadEventLogW function states, when getting the events, a buffer of max 512K can be used. Now (starting with PyWin32 version 220), it's possible to pass it as an argument (the last one) to win32evtlog.ReadEventLog. Check [SourceForge.hg]: mhammond/pywin32 - Add buffer size parameter for ReadEventLog (patch #143 from cristi fati) for more details. By default, there was a limitation so that the buffer size was hardcoded to 1K. Since every ReadEventLog was accessing the disk, with the new buffer size I got a 10X speed improvement (for ~180K events)
• Since I'm storing all the data in the 2 lists (instead of inplace data processing), I am choosing speed over memory consumption. For ~20K events, the 2 lists are taking ~30MB of RAM (which nowadays I think it's decent enough)

@EDIT0: I couldn't find a way to get all the required info with the Evt* functions family, so I'm getting it from both sources (I enhanced the script that I've previously posted):

@EDIT1: According to [MS.Docs]: OpenEventLogW function:

If you specify a custom log and it cannot be found, the event logging service opens the Application log; however, there will be no associated message or category string file.

[MS.Docs]: Eventlog Key lists the 3 standard ones. So, that's why it opens the Application log. I've done some small changes to the script to test the sources. I don't know where mmc gets the Setup events from.