How to troubleshoot ePolicy Orchestrator event and report content


Environment

McAfee ePolicy Orchestrator (ePO) 5.x

Summary

This article describes the path that events take from a managed product (client) to the ePO database. You can use these steps to troubleshoot event issues for incorrect and unexpected event query results.

The following describes the path from creation of the event by the managed product to the database:
 
Managed product --> AgentEvents folder or MA database --> agent-to-server communication interval --> ePO Server Service --> ePO Events folder --> ePO Events Parser Service --> database or syslog server

Events are generated by the managed products, such as Endpoint Security (ENS), and passed to the McAfee Agent. Some products that use a legacy plug-in write events to the AgentEvents folder. Newer products, such as McAfee Active Response, Data Loss Prevention 11.4 and later, use msgbus instead of the legacy LPC plug-in. When these newer msgbus products generate an event, the event is stored in the Agent database instead of the AgentEvents folder. The events are then uploaded to the Agent Handler (Apache service), either immediately or at the next agent-to-server communication interval (ASCI). The Agent Handler saves them to the Events folder on the server. The EventParser Service processes the events, writing them to the database. If configured, EventParser service forwards the event to a Syslog Server after the Apache service receives the event.

See the ePolicy Orchestrator Product Guide for instructions about how to configure a Registered syslog server:

ePO 5.10:
https://docs.mcafee.com/bundle/epolicy-orchestrator-5.10.0-interface-reference-guide/page/GUID-092B5092-305D-400D-BE22-8973139B2B7B.html

ePO 5.9:
https://docs.mcafee.com/bundle/epolicy-orchestrator-5.9.x-product-guide/page/GUID-5C5332B3-837A-4DDA-BE5C-1513A230D90A.html

Solution

The following contains specific information about the path an event takes. Becoming familiar with the specific details helps you understand where a potential problem might be occurring.

Event Filtering
Events are generated via either the managed products, such as ENS or McAfee Agent. The configuration within the Event Filtering settings determines the following:
  • Whether the event is generated on the endpoint
    And
  • If the event is also forwarded to a registered Syslog Server
Event Filtering is a single global configuration that affects all agents, regardless of managed product or platform. Use the following steps within the ePO console and enable or disable events. Or, send the event to SIEM (send to a syslog server).
  1. Log on to the ePO console.
  2. Click Menu, Configuration, Server Settings.
  3. Click Event Filtering under the Setting Categories column, and click Edit.
  4. Select or deselect the Event IDs, as needed. Or, choose to store only in ePO, in SIEM, or both, then click Save.
NOTE: For a list of McAfee managed product-generate Event IDs listed in ePO, see KB54677.

After a change has been made to the list of enabled or disabled events, this change is saved on the ePO Server. The clients then downloaded it via a file named EvtFiltr.ini. The EvtFiltr.ini file contains a list of all events that have been disabled within the Event Filtering configuration in ePO. If the copy of EvtFiltr.ini file stored on the client does not contain a specific event ID, then this event can be generated on the client.

The EvtFiltr.ini file is located under the McAfee Agent Data directory. That location differs depending on the operating system:
  • For Windows: C:\ProgramData\McAfee\Agent\
  • For Linux and macOS: /var/McAfee/agent/scratch
Generate an event
Events are generated via one of the following methods:
  • A managed product, such as ENS, by calling an MA dll (ma_event_service.dll).
    Or
  • Products that use the older legacy LPC plug-in, store events in the AgentEvents folder.
With newer products, such as MAR, EDR, DLP 11.4 and later, the events are stored in the MA database (..\ProgramData\McAfee\Agent\DB). If the event is low priority, it is held in the agent events folder, and is uploaded to the ePO server at the next ASCI.
  • If the event is high priority, it is uploaded immediately. (Priority event forwarding configured under MA General policy>Events tab).

    NOTE: The agent events folder is located under the agent directory structure, which differs depending on operating system:
     
    • For Windows: C:\ProgramData\McAfee\Agent\AgentEvents
    • For Linux, macOS:  /var/McAfee/agent/AgentEvents

Troubleshooting

ENDPOINT
First verify that the event is being generated on the endpoint in either of the following:
  • AgentEvents folder
  • MA database, if the product generating the event is a newer msgbus product.
  1. To make it easier to capture the event on the endpoint, temporarily prevent the agent from uploading the events to the ePO server. Prevent the upload by doing one of the following.
    NOTE: Usually, there is plenty of time to capture a copy of the event on the endpoint without doing one of these things.
     
    • Stop the McAfee Agent service on the client.
    • Stop the ePolicy Orchestrator Server service on the ePO server and Agent Handler (Apache service).
    • Disable the network adapter on the client system
NOTE: VSE/ENS Access Protection, or MA Self-Protect, might prevent you from stopping MA services.
 
  1. Reproduce the action and generate the specific event or a generic detection event on the system. See the following:
    • How to use the EICAR antimalware test file with McAfee products (KB59742)
    • For SPAM products, use a similar test file called GTUBE. Email it from an external email address to your company address so it goes through the SPAM scanner.
      The GTUBE test file is available from: http://spamassassin.apache.org/gtube/.
    • Look in the AgentEvents folder for randomly named files that are waiting to be uploaded to the ePO server. 
    • For products such as MAR, EDR, DXL, DLP 11.4 or later, which use msgbus, you need to collect MER results from the client. Or, collect the MA \db and \keystore files, and use the MER Analyzer to view the event.
    • Open the event to review the properties of the event. Most events are typical xml files. You can open these files and view the details of the event using a browser or text editor such as Notepad++.
       
  2. Re-enable the connection to the ePO server using one of the following methods (depending on the action taken in Step 1):
    • Start the McAfee Agent service on the client.
    • Start the Agent Handler service.
    • Enable the network adapter on the client.
       
  3. Normal priority events are uploaded during the normal Agent to Server communication interval. You can upload these immediately using the Send Events button on the McAfee Agent Status Monitor.
     

SERVER (Agent Handler on ePO server, or other server)

Receiving the event:
After the McAfee Agent has uploaded the event, the Apache service receives it. It is then handled on either the ePO server, or on a Remote Agent Handler. The handler then stores the event in the Events folder, and we see logging similar to the following in the Server_.log:

I #10244 NAIMSERV Received [Event] from <name of client system> :{75BCADA0-16B4-11EA-27C6-005056014A0F}
I #10244 NAIMSERV Processed [Event] from <name of client system>:{75BCADA0-16B4-11EA-27C6-005056014A0F} in 0ms


Events folder location: <ePO or Agent Handler installation folder>\DB\Events


Parsing or Forwarding the event:
The eventparser_.log file has a similar log activity when the event is successfully parsed to the database:
 
#07008 EVNTPRSR Succeeded , C:\PROGRA~2\McAfee\EPOLIC~1\DB\Events\0b8af8eb-1340-470b-8d30-77cfde8bac61-mc_20200402175859651368200001688.txml.

If configured, the event might be forwarded to the syslog receiver. In this case, if LogLevel 8 (see KB56207) is enabled on the ePO server or remote Agent Handler, we see activity similar to the following in the eventparser_.log file:

#07008 EVNTPRSR source\SyslogForwarder.cpp(371): Found cached work item data for tenant 1: bpsid=1, guid={00000000-0000-0000-0000-000000000000}, nodepath=1\2
#07008 EVNTPRSR source\SyslogForwarder.cpp(376): Construct new work item for tenant 1: bpsid=1, guid={00000000-0000-0000-0000-000000000000}, nodepath=1\2
#07008 MFEFIPS mfefips_SSLSubSys.cpp(230): Using cached connection for :6514



For threat type events (Malware detected, Access Protection rule triggered), we see the event show up in the Threat Event Log. It appears under the Reporting menu in the ePO console. Any event that has been successfully parsed can be queried using an Events query under Queries and Reports.


Troubleshooting event flow:
  1. Event is not generated on the endpoint:
    1. Verify that the event is not listed as one of the disabled events in the EvtFiltr.ini file.
    2. Attempt to reproduce the action that can generate the event, and look in the masvc_.log file for errors.
    3. Verify the managed product that generates the event is running.
       
  2.  Event is not sent to the ePO Server or Agent Handler:
    1. Look in the masvc_.log file for errors just after an attempt to communicate to the handler or send the event.
    2. Verify that the Apache service is running on the ePO server or Agent Handler.
       
  3. Event is not visible in ePO after verifying it was successfully uploaded from the endpoint.
    1. Verify if the client is communicating to the ePO server or a remote Agent Handler.
    2. Look in the eventparser_.log file for errors that might have occurred just after the client forwarded the event.
    3. If there are errors found in the eventparser log file, try to obtain a copy of the event. Look to see if it can be successfully parsed on a lab ePO server.
    4. Verify that the latest extension is installed for the managed product that generated the event.
    5. Look to see if there is a separate Reporting extension for the managed product. Install the extension if applicable.
       
  4. The Syslog Server does not receive the event:
    1. Verify if the client is communicating to the ePO Server or a remote Agent Handler.
    2. Look in the eventparser_.log file for errors that might have occurred just after the event was forwarded from the client.
    3. In the ePO console, edit the Registered Syslog Server, and verify that the Test connection option under the Details tab is successful

 
Additional information about events that are not parsed by EventParser-

Event plug-ins:
Events that are generated via VSE, HIPs, or DLP, are parsed via an event parser plug-in. The plug-in is installed when you install the extension for that managed product. These plug-ins are responsible for writing the event to the database and are installed within the ePO or Agent Handler installation folder under .\DB\Plugin folder. 

NOTE: Some managed products contain a separate Reporting extension that contains this plug-in. For example, VirusScan Enterprise has a management extension and a reporting extension. The event parsing plug-in is added only via the reporting extension.

When the EventParser Server tries to process an event, it looks to see if the required event parser plug-in exists for this event under the \DB\Plugin\ folder. If not, it tries to download it from the database according to the information contained in the EPORegisteredEventPlugins table. When downloaded, this event parser plug-in can then be called to parse the event.


Unknown Events:
ePO does not know how to parse the event if ePO receives a butL event.
  • No event parser plug-in is available for this event under the \DB\Plugin folder
    Or 
  • Exists within the EPORegisteredEventPlugins table 

In the above situation, it is considered an Unknown event. When this event happens, we see log activity like this in the eventparser_.log file:

W #02796 EVNTPRSR Skipping virus_detection_event no plugin available.

By default, all unknown events are deleted. But, it is possible to configure ePO, or a remote handler, so that it stores these events in a temporary folder.

To keep Unknown Events, follow these steps on the ePO server and any remote Agent Handlers:

CAUTION: This article contains information about opening or modifying the registry.
  • The following information is intended for System Administrators. Registry modifications are irreversible and could cause system failure if done incorrectly.
  • Before proceeding, Technical Support strongly recommends that you back up your registry and understand the restore process. For more information, see the Microsoft Windows registry information for advanced users article.
  • Do not run a REG file that is not confirmed to be a genuine registry import file.
  1. To open the Windows registry, press Windows+R, type regedit.exe, and click OK.
  2. Navigate to the following key:

    HKEY_LOCAL_MACHINE\SOFTWARE\Network Associates\ePolicy Orchestrator\EventParser\
     
  3. Add the [DWORD] DeleteUnknownEvents. If not already present.
  4. Set the value to 0. Default value is 1.
  5. Close the registry editor.
  6. Restart the EventParser service.
Once value is set, all unknown events go to the \DB\Events\Unknown folder.

Failed Events:
Anytime the EventParser fails to parse an event, it goes into the \DB\Events\Debug folder. 

NOTE: The above does not include events that the EventParser skips, as for Unknown events mentioned above.

These events remain in the Debug folder, unless they are deleted, or moved. You can try to reparse these events by moving them from the Debug folder back to the root of the Events folder.

 
 

Previous Document ID

615924

Glossary of Technical Terms

 Highlight Glossary Terms

Please take a moment to browse our Glossary of Technical Terms.