Tips for Parsing Syslog to Azure Sentinel

Azure Sentinel 3 Minutes

In this blog post, I don’t want to spend a lot of time digging through the specifics of how to setup and configure a Palo Alto device for forwarding rules and parsing, but I do want to share some resources and recent experience to help those that may have difficulties with identifying that data is being ingested correctly and accurately for whatever device it happens to be.

We’ve made it extremely easy to connect data to the Log Analytics workspace for Azure Sentinel through “official” connectors in the product, but there’s still some device-specific configuration necessary that our connectors and connector guidance can’t cover. This was made clear during a recent experience with a customer. The customer had team members that were fully adept at managing and configuring Palo Alto devices, however, not all of the data shown in the device logs were making it to the Palo Alto table in Azure Sentinel. For example, the customer could see that there had been URL activity, but that actual URL in question (and subsequent column) was not available in the results, hence, Azure Sentinel analysts couldn’t report on the offending URL and then also could not generate Analytics Rules to alert on specific occurrences.

This is just one example, but after digging in and understanding the situation, I helped the customer realize that even more data was missing.

Eventually, we determined that the issue boiled down to accurate parsing. The parsing rules for the Syslog Forwarding profile were off just an inch or two, which meant that some of the data couldn’t be parsed correctly and was being skipped. No error. No big red flag indicator that something was wrong. It took comparing the Threat and Traffic log fields from the Palo Alto devices against the existing data in the Log Analytics workspace to determine that things were missing.

So, why were the rules off? Well…and here’s a good reminder…NEVER copy/paste from a web page or a PDF document. Yes…the customer had been copying the rules from a PDF document provided by the device vendor, which led to missing and hidden characters and poorly formatted parsers. There’s a note about this on our Palo Alto connector docs page, but as I said before, we make it extraordinarily easy to connect data to Azure Sentinel that most customers never think they need to check the docs page. I’ve now been told that this guidance will soon be included on the self-help section on the actual connector page to avoid more confusion.

So…

Tip 1

Never copy/paste from a web page or PDF document directly into a rule. Always obtain the cleanest version. If it’s not available from the vendor, you’ll need to invest the time yourself to clean it up. For our Palo Alto example, we were lucky to locate the following on GitHub…

…but, this was after searching for quite a while and a lot of troubleshooting in the meantime.

For Palo Alto, at least, separating each parsing command into single lines helped a heap. Due to copying/pasting, the rules were a jumbled mess. Just separate all the log values onto individual lines. And, while this seems to be a Palo Alto-specific recommendation, it’s not. Ensuring each line of a parsing rule has its own precedent is important to the parsing engine.

Tip 2

Always take the time to compare the vendor’s log field reference resources against what you are actually seeing in the Azure Sentinel data table. Just doing a quick review, you might realize that you’re not getting everything you should be.

For Palo Alto, we compared against both the Threat and Traffic log tables and determined that a lot more was missing that originally thought.

And, unfortunately, you may want to ensure to do this over and over as firmware versions are incremented through updates on the devices, because we also found during this exercise that the parsing requirements sometimes change between firmware versions.

Have any other recommendations, tips, or suggestions for ensuring accurate data for Azure Sentinel? Let me know: @rodtrent

Author

Published