We’ve made some significantly important announcements recently around additional cost-cutting measures (and other things) through new log ingestion and log retention opportunities. If you’re not caught up on this yet, please see the following:
- Ingest, Archive, Search, and Restore Data in Microsoft Sentinel
- FAQ: Search, Basic Ingestion, Archive, and Data Restoration
While these resources provide some great information – particularly the additional information included in the FAQ – there continues to be confusion over the Basic Logs option. So, I believe this feature requires some clarification.
Basic Logs can be a definite cost-saving measure, but many customers are attempting to include it in general Microsoft Sentinel planning. Basic Logs has very specific use cases and very specific limitations. Many customers may never need or use this option.
Consider those massive log files like Netflow or Storage services. When you are tasked with looking through those log files, its generally not as part of a Hunting operation. When you need to surface and expose the data in those types of logs, it’s because you have identified that a critical situation already exists, and you need the data from those logs to confirm the suspicious activity and to add additional context to the investigation. Netflow or Storage logs are probably not going to provide the first sign that you have an issue. That will probably come from device or user activities, possibly through monitoring Security events, Office or Defender alerts.
The logs intended for Basic Logs are massive in size. They are not something you would generally want to ingest on a constant basis particularly when the regular Analytics Logs ingestion cost is around $2.00-$2.50 per GB depending on the Azure region being used.
Using Basic Logs for these enormous logs, there’s a clear cost advantage because it cuts the normal price almost in half at $1.00 per GB (50% Log Analytics charge and 50% Sentinel charge). But there’s a catch. Actually, there’s quite a few. With Basic Logs you give up the ability to create Analytics Rules (alerts are not supported), KQL language access is limited and there’s a charge for interactive queries ($0.005/GB-scanned), the supported log types are limited (see the FAQ), and the data from these log files can only be stored as active or “hot” data for 8 days at a time.
For those instances where Basic Logs makes sense, this option is hugely valuable. But Basic Logs should be used as just another tool and a tool that’s employed after-the-fact.
What I mean by that is that when planning a new Microsoft Sentinel deployment or adjusting an existing one to incorporate the new potentially cost-saving measures, focus first on what Microsoft Sentinel already provides – which is already amazing with the low ingestion costs, free 90-day retention for all data, commitment tier savings for ingestion, and no cost for queries with full access to KQL – and also maybe incorporate the new Archived Logs feature to store data longer than the free 90 days for compliance and audit purposes.
Standing up and using Microsoft Sentinel has always been and always should be easy. Keep it that way.
[Subscribe to the RSS feed for this blog]
[Subscribe to the Weekly Microsoft Sentinel Newsletter]
[Subscribe to the Weekly Microsoft Defender Newsletter]
[Learn KQL with the Must Learn KQL series and book]