Weekend Learning - SIEM Costs

There are several events recently that slowed down my progress on this series but it's good learning experience and give me even more encouragement.

In this article, I would like to address the question: "How much do you need to pay for using Sentinel?" including "Are there any other significant cost?"

There are several ways to explore this topic, in my opinion, I will choose "data" (as a currency) as my approach to explain about core components in Sentinel cost and pricing. Microsoft offers various options for data ingestion and storage to help them maximize the TCO based on your business needs and cybersecurity maturity. The recommendation is considering data in 2 classification: Primary security data (data required for alerting, which can be ingested into Standard Logs) and Secondary security data (data used for operations/ investigations, which can leverage Basic Logs, Archived Tier, ADX or Azure Storage to reduce cost).

Before we jump into the details of it, I would like to bring up an important point that we should plan and do architecture design for a cloud SIEM (goal) not Sentinel (tool). It will help us to have a clear understanding of what we need to have but not what Sentinel's features are.

1. Core components

• Daily Data Ingestion into Microsoft Sentinel & Log Analytics for security monitoring and alerting; previously, there were separate prices for Log Analytics and Microsoft Sentinel. Now there is a single combined price for both components which simplifies budgeting, billing, and cost management. You can read more at Introducing the new Microsoft Sentinel simplified pricing.
• Monthly Data Storage for Log Retention
• Event-based Workflow triggered for SOAR, for example Sentinel Playbooks with Azure Logic Apps (in common, this is not a major cost consideration for Sentinel customers.)

2. Other significant costs

• Data Transformation Cost: Resource logs will go into a data ingestion pipeline but you should consider in some use-case such as redact sensitive data or enrich the logs, pre-ingestion log parsing may incur processing fee. To address this component, Microsoft Sentinel gives you two tools to control this process: using Log Ingestion API and DCRs. You have options to apply filtering process or reducing data ingestion (could be dropping rows or events or column) as part of cost optimization strategy.
• Network Transfer Cost (aka Bandwidth Cost): The key concept is that data ingress is free (please note that this is not about ingestion) but data egress will be charged including data moving out of Azure data centers as well as moving between Azure data centers.
• Syslog Forwarding VM: it requires at least one Linux Server. If you run it on Azure VM, in some scenarios, you may need more than one VM for load balancing or creating isolated data collection channel. You also need to have some protection layers implemented for these resources.
• Security Notebook and Machine Learning Compute: the notebooks in case you need will require ML Compute, also Azure Storage and Azure Key Vault. Until #Security #Copilot come to the public (and even when it happens we had Azure Fusion and Machine Learning Notebooks) , if you still want to bring your own ML into the Sentinel platform, one of Microsoft Security partners in APAC, NCS showed case the capabilities of MS security platform and their BYOD ML NCS Case Study - End-to-End Integration of Custom BYOML model with Sentinel - Microsoft Community Hub. You can explore more about #MicrosoftSentinel Enhancements in #MachineLearning and Productivity in this article.

• Azure Open AI: recently, when we talk about simplifying the incident handling process and improving our SOC performance, integrate Azure OpenAI with Sentinel becomes a hot topic. If you are interested, you can find real good information from this blog post: Microsoft Sentinel — Azure OpenAI Incident Response Playbook | by Antonio Formato | Microsoft Azure | Medium
• Azure Monitor: you may need to have additional capabilities such as alert rules for activity logs, metrics, log query or push notification on email, SMS or some mobile apps.
• Data export: in many cases, data exporting is necessary for historical data archiving or side-by-side/ hybrid SIEM scenarios. There are 2 approaches that you can consider: Continuous Export, Event Hubs (data pipeline). Read more at Moving Azure Sentinel Data to ADX for Long Term Storage - Azure Cloud & AI Domain Blog (azurecloudai.blog) and Sentinel data export tool Export Historical Data from Log Analytics (microsoft.com). If you use Event Hub, you will want to consider Cost based on Throughput Unit.

3. What is the strategy to optimize or reduce the cost?

• Prioritize free data sources: Azure Activity Logs, Office 365 Audit Logs, Alerts from Microsoft Defender such as MDE, MDCA, MDI, Microsoft Defender for Cloud (please note that only alerts are free not the raw logs from these services; if you enforce daily cap limit, these free data source is not limited.)
• 31-day Free Trial is applied to new created Sentinel instances.
• If the customer is having M365 E5, they can receive a data grant of up to 5MB per user per day to ingest Microsoft 365 data. Read more at Microsoft 365 E5 benefit offer with Microsoft Sentinel
• Customers have Sentinel enabled on the Defender for Server workspace will get the benefit from the free 500mb grant in Microsoft Defender for Cloud Plan 2, which includes only specific log types, and ingestion over that daily aggregated limit will be charged, as well as additional retention (1+ month). Read more at Microsoft Defender for Server Reference Architecture and Deployment Guide - Azure Cloud & AI Domain Blog (azurecloudai.blog) (please note that this is not Microsoft original blog).

4. How to keep track on Sentinel Cost

• Workspace Usage Report workbook provides your workspace's data consumption, cost, and usage statistics.
• Control your Microsoft Sentinel budget using a Cost management playbook. 
• Detect the Ingestion Data Cost Spike using workbook leveraging series_decompose_anomalies().
• Run queries to understand your ingestion data
• Using Azure Cost Analytics and filter Microsoft Sentinel resource
• Applying Daily cap in Microsoft Sentinel setting and create an analytics rule that triggers an alert when the daily cap is reached. (Note: you should note that this is only recommended for avoiding unplanned cost incur but should not be consider at the first sight or a cost reduction or optimization feature.)

You can go through this article to explore the recommendations to reduce the cost for Microsoft Sentinel Reduce costs for Microsoft Sentinel and I also learn a lot from this webinar https://youtu.be/0cIYB92Qb60

Thanks for all comments, sharing and encouraging. I am looking forward to have your sharing and thoughts on this article. If you have interest in any topic, please let me know in the comments. I plan to have next article about SIEM Use Cases and improving operational efficiency.