Live Workshop icon

July 16: Financial Services Roundtable

Top 12 Log Management Security Tips

Detect, investigate, and respond within seconds of a security incident

June 3rd, 2020

Security departments have always been the most active users of log data. As log management modernizes with index-free logging solutions, real-time access to data and deeper historical searches become possible, changing the best practices.

“Central log management is an important, but often undervalued, tool for an organization’s threat management capabilities and compliance requirements. Security and risk management leaders can benefit from the adoption of a log management tool for multiple security operations use cases.” 1

Stay up to date by following these 12 Tips for Security Log Management:

  1. Plan out all security use cases in advance of log management deployment

  2. Store data for the appropriate amount of time

  3. Account of hidden costs of open source tools

  4. Log everything

  5. Run log management alongside a SIEM

  6. Standardize log data

  7. Avoid alert fatigue by customizing notifications

  8. Choose platform-agnostic tools

  9. Optimize storage costs

  10. Choose tools with real-time capabilities

  11. Deploy log management in the same environment as the logs

  12. Share log data with other departments


1. Plan out all security use cases in advance of log management deployment

To build an effective security system, it’s important to start by understanding the use cases the system must address. Take account of compliance needs as well as sources of external and internal threats. Build a roadmap from the start that will govern which data you will build alerts, dashboards, and metrics from. For additional ideas about sources of protection, consult MITRE ATT&CK’s Getting Started with ATT&CK guide to the adversary tactics and techniques used by attackers to compromise a security system.

2. Store data for the appropriate amount of time

Determine what data is needed, and the time needed for active dashboards and alerts. Breaches can occur months after a system has been compromised, so determine how long data should be collected to assist in investigations. Identify the required retention time of data stored for compliance purposes — some data stored for compliance must be available for at least a year. A sophisticated log management platform that optimizes storage with compression may be able to store significantly more than traditional log management or Security Information and Event Management (SIEM) tools. The most advanced log management tools offer a pricing option that makes collecting unlimited data an affordable option.

3. Account for the hidden costs of open source tools

While SIEM systems that are based on open-source platforms may look attractive at the onset, they come with additional requirements to install, manage, and customize them that may end up being much more expensive than what was saved on the license. When calculating the cost of a solution, take into account the total cost of operations (TCO).


4. Log everything

Collect all data using data shippers and parsers in the log management system. Don’t leave out any potential sources of data that may be useful in the future. Collecting all data gives you a complete picture of an incident, so you can search all of it and determine the root cause. Modern log management is available that collects data without having to index it or set up a schema beforehand — making it much easier to conduct a more thorough search.


5. Run log management alongside a SIEM

Organizations with a SIEM installed should consider the benefits and costs of installing a modern log management system to offload the burden of log aggregation and storage from the SIEM. This will enhance the ability to conduct a more comprehensive search with more data sources and longer-retained data.

“Many clients are unaware that log management tools can be leveraged for use cases beyond just collecting logs in a central repository to have them available for after-event analysis.” 1

6. Standardize log data

If possible, a standard format for data can be normalized at the collection layer to reduce the load on systems further downstream. It brings benefits of consistent reporting from all systems and applications, and reduced work each time you start configuring something new. It also helps to futureproof your system. To get started, consider using a service such as FluentD or Vector.

7. Avoid alert fatigue by customizing notifications

Modern log management tools provide a wide variety of ways to cut down on the volume of alerts. You can customize alerting events to notify just once, and also decrease worker load by automating a webhook response to an alert-triggering event.

8. Choose platform-agnostic tools

Don’t be locked into specific vendors with point-to-point integrations. Adding additional services should be as simple as extending collector configurations. Modern log management solutions have open APIs and use industry-standard data shippers.


9. Optimize storage costs

Modern log management solutions can compress data to save on storage and data transfer costs. The results can be dramatic. Sources such as Windows Event Logs can be compressed as much as 80x. Typical overall compression can range from 5-20x. If deploying in a cloud environment, save on storage costs by using a solution that works with object storage rather than the more typical and expensive network block storage.

10. Choose tools with real-time capabilities

Choose solutions that decrease mean time to resolution (MTTR). Choose a log management option that supports real-time searches, alerts, and dashboards to enable your security team to respond as quickly as possible.

11. Deploy log management in the same environment as the logs

If logs are being generated from platforms, services, or apps in the cloud, it may be required to pay to send logs to or from the cloud. Deploy a log management system in the same cloud platform to minimize data transfer costs.

12. Share log data with other departments

Don’t use log management just for security! Real-time log analytics is a useful tool for a variety of departments. For example, logs can show how customers are using applications, or how materials are moving in the manufacturing supply chain. Data can be made accessible to non-technical users through shared dashboards and widgets.


Learn more


Continue learning about the security use cases of modern log management by exploring the following options:

  • Reserve your spot for our upcoming Higher Education Roundtable, June 24, 2020. Explore how three universities use log data and alerts to improve their system resilience and security.

  • Read about how a Security Operations Center (SOC) provider uses log management in every department from security to sales, marketing, finance, and operations. Vijilan uses Humio to monitor system security, and also applies it to create advanced business analytics. Read more in our customer study, or listen to a podcast.

  • Take an in-depth look at the security use cases of modern log management in our How-to Guide: Use log management as the foundation of the security stack.

  • Explore how to combine high-performant log management with network monitoring by viewing a replay of our Modern Observability Stack workshop.

For more information, visit humio.com/secops.




1. Gartner: Use Central Log Management for Security Operations Use Cases, 20 March 2020, Toby Bussa, Kelly Kavanagh, Mitchell Schneider (Requires Gartner subscription).

Start your free trial now, available Self-hosted and SaaS, or request a demo.