Log Files Explained

Contents

What is a Log File?

A log file is an event that took place at a certain time and might have metadata that contextualizes it. 

Logs files are a historical record of everything and anything that happens within a system, including events such as transactions, errors and intrusions. That data can be transmitted in different ways and can be in both structured, semi-structured and unstructured format.

The basic anatomy of a log file includes: 

  • The timestamp – the exact time at which the event logged occurred 

  • User information

  • Event information – what was the action taken

However, depending on the type of log source, the file will also contain a wealth of relevant data. For example, server logs will also include the referred webpage, http status code, bytes served, user agents, and more.

Where do Log Files Come From?

Just about everything produces some version of a log, including:

  • Apps

  • Containers

  • Databases

  • Firewalls

  • Endpoints

  • IoT devices

  • Networks

  • Servers

  • Web Services

The list goes on, but the point is, almost all infrastructure that you interact with on a daily basis produces a log file. 

Who uses Log Files?

Log files can provide almost every role at an organization with valuable insights. Below are some of the most common use cases by job function:

Types of Logs

Nearly every component in a network generates a different type of data and each component collects that data in its own log. Because of that, many types of logs exist, including:

Event logs

An event log is a high-level log that records information about network traffic and usage, such as login attempts, failed password attempts, and application events.

Server logs

A server log is a text document containing a record of activities related to a specific server in a specific period of time. 

System logs

A system log, or syslog, is a record of operating system events. It includes startup messages, system changes, unexpected shutdowns, errors and warnings, and other important processes. Windows, Linux, and macOS all generate syslogs.

Authorization logs and access logs

Authorization logs and access logs include a list of people or bots accessing certain applications or files. 

Change logs

Change logs include a chronological list of changes made to an application or file. 

Availability logs

Availability logs track system performance, uptime, and availability.

Resource logs

Resource logs provide information about connectivity issues and capacity limits. 

Threat logs

Threat logs contain information about system, file, or application traffic that matches a predefined security profile within a firewall. 

The importance of modern log management

While there are seemingly infinite insights to be gained from log files, there are a few core challenges that prevent organizations from unlocking the value offered in log data.

Humio's modern log management solution

Humio is a log management solution purpose-built to handle the scale of today’s data volumes. Whether self-hosted or run in the cloud, Humio offers index-free logging that enables searches of any part of the logs, metrics, or other data, as well as data streaming and streaming analytics that make it possible to conduct searches and analyses in real-time.

Humio delivers streaming observability with sub-second latency and data-burst capabilities. Users can also search multiple datasets with a single query, resulting in enriched data and better insights.

Learn more from other related content

Humio makes the 2020 Accel Euroscape

We’re proud to share that Humio is included in the 2020 Accel Euroscape. The Accel Euroscape is a list compiled by American venture capital firm Accel of the top cloud companies started in Europe a...

Read more

Episode 39 - Data compression and index-free logging with Jerald Perry

In this week’s podcast we talk with Jerald Perry, Senior Technical Marketing Engineer at Humio about how Humio’s index-free architecture and compression translate to cost savings for our customers.

Read more

Integrating security throughout your infrastructure

Discover how to increase resilience by enabling security workflows, threat modeling, compliance requirements, and real-time threat hunting

Read more