Log Files Explained
September 22, 2021
What is a Log File?
A log file is an event that took place at a certain time and might have metadata that contextualizes it.
Logs files are a historical record of everything and anything that happens within a system, including events such as transactions, errors and intrusions. That data can be transmitted in different ways and can be in both structured, semi-structured and unstructured format.
The basic anatomy of a log file includes:
The timestamp – the exact time at which the event logged occurred
User information
Event information – what was the action taken
However, depending on the type of log source, the file will also contain a wealth of relevant data. For example, server logs will also include the referred webpage, http status code, bytes served, user agents, and more.
Where do Log Files Come From?
Just about everything produces some version of a log, including:
Apps
Containers
Databases
Firewalls
Endpoints
IoT devices
Networks
Servers
Web Services
The list goes on, but the point is, almost all infrastructure that you interact with on a daily basis produces a log file.
Who uses Log Files?
Log files can provide almost every role at an organization with valuable insights. Below are some of the most common use cases by job function:
Types of Logs
Nearly every component in a network generates a different type of data and each component collects that data in its own log. Because of that, many types of logs exist, including:
Event logs
An event log is a high-level log that records information about network traffic and usage, such as login attempts, failed password attempts, and application events.
Server logs
A server log is a text document containing a record of activities related to a specific server in a specific period of time.
System logs
A system log, or syslog, is a record of operating system events. It includes startup messages, system changes, unexpected shutdowns, errors and warnings, and other important processes. Windows, Linux, and macOS all generate syslogs.
Authorization logs and access logs
Authorization logs and access logs include a list of people or bots accessing certain applications or files.
Change logs
Change logs include a chronological list of changes made to an application or file.
Availability logs
Availability logs track system performance, uptime, and availability.
Resource logs
Resource logs provide information about connectivity issues and capacity limits.
Threat logs
Threat logs contain information about system, file, or application traffic that matches a predefined security profile within a firewall.
The importance of modern log management
While there are seemingly infinite insights to be gained from log files, there are a few core challenges that prevent organizations from unlocking the value offered in log data.
Humio's modern log management solution
Humio is a log management solution purpose-built to handle the scale of today’s data volumes. Whether self-hosted or run in the cloud, Humio offers index-free logging that enables searches of any part of the logs, metrics, or other data, as well as data streaming and streaming analytics that make it possible to conduct searches and analyses in real-time.
Humio delivers streaming observability with sub-second latency and data-burst capabilities. Users can also search multiple datasets with a single query, resulting in enriched data and better insights.
Learn more from other related content
Humio makes the 2020 Accel Euroscape
We’re proud to share that Humio is included in the 2020 Accel Euroscape. The Accel Euroscape is a list compiled by American venture capital firm Accel of the top cloud companies started in Europe a...
Episode 39 - Data compression and index-free logging with Jerald Perry
In this week’s podcast we talk with Jerald Perry, Senior Technical Marketing Engineer at Humio about how Humio’s index-free architecture and compression translate to cost savings for our customers.
Integrating security throughout your infrastructure
Discover how to increase resilience by enabling security workflows, threat modeling, compliance requirements, and real-time threat hunting