You’ve probably heard that logs are one of the “pillars” of observability.

Well, logs are also a pillar of cybersecurity. Indeed, logs arguably play just as important a role in helping to identify and address security risks as they do in driving observability. That’s because logs record data about potential security events that you can’t obtain from other sources.

To prove the point, let’s walk through the role that logs play in cybersecurity.

How Logs Enable Security

The relationship between logs and cybersecurity is straightforward enough: because logs record data about who did what inside applications and infrastructure, logs are a critical source of insight for detecting and investigating security risks.

More specifically, logs provide security insights such as the following:

• When, how, and who accessed a system.
• The changes made to a system, and who or what caused the changes.
• Which other systems were connected to or integrated with a given system.
• The enabled security protections on a system and which actions those protections took to defend against a threat.
• The compromised resources or system components during a breach.

As an example of leveraging logs for security purposes, you could use logs from a Web server to detect which remote hosts were attempting to connect to a website or app. If you notice a large and sudden spike in requests from unknown hosts, it could be your first sign that someone is attempting to launch a Distributed Denial of Service (DDoS) attack against your server.

Likewise, most operating systems generate authentication logs (or, in the case of Windows, they record authentication events within generic system logs). Authentication logs track who tried to log into a server, how they tried to connect, and whether they were successful. This data is crucial for identifying security issues such as attempts to brute-force passwords (repeatedly trying to log in using different password combinations until attackers happen upon the right one). Auth logs can also help investigate a security incident after the fact to determine precisely how an attacker breached a system.

In cloud-native environments, too, logs provide essential visibility into security risks and events. Kubernetes, for instance, can generate audit logs that track interactions with the Kubernetes API, providing unparalleled visibility into which actions human and machine users take within a Kubernetes cluster. If you want to detect unusual behavior that could reflect a security breach inside Kubernetes, audit logs are often the best data source.

All Logs Are Security Logs

Although some logs (like Linux authentication logs and Kubernetes audit logs) are generated primarily for tracking events relevant to security purposes, you shouldn’t limit yourself to these types of logs for security monitoring purposes.

Indeed, every type of log – whether or not it is designed primarily to assist in security operations – is a security log.

As we noted above, for example, Web server logs can provide critical security insights, even though recording potential security threats is not the primary purpose of Web server logs. Likewise, generic operating system logs, like Linux Syslog files (which record all manner of information about Linux operating system processes and applications), can be a great source of security visibility and context.

Generic logs also help you identify which applications and processes were doing what when a security incident occurred. Doing this is valuable if you want to gain context to help investigate how a security breach happened or what the breach impacted.

Finally, logs of all types allow you to establish baselines of “normal” system behavior, then identify events or patterns that deviate from that behavior – and that, in some cases, could reflect security risks.

Other Data Sources for Security Monitoring

Logs are not, of course, the sole source of insight into security risks. IT security analysts use a range of additional data sources to detect and investigate threats, such as:

• Vulnerability Databases record known security risks inside applications, libraries, and services.
• Threat Intelligence Reports detail exploit techniques used by attackers.
• Metrics. You can use the metrics from applications and infrastructure alongside logs to establish a baseline of normal behavior and measure anomalies against it.

However, logs provide data that these other sources of security visibility lack. You can only get a list of authentication events or requests made to an API, for example, by looking at log files.

Thus, while it’s common to use other data sources to provide additional security insights, logs offer unique types of information that you can’t obtain in any other way.

Conclusion

That’s what makes logs so crucial as the foundation of security: logs offer details about how systems were accessed and used that you can’t collect through other means. It’s almost impossible to imagine a security strategy that doesn’t rely on logs as its central source of insight into potential security risks and their context.

‍