As cybersecurity attack vectors evolve, security logging and monitoring are becoming even more important. Effective logging and monitoring enables organizations to detect and investigate security incidents quickly. Cloud-based attackers are getting more sophisticated, and often rely on stolen credentials to escalate privileges and move laterally within corporate IT networks. Many do so undetected, which is why modern IT systems require a watchful eye on log data to detect suspicious activity and inform incident response efforts.

The OWASP Top 10 provides crucial guidelines to enhance security practices, including addressing security logging and monitoring failures. This blog post will cover some of the most common security logging and monitoring mistakes, and provide actionable recommendations to avoid them.

Mistake 1: Insufficient Logging

Insufficient logging involves recording too little data or missing critical events. When essential information is not logged, it becomes challenging to detect and investigate security incidents effectively. The primary consequence of insufficient logging is the difficulty in detecting and investigating security incidents. Without comprehensive logs, identifying the root cause of an attack or breach can be near impossible, leaving the organization vulnerable to further exploits.

For example, many systems support logging for network devices, operating systems, web servers, mail servers, and database servers. However, custom application event logging is often overlooked, disabled, or improperly configured. This type of logging provides significantly deeper insights than infrastructure logging alone. Logging for web applications (such as websites or web services) extends far beyond merely enabling web server logs (e.g., using the Extended Log File Format).

Recommendations

• Log all authentication attempts: This includes successful and failed login attempts. Tracking both successful and failed attempts allows you to identify potential brute force attacks and unauthorized access attempts.
• Include details like timestamps, user IDs, and IP addresses: Such details provide valuable context for understanding security events. This information can help trace the origin of an attack, determine the time of occurrence, and identify the user involved, enhancing your ability to respond appropriately.
• Follow OWASP recommendations on application logging: Application logging should maintain consistency within the application and across an organization’s application portfolio. It should also adhere to industry standards where applicable. This ensures that the logged event data can be easily consumed, correlated, analyzed, and managed by a variety of systems.

Mistake 2: Poor Log Storage Practices

Storing logs in insecure locations or for insufficient durations compromises the integrity and availability of valuable forensic data. Poor log storage practices can lead to log tampering and the loss of essential forensic data, undermining your ability to investigate and respond to incidents. Without secure storage, attackers can alter or delete logs to cover their tracks, making it difficult to trace security incidents and assess the full extent of a breach.

What’s more, insufficient log retention periods can result in the premature deletion of logs, depriving your organization of crucial historical data needed for compliance audits and long-term security analyses and best-practices like threat hunting.

Recommendations

• Use secure, tamper-evident storage for logs: Ensure that logs cannot be altered without detection. If you’re relying on cloud storage like Amazon S3 or Google Cloud Platform, adhere to cloud security and configuration best-practices to lock down your logs.
• Retain logs for an adequate period: Retention policies should align with compliance and operational needs. Industry research shows that enterprise organizations take more than 200 days on average to identify a data breach. Based on that figure, even a data retention window of six months would only help you identify around 50% of data breaches. Retaining security logs for longer periods of time enables a more comprehensive approach to security log analysis and helps your security team detect advanced persistent threats, as well as conduct thorough root cause analysis.
• Implement access controls and encryption: Only authorized personnel should access logs via role-based access control (RBAC), and logs should be encrypted both in transit and at rest.

Mistake 3: Inadequate Monitoring and Alerting

Failing to actively monitor logs or set up effective incident alerting for visibility can result in delayed detection of security breaches and attacks. Inadequate monitoring and alerting systems lead to delayed responses to security incidents. Without timely alerts, breaches can go unnoticed, allowing attackers more time to exploit vulnerabilities.

Recommendations

• Use automated tools to monitor logs in real time: Automated logging and monitoring systems ensure continuous vigilance. Consider a modular observability stack to take advantage of best-of-breed tooling at a lower cost.
• Configure alerts for suspicious activities: For instance, set up alerts for multiple failed login attempts.
• Regularly review and update alert thresholds and criteria: Ensure that alert settings remain relevant and effective to avoid alert fatigue.

Mistake 4: Ignoring Log Analysis

Not analyzing logs regularly or comprehensively means missing opportunities to identify patterns and prevent future incidents. Ignoring log analysis can result in missed chances to detect emerging threats and patterns, as subtle indicators of malicious activity may go unnoticed. Regular analysis helps in understanding attack trends, enabling organizations to proactively improve their security measures and defenses.

By thoroughly examining logs, you can identify recurring issues, predict potential vulnerabilities, and develop strategies to mitigate future risks. Additionally, comprehensive log analysis supports compliance with regulatory requirements and enhances overall incident response capabilities, ensuring a more robust security posture.

Recommendations

• Conduct regular log analysis to identify trends and anomalies: Regular analysis of your application and infrastructure log files can highlight unusual patterns indicative of potential threats.
• Complement your SIEM or Security Data Lake with dedicated log management solutions: Tools like ChaosSearch can enhance your ability to analyze logs effectively. These tools work alongside a SIEM like Splunk, or a data lake like Amazon Security Lake, so you can conduct comprehensive log analysis without moving data outside of your cloud storage environment.
• Utilize these tools for threat hunting: Proactively search for threats to enhance your security posture. For example, you can threat hunt in Amazon Security Lake using ChaosSearch to parse through historical log and telemetry data at a far lower cost, enhancing the threat detection process.

Mistake 5: Lack of Log Integrity and Confidentiality

Logs containing sensitive information can be altered or accessed by unauthorized individuals if proper integrity and confidentiality measures are not in place. Compromised log integrity and unauthorized access to logs can lead to significant security breaches and data leaks. Ensuring log integrity and confidentiality is essential to maintaining a secure environment.

Recommendations

• Implement checksums and cryptographic signatures for logs: These measures help detect any alterations to logs. Leverage the OWASP security logging and monitoring checklist to ensure that you’ve implemented security logging design best practices.
• Ensure log files are encrypted both in transit and at rest: Encryption protects logs from unauthorized access.
• Restrict log access to authorized personnel only: Access controls should be strictly enforced to prevent unauthorized access.

Effective Security Logging and Monitoring with OWASP

Effective security log analysis is crucial for protecting your organization. Utilizing resources like OWASP can ensure you maintain proactive security controls. In addition, tools like a security data lake (e.g., Amazon Security Lake) complemented by centralized log management solutions like ChaosSearch enable you to analyze logs at scale and at low cost. This approach allows for longer log retention periods and more comprehensive analysis.

By avoiding the common mistakes above and following best practices, organizations can significantly enhance their security logging and monitoring capabilities, ultimately leading to a stronger overall security posture.