Archiving is in and your logs are here to stay!

We develop features that streamline the log management processes for our users. Logs are information assets, and we understand that you need to retrieve, re-asses and draw insights from your historic logs. observIQ offers a simple integration with Amazon Web Services (AWS) for Extended Retention. It takes less than 30 seconds to set up and archive logs directly to an S3 Bucket in your AWS account. Once compressed and uploaded from observIQ, your logs are retained indefinitely – as long as your cloud bucket exits. Your data in the S3 bucket is backed up in a SOC 2 compliant data center. 

On the topic of archiving, we want to highlight some scenarios where you may turn to your S3 bucket to pull up historic logs:

1. Audits
2. Analyzing security status
3. Business insights

Audits and Compliance:

For businesses with strict compliance and regulatory requirements, extended log retention is critical, but that doesn’t mean it should be a hassle. The period of retention and the type of logs retained varies based on the business’s industry standards. Creating an audit trail for every event in the network or applications is mandatory for common compliance standards such as HIPAA and PCI. Most mandated retention standards lean toward the one year time frame, but some businesses choose to retain logs for longer periods to err on the side of caution in case their compliance certification norms change. 

Every action in the cyber realm generates logs. Businesses choose log data that are most necessary for their compliance. Common log types that are retained for the purpose of compliance are:

1. Network and application access credentials such as user ID, access locations, user information, time and date of access, terminal access identification, etc. 
2. Changes to application and infrastructure such containerized pod additions and deletions; activating and deactivating firewalls, malware, etc.
3. Changes the admin makes to the log data, or log management tool. This could be importing log files, deleting a batch of logs or changes to the log management tool. 

Analyzing security status

Cyber security is a top priority for all businesses, even outside the tech industry. Logs give you an unobstructed view of your application and infrastructure’s health and security. Cybersecurity auditors want to read through present and past logs to analyze security performance over time, identifying potential vulnerabilities and formulating process changes to tighten security. This is also tied into the compliance aspect of log retention. Ahead of applying for a compliance certification, most companies employ legal and cybersecurity counsel to assess their security and infrastructure and make recommendations to navigate the certification process. A business’s logs is the first resource an independent auditor wants to look at. 

Business Insights

“Data-driven decisions” is a pervasive buzz phrase in the business world today. Some of your most critical data lives in your logs. Generating deep, actionable insights from data requires deep, accurate, and organized data infrastructures. Retaining logs over long periods of time gives you visibility into the trends, usage, performance, and security of your applications and infrastructures. Formulating data-driven decisions around development, security, and user experience is near impossible without extended retention. 

Businesses pull up historic logs for trend analysis. Metrics such as response time, response size, source of traffic, volume of traffic, pod status, etc. are studied over a period of time to chart a performance pattern, and decision making ensues. 

The Archiving Process in observIQ

The extended retention features in observIQ are available to all users, including users on the free tier. Free extended retention can go a long way for any business, and anyone with an S3 bucket can start archiving their logs with observIQ today!

1 Configure an AWS S3 Bucket

Within AWS:

1. Access your S3 Console 
2. Use an existing bucket or create a new one

2 Enable Archiving in observIQ

You should be signed in as an Owner or Admin to access the archive functionality

1. Head to Settings → Archive
2. Enable archiving.
3. Select the region, the region selection in both observIQ and AWS S3 should be the same.
4. Enter the name of the S3 bucket where the logs will be archived. Your logs are saved as zipped files that are time stamped with the date and time of export.
5. Enter the access ID and the secret access key for your AWS account.

Your logs are saved in the S3 bucket periodically. They are saved in batches and each batch is enclosed within a zipped file. The zipped files are named with a timestamp. We make sure that all your logs are saved to the S3 bucket before the end of the retention period. 

The list below the extended retention setup displays all the zipped files saved to the linked S3 bucket. Clicking the filename navigates you to the file within AWS. 

observIQ makes extended retention a painless process for you and your team. Try out our log management platform today. Every feature is included in every plan, even the free tier. Reach out to our support team with any questions, and discuss how observIQ can best fit your log management needs.

The post Archiving Is In, And Your Logs Are Here To Stay! appeared first on observIQ.