| Author |
Topic: How to Rotate Apache Log Files in AWS Linux EC2 Instance |
|
AwsEC2
member
offline  |
| |
| posts: |
39 |
| joined: |
08/28/2012 |
| from: |
CA |
|
|
 |
|
|
| How to Rotate Apache Log Files in AWS Linux EC2 Instance |
On even a moderately busy server, the quantity of information stored in the log files is very large. The access log file typically grows 1 MB or more per 10,000 requests. It will consequently be necessary to periodically rotate the log files by moving or deleting the existing logs. This cannot be done while the server is running, because Apache will continue writing to the old log file as long as it holds the file open. Instead, the server must be restarted after the log files are moved or deleted so that it will open new log files.
A typical scenario that simply rotates the logs and compresses the old logs to save space is:
mv access_log access_log.old
mv error_log error_log.old
apachectl graceful
sleep 600
gzip access_log.old error_log.old
Another way to perform log rotation is using logrotate utility tool:
logrotate [-dv] [-f|--force] [-s|--state file] config_file+
|
|
|
|
|
|
|
AwsEC2
member
offline  |
| |
| posts: |
39 |
| joined: |
08/28/2012 |
| from: |
CA |
|
|
|
|
|
| Logrotate configuration file |
A simple configuration file looks like this:
# sample logrotate configuration file compress
/var/log/messages {
rotate 5
weekly
postrotate
/sbin/killall -HUP syslogd
endscript
}
"/var/log/httpd/access.log" /var/log/httpd/error.log {
rotate 5
mail www@my.org
size=100k
sharedscripts
postrotate
/sbin/killall -HUP httpd
endscript
}
/var/log/news/news.crit {
monthly
rotate 2
olddir /var/log/news/old
missingok
postrotate
kill -HUP cat /var/run/inn.pid
endscript
nocompress
}
The first section of the config files defined how to handle the log file /var/log/messages. The log will go through five weekly rotations before being removed. After the log file has been rotated (but before the old version of the log has been compressed), the command /sbin/killall -HUP syslogd will be executed.
The next section defines the parameters for both /var/log/httpd/access.log and /var/log/httpd/error.log. They are rotated whenever is grows over 100k is size, and the old logs files are mailed (uncompressed) to www@my.org after going through 5 rotations, rather then being removed. The sharedscripts means that the postrotate script will only be run once(after the old logs have been compressed), not once for each log which is rotated. Note that the double quotes around the first filename at the beginning of this section allows logrotate to rotate logs with spaces in the name. Normal shell quoting rules apply, with , ", and \ characters supported.
The last section defines the parameters for all of the files in /var/log/news. Each file is rotated on a monthly basis. This is considered a single rotation directive and if errors occur for more then one file, the log files are not compressed.
Please use wildcards with caution. If you specify *, logrotate will rotate all files, including previously rotated ones. A way around this is to use the olddir directive or a more exact wildcard (such as *.log).
|
|
|
|
|
|
|
AwsEC2
member
offline |
| |
| posts: |
39 |
| joined: |
08/28/2012 |
| from: |
CA |
|
|
|
|
|
| Working example -- Manually |
Let's demo the utility with an example.
Step 1) Log file before rotation
The file which is about to be rotated is /home/ec2-user/example1.com/access_log which has the info as the following:
$ cd /home/ec2-user/example1.com
$ ls -l access_log
-rw-r--r-- 1 root root 173608 Sep 4 22:43 access_log
Step 2) Configuring how to rotate it
If you want to rotate the above log file for every 1KB, create the logrotate.conf as shown below.
$ vi my_logrotate.conf
/home/ec2-user/example1.com/access_log {
size 1k
create 644 root root
rotate 3
}
This logrotate configuration has following three options:
size 1k logrotate runs only if the filesize is equal to (or greater than) this size.
create rotate the original file and create the new file with specified permission, user and group.
rotate limits the number of log file rotation. So, this would keep only the recent 3 rotated log files.
Step 3) Manually run the utility tool
$ sudo logrotate my_logrotate.conf
Step 4) Log files after rotation
After the logrotation, following is the size of access_log
$ ls -l access_log*
-rw-r--r-- 1 root root 0 Sep 7 00:04 access_log
-rw-r--r-- 1 root root 173608 Sep 4 22:43 access_log.1
Note: After rotation, the running instance, Apache httpd, is holding the file descriptor pointing to access_log.1, rather than access_log. This can be verified by witnessing the growth of access_log.1. That's why you have to restart your instance after rotation for most cases. The alternative is to use copytruncate rather than create.
$ vi my_logrotate.conf
/home/ec2-user/example1.com/access_log {
size 1k
# create 644 root root
copytruncate
rotate 3
}
|
|
|
|
|
|
|
AwsEC2
member
offline |
| |
| posts: |
39 |
| joined: |
08/28/2012 |
| from: |
CA |
|
|
|
|
|
| Working example -- Automatically |
The above log file rotation can be done automatically by system cron. Following are the key files that you should be aware of for logrotate to work properly.
Key file #1). /usr/sbin/logrotate The logrotate command itself.
Key file #2). /etc/cron.daily/logrotate The shell script executes the logrotate command daily.
The content should look like this:
$ cat /etc/cron.daily/logrotate
#!/bin/sh
/usr/sbin/logrotate /etc/logrotate.conf
EXITVALUE=$?
if [ $EXITVALUE != 0 ]; then
/usr/bin/logger -t logrotate "ALERT exited abnormally with [$EXITVALUE]"
fi
exit 0
Key file #3). /etc/logrotate.conf Log rotation configuration for all the log files are specified in this file.
The content should look like this:
$ cat /etc/logrotate.conf
# see "man logrotate" for details
# rotate log files weekly
weekly
# keep 4 weeks worth of backlogs
rotate 4
# create new (empty) log files after rotating old ones
create
# use date as a suffix of the rotated file
dateext
# uncomment this if you want your log files compressed
#compress
# RPM packages drop log rotation information into this directory
include /etc/logrotate.d
# no packages own wtmp and btmp -- we'll rotate them here
/var/log/wtmp {
monthly
create 0664 root utmp
minsize 1M
rotate 1
}
/var/log/btmp {
missingok
monthly
create 0600 root utmp
rotate 1
}
# system-specific logs may be also be configured here.
Key directory #4). /etc/logrotate.d A folder to hold individual program's log rotation configuration.
For example, an Apache's vHost example1.com
$ cat /etc/logrotate.d/example1.com.rotate
'/home/ec2-user/example1.com/access_log' {
nomail
missingok
notifempty
compress
nosharedscripts
weekly
copytruncate
}
'/home/ec2-user/example1.com/error_log' {
nomail
missingok
notifempty
compress
nosharedscripts
weekly
copytruncate
}
and Apache httpd itself:
$ cat /etc/logrotate.d/httpd
/var/log/httpd/*log {
missingok
notifempty
sharedscripts
postrotate
/bin/kill -HUP `cat /var/run/httpd.pid 2>/dev/null` 2> /dev/null || true
endscript
}
If you have Tomcat running to serve the dynamic content, you may want to put your Tomcat's rotation configuration file here as well.
$ cat /etc/logrotate.d/tomcat5
/var/log/tomcat5/*.txt {
copytruncate
weekly
rotate 52
compress
missingok
}
/var/log/tomcat5/catalina.out {
copytruncate
weekly
rotate 52
compress
missingok
}
|
|
|
|
|
|
|
|
Email To Friend | 
Set Alert To This Topic
New Topic | 
Post Reply 
posted on: 09/06/2012 02:10:31 AM
Edit
Quote
Report
Profile
Reply