12.1. syslogsyslog is the tried-and-true workhorse of Unix logging utilities. It accepts log data from the kernel (by way of klogd), from any and all local process, and even from processes on remote systems. It's flexible as well, allowing you to determine what gets logged and where it gets logged to. A preconfigured syslog installation is part of the base operating system in virtually all variants of Unix and Linux. However, relatively few system administrators customize it to log the things that are important for their environment and disregard the things that aren't. Since, as few would dispute, information overload is one of the major challenges of system administration, this is unfortunate. Therefore, we begin this chapter with a comprehensive discussion of how to customize and use syslog. 12.1.1. Configuring syslogWhenever syslogd, the syslog daemon, receives a log message, it acts based on the message's type (or "facility") and its priority. syslog's mapping of actions to facilities and priorities is specified in /etc/syslog.conf. Each line in this file specifies one or more facility/priority selectors followed by an action; a selector consists of a facility or facilities and a (single) priority. In the following syslog.conf line in Example 12-1, mail.notice is the selector and /var/log/mail is the action (i.e., "write messages to /var/log/mail"). Example 12-1. Sample syslog.conf linemail.notice /var/log/mail Within the selector, mail is the facility (message category) and notice is the level of priority. 12.1.1.1 FacilitiesFacilities are simply categories. Supported facilities in Linux are auth, auth-priv, cron, daemon, kern, lpr, mail, mark, news, syslog, user, uucp, and local0 through local7. Some of these are self-explanatory, but the following are of special note: Used for many security events. auth-priv Used for access-control-related messages. daemon Used by system processes and other daemons. kern mark Messages generated by syslogd itself, which contain only a timestamp and the string --MARK--; to specify how many minutes should transpire between marks, invoke syslogd with the -m [minutes] flag. user The default facility when none is specified by an application or in a selector.
The default facility for OpenLDAP daemon (slapd) messages.
The default facility for Cyrus Imapd messages.
* Wildcard signifying "any facility." Wildcard signifying "no facility." 12.1.1.2 PrioritiesUnlike facilities, which have no relationship to each other, priorities are hierarchical. Possible priorities in Linux are (in increasing order of urgency): debug, info, notice, warning, err, crit, alert, and emerg. Note that the "urgency" of a given message is determined by the programmer who wrote it; facility and priority are set by the programs that generate messages, not by syslog. As with facilities, the wildcards * and none may also be used. Only one priority or wildcard may be specified per selector. A priority may be preceded by either or both of the modifiers, = and !. If you specify a single priority in a selector (without modifiers), you're actually specifying that priority plus all higher priorities. Thus the selector mail.notice translates to "all mail-related messages having a priority of notice or higher," i.e., having a priority of notice, warning, err, crit, alert, or emerg. You can specify a single priority by prefixing a = to it. The selector mail.=notice translates to "all mail-related messages having a priority of notice." Priorities may also be negated: mail.!notice is equivalent to "all mail messages except those with priority of notice or higher," and mail.!=notice corresponds to "all mail messages except those with the priority notice." 12.1.1.3 ActionsIn practice, most log messages are written to files. If you list the full path to a filename as a line's action in syslog.conf, messages that match that line will be appended to that file. (If the file doesn't exist, syslog will create it.) In Example 12-1, we instructed syslog to send matched messages to the file /var/log/mail. You can send messages other places, too. An action can be a file, a named pipe, a device file, a remote host, or a user's screen. Pipes are usually used for debugging. Device files that people use are usually TTYs. Some people also like to send security information to /dev/lp0i.e., to a local line printer. Logs that have been printed out can't be erased or altered by an intruder, but they also are subject to mechanical problems (paper jams, ink depletion, etc.) and are harder to parse if you need to find something in a hurry. Remote logging is one of the most useful features of syslog. If you specify a hostname or IP address preceded by an @ sign as a line's action, messages that match that line will be sent to UDP port 514 on that remote host. For example, the line: *.emerg @mothership.mydomain.org will send all messages with emerg priority to UDP port 514 on the host named mothership.mydomain.org. Note that the remote host's (in this example, mothership's) syslogd process will need to have been started with the -r flag for it to accept your log messages. By default, syslogd does not accept messages from remote systems. If you run a central log server, which I highly recommend, you'll want to consider some sort of access controls on it for incoming messages. At the very least, you should consider TCPwrappers' hosts access (source-IP-based) controls or maybe even local firewall rules (ipchains or iptables). For more information on using iptables, see "Every System Can Be Its Own Firewall: Using iptables for Local Security" in Chapter 3. For an introduction to TCPwrappers, see the sidebar "What are `TCPwrappers-Style Access Controls,' and How Do You Use Them?" in Chapter 5. 12.1.1.4 More sophisticated selectorsYou can list multiple facilities separated by commas in a single syslog.conf selector. To extend Example 12-1 to include both mail and uucp messages (still with priority notice or higher), you could use the line shown in Example 12-2. Example 12-2. Multiple facilities in a single selectormail,uucp.notice /var/log/mail The same is not true of priorities. Remember that only one priority or priority wildcard may be specified in a single selector. You may, however, specify multiple selectors separated by semicolons. When a line contains multiple selectors, they're evaluated from left to right; you should list general selectors first, followed by more specific selectors. You can think of selectors as filters: as a message is passed through the line from left to right, it passes first through coarse filters and then through more granular ones. Continuing our one-line example, suppose we still want important mail and uucp messages to be logged to /var/log/mail, but we'd like to exclude uucp messages with priority alert. Our line then looks like Example 12-3. Example 12-3. Multiple selectors in a single linemail,uucp.notice;uucp.!=alert /var/log/mail Note that in the second selector (uucp.!=alert), we used the prefix != before the priority to signify "not equal to." If we wanted to exclude uucp messages with priority alert and higher (i.e., alert and emerg), we could omit the = (see Example 12-4). Example 12-4. Selector list with a less specific exceptionmail,uucp.notice;uucp.!alert /var/log/mail You might wonder what will happen to a uucp message of priority info: this matches the second selector, so it should be logged to /var/log/mail, right? No: since the line's first selector matches only mail and uucp messages of priority notice and higher, such a message wouldn't be evaluated against the same line's second selector.
There's nothing to stop you from having a different line for dealing with info-level uucp messages, though. You can even have more than one line deal with these if you like. Unlike a firewall rule base, each log message is tested against all lines in /etc/syslog.conf and acted on as many times as it matches. Suppose we want emergency messages broadcast to all logged-in users, as well as written to their respective application logs. We could use something like Example 12-5. Example 12-5. A sample syslog.conf file# Sample syslog.conf file that sorts messages by mail, kernel, and "other," # and broadcasts emergencies to all logged-in users # print most sys. events to tty10 and to the xconsole pipe, and emergencies to everyone kern.warn;*.err;authpriv.none |/dev/xconsole *.emerg * # send mail, news (most), & kernel/firewall msgs to their respective logfiles mail.* -/var/log/mail kern.* -/var/log/kernel_n_firewall # save the rest in one file *.*;mail.none -/var/log/messages Did you notice the - (minus) sign in front of the write-to-file actions? This tells syslogd not to synchronize the specified logfile after writing a message that matches that line. Skipping synchronization decreases disk utilization and thus improves performance, but it also increases the chances of introducing inconsistencies, such as missing or incomplete log messages, into those files. Use the minus sign, therefore, only in lines that you expect to result in numerous or frequent file writes. Besides performance optimization, Example 12-5 also contains some useful redundancy. Kernel warnings plus all messages of error-and-higher priority, except authpriv messages, are printed to the X-console window. All messages having priority of emerg and higher are, too, in addition to being written to the screens of all logged-in users. Furthermore, all mail messages and kernel messages are written to their respective logfiles. All messages of all priorities (except mail messages of any priority) are written to /var/log/messages. Example 12-5 was adapted from the default syslog.conf that the SUSE installer put on one of my systems. But why shouldn't such a default syslog.conf file be fine the way it is? Why change it at all? Maybe you needn't, but you probably should. In most cases, default syslog.conf files either: Assign to important messages at least one action that won't effectively bring those messages to your attention (e.g., by sending messages to a TTY console on a system you access only via SSH). Handle at least one type of message with too much or too little redundancy to meet your needs. We'll conclude our discussion of syslog.conf with Tables Table 12-1 through Table 12-4, which summarize syslog.conf's allowed facilities, priorities, and types of actions. Note that numeric codes should not be used in syslog.conf on Linux systems. They are provided here strictly as a reference, should you need to configure a non-Linux syslog daemon that uses numeric codes (e.g., Cisco IOS) or to send syslog messages to your log server because they're used internally (i.e., in raw syslog packets). You may see them referred to elsewhere.
12.1.1.5 Running syslogdJust as the default syslog.conf may or may not meet your needs, the default startup mode of syslogd may need tweaking as well. Table 12-5 and subsequent paragraphs touch on some syslogd startup flags that are particularly relevant to security. For a complete list, you should refer to the manpage sysklogd (8). In addition, note that when you're changing and testing syslog's configuration and startup options, it usually makes sense to start and stop syslogd and klogd in tandem (see the "What About klogd?" sidebar at the beginning of this chapter if you don't know what klogd is). Since it also makes sense to start and stop these the same way your system does, I recommend that you use your system's syslog/klogd startup script. On most Linux systems, both facilities are controlled by the same startup script, named either /etc/init.d/syslog or /etc/init.d/sysklog (sysklog is shorthand for "syslog and klogd"). On SUSE, Red Hat, and Fedora systems, you can edit the file /etc/sysconfig/syslog to control which flags are sent to syslog via the startup script. On other distributions, you may need to edit the startup script directly to change syslog's startup flags. See Table 12-5 for a list of some of those flags.
The first syslogd flag we'll discuss is the only one used by default in Red Hat 7.x in its /etc/init.d/syslog script. This flag is -m 0, which disables mark messages. mark messages contain only a timestamp and the string --MARK--, which some people find useful for navigating lengthy logfiles. Others find them distracting and redundant, given that each message has its own timestamp anyhow. To turn mark messages on, specify a positive nonzero value after -m that tells syslogd how many minutes should pass before it sends itself a mark message. Remember that mark has its own facility (called, predictably, mark) and that you must specify at least one selector that matches mark messages (such as mark.*, which matches all messages sent to the mark facility, or *.*, which matches all messages in all facilities). For example, to make syslogd generate mark messages every 30 minutes and record them in /var/log/messages, you would first add a line to /etc/syslog.conf similar to Example 12-6. Example 12-6. syslog.conf selector for mark messagesmark.* -/var/log/messages You would then need to start syslogd, as shown in Example 12-7. Example 12-7. Invoking syslogd with 30-minute marksmylinuxbox:/etc/init.d# ./syslogd -m 30 Another useful syslogd flag is -a [socket]. This allows you to specify one or more sockets (in addition to /dev/log for syslogd) from which to accept messages. In Chapter 6, we used this flag to allow a chrooted named process to bounce its messages off of a dev/log socket (device file) in the chroot jail to the nonchrooted syslogd process. In that example, BIND was running in a "padded cell" (subset of the full filesystem) and had its own log socket, /var/named/dev/log. We therefore changed a line in /etc/init.d/syslog that reads as shown in Example 12-8. Example 12-8. init.d/syslog line invoking syslogd to read messages from a chroot jaildaemon syslogd -m 0 -a /var/named/dev/log More than one -a flag may be specified (Example 12-9). Example 12-9. Invoking syslogd with multiple "additional log device" directivessyslogd -a /var/named/dev/log -a /var/otherchroot/dev/log -a /additional/dev/log Continuing down the list of flags in Table 12-5, suppose you need to test a new syslog configuration file named syslog.conf.test, but you prefer not to overwrite /etc/syslog.conf, which is where syslogd looks for its configuration file by default. Use the -f flag to tell syslogd to use your new configuration file (Example 12-10). Example 12-10. Specifying the path to syslogd's configuration filemylinuxbox:/etc/init.d# ./syslogd -f ./syslog.conf.test We've already covered use of the -r flag, which tells syslogd to accept log messages from remote hosts, but we haven't talked about the security ramifications of this. On the one hand, security is clearly enhanced when you use a centralized log server or do anything else that makes it easier for you to manage and monitor your logs. On the other hand, you must take different threat models into account. Are your logs sensitive? If log messages traverse untrusted networks and if the inner workings of the servers that send those messages are best kept secret, then the risks may outweigh the benefit (at least, the specific benefit of syslogd's unauthenticated cleartext remote logging mechanism). If this is the case for you, skip to this chapter's section on Syslog-ng. Syslog-ng can send remote messages via the TCP protocol and can therefore be used in conjunction with stunnel, ssh, and other tools that greatly enhance its security. Since syslog uses only the connectionless UDP protocol for remote logging and therefore can't "tunnel" its messages though stunnel or ssh, syslog is inherently less securable than Syslog-ng. If your log messages aren't sensitive (at least the ones you send to a remote logger), then there's still the problem of Denial of Service and message forgery attacks. If you invoke syslogd with the -r flag, it will accept all remote messages without performing any checks whatsoever on the validity of the messages themselves or on their senders. Again, this risk is most effectively mitigated by using Syslog-ng. But one tool you can use with syslog to partially mitigate the risk of invalid remote messages is TCPwrappers. Specifically, TCPwrappers' hosts access authentication mechanism provides a simple means of defining which hosts may connect to your log server and via which protocols. Hosts-access authentication is easily tricked by source-IP spoofing (especially since syslog transactions are strictly one-way), but it's better than nothing, and it's probably sufficient to prevent mischievous but lazy attackers from interfering with syslog. If you're willing to bet that it is, obtain and install TCPwrappers and refer to its hosts_access(5) manpage for details. Note that despite its name, TCPwrappers' hosts access can be used to control UDP-based applications. |