Lesson 2 Logging ISA Server Activity

ISA Server features detailed security and access logs that can be generated in standard data formats like World Wide Web Consortium (W3C) and Open Database Connectivity (ODBC). The three logs built into ISA Server, monitor activity of the Firewall service, the Web Proxy service, and packet filtering. New logs can be created daily, weekly, monthly, or yearly.

After this lesson, you will be able to


Locate and view ISA Server logs for the Firewall service, for the Web Proxy service, and for packet filtering

Modify ISA Server logging properties such as log location, log fields, and log file compression.

Configure ISA Server to use an ODBC data source for logging.


Estimated lesson time: 75 minutes

Managing ISA Server Logs

ISA Server logs packet filtering activity, Firewall service activity, and Web Proxy service activity. By default, one new log file is generated for each service per day. Each new log file is stored in the ISALogs folder in the ISA Server installation folder. A typical path for this location is %ProgramFiles%\Microsoft ISA Server\ISALogs\.

You can modify many of the default settings for these three service logs. For example, you can modify the logging format. By default, ISA Server logs to a file in W3C format, but you can also choose to log to a file in ISA format. A third option is to log to a database in ODBC format. Another property you can modify is the frequency of log reports. You can configure ISA Server to generate a log report daily, monthly, weekly, or yearly. Other properties you can modify in ISA Server logging include the specific fields to generate in the log report, and the location to store the log file.

Logging properties are modified in ISA Management in the Logs folder of the Monitoring Configuration node. Figure 9.6 shows the three service logs you can configure when the Logs folder is selected.

Figure 9.6 Managing logs in ISA Management

Properties for each service log can be configured by double-clicking any one of the service log icons in the details pane. Figure 9.7 shows the Log tab associated with the Properties dialog box of a service log. This tab allows you to configure the log format and storage location.

Figure 9.7 The Log tab of a service log's properties

Figure 9.8 shows the Fields tab associated with the Properties dialog box of a service log. This tab allows you to specify which fields are included in the log report.

Figure 9.8 The Fields tab of a service log's properties

When a service log is enabled, logs are generated for each server in the array. The ISA Server reporting system then centralizes the logs, collecting data from all the servers and combining them into a single report.

Logging to a File

You can save ISA Server logs to a file in either of the following formats:

W3C format

ISA format

Although the logging configuration is array-wide, log files are created on every ISA Server in the array. You can specify the location of a log file in the Options dialog box, as shown in Figure 9.9, which you access by clicking the Options button in the specific service log's Properties dialog box.

Figure 9.9 Specifying a log file location

The default location for the log file is the ISALogs folder. When you leave this option selected, the log file is saved to the same location within the ISA Server installation folder on every ISA Server computer in the array. If you specify another folder, however, the path you specify for the log file must exist on every server in the array.

It is recommended that you log to NT file system (NTFS) partitions so that the log files benefit from advanced file system features like NTFS security and data compression.

W3C Format

W3C logs contains both data and directives describing the version, date and logged fields. Because the fields are described in the file, unselected fields are not logged. The tab character is used as delimiter. Date and time are displayed in Greenwich Mean Time (GMT). Figure 9.10 shows part of a Web Proxy log file in W3C format.

Figure 9.10 A Web Proxy log file in W3C log format

ISA Format

ISA format contains only data with no directives. All fields are always logged; unselected fields are logged with a dash to indicate that they are empty. The comma character is used as delimiter. The Date and Time fields display local time. A sample Firewall service log file in ISA format is shown in Figure 9.11.

Figure 9.11 A Firewall service log file in ISA format

Log File Names

A log file name is derived from the name of the service being logged, the log file format, and the date of the log.

The first three letters of the service log file name indicate the service being logged. FWS indicates the Firewall service, WEB indicates the Web Proxy service, and IPP indicates IP packet filters. This is followed by the letters EXT if the file is in the W3C extended log file format. (A file in ISA Server file format is designated by the absence of the letters EXT.) Next in the file name is a letter indicating the frequency of the log: the letter D represents a daily log, the letter W indicates a weekly log, the letter M represents monthly, and the letter Y indicates a yearly log. Finally, the date of the log file follows in the form yyyymmdd. The date May 21, 2002 would thus be represented as 20020521, and a daily W3C Web Proxy service log created on that date would be named WEBEXTD20020521.log. A monthly Firewall service log file in ISA format created on the same date would be named FWSM20020521.log.

Log File Options

In order to save the amount of disk space required for the log files, ISA Server allows you to configure the following options:

Compress log files to reduce the disk space required. Files are compressed only if stored on NTFS partitions.

Limit the number of log files saved on all the servers in the array.


To configure logging to a file


In the console tree of ISA Management, expand the Monitoring Configuration node and click the Logs folder.
In the details pane, right-click the applicable service, and then click Properties.
On the Log tab, click the File radio button.
In the Format drop-down list box, select the log format.
In the Create A New File drop-down list box, click a time period that specifies how often a new log file will be created.
To modify the location of the log file, click the Options button and then do one of the following:
To save the file to the default folder, click the ISALogs Folder (In The ISA Server Installation Folder) radio button.
To save the file to another folder, click the Other Folder radio button and then type a folder name in the text box or click Browse and find a storage location.


To set the maximum number of log files, click the Limit Number Of Log Files check box, and in the associated text box, type the maximum number of log files to keep for the array.
To compress the log files, click the Compress Log Files check box.

Logging to a Database

You can store ISA Server logs to an ODBC database instead of a file. You configure this option by selecting the Database radio button on the Log tab of a service log's Properties dialog box.

Before configuring ISA Server to log data to a database, you must create a database and tables to support logging. The root folder of the ISA Server CD-ROM includes the following sample scripts that create tables and indexes to support database logging:

Pf.sql defines the packet filter log table called PacketFilterLog and indexes to support table queries.

W3PROXY.sql defines the Web Proxy service log table WebProxyLog and indexes to support table queries.

FWSRV.sql defines the Firewall service log table called FirewallLog and indexes to support table queries.

For example, to create a SQL Server 2000 log database named ISALogs that can grow to a maximum size of 100 MB and a transaction log file that can grow to a maximum size of 50 MB, perform the following steps:

Open SQL Query Analyzer and connect to the SQL Server where you will create the log database.
In the Editor pane of the Query window, enter the following Transact-SQL code:

USE master
GO
CREATE DATABASE ISALogs

This code creates a database with the name ISALogs.

Directly beneath the code that you just entered in the Editor pane, enter the following code:

ON PRIMARY
(
NAME = ISALog_dat,
FILENAME = 'C:\Program Files\Microsoft SQL Server\MSSQL\Data\ISALog.mdf',
SIZE = 50,
MAXSIZE = 100,
FILEGROWTH = 1
)

This code defines the primary file. The logical name of the file is ISALog_dat. The path and file name used by the operating system is C:\Program Files\Microsoft SQL Server\MSSQL\Data\ISALog.mdf. The initial size of the file is 50 MB, and the maximum size that the file can grow is 100 MB. The growth increment of the file is 1 MB.

If the path specified in the code is different on your computer, you must specify the correct path in the code, or leave the code as is and create the file path on the C: drive before running this code in step 5.

Directly beneath the code that you just entered in the Editor pane, enter the following code:

LOG ON
(
NAME = ISALog_log,
FILENAME = 'C:\Program Files\Microsoft SQL Server\MSSQL\Data\ISALog.ldf',
SIZE = 25,
MAXSIZE = 50,
FILEGROWTH = 1
)
GO

This code defines the log file. The logical name of the file is ISALog_log. The path and file name used by the operating system is C:\Program Files\Microsoft SQL Server\MSSQL\Data\ISALog.ldf. The initial size of the file is 25 MB, and the maximum size that the file can grow is 50 MB. The growth increment of the file is 1 MB.

If the path specified in the code is different on your computer, you must specify the correct path in the code, or leave the code as is and create the file path on the C: drive before running this code in step 5.

Execute all the code as one statement.

The Messages tab of the Results pane displays two messages, one saying that 100 MB of disk space has been allocated to the primary file and the other saying that 50 MB of disk space has been allocated to the transaction log file.

Verify that the current database is ISALogs, then open and execute each of the .sql files appearing on the root of the ISA Server CD-ROM.
The log tables and indexes are created for each service.

After you create the database and the log tables and indexes, you must set up the ISA Server computer to use the data source name. Perform the following steps:

On the ISA Server computer, click Start, point to Programs, point to Administrative Tools, and then click Data Sources (ODBC).
On the System DSN tab, click the Add button.
In the Create New Data Source screen, select the appropriate driver for the database.

If you are using SQL Server 2000, select SQL Server.

Follow the on-screen instructions to create the database.

The data source name is the one you enter in the ODBC data source (DSN) text box when you configure the log database properties.

Do not use spaces in the data source name or ISA Server services will stop.

Follow these steps to configure logging to a database:

In the console tree of ISA Management, expand the Monitoring Configuration node and click the Logs folder.
In the details pane, right-click the applicable service, and then click Properties.
On the Log tab, click the Database radio button.
Confirm or modify the following parameters:
ODBC data source (DSN)
Table name

This is the name of each table as specified in the .sql files on the root of the ISA Server CD-ROM. For example, the table name for the Packet Filters log is PacketFilterLog.

User account


Click the Set Account button, type the user name in the User text box, and type the password in the Password and Confirm Password text boxes. You can use a SQL Server logon ID or a Windows user account, depending on how you configure SQL Server authentication.

Logging Packets

All packets that pass through ISA Server can be logged to the packet filter log. You can configure exactly which packets are logged by following these guidelines:

By default, when you install ISA Server, all dropped packets are logged to the packet filter log. When you disable packet filtering, logging is turned off altogether.

You can configure ISA Server to disable logging for packets that are dropped due to any specific block-mode IP packet filter.

You can configure ISA Server to log all packets—allowed and dropped—that are communicated by way of ISA Server. When you enable logging of allowed packets, all packets that pass through ISA Server are logged in the packet filter log.

Logging allowed packets and blocked packets causes a considerable load on the server.

Follow these steps to log allowed packets:


In the console tree of ISA Management, expand the Access Policy node, right-click the IP Packet Filters folders, and then click Properties.

On the Packet Filters tab, select the Log Packets From 'Allow' Filters check box.

You can only log allowed packets if packet filtering is enabled.

Follow these steps to log blocked packets:


In the console tree of ISA Management, expand the Access Policy node and click the IP Packet Filters folder.

On the View menu, confirm that the Advanced option is selected.

In the details pane, right-click the block-mode IP packet filter whose packets you want to log and then click Properties.

On the General tab, select the Log Any Packets Matching This Filter check box.


Firewall and Web Proxy Log Fields

When you select the Fields tab of a service log's properties, you can select any of the available number of fields to log. By default, the packet filters log reports nine of a possible twelve fields. Also by default, the Firewall service and Web Proxy service logs report 18 of a possible 27 fields. You can change these default selections at any time or restore the defaults by clicking the Restore Defaults button.

Follow these steps to specify fields to log:


In the console tree of ISA Management, expand the Monitoring Configuration node and click the Logs folder.
In the details pane, right-click the applicable service, and then click Properties.
Click the Fields tab.

You complete the following tasks from this tab:

To select specific fields for logging, enable the appropriate check box.
To clear all the check boxes in the field list, click the Clear All button.
To enable all the check boxes in the field list, click the Select All button.
To enable a default set of fields in the ISA Server log file, click the Restore Defaults button.

Table 9.2 lists the fields that you can include in each of the ISA Server Firewall and Web Proxy log files. The field name noted in parentheses is relevant when you use the 3C extended log file format. For W3C names, note that the prefix "c" represents client actions, "s" represents server actions, "cs" represents client-to-server actions, "sc" represents server-to-client actions, and "r" represents remote actions.

Some fields are relevant for either Web Proxy Service or Firewall Service, but not both. For each of these fields, the table indicates which service the field applies to. Note that, in ISA Server log format, an empty field appears in the log with a hyphen (-). In W3C log file format, the field does not appear at all if it is not applicable to the service.

Tables 9.2 through 9.6 list possible values for some of these fields. You can view these field names through the Monitoring Configuration node in ISA Management. To do so, click on the Logs folder, and then double-click on the log files belonging to the ISA Server Firewall Service and the ISA Server Web Proxy Service. Finally, click on the Fields tab to review the fields available for logging.

Table 9.2 Available Logging Fields for Firewall and Web Proxy Services

Table 9.3 presents the possible values for the client agent (c-agent in W3C format) parameter reported by the ISA Server Firewall service log. This expands on the information presented in the third row of Table 9.2. When all fields are selected to be logged, one of the values listed in Table 9.3 appears in the third field position of the log. This value indicates the operating system used by the client.

Table 9.3 Operating System Values

Table 9.4 lists the possible values for the object source (s-object-source in W3C format) parameter reported by the ISA Server Web Proxy service log. This expands on the information presented in row 21 of Table 9.2. When all fields are selected to be logged, one of the values listed in Table 9.4 appears in the 21st field position of the log. This value indicates which source was used to retrieve the current object.

Table 9.4 Object Source Values

Table 9.5 lists some of the possible values for the result code (or sc-status in W3C format) parameter reported by the ISA Server Firewall and Web Proxy service logs. This expands on the information presented in row 22 of Table 9.2.

This value is often used to indicate a Windows error code (for values less than 100), an HTTP status code (for values between 100 and 1,000), or a Winsock error code (for values between 10,000 and 11,004). A connection usually has two log entries: the first entry is for when the connection is established; the second entry is for when the connection is terminated. The first entry is logged with a result code of either 0 (Successful Connection) or 13301 (Connection Refused) and a byte count of 0. The second entry is logged with a result code of either 20000 (Normal Connection Termination) or 20001 (Unexpected Connection Termination) and the corresponding byte count.

For additional information about error codes, consult the Microsoft Developers Network online at http://msdn.microsoft.com, or Microsoft Technet at http://www.microsoft.com/technet.

Table 9.5 Sample Result Code Values

Table 9.6 lists the possible values for the cache info (s-cache info in W3C format) parameter reported by the ISA Server Web Proxy service log. This expands on the information presented in row 23 of Table 9.2. This value indicates why the object was or was not cached.

Table 9.6 Cache Info Values

Packet Filter Log Fields

You can use the ISA Server log to monitor and analyze the status of the packet filters. Table 9.7 lists the fields that you can include in each of the Packet Filters log files. You can view these field names through the Monitoring Configuration node in ISA Management. To do so, click on the Logs folder, and then double-click on Packet Filters log file. Finally, click on the Fields tab to review the fields available for logging.

Table 9.7 Log Fields for Packet Filtering

Practice: Reading Web Logs

Exercise: Analyzing a Web Log

To prepare for this exercise, open the sample Web Proxy log named WEBEXTD20010129, which can be found in the \Exercises\Chapter9\ folder of the Supplemental Course Materials CD-ROM accompanying this book. The log file is in W3C format. Once you have opened the file, use the information in this chapter to answer the following questions. Be sure to provide the specific code value or values that are essential to answer each question.

For the first Web connection in the log (initiated at 02:31:35), was the traffic heavier from the client to the server, or vice-versa?

For this same connection, was the Web request fulfilled from the cache or from the Internet?

At what time was a connection reset by the remote side?

Was the object requested at 02:31:45 found in the cache? Had the object been modified? What source was ultimately used to fulfill the request?

Which Web operation was conducted at 04:55:14? Was the destination found?


Lesson Summary

Three logs are built into ISA Server that monitor the activity of ISA Server packet filters, the Firewall service, and the Web Proxy service. By default, one new log file is generated for each service per day. Also by default, each new log file is stored in the ISALogs folder below the ISA Server installation folder. Logging properties are modified in ISA Management in the Logs folder of the Monitoring Configuration node.

You can save ISA Server logs either to a file or to an ODBC database. When you save to a file, you can specify either the W3C format or the ISA format. To conserve disk space required for the log files, ISA Server allows you to compress log files and to limit the number of log files saved on all the servers in the array.

ISA Server allows you to specify which fields to report to the service logs. You should consult ISA Server Help to determine which fields are available for each service and what each field value represents.