Guide to Web Server
Logs and Errors

When you want to track Web site stats, server log files are essential resources. Because the location of server log files is defined when a Web server is installed, my server log files and your server log files are probably stored in a completely different directory. The best way to find the server log files is to ask your service provider or the webmaster where the server logs are stored.

Access Log

The access log is the key to discovering who is visiting your Web site and why. Every time someone requests a file from your Web site, an entry is made in the access log, making the access log a running history of every successful and unsuccessful attempt to retrieve information from your Web site. Because each entry has its own line, entries in the access log can be extracted easily when you want to compile stats for your Web site. Compiling stats from access log entries is covered inthe Saturday Morning section of the book under "Understanding Visits and Page Views." Entries in the access log look like this:

Common Log Format

The most prevalent format for server access logs is the Common Log Format. As summarized in Table 1, access log entries in the Common Log Format have seven fields.

Table 1: Fields in the Common Log Format

Extended Log Format

A log format growing in popularity is the extended log format. Access log entries in the Extended Log Format have nine fields. Because the first 7 fields are the same as those shown in Table 1, Table 2 summarizes only the two additional fields.

Table 2 Fields in the Extended Log Format

Referrer Log

The referrer log tells you exactly where a client was before arriving at your Web site. Entries in the referrer log look like this:

In the combined or extended log format, this referrer information is added to the access log. Table 3 summarizes the fields in the referrer log.

Table 3 Fields in the Referrer Log

Agent Log

The agent log tells you the name and version of the browser that requested a file on your server. The values placed in the agent log are taken from the User_Agent field that all browsers supply in the HTTP header accompanying a file request. Although entries in the agent log don't follow a strict format, a general format for entries is the following:

Entries in an actual agent log look like this:

Status Code Classes

Status codes are defined in the HTTP specification and are universal to all Web servers. All status codes are three-digit numbers. Because the first digit of the status code indicates the class of the code, you can often tell at a glance what has happened. Table 4 shows the general classes for status codes. Table 5 shows all the status codes in the HTTP 1.1 specification.

Table 4 - General Classes of Status Codes