What the results mean

• How the web works. This section discusses what happens when somebody connects to your web site, and what you can and can't find out about them. If you think that you can get statistics on how many people have visited your web site (or want to know why you can't), then this section is for you.
• Analog's reports. This section gives a summary of analog's reports, what they contain, and which commands influence each one.
• Analog's definitions. This section gives precise details on all of analog's terminology, exactly what is counted in each report, and so on.

How the web works

This section is fairly long, but it's worth reading carefully. If you understand the basics of how the web works, you will understand what your web statistics are really telling you.

So, what do you know about it? First, I make one request for your front page. You know the date and time of the request and which page I asked for (of course), and the internet address of my computer (my host). I also usually tell you which page referred me to your site, and the make and model of my browser. I do not tell you my username or my email address.

Next, I look at the page (or rather my browser does) to see if it's got any graphics on it. If so, and if I've got image loading turned on in my browser, I make a separate connection to retrieve each of these graphics. I never log into your site: I just make a sequence of requests, one for each new file I want to download. The referring page for each of these graphics is your front page. Maybe there are 10 graphics on your front page. Then so far I've made 11 requests to your server.

After that, I go and visit some of your other pages, making a new request for each page and graphic that I want. Finally, I follow a link out of your site. You never know about that at all. I just connect to the next site without telling you.

The other sort of cache is on a larger scale. Almost all ISP's now have their own cache. This means that if I try to look at one of your pages and anyone else from the same ISP has looked at that page recently, the cache will have saved it, and will give it out to me without ever telling you about it. (This applies whatever my browser settings.) So hundreds of people could read your pages, even though you'd only sent it out once.

You can also know what people told you their browsers were, and what the referring pages were. You should be aware, though, that many browsers lie deliberately about what sort of browser they are, or even let users configure the browser name. Also, a few browsers send incorrect referrers, telling you the last page that the user was on even if they weren't referred by that page. And some people use "anonymizers" which deliberately send false browsers and referrers.

1. You can't tell the identity of your readers. Unless you explicitly require users to provide a password, you don't know who connected or what their email addresses are.
2. You can't tell how many visitors you've had. You can guess by looking at the number of distinct hosts that have requested things from you. Indeed this is what many programs mean when they report "visitors". But this is not always a good estimate for three reasons. First, if users get your pages from a local cache server, you will never know about it. Secondly, sometimes many users appear to connect from the same host: either users from the same company or ISP, or users using the same cache server. Finally, sometimes one user appears to connect from many different hosts. AOL now allocates users a different hostname for every request. So if your home page has 10 graphics on, and an AOL user visits it, most programs will count that as 11 different visitors!
3. You can't tell how many visits you've had. Many programs, under pressure from advertisers' organisations, define a "visit" (or "session") as a sequence of requests from the same host until there is a half-hour gap. This is an unsound method for several reasons. First, it assumes that each host corresponds to a separate person and vice versa. This is simply not true in the real world, as discussed in the last paragraph. Secondly, it assumes that there is never a half-hour gap in a genuine visit. This is also untrue. I quite often follow a link out of a site, then step back in my browser and continue with the first site from where I left off. Should it really matter whether I do this 29 or 31 minutes later? Finally, to make the computation tractable, such programs also need to assume that your logfile is in chronological order: it isn't always, and analog will produce the same results however you jumble the lines up.
4. Cookies don't solve these problems. Some sites try to count their visitors by using cookies. This reduces the errors. But it can't solve the problem unless you refuse to let people read your pages who can't or won't take a cookie. And you still have to assume that your visitors will use the same cookie for their next request.
5. You can't follow a person's path through your site. Even if you assume that each person corresponds one-to-one to a host, you don't know their path through your site. It's very common for people to go back to pages they've downloaded before. You never know about these subsequent visits to that page, because their browser has cached them. So you can't track their path through your site accurately.
6. You often can't tell where they entered your site, or where they found out about you from. If they are using a cache server, they will often be able to retrieve your home page from their cache, but not all of the subsequent pages they want to read. Then the first page you know about them requesting will be one in the middle of their true visit.
7. You can't tell how they left your site, or where they went next. They never tell you about their connection to another site, so there's no way for you to know about it.
8. You can't tell how long people spent reading each page. Once again, you can't tell which pages they are reading between successive requests for pages. They might be reading some pages they downloaded earlier. They might have followed a link out of your site, and then come back later. They might have interrupted their reading for a quick game of Minesweeper. You just don't know.
9. You can't tell how long people spent on your site. Apart from the problems in the previous point, there is one other complete show-stopper. Programs which report the time on the site count the time between the first and the last request. But they don't count the time spent on the final page, and this is often the majority of the whole visit.

Defenders of counting visits etc. claim that these are just small approximations. I disagree. For example, almost everyone is now accessing the web through a cache. If the proportion of requests retrieved from the cache is 50% (a not unrealistic figure) then half of the users' requests aren't being seen by the servers.

Other defenders of these methods claim that they're still useful because they measure something which you can use to compare sites. But this assumes that the approximations involved are comparable for different sites, and there's no reason to suppose that this is true. Pirolli & Pitkow's results show that the figures you get depend very much on how you count them, as well as on your server configuration. And even once you've agreed on methodology, different users on different sites have different patterns of behaviour, which affect the approximations in different ways: for example, Pirolli & Pitkow found different characteristics of weekday and weekend users at their site.

I've presented a somewhat negative view here, emphasising what you can't find out. Web statistics are still informative: it's just important not to slip from "this page has received 30,000 requests" to "30,000 people have read this page." In some sense these problems are not really new to the web -- they are present just as much in print media too. For example, you only know how many magazines you've sold, not how many people have read them. In print media we have learnt to live with these issues, using the data which are available, and it would be better if we did on the web too, rather than making up spurious numbers.

Another, extremely well-written document on these ideas is Measuring Web Site Usage: Log File Analysis by Susan Haigh and Janette Megarity. Being on a Canadian government site, it's available in both English and French. Or for an even more negative point of view, you could read Why Web Usage Statistics are (Worse Than) Meaningless by Jeff Goldberg.

Analog's reports

Top lines

General Summary

The figures in parentheses represent the seven days given at the top of this report: the seven days before the program was run.

You can't find out the number of visitors or visits you've had, and don't believe any program which tells you that you can. See the section on How the web works for a discussion of this.

Time reports

week beg.: #reqs: pages: 
---------: -----: -----: 
13/Sep/98: 69614: 25277: 
20/Sep/98: 10032:  5861: 

The timezone is whatever your server records time in -- usually your server's local time, or sometimes GMT.

Time summaries

day: #reqs: pages: 
---: -----: -----: 
Sun:  2031:  1193: 
Mon:  8001:  4668: 
Tue:     0:     0: 
Wed: 13934:  5915: 
[etc.]

Other reports

#reqs: %bytes:       last date: file
-----: ------: ---------------: ----
 4123:  2.29%: 21/Sep/98 01:57: /~sret1/analog/
 3064:  0.15%: 21/Sep/98 01:54: /~sret1/analog/analogo.gif
 1737:  0.01%: 21/Sep/98 01:53: /~sret1/images/bar1.gif
 1692:  0.01%: 21/Sep/98 01:53: /~sret1/images/bar16.gif
 1685:  0.01%: 21/Sep/98 01:53: /~sret1/images/bar8.gif
67345: 97.54%: 21/Sep/98 02:04: [not listed: 8,175 files]

• The Host Report lists all computers which downloaded files from you.
• The Domain Report lists which countries those computers came from. (If you only get "unresolved numerical addresses", see the FAQ.)
• The Organisation Report attempts to list the organisations (companies, institutions, ISPs etc.) which the computer was registered under.
• The Host Redirection Report and Host Failure Report list all computers which encountered redirections or errors.
• The Request Report (the example above) lists which files were downloaded.
• The Directory Report lists which directories those files came from.
• The File Type Report lists the file types (actually, extensions) of those files.
• The File Size Report breaks them down by size.
• The Processing Time Report shows the time taken to serve each file.
• The Redirection Report lists the filenames which resulted in redirections: mainly directories without the final slash, and "click-thru"'s.
• The Failure Report lists the filenames which caused errors.
• The Referrer Report lists which pages linked to your files (and also pages which included your images).
• The Referring Site Report lists the servers those referrers were on.
• The Search Query Report and the Search Word Report list which search terms people used to find your site.
• The Internal Search Query Report and Internal Search Word Report list the search terms people used on scripts within your site.
• The Redirected Referrer Report lists the referrers which led to redirections.
• The Failed Referrer Report is essentially a broken link report.
• The Browser Report lists the detailed versions of browsers used, and the Browser Summary collects them by vendor. You should be aware that browsers can lie about what sort of browser they are.
• The Operating System Report lists the operating systems of the visitors whose browser types you know (as far as possible: it's not always possible to distinguish accurately between different Windows versions, for example, because the same browser can run on more than one Windows version).
• The Virtual Host Report lists the activity of your various virtual domains.
• The Virtual Host Redirection Report and Virtual Host Failure Report give the number of redirections and errors on each of those domains.
• The User Report lists your visitors if your server requires authentication, or perhaps the visitors' cookies.
• The User Redirection Report and User Failure Report list the users who encountered redirections or errors.
• The Status Code Report lists the number of each HTTP status code that you had.

Most of these reports have a hierarchical structure, like this example for the Domain Report:

no.: #reqs: %bytes: domain
---: -----: ------: ------
  1: 13243: 16.23%: .com (Commercial)
   :  1262:  1.26%:   aol.com
  2: 11783: 25.64%: .jp (Japan)
   :  9592: 22.19%:   ad.jp
   :  1043:  1.97%:   co.jp
  3: 10073: 11.62%: .net (Network)
   :  1926:  1.71%:   uu.net
  4:  9657: 13.31%: [unresolved numerical addresses]
  5:  7388:  8.04%: .uk (United Kingdom)
   :  5792:  5.74%:   ac.uk
   :  1510:  1.99%:   co.uk
   : 18502: 25.16%: [not listed: 82 domains]

Bottom lines

Analog's definitions

We start with some basic definitions. The host is the computer which has asked you for a file (often called the "client"). The file might be a page (i.e., an HTML document) or it might be something else, such as an image. By default filenames ending in (case insensitive).html, .htm, or / count as pages.

The total requests counts all the files which have been requested, including pages, graphics, etc. (Some people call this the number of hits, but that word is also used in other ways by other people, so I avoid it). The requests for pages obviously only counts pages. One user can generate many requests by requesting lots of different files, or the same file many times.

The referrer for a request is the place that the user (or his computer) heard about your file from. If he followed a link to reach a page, it will be the previous page. In the case of a graphic on a page, the referrer will be the page containing the graphic.

Analog's kilobytes are 1024 bytes.

First, successful requests are those with HTTP status codes in the 200's (where the document was returned) or with code 304 (where the document was requested but was not needed because it had not been recently modified and the user could use a cached copy). Successful requests for pages refers to those lines on which the file requested was named and was a page.

Redirected requests are those with other codes in the 300's, indicating that the user was directed to a different file instead. The most common cause of these requests is that the user has incorrectly requested a directory name without the trailing slash. The server replies with a redirection ("you probably mean the following") and the user then makes a second connection to get the correct document (although usually the browser does it automatically without the user's intervention or knowledge). The other common cause of redirected requests is their use as "click-thru" advertising banners.

Failed requests are those with codes in the 400's (error in request) or 500's (server error). They come about for a variety of reasons, but the most common are when the requested file is not found or is read-protected.

Finally, requests returning informational status code are those with status codes in the 100's. These are very rare at the moment.

The "not listed" line at the bottom of each of the non-time reports represents those items which were not listed because they were below the floor for the report.

The figures in parentheses in the General Summary are for the last seven days: either the seven days before the TO time, or if no TO time is given, the seven days before the time of the program start. (It would be nicer to use the seven days before the last time in the logfile, but we don't know when this is until we've read the whole logfile, and by then it's too late.) The figures for the last seven days are not included if all, or none, of the requests fall in the last seven days.

Frequently asked questions

List of Questions

1. Understanding the Output
   See also What the results mean.
   1. How do I find out the number of hits from your data?
   2. Why are there so many referrers from my own site?
   3. The analysis covers exactly a week, but the figures for the last seven days don't agree with the totals.
   4. I only have 240 requests in total. Why does analog think there are 840 requests per week?
   5. The pie charts don't agree with the figures in the tables.
   6. Why doesn't analog agree with the counter on my page?
   7. Why doesn't analog agree with grepping the logfile?
   8. Why doesn't analog agree with my other logfile analysis program?
   9. Why do I only get "unresolved numerical addresses" in the Domain Report?
   10. Why are directories listed in the Request Report?
   11. When someone reads one of my PDF files, it scores dozens of hits.
   12. Kilobytes should be 1000 bytes, not 1024 bytes.
   13. The Organisation Report doesn't identify organisations correctly.
   14. "Organization" isn't spelled correctly.

Understanding the Output

1. How do I find out the number of hits from your data?
   I don't use the word hits, because people use it in different ways, so it's misleading. I use requests for the number of transfers of any type of file (text, graphics, ...), and page requests for the number of transfers of HTML pages. See the section on Analog's definitions for more information.
2. Why are there so many referrers from my own site?
   These come from all the internal links on your site, and all the graphics on your pages. See the section on How the web works for more information.
3. The analysis covers exactly a week, but the figures for the last seven days don't agree with the totals.
   The figures in parentheses are for the seven days before the time the program was run. They are never for the seven days before the end of the logfile.
4. I only have 240 requests in total. Why does analog think there are 840 requests per week?
   If you have 240 requests in two days, that's a rate of 840 requests per week. Just like if you drove 28 miles in 20 minutes, you'd have driven at 84 miles per hour.
5. The pie charts don't agree with the figures in the tables.
   Possibly you are looking at out-of-date images. Make sure to reload the images as well as the text
6. Why doesn't analog agree with the counter on my page?
   There are lots of possible reasons. Do they both start from the same date? Are you just looking at requests for that one page with analog, not for all your other pages and graphics? Also, analog will record all requests to that page; if it's a graphic, your counter will only measure requests from people on graphical browsers that reached that place on the page.
7. Why doesn't analog agree with grepping the logfile?
   Have you understood what analog includes in its counts? In particular, most reports only list "successful" requests (HTTP status codes 200-209 & 304). A naïve grep would count failures too.
8. Why doesn't analog agree with my other logfile analysis program?
   Small differences can be put down to different parsing. But if you are seeing large differences, you have to understand what analog counts, and what the other program counts. For example, some programs count HTTP status codes 301 & 302 as successes, whereas I think that to do so gives extremely misleading results.
9. Why do I only get "unresolved numerical addresses" in the Domain Report?
   Your server only records the numerical IP address of the hosts that contact you, not their names.
10. Why are directories listed in the Request Report?
    They are not directories, they are pages with the same name as the directory. For example, I have both a directory called /analog/ and a page called /analog/ (which happens to be the same as /analog/index.html).
11. When someone reads one of my PDF files, it scores dozens of hits.
    PDF files are often downloaded and read one page at a time, and each page will then count as a separate request. Although this is not ideal, it's much less clear what to do about it. Analog has no way of knowing how many pages constituted a single download in the reader's mind. As usual, we can only reliably report how many requests there were at the server, not guess what users did with the file later.
12. Kilobytes should be 1000 bytes, not 1024 bytes.
    Personally I think that 1024 bytes is a kilobyte.
13. The Organisation Report doesn't identify organisations correctly.
    I admit they aren't perfect, but this is because in domains in which organisations aren't all at the same level in the domain hierarchy, there is no way to identify them perfectly without long lists.
14. "Organization" isn't spelled correctly.
    Yes it is. If you want American spellings, you have to specify it.