[ILUG] (OT) - http Log files

[Ciaran]

There is a good little prog called webalizer you can run to convert your log
files to nicely formatted HTML - http://www.mrunix.net/webalizer/ . I have
altered the code to allow it to be run from a browser as a cgi-script, if you
want it mail me. It is a very fast little prog., does upwards of 20,000 entries
a second on my remote server (don't know what spec it is, it's with webfusion
in England so I reckon fairly good). Performs similarly well on my home machine
(pentium 500), so you could download the files and run it locally or whatever.
To be honest though, taking up the bandwidth necessary to download 28 Meg may
hurt your ISP more than a few seconds of CPU time. A log file that size would
have about 300,000 entries in it, would take ~15 - 20 seconds to analyse at a
modest estimate. Your call, maybe cut the file up before analysing?

Regards,

Ciaran.

On Fri, 08 Dec 2000, Garret Flynn wrote:
> [snip]
> > Is there an easier way.. Can I run log analysis on a remote file.. I can
> > telnet to my isp, and get a bash prompt, but it seems to be a *very* limited
> > shell.. no grep, or tail, or anything I recognise like that.
> 
> 
> 
> If it doesn't have the grep command you would try using a perl 'one liner' 
> from the command line to grep the file
> ie
> 
> perl -e 'open(I,"access.log");while(<I>){print $_ if($_=~/SOMETHING/);}' >results.txt
> 
> replace the word SOMETHING with the string you want to grep for.
> 
> [snip]
> > Can I put a some sort of script up there to do the analysis, and send me
> > the results ?
> 
> You could, but i don't think IOL would be to happy about it.
> You have to remember that your site is basically a <virtualhost> entry in
> apache and that they're also hosting other sites on the same physical
> server. If you are taking all the CPU time to do the analysis , it would
> slow down everyone elses sites. Also you may have a CPU time limit, in
> which case you may not be able to run a full analysis..
> 
> My advice would be to make a copy of the log file (maybe use the perl grep
> trick I did above) , gzip it and download it.
> 
> -Garret
> 
> 
> 
> -- 
> Irish Linux Users' Group: ilug at linux.ie
> http://www.linux.ie/mailman/listinfo/ilug for (un)subscription information.
> List maintainer: listmaster at linux.ie
-- 
---
"Anyone can make mistakes, but for very big mistakes, you need a computer."

cj at x21.org
RESIST AND CREATE . . .