Forum: Size logs - bigger than 2Gb

 

« Back To Thread List

jackadamantina
2799 days ago
I had multiple logs access.log - size 2Gb or bigger. This logs in Awstats is compiled with success, but in Jawstats this logs do not compiler, and the stay blank page.

What the problem? Any sugestion?
mbsouth
2770 days ago
Does your copy of JAWStats works with less data?
Have you tested this?
tchung61
2241 days ago
Yes, this is a real problem. JAWStats (version 0.7) cannot process very large data files generated by AWStats.
The AWStats data files (awstatsMMYYY..txt) that are over 50MB in size, JAWStats cannot process them. The page just come up blank.
If you trace the PHP code, you will find that the PHP script is hung at this line:
$this->sAWStats = htmlspecialchars(file_get_contents($sFilePath));
in the "clsAWStats.php" script.
I believe it is because the "file_get_contents" that tries to convert the AWStats data file into one big string, cannot handle it when this data file is too big.
The fix is to not to use that built-in function "file_get_contents".
Will come up with some solution later.
mbsouth
2226 days ago
JAWStats is brocken by design!

Firstly you should understand that when using "file_get_contents" your fetching the entire string of data into a variable, that variable is stored in the hosts memory. accepted

Firstly you should understand that when using file_get_contents your fetching the entire string of data into a variable, that variable is stored in the hosts memory.
If that string is greater then the size dedicated to the PHP process then PHP will halt.

You can boost the memory limit at run-time with

ini_set('memory_limit', $new_limit_here);

though some servers do not allow this override. But slurping an entire 50mb file into memory is generally not a good idea. Processing it line-by-line would be easier for the most part.

just my 2 cents
mbsouth
2226 days ago
Missing exception handling too:


if( is_file(...) && is_readable(...) ){...}
or/and
if( file_get_contents($sFilePath) ){...}
or/and
htmlspecialchars(...,[ENT_SUBSTITUTE,[ENT_DISALLOWED]]);


$this->sAWStats = htmlspecialchars(file_get_contents($sFilePath));


jm2c

« Back To Thread List

Participate in this discussion

 

You must be logged in to post in this thread.

 

Version 0.7 beta available now

Released 25th Janury 2009

Try the online demo or download now.

 
Latest JAWStats News:

Version 0.7 Beta Released

Sprechen Sie Deutsch? Multi-lingual JAWStats v0.7 is finally here.

posted 25th Jan '09. read more »
 

Help JAWStats

Please donate a dollar (or more!) towards the hosting costs for this site. This money doesn't go anywhere near my pocket, it is purely used against my hosting bill.

 

Get $50 off your webhosting

You could do far worse than choose Dreamhost. I've used them for 6 years and they're great. Simply enter the promo code: JAWSTATS on signup!