User:Reorx/Log Parser

The EQWiki - Your source for random EverQuest knowledge

Design and thoughts on the creation of a log parser.

OL7euU See it for the first time!!...

VtTySm As usual, the webmaster posted correctly..!!

Contents 1 Performance 1.1 Test Timings 2 Design 2.1 Basic Operation

Performance

Test Timings

Some very rough timings of the loading and parsing of various sized files.

File Size	Load Time	Line Parse	Simple Type Parse
0.1 Mb	3 ms	1 ms	3 ms
1 Mb	40 ms	6 ms	4 ms
10 Mb	300 ms	100 ms	50 ms
100 Mb	2500 ms	700 ms	400 ms

As can be seen, most of the time for larger files is spent on the actual loading rather than the parsing.

Design

Basic Operation

• Read part of log file (ex: 10 kb)
• Parse as much of the log file as possible
• Update display with current parse data
• Repeat until all of log file loaded and parsed