|Subject:||Logparser to parse IIS logs and output to (multiple) new files|
|Date:||Thu, 21 Jun 2007|
I just started reading about logparser and I would like to use it to
rid IIS logs of entries containing bot names listed in a text file.
I'll be using autoit and Logparser COM.
I was wondering what the best way to go about this would be. From what
I have read in the logparser documentation, it seems that the
following would not be possible:
1. Logparser query cannot be setup such that: it is passed a file (say
a list of bot names) and told to filter the inputfiles of all lines
containing any of the terms in said file.
2. Logparser cannot produce output upon running a query on a set of
multiple files such that the output is organized into files that
correspond to the original files. (So if files A, B, C are to be
filtered, the output cannot automatically be placed in files A1, B1,
C1 such that the contents of A1 = filtered contents of A, and
similarly for B1 and C1).
Given the above assumptions, I have concluded that If I want to filter
a large set of files and if the filtering involves a large set of
filter terms, the best way to utilize logparser would be to run a
query multiple times in nested loops (inner loop traverses the list of
bots, and outer loop traverses the list of files. [In effect: For each
file A, [for each filter term 'bot' run a query that retreives all
lines NOT containing "bot", placing the output in A1. Repeat query on
file A for all bots names] Repeat for all files].
I'd like to confirm whether or not my assumptions 1 and 2 above are
mistaken, and if my suggested way of completing this task is over
complicating things. Also any further tips or tool /scripting
recommendations are most welcome. Thanks.