Logstash filter and patterns for syslog and filebeat logs receive

testsoft.net syslog index patterns

This page shows the logstash settings and the patterns that I use in my project. The configuration is being updated, if you notice errors in the parser, please write in the comments. For linux I use syslog, of course it takes some time to configure parsing, but this configuration seems to me more correct. Using… Continue reading Logstash filter and patterns for syslog and filebeat logs receive

Arcsight flexConnector for Fail2Ban

flexconnector-fail2ban

#fail2ban 2020-12-01 05:01:08,966 fail2ban.filter [893]: INFO [sshd] Found 1.1.1.1 – 2020-12-01 05:01:08 regex=(fail2ban)\\s+(.*) token.count=2 token[0].name=type token[0].type=String token[1].name=body token[1].type=String event.deviceVendor=__stringConstant(“fail2ban”) event.deviceProduct=__stringConstant(“fail2ban”) event.sourceUserPrivileges=__stringConstant(“fail2ban”) event.deviceProcessName=__stringConstant(“fail2ban”) event.flexString2=body event.flexString2Label=__stringConstant(“raw”) event.name=type event.message=body submessage.messageid.token=type submessage.token=body submessage.count=1 submessage[0].messageid=fail2ban submessage[0].pattern.count=1 #2020-12-01 06:23:17,883 fail2ban.filter [893]: INFO [sshd] Found 1.1.1.1 – 2020-12-01 06:23:17 submessage[0].pattern[0].regex=(\\d+\\-\\d+\\-\\d+\\s+\\d+\\:\\d+\\:\\d+\\,\\d+)\\s+([^\\s]+)\\s+\\[(\\d+)\\]\\:\\s+(\\w+)\\s+\\[([^]]+)\\]\\s+((Found|Ban|Unban)?\\s+(\\d+.\\d+.\\d+.\\d+)?.*) submessage[0].pattern[0].mappings=$1|$2|$3|$4|$5|$6|$7|$8 submessage[0].pattern[0].fields=event.deviceCustomString1,event.name,event.deviceCustomString4,event.deviceSeverity,event.targetProcessName,event.message,event.deviceAction,event.attackerAddress submessage[0].pattern[0].extramappings=event.name\=__concatenate($2,” “,$5)|event.deviceSeverity=__toLowerCase($4)|event.deviceCustomString1Label=__stringConstant(“time_local”)|event.deviceCustomString4Label=__stringConstant(“pid”) #$1 – event.deviceCustomString1 – time_local #$2 – event.name… Continue reading Arcsight flexConnector for Fail2Ban

Arcsight flexConnector for openVPN

flexconnector openvpn

regex=(openvpn)\\[.*\\]:\\s(.*) token.count=2 token[0].name=type token[0].type=String token[1].name=body token[1].type=String event.deviceVendor=__stringConstant(“openvpn”) event.deviceProduct=__stringConstant(“openvpn”) event.sourceUserPrivileges=__stringConstant(“openvpn”) event.deviceProcessName=__stringConstant(“openvpn”) event.flexString2=body event.name=type event.message=body submessage.messageid.token=type submessage.token=body submessage.count=1 submessage[0].messageid=openvpn submessage[0].pattern.count=3 submessage[0].pattern[0].regex=([^\\/]+)\\/(\\d+.\\d+.\\d+.\\d+):(\\d+)\\s+(.*) submessage[0].pattern[0].mappings=$1|$2|$3|$4 submessage[0].pattern[0].fields=event.targetUserName,event.attackerAddress,event.attackerPort,event.message submessage[0].pattern[1].regex=(\\d+.\\d+.\\d+.\\d+):(\\d+)\\s+(.*) submessage[0].pattern[1].mappings=$1|$2|$3 submessage[0].pattern[1].fields=event.attackerAddress,event.attackerPort,event.message submessage[1].pattern.count=1 submessage[1].pattern[0].regex=(.*) submessage[1].pattern[0].fields=event.message submessage[1].pattern[0].extramappings=event.reason=__stringConstant(“unparsed”)

Arcsight flexConnector for WordPress

flexconnector wordpress

##################### # ### TESTSOFT.NET ### # ############################ ### ArcSight Parser For WordPress Stock Logformat v1.0 ### #wordpress(testsoft.net)[3673]: Authentication attempt for unknown user admin from 1.1.1.1 regex=(wordpress)(.*) token.count=2 token[0].name=type token[0].type=String token[1].name=body token[1].type=String event.deviceVendor=__stringConstant(“wordpress”) event.deviceProduct=__stringConstant(“applog”) event.sourceUserPrivileges=__stringConstant(“wordpress”) event.deviceProcessName=__stringConstant(“wordpress”) event.flexString2=body event.flexString2Label=__stringConstant(“raw”) event.name=type event.message=body submessage.messageid.token=type submessage.token=body submessage.count=1 submessage[0].messageid=wordpress submessage[0].pattern.count=4 #(testsoft.net)[3673]: Authentication failure for testsoft from 1.1.1.1 submessage[0].pattern[0].regex=\\(([^)]+)\\)\\[([^]]+)\\]\\:\\s+(.*failure\\s+for\\s+([^\\s+]+)\\s+from\\s+(\\d+.\\d+.\\d+.\\d+).*) submessage[0].pattern[0].mappings=$1|$2|$3|$4|$5 submessage[0].pattern[0].fields=event.name,event.deviceCustomString4,event.message,event.targetUserName,event.attackerAddress submessage[0].pattern[0].extramappings=event.deviceCustomString4Label=__stringConstant(“pid”)|event.deviceSeverity=__stringConstant(“warning”)… Continue reading Arcsight flexConnector for WordPress

Arcsight flexConnector for Suricata

flexconnector suricata

regex=(suricata)\\[.*\\]:\\s(.*) token.count=2 token[0].name=type token[0].type=String token[1].name=body token[1].type=String event.deviceVendor=__stringConstant(“suricata”) event.deviceProduct=__stringConstant(“ids”) event.sourceUserPrivileges=__stringConstant(“suricata”) event.deviceProcessName=__stringConstant(“suricata”) event.flexString2=body submessage.messageid.token=type submessage.token=body submessage.count=1 submessage[0].messageid=suricata submessage[0].pattern.count=1 submessage[0].pattern[0].regex=\\[\\d+:\\d+:\\d+\\]\\s+(.*)\\[Classification:\\s+([^]]+)\\].*}\\s+(\\d+.\\d+.\\d+.\\d+):\\d+\\s+\\-\\>\\s+(\\d+.\\d+.\\d+.\\d+)\\:(\\d+).* submessage[0].pattern[0].mappings=$1|$2|$3|$4|$5 submessage[0].pattern[0].fields=event.message,event.name,event.attackerAddress,event.targetAddress,event.targetPort

Arcsight flexConnector for Nginx with Bitrix

flexconnector nginx

##################### # ### TESTSOFT.NET ### # ############################ ### ArcSight Parser For Nginx and Bitrix Stock Logformat v1.0 ### regex=(nginx):\\s(.*) token.count=2 token[0].name=type token[0].type=String token[1].name=body token[1].type=String event.deviceVendor=__stringConstant(“nginx”) event.deviceProduct=__stringConstant(“webserver”) event.sourceUserPrivileges=__stringConstant(“nginx”) event.deviceProcessName=__stringConstant(“nginx”) event.flexString2=body event.name=type event.message=body submessage.messageid.token=type submessage.token=body submessage.count=1 submessage[0].messageid=nginx submessage[0].pattern.count=5 #1.1.1.1 – – [29/Aug/2020:15:38:17 -0400] “GET /test/test/ HTTP/1.1” 401 597 “-” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML,… Continue reading Arcsight flexConnector for Nginx with Bitrix

Arcsight flexConnector for Apache with modsecurity

flexconnector apache with modsecurity

The parser requires a little improvement, but in general, it processes 90% of incoming events. ##################### # ### TESTSOFT.NET ### # ############################ ### ArcSight Parser For Apache and ModSecurity Stock Logformat v1.0 ### regex=(httpd)(?:\\[.*\\])?:\\s(.*) token.count=2 token[0].name=type token[0].type=String token[1].name=body token[1].type=String event.deviceVendor=__stringConstant(“apache”) event.deviceProduct=__stringConstant(“webserver”) event.sourceUserPrivileges=__stringConstant(“apache”) event.deviceProcessName=__stringConstant(“apache”) event.flexString2=body event.flexString2Label=__stringConstant(“raw”) event.name=type event.message=body submessage.messageid.token=type submessage.token=body submessage.count=1 submessage[0].messageid=httpd submessage[0].pattern.count=3 ### Example ###… Continue reading Arcsight flexConnector for Apache with modsecurity