Skip to content

Commit

Permalink
Add a test for MariaDB and fix typos + MariaDB collection docs (#875)
Browse files Browse the repository at this point in the history
* Add one more test for MariaDB

* Fix the MariaDB documentation

* Fix typos in successfully

* Doc improvements
  • Loading branch information
williamdes authored Dec 12, 2023
1 parent 22de5e3 commit c0b48eb
Show file tree
Hide file tree
Showing 9 changed files with 37 additions and 19 deletions.
3 changes: 2 additions & 1 deletion .tests/mariadb-logs/mariadb-logs.log
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,5 @@
2021-11-09 5:13:13 8 [Warning] Access denied for user 'example-user'@'172.17.0.1' (using password: NO)
2022-05-01 11:38:05 356 [Warning] Access denied for user 'root'@'127.0.0.1' (using password: NO)
221128 14:49:17 [Warning] Access denied for user 'root'@'192.168.1.192' (using password: YES)
220109 14:49:17 [Warning] Access denied for user 'root'@'192.168.1.192' (using password: NO)
220109 14:49:17 [Warning] Access denied for user 'root'@'192.168.1.192' (using password: NO)
2023-11-21 19:42:20 134334 [Warning] Access denied for user 'root'@'34.140.248.32' (using password: NO)
25 changes: 21 additions & 4 deletions .tests/mariadb-logs/parser.assert
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
len(results) == 4
len(results["s00-raw"]["crowdsecurity/non-syslog"]) == 5
len(results["s00-raw"]["crowdsecurity/non-syslog"]) == 6
results["s00-raw"]["crowdsecurity/non-syslog"][0].Success == true
results["s00-raw"]["crowdsecurity/non-syslog"][0].Evt.Parsed["message"] == "2021-11-09 5:13:12 8 [Warning] Access denied for user 'example-user'@'172.17.0.1' (using password: YES)"
results["s00-raw"]["crowdsecurity/non-syslog"][0].Evt.Parsed["program"] == "mariadb"
Expand All @@ -25,13 +25,13 @@ results["s00-raw"]["crowdsecurity/non-syslog"][4].Evt.Parsed["message"] == "2201
results["s00-raw"]["crowdsecurity/non-syslog"][4].Evt.Parsed["program"] == "mariadb"
results["s00-raw"]["crowdsecurity/non-syslog"][4].Evt.Meta["datasource_type"] == "file"
results["s00-raw"]["crowdsecurity/non-syslog"][4].Evt.Meta["datasource_path"] == "mariadb-logs.log"
len(results["s00-raw"]["crowdsecurity/syslog-logs"]) == 5
len(results["s00-raw"]["crowdsecurity/syslog-logs"]) == 6
results["s00-raw"]["crowdsecurity/syslog-logs"][0].Success == false
results["s00-raw"]["crowdsecurity/syslog-logs"][1].Success == false
results["s00-raw"]["crowdsecurity/syslog-logs"][2].Success == false
results["s00-raw"]["crowdsecurity/syslog-logs"][3].Success == false
results["s00-raw"]["crowdsecurity/syslog-logs"][4].Success == false
len(results["s01-parse"]["crowdsecurity/mariadb-logs"]) == 5
len(results["s01-parse"]["crowdsecurity/mariadb-logs"]) == 6
results["s01-parse"]["crowdsecurity/mariadb-logs"][0].Success == true
results["s01-parse"]["crowdsecurity/mariadb-logs"][0].Evt.Parsed["day"] == "09"
results["s01-parse"]["crowdsecurity/mariadb-logs"][0].Evt.Parsed["source_ip"] == "172.17.0.1"
Expand Down Expand Up @@ -113,7 +113,7 @@ results["s01-parse"]["crowdsecurity/mariadb-logs"][4].Evt.Meta["log_type"] == "m
results["s01-parse"]["crowdsecurity/mariadb-logs"][4].Evt.Meta["source_ip"] == "192.168.1.192"
results["s01-parse"]["crowdsecurity/mariadb-logs"][4].Evt.Meta["user"] == "root"
results["s01-parse"]["crowdsecurity/mariadb-logs"][4].Evt.Meta["datasource_path"] == "mariadb-logs.log"
len(results["s02-enrich"]["crowdsecurity/dateparse-enrich"]) == 5
len(results["s02-enrich"]["crowdsecurity/dateparse-enrich"]) == 6
results["s02-enrich"]["crowdsecurity/dateparse-enrich"][0].Success == true
results["s02-enrich"]["crowdsecurity/dateparse-enrich"][0].Evt.Parsed["source_ip"] == "172.17.0.1"
results["s02-enrich"]["crowdsecurity/dateparse-enrich"][0].Evt.Parsed["thread_id"] == "8"
Expand Down Expand Up @@ -205,4 +205,21 @@ results["s02-enrich"]["crowdsecurity/dateparse-enrich"][4].Evt.Meta["timestamp"]
results["s02-enrich"]["crowdsecurity/dateparse-enrich"][4].Evt.Meta["user"] == "root"
results["s02-enrich"]["crowdsecurity/dateparse-enrich"][4].Evt.Meta["datasource_path"] == "mariadb-logs.log"
results["s02-enrich"]["crowdsecurity/dateparse-enrich"][4].Evt.Enriched["MarshaledTime"] == "2022-01-09T14:49:17Z"
results["s01-parse"]["crowdsecurity/mariadb-logs"][5].Success == true
results["s01-parse"]["crowdsecurity/mariadb-logs"][5].Evt.Parsed["user"] == "root"
results["s01-parse"]["crowdsecurity/mariadb-logs"][5].Evt.Parsed["year"] == "2023"
results["s01-parse"]["crowdsecurity/mariadb-logs"][5].Evt.Parsed["message"] == "2023-11-21 19:42:20 134334 [Warning] Access denied for user 'root'@'34.140.248.32' (using password: NO)"
results["s01-parse"]["crowdsecurity/mariadb-logs"][5].Evt.Parsed["day"] == "21"
results["s01-parse"]["crowdsecurity/mariadb-logs"][5].Evt.Parsed["month"] == "11"
results["s01-parse"]["crowdsecurity/mariadb-logs"][5].Evt.Parsed["program"] == "mariadb"
results["s01-parse"]["crowdsecurity/mariadb-logs"][5].Evt.Parsed["source_ip"] == "34.140.248.32"
results["s01-parse"]["crowdsecurity/mariadb-logs"][5].Evt.Parsed["thread_id"] == "134334"
results["s01-parse"]["crowdsecurity/mariadb-logs"][5].Evt.Parsed["time"] == "19:42:20"
results["s01-parse"]["crowdsecurity/mariadb-logs"][5].Evt.Parsed["using_password"] == "NO"
results["s01-parse"]["crowdsecurity/mariadb-logs"][5].Evt.Parsed["date"] == "2023-11-21"
results["s01-parse"]["crowdsecurity/mariadb-logs"][5].Evt.Meta["datasource_type"] == "file"
results["s01-parse"]["crowdsecurity/mariadb-logs"][5].Evt.Meta["log_type"] == "mariadb_failed_auth"
results["s01-parse"]["crowdsecurity/mariadb-logs"][5].Evt.Meta["source_ip"] == "34.140.248.32"
results["s01-parse"]["crowdsecurity/mariadb-logs"][5].Evt.Meta["user"] == "root"
results["s01-parse"]["crowdsecurity/mariadb-logs"][5].Evt.Meta["datasource_path"] == "mariadb-logs.log"
len(results["success"][""]) == 0
8 changes: 4 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -117,8 +117,8 @@ Now that we have config & logs, let's run it for the first time :
```bash
▶ cscli hubtest run dovecot-logs
INFO[27-09-2021 06:13:59 PM] Running test 'dovecot-logs'
INFO[27-09-2021 06:13:59 PM] parser 'crowdsecurity/dovecot-logs' installed succesfully in runtime environment
INFO[27-09-2021 06:13:59 PM] parser 'crowdsecurity/syslog-logs' installed succesfully in runtime environment
INFO[27-09-2021 06:13:59 PM] parser 'crowdsecurity/dovecot-logs' installed successfully in runtime environment
INFO[27-09-2021 06:13:59 PM] parser 'crowdsecurity/syslog-logs' installed successfully in runtime environment
WARN[27-09-2021 06:14:02 PM] Assert file '/home/bui/github/hub/.tests/dovecot-logs/parser.assert' is empty, generating assertion:

results["s00-raw"]["crowdsecurity/syslog-logs"][0].Success == true
Expand Down Expand Up @@ -196,8 +196,8 @@ results["s01-parse"]["crowdsecurity/dovecot-logs"][2].Evt.Meta["datasource_type"
```bash
▶ cscli hubtest run dovecot-logs
INFO[27-09-2021 06:19:33 PM] Running test 'dovecot-logs'
INFO[27-09-2021 06:19:33 PM] parser 'crowdsecurity/syslog-logs' installed succesfully in runtime environment
INFO[27-09-2021 06:19:33 PM] parser 'crowdsecurity/dovecot-logs' installed succesfully in runtime environment
INFO[27-09-2021 06:19:33 PM] parser 'crowdsecurity/syslog-logs' installed successfully in runtime environment
INFO[27-09-2021 06:19:33 PM] parser 'crowdsecurity/dovecot-logs' installed successfully in runtime environment
Test 'dovecot-logs' passed successfully (39 assertions) 🟩
```

Expand Down
2 changes: 1 addition & 1 deletion collections/crowdsecurity/mariadb.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ Example acquisition for this collection :
filenames:
- /var/log/mysql/error.log
labels:
type: mysql
type: mariadb
```
notes :
Expand Down
2 changes: 1 addition & 1 deletion parsers/s00-raw/crowdsecurity/unifi-logs.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Those logs are sligthly non-conformant to the syslog standard, hence the need fo

As crowdsec does not run easily directly on an UDM, you'll likely want to setup syslog export on your UDM, and use the following acquisition config:

```
```yaml
source: syslog
listen_addr: 0.0.0.0
listen_port: 4242
Expand Down
6 changes: 3 additions & 3 deletions parsers/s01-parse/corvese/apache-guacamole-logs.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ The default log format used by quacamole is not supported by crowdsec as there i

- Create the following 'logback.xml' file in guacamole home dir to log with the correct Date format.
STDOUT
```
```xml
<configuration>
<!-- Default appender -->
<appender name="GUAC-DEFAULT" class="ch.qos.logback.core.ConsoleAppender">
Expand All @@ -36,7 +36,7 @@ STDOUT
```

FILE:
```
```xml
<configuration>
<!-- Default appender -->
<appender name="GUAC-DEFAULT" class="ch.qos.logback.core.FileAppender">
Expand All @@ -55,7 +55,7 @@ FILE:
```

If using docker you will need to mount the following volume that links to the logback.xml file and set the GUACAMOLE_HOME variable
```
```yaml
volumes:
- ./logback.xml:/home/guacamole/temp/logback.xml:ro
environment:
Expand Down
6 changes: 3 additions & 3 deletions parsers/s01-parse/crowdsecurity/aws-cloudtrail.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ crowdsec with the following configuration:


Example of `acquis.yaml` using s3 s3notifications through sqs:
```
```yaml
source: s3
polling_method: sqs
sqs_name: <sqs_queue>
Expand All @@ -33,7 +33,7 @@ resources.
Cloudtrail logs are arriving every few minutes, thus, we can't use the
real time feature of crowdsec. That's the reason we are suggesting
to use the time machine feature, to take into account the time when
they occured and not when they are sent to CrowdSec.
they occurred and not when they are sent to CrowdSec.
Please have a look at the documentation
https://docs.crowdsec.net/docs/next/data_sources/s3
Expand All @@ -43,7 +43,7 @@ https://docs.crowdsec.net/docs/next/data_sources/s3
Cloudtrail logs can be sent to kinesis as well, and crowdsec supports
such a source for cloudtrail logs:
```
```yaml
source: kinesis
stream_name: cloutrail_stream
aws_region: eu-west-1
Expand Down
2 changes: 1 addition & 1 deletion parsers/s01-parse/crowdsecurity/sysmon-logs.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
A parser for [sysmon](https://docs.microsoft.com/en-us/sysinternals/downloads/sysmon) events.

Example acquisition config:
```
```yaml
source: wineventlog
pretty_name: sysmon
event_channel: "Microsoft-Windows-Sysmon/Operational"
Expand Down
2 changes: 1 addition & 1 deletion parsers/s01-parse/jusabatier/apereo-cas-audit-logs.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@ Use Apereo CAS audit logs : https://apereo.github.io/cas/6.4.x/audits/Audits-Fil

Need to :
* ativate `cas.audit.slf4j.use-single-line=true` in CAS configuration
* add cas_audit.log generated file to CrowdSec aquisitions
* add cas_audit.log generated file to CrowdSec acquisitions

Sample log4j config :

Expand Down

0 comments on commit c0b48eb

Please sign in to comment.