Logspout to Signoz, no severity_number or severity_text #3968

Hellalol · 2023-11-15T08:29:01Z

Hellalol
Nov 15, 2023

{
"body": "2023-11-15 08:46:50.012 INFO 1 --- [AsyncResolver-bootstrap-executor-0] c.n.d.s.r.aws.ConfigClusterResolver : Resolving eureka endpoints via configuration",
"id": "2Y2BBfmM72xDo22HcMx19L2gZsy",
"timestamp": "2023-11-15T07:46:50Z",
"attributes": {
"container_id": "338710e49d90",
"container_name": "opt_example_01",
"severity_field": ""
},
"resources": {},
"severity_text": "",
"severity_number": 0,
"span_id": "",
"trace_flags": 0,
"trace_id": ""
}

Even though I am getting the INFO and 1 in my logs they wont seem to populate the fields severity_text and severity_number. I've looked into the otel-collector and played around a bit with the regex but cant seem to get it to work. Did anyone get this to work? Help would be appreciated! :)

srikanthccv · 2023-11-18T05:14:23Z

srikanthccv
Nov 18, 2023
Maintainer

regex parser is widely used and should support this. What regex did you try?

0 replies

Hellalol · 2023-11-22T10:24:40Z

Hellalol
Nov 22, 2023
Author

receivers:
  tcplog/docker:
    listen_address: "0.0.0.0:2255"
    operators:
      - type: regex_parser
        regex: '^<([0-9]+)>[0-9]+ (?P<timestamp>[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}(\.[0-9]+)?([zZ]|([\+-])([01]\d|2[0-3]):?([0-5]\d)?)?) (?P<container_id>\S+) (?P<container_name>\S+) [0-9]+ - -( (?P<body>(\n|.)*))?'
        #regex: '^<([0-9]+)>[0-9]+ (?P<timestamp>[0-9]{4}-[0-9]{2}-[0-9]{2}T[0-9]{2}:[0-9]{2}:[0-9]{2}(\.[0-9]+)?([zZ]|([\+-])([01]\d|2[0-3]):?([0-5]\d)?)?) (?P<container_id>\S+) (?P<container_name>\S+) [0-9]+ - -( (?P<date>\S+))?( (?P<time>\S+))?(\s+(?P<level>\S+))?(\s+(?P<body>.*))?'
        timestamp:
          parse_from: attributes.timestamp
          layout: '%Y-%m-%dT%H:%M:%S.%LZ'
      - type: regex_parser
        id: get_severity
        parse_from: attributes.body
        regex: '(?P<severity_field>(INFO|DEBUG|TRACE|WARN(ING)?|ERROR|SEVERE))?'
#        severity:
#          parse_from: attributes.severity_field
#          mapping:
#            warn: ERROR
#            error: WARNING
#            info: INFO
#            debug: DEBUG
      - type: move
        from: attributes["body"]
        to: body
      - type: remove
        field: attributes.timestamp
        # please remove names from below if you want to collect logs from them
      - type: filter
        id: signoz_logs_filter
        expr: 'attributes.container_name matches "^signoz-(logspout|frontend|alertmanager|query-service|otel-collector|otel-collector-metrics|clickhouse|zookeeper)"'

The commented out parts are the ones I have been experimenting with. I think its my knowledge in regex that is lacking. Help or example how to parse would be helpful.

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Logspout to Signoz, no severity_number or severity_text #3968

{{title}}

Replies: 2 comments

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Select a reply

Logspout to Signoz, no severity_number or severity_text #3968

Hellalol Nov 15, 2023

Replies: 2 comments

srikanthccv Nov 18, 2023 Maintainer

Hellalol Nov 22, 2023 Author

Hellalol
Nov 15, 2023

srikanthccv
Nov 18, 2023
Maintainer

Hellalol
Nov 22, 2023
Author