Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

4.6 destination flags #54

Draft
wants to merge 6 commits into
base: main
Choose a base branch
from
Draft

4.6 destination flags #54

Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
62d80ea
Formatting fixes
eldarnash Jan 29, 2024
0c296d3
[4.6] Adds seqnum-all destination flag
eldarnash Feb 4, 2024
c9dd9bd
Fix review comments
eldarnash Feb 5, 2024
a43c7a5
threaded flag clarification
eldarnash Feb 5, 2024
34bd8f7
Includes flags option in a bunch of destinations where it was missing
eldarnash Feb 5, 2024
037189a
Formatting fix
eldarnash Feb 5, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 23 additions & 4 deletions content/headless/chunk/option-destination-flags.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,31 @@

| | |
| -------- | ------------------------------ |
| Type: | no-multi-line, syslog-protocol |
| Type: | `ignore-errors`, `no-multi-line`, `no-seqnum-all`, `seqnum-all`, `syslog-protocol`, `threaded` |
| Default: | empty set |

*Description:* Flags influence the behavior of the destination driver.

- `ignore-errors`: <!-- FIXME -->
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ignore-error flag silently drops messages which have not been successfully sent out because of unrecoverable low-level errors (e.g. I/O errors).

- `no-multi-line`: The `no-multi-line` flag disables line-breaking in the messages: the entire message is converted to a single line.
- `no-seqnum-all`: <!-- FIXME -->
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The no-seqnum-all flag explicitly disables new seqnum generation for forwarded messages, which is the default behavior.

@bazsi Do I understand this right? I feel like this should return 0 seqnum even for locally generated messages, but the flag does not clear the LWO_SEQNUM field:
https://github.com/syslog-ng/syslog-ng/blob/1f0a854d0227dd95410fbc094997d8b0b225cd41/lib/logwriter.c#L2037

Do we even need this flag? The no-seqnum flag (not yet released) does the full disabling, that I have mentioned. I think it is confusing to have both.

Also, why do we need the seqnum flag, which sets the default behavior?

I feel like there are 3 cases:

  1. local messages get a generated seqnum, forwarded messages keep their own.
  2. all messages get a generated seqnum.
  3. neither local, nor forwarded messages get a seqnum (SEQNUM = 0).

The first is the default behavior, which does not necessarily need an explicit flag.
The second is enabled by seqnum-all.
The third is enabled by no-seqnum.

I don't see the point of having any other SEQNUM related flags.

- `seqnum-all`: Available in {{% param "product.abbrev" %}} version 4.6 and later. By default, {{% param "product.abbrev" %}} follows the logic of the RFC5424 `meta.sequenceId` structured data element: it adds a sequence number to local messages, forwarded messages retain their original sequenceId.

*Description:* Flags influence the behavior of the destination driver.
The `seqnum-all` flag adds a sequence number to every message sent to the destination, not just local messages. This also changes the behavior of the `${SEQNUM}` macro. For example:

```shell
destination { syslog("127.0.0.1" port(2001) flags(seqnum-all)); };
```

The output messages have increasing sequence numbers:

```shell
<13>1 2023-12-09T21:51:30+00:00 localhost sdff - - [meta sequenceId="1"] f sdf fsd
<13>1 2023-12-09T21:51:32+00:00 localhost sdff - - [meta sequenceId="2"] f sdf fsd
<13>1 2023-12-09T21:51:32+00:00 localhost sdff - - [meta sequenceId="3"] f sdf fsd
<13>1 2023-12-09T21:51:32+00:00 localhost sdff - - [meta sequenceId="4"] f sdf fsd
<13>1 2023-12-09T21:51:32+00:00 localhost sdff - - [meta sequenceId="5"] f sdf fsd
```

- *no-multi-line*: The `no-multi-line` flag disables line-breaking in the messages: the entire message is converted to a single line.
- *syslog-protocol*: The `syslog-protocol` flag instructs the driver to format the messages according to the new IETF syslog protocol standard (RFC5424), but without the frame header. If this flag is enabled, macros used for the message have effect only for the text of the message, the message header is formatted to the new standard. Note that this flag is not needed for the `syslog` driver, and that the `syslog` driver automatically adds the frame header to the messages.
- `syslog-protocol`: The `syslog-protocol` flag instructs the driver to format the messages according to the new IETF syslog protocol standard (RFC5424), but without the frame header. If this flag is enabled, macros used for the message have effect only for the text of the message, the message header is formatted to the new standard. Note that this flag is not needed for the `syslog` driver, and that the `syslog` driver automatically adds the frame header to the messages.
- `threaded`: <!-- FIXME -->
Copy link
Member

@alltilla alltilla Feb 5, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The threaded flag enables multithreading for the current destination, even if the global threaded(no) option is set. For more information about multithreading, see: https://axoflow.com/docs/axosyslog-core/chapter-multithreading/configuring-multithreading/#example-enabling-multithreading

Loading