Skip to content

Fix CEF export serialization and standard compliance - #963

Merged
adulau merged 1 commit into
mainfrom
codex/cef-export-standard
Oct 22, 2026
Merged

adulau merged 1 commit into
mainfrom
codex/cef-export-standard

Conversation

@adulau

@adulau adulau commented Oct 22, 2026

Copy link
Copy Markdown
Member

The CEF exporter could produce malformed records from ordinary URL query strings, backslashes, pipes in device settings, and CR/LF in attribute values. It also emitted IPv6 in CEF:0 address fields and prepended a fabricated syslog host to file output.

This change aligns file serialization with the OpenText ArcSight CEF Implementation Standard, version 27, covering header definitions, file output, character encoding and the extension dictionary.

  • Escape header backslashes/pipes and extension backslashes/equals signs; encode extension CR/LF while preserving UTF-8, spaces and extension pipes. For example, a URL query now renders as request=https://example.org/?a\=1.
  • Validate severity (0–10 or the five standard labels), header/string lengths, addresses and timestamps. Return module errors without partial files for malformed input; merge missing configuration keys with existing defaults.
  • Start file records with CEF:. Preserve attribute timestamps as epoch milliseconds in labeled deviceCustomDate1 fields, avoiding year loss, locale/timezone dependence, and assumptions that modification time is incident time.
  • Retain existing extension mappings, normalize IP addresses, and emit CEF:1 for IPv6 address records while retaining CEF:0 for other records.
  • Use the attribute type as the event-class ID to distinguish source/destination indicators and hash algorithms; retain the category as the name and cat.
  • Support bare, wrapped and legacy event collections plus object attributes, expose event input metadata, and correct the MIME type to text/plain.

The usage guide documents configuration, mappings, an example, and migration implications. Collectors expecting the old syslog prefix, category-based event-class IDs, or only CEF:0 require corresponding updates. All supplied supported attributes remain included regardless of to_ids; unsupported types remain ignored.

Validation:

  • .venv/bin/python -m pytest -q tests/test_cef_export.py: 92 passed, including independent decoding/escaping checks, field limits, invalid-input handling, IPv4/IPv6, timestamps, event shapes and HTTP discovery/export.
  • Black formatting with repository settings, isort, flake8 for the changed module, and git diff --check passed.
  • Regenerated only the CEF sections of both documentation catalogs.
  • No live ArcSight collector integration or certification test was performed.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 22, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-22T08:13:09.373485Z 8dfd67d PR opened
🔒 Security Review ✅ Completed 2026-10-22T08:13:31.068904Z 8dfd67d PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@adulau
adulau merged commit 342ca51 into main Oct 22, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant