Skip to content

Conversation

hilmarf
Copy link
Member

@hilmarf hilmarf commented Sep 5, 2025

This pull request introduces a mechanism for handling log export errors via a user-provided error consumer, which can be passed through the OpenTelemetry context. The changes are focused, minimal, and primarily affect the log record processing pipeline.


Key Changes

1. BatchLogRecordProcessor.java

  • Error Consumer Support:
    • Adds a Consumer<Collection<LogRecordData>> errorConsumer field to the processor's worker.
    • The worker's constructor and logic are updated to accept and use this error consumer.
    • When a log record cannot be queued (i.e., dropped), or when a batch export fails, the error consumer is invoked with the failed records.
    • The error consumer can also be retrieved from the current context (ExportErrorContext.KEY), allowing per-call customization.

2. BatchLogRecordProcessorBuilder.java

  • Builder Extension:
    • Adds a setErrorConsumer method to allow users to configure a global error consumer for the processor.
    • Passes the error consumer to the processor during build.

3. ExportErrorContext.java (New File)

  • Context Key Definition:
    • Introduces a new immutable class that defines a ContextKey for a Consumer<Collection<LogRecordData>>.
    • This key allows users to attach a custom error consumer to the OpenTelemetry context, enabling dynamic error handling strategies.

4. SimpleLogRecordProcessor.java

  • Error Consumer Integration:
    • On export failure, checks the context for an error consumer and invokes it with the failed log record if present.

Motivation & Impact

  • Custom Error Handling:
    This PR enables applications to react programmatically to log export failures, such as by logging, alerting, or retrying failed exports.
  • Context-Aware:
    By leveraging the OpenTelemetry context, error handling can be customized per operation or request, not just globally.
  • Minimal API Surface:
    The change is backward-compatible and opt-in; existing users are unaffected unless they choose to use the new error consumer feature.

Summary

This PR adds a simple but powerful extension point for error handling in log export scenarios. By allowing a Consumer<Collection<LogRecordData>> to be attached via context or builder, it gives users fine-grained control over how export failures are managed, without complicating the main processing logic. The implementation is clean, well-contained, and leverages existing OpenTelemetry patterns.


Sample usage

This how the client coding could look like:

    // for your regular debug logging use your favorite logging framework, e.g. SLF4J
    org.slf4j.Logger slf4jLogger = LoggerFactory.getLogger(getClass());

    // create your custom error handler to act on log records, which failed to be exported
    java.util.function.Consumer<Collection<LogRecordData>> myFaildLogMessageHandler = collection -> {
      // Handle the collection of failed log records
      collection.forEach(otelLogRecordData -> {
        // Process each log record, e.g., log it or send it to a monitoring system
        slf4jLogger.error("Failed to export log record: {}", otelLogRecordData);
        // TODO: Implement your error handling logic here. E.g., roll back transaction, retry, etc.
      });
    };

    // Now let's setup OpenTelemetry logging with OTLP exporters and a custom error handler.

    // create the LogRecordExporter of your choice
    LogRecordExporter otelLogRecordExporter = OtlpGrpcLogRecordExporter.builder().build();
    // create the LogRecordProcessor of your choice
    LogRecordProcessor otelLogRecordProcessor = BatchLogRecordProcessor.builder(otelLogRecordExporter)
        // in case of BatchLogRecordProcessor use the builder and add your custom error handler
        .setErrorConsumer(myFaildLogMessageHandler).build();
    // create the LoggerProvider with the LogRecordProcessor
    LoggerProvider otelSdkLoggerProvider = SdkLoggerProvider.builder().addLogRecordProcessor(otelLogRecordProcessor).build();
    // create the OpenTelemetry Logger using the LoggerProvider
    io.opentelemetry.api.logs.Logger otelLogger = otelSdkLoggerProvider.loggerBuilder("MY-REALLY-RELEVANT-LOGS").build();

    // create a context with the new custom error handler
    Context otelContext = Context.current().with(ExportErrorContext.KEY, myFaildLogMessageHandler);

    // now you can use the OpenTelemetry Logger to log messages
    otelLogger.logRecordBuilder().setBody("my really relavant log message").setContext(otelContext).emit();

    // of course you can handle the error individually by each log message in your own way, e.g.:
    otelLogger.logRecordBuilder().setBody("my really relavant log message with custom error handler")
        .setContext(Context.current().with(ExportErrorContext.KEY, failedLogRecordDataCollection -> {
          // Handle the failed log record export, e.g. log it with SLF4J
          slf4jLogger.error("Failed to export log record: {}",
              failedLogRecordDataCollection.stream().map(LogRecordData::getBodyValue).findFirst());
          // TODO: Implement your error handling logic here. E.g., roll back transaction, retry, etc.
        })).emit();

@hilmarf hilmarf moved this to In review in OTel-Audit-Logging Sep 5, 2025
@hilmarf hilmarf marked this pull request as ready for review September 5, 2025 13:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: In review
Development

Successfully merging this pull request may close these issues.

1 participant