Metric to Log Timestamp gets overwritten? How to preserve log timestamp when converting to metric and sending using prometheus remote write?

[generate-me12]

Hi, I'm trying to use log_to_metric transformation. These are logs have timestamps that aren't current such as 2025-03-24T18:53:01.419106546Z These are metrics that were converted to logs to be stored in an s3 bucket, I am then ingesting these logs from s3 and I converting them to metrics and then sending them to Amazon Prometheus using prometheus_remote_write.

This is how I'm converting the log to metrics:

convert_s3:
    type: log_to_metric
    inputs:
      - s3_metrics_as_logs
    metrics: []
    all_metrics: true

I notice when I notice that log_to_metric transform sets the metric timestamps to the current timestamp the metric is converted. I tried adding a remap after this to change the timestamp back using
.timestamp = parse_timestamp!(.tags.log_timestamp, "%Y-%m-%dT%H:%M:%S%.9fZ")
but I notice that when I use prometheus_remote_write to send to prometheus, the timestamp shows as the current timestamp.
Does both log_to_metric and prometheus_remote_write alter the timestamp?

[pront]

Your approach of restoring the timestamp after conversion works. Use %+ to parse the RFC 3339 timestamp, including its Z timezone:

transforms:
  convert_s3:
    type: log_to_metric
    inputs: [s3_metrics_as_logs]
    metrics: []
    all_metrics: true

  restore_timestamp:
    type: remap
    inputs: [convert_s3]
    source: |
      .timestamp = parse_timestamp!(.tags.log_timestamp, "%+")

The Remote Write sink should consume restore_timestamp. The literal Z in your original format does not parse a timezone, so it can interpret the time using Vector's configured/default timezone. See parse_timestamp.

I tested this on Vector 0.60.0 with a local Vector Remote Write receiver. Your timestamp arrived as 2025-03-24T18:53:01.419Z, consistent with Remote Write's millisecond precision.

I couldn't reproduce the sink replacing a restored timestamp with the current time. If that still happens, please share your Vector version, a sample input event, and the full transform/sink wiring.

Please don't hesitate to followup in this thread if you have further questions.