Skip to content

Add mutli-team dimensions to provider metrics #68996

Description

@ferruzzi

Follow-up to #68294 which added team name to all relevant core metrics, this one is for mob-coding the providers.

Provider Assignee Status PR
amazon @justinpakzad 👀
celery
edge3 worker @shivaam 👀
edge3 execurtor @shivaam 🚧 #69005
k8s @SameerMesiah97 🚧 #69046
openlineage

Assigned: 👀
PR: 🚧
Done: ✅


Provider metrics to be updated:

amazon (3):

  • batch_executor.adopt_task_instances.duration
  • ecs_executor.adopt_task_instances.duration
  • lambda_executor.adopt_task_instances.duration

celery (2):

  • celery.task_timeout_error
  • celery.execute_command.failure

edge3 worker (10):

  • edge_worker.status
  • edge_worker.connected
  • edge_worker.maintenance
  • edge_worker.jobs_active
  • edge_worker.concurrency
  • edge_worker.free_concurrency
  • edge_worker.num_queues
  • edge_worker.heartbeat_count
  • edge_worker.ti.start
  • edge_worker.ti.finish

edge3 executor (1):

  • edge_executor.sync.duration

kubernetes (8):

  • kubernetes_executor.pod_creation_status
  • kubernetes_executor.pod_deletion_status
  • kubernetes_executor.pod_patching_status
  • kubernetes_executor.clear_not_launched_queued_tasks.duration
  • kubernetes_executor.adopt_task_instances.duration
  • kubernetes_executor.pod_creation
  • kubernetes_executor.pod_deletion
  • kubernetes_executor.pod_patching

openlineage (4):

  • ol.emit.failed
  • ol.event.size
  • ol.emit.attempts
  • ol.extract

Prior work examples:

Assets: #68367
Deadlines: #68589
Executors: #68593
Scheduler: #68594
Dag Processor: #68599

Metadata

Metadata

Assignees

No one assigned

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions