DO-2075 Added fenix and desktop baseline city seen tables

[wwyc]

Description

Initialize the *baseline_city_seen tables by deriving each client’s first-seen and last-seen city, subdivision and country fields from the stable tables.
Note: This one-time initialization logic will no longer apply once city/subdivision/country fields are nulled in the stable tables.
Ongoing updates: After initialization, the tables will be updated daily via ETL using live tables (appending new clients and advancing last-seen values).

Related Tickets & Documents

• DO-2075

Reviewer, please follow this checklist

[BenWu]

You should remove all the generated files in sql/ before merging because generated sql doesn't need to be checked in.

The is_init query with a 100% sample might be too expensive run in a single query and timeout. This is fine for the POC but something to prepare for later

[BenWu]

You should pay attention to how long this takes to init with a 1% sample since it blocks artifact deployment during the init. It should be fine for now but we might need to do the full initial backfill another way.

[wwyc]

I just did an init with a 2% sample and it took:

• firefox_desktop: 319s
• fenix: 111s
  However, if I were to add @sample_id then the table would be initialized in parallel per sample_id:

  bigquery-etl/bigquery_etl/cli/query.py

  Line 1460 in 59139fd

  To run in parallel per sample_id, include a @sample_id parameter in the query.

Would that be sufficient or is there another way to do the full initial backfill that you would recommend? Thanks!

[dataops-ci-bot]

Integration report for "Merge branch 'main' into do-2075-initialize-client-city-seen"

sql.diff

Click to expand!

Only in /tmp/workspace/generated-sql/dags/: bqetl_baseline_clients_city_seen.py
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/dags/bqetl_baseline_clients_city_seen.py /tmp/workspace/generated-sql/dags/bqetl_baseline_clients_city_seen.py
--- /tmp/workspace/main-generated-sql/dags/bqetl_baseline_clients_city_seen.py	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/dags/bqetl_baseline_clients_city_seen.py	2025-10-03 20:53:31.000000000 +0000
@@ -0,0 +1,77 @@
+# Generated via https://github.com/mozilla/bigquery-etl/blob/main/bigquery_etl/query_scheduling/generate_airflow_dags.py
+
+from airflow import DAG
+from airflow.sensors.external_task import ExternalTaskMarker
+from airflow.sensors.external_task import ExternalTaskSensor
+from airflow.utils.task_group import TaskGroup
+import datetime
+from operators.gcp_container_operator import GKEPodOperator
+from utils.constants import ALLOWED_STATES, FAILED_STATES
+from utils.gcp import bigquery_etl_query, bigquery_dq_check, bigquery_bigeye_check
+
+docs = """
+### bqetl_baseline_clients_city_seen
+
+Built from bigquery-etl repo, [`dags/bqetl_baseline_clients_city_seen.py`](https://github.com/mozilla/bigquery-etl/blob/generated-sql/dags/bqetl_baseline_clients_city_seen.py)
+
+#### Description
+
+Scheduled queries for client city seen tables
+#### Owner
+
+[email protected]
+
+#### Tags
+
+* impact/tier_2
+* repo/bigquery-etl
+"""
+
+
+default_args = {
+    "owner": "[email protected]",
+    "start_date": datetime.datetime(2025, 9, 20, 0, 0),
+    "end_date": None,
+    "email": ["[email protected]", "[email protected]"],
+    "depends_on_past": False,
+    "retry_delay": datetime.timedelta(seconds=1800),
+    "email_on_failure": True,
+    "email_on_retry": True,
+    "retries": 2,
+    "max_active_tis_per_dag": None,
+}
+
+tags = ["impact/tier_2", "repo/bigquery-etl"]
+
+with DAG(
+    "bqetl_baseline_clients_city_seen",
+    default_args=default_args,
+    schedule_interval="0 4 * * *",
+    doc_md=docs,
+    tags=tags,
+    catchup=False,
+) as dag:
+
+    fenix_baseline_clients_city_seen_v1 = bigquery_etl_query(
+        task_id="fenix_baseline_clients_city_seen_v1",
+        destination_table="baseline_clients_city_seen_v1",
+        dataset_id="fenix_derived",
+        project_id="moz-fx-data-shared-prod",
+        owner="[email protected]",
+        email=["[email protected]", "[email protected]"],
+        date_partition_parameter=None,
+        depends_on_past=True,
+        parameters=["submission_date:DATE:{{ds}}"],
+    )
+
+    firefox_desktop_baseline_clients_city_seen_v1 = bigquery_etl_query(
+        task_id="firefox_desktop_baseline_clients_city_seen_v1",
+        destination_table="baseline_clients_city_seen_v1",
+        dataset_id="firefox_desktop_derived",
+        project_id="moz-fx-data-shared-prod",
+        owner="[email protected]",
+        email=["[email protected]", "[email protected]"],
+        date_partition_parameter=None,
+        depends_on_past=True,
+        parameters=["submission_date:DATE:{{ds}}"],
+    )
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix: baseline_clients_city_seen
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived: baseline_clients_city_seen_v1
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop: baseline_clients_city_seen
Only in /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/firefox_desktop_derived: baseline_clients_city_seen_v1
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_daily_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_daily_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_daily_v1/metadata.yaml	2025-10-03 20:31:04.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_daily_v1/metadata.yaml	2025-10-03 20:47:19.000000000 +0000
@@ -39,7 +39,7 @@
   enabled: false
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_first_seen_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_first_seen_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_first_seen_v1/metadata.yaml	2025-10-03 20:31:04.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_first_seen_v1/metadata.yaml	2025-10-03 20:47:19.000000000 +0000
@@ -49,7 +49,7 @@
   enabled: false
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_last_seen_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_last_seen_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_last_seen_v1/metadata.yaml	2025-10-03 20:31:04.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/baseline_clients_last_seen_v1/metadata.yaml	2025-10-03 20:47:19.000000000 +0000
@@ -48,7 +48,7 @@
   enabled: false
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/metrics_clients_daily_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/metrics_clients_daily_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/metrics_clients_daily_v1/metadata.yaml	2025-10-03 20:31:04.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/accounts_cirrus_derived/metrics_clients_daily_v1/metadata.yaml	2025-10-03 20:47:19.000000000 +0000
@@ -34,7 +34,7 @@
   enabled: false
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_daily_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_daily_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_daily_v1/metadata.yaml	2025-10-03 20:31:04.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_daily_v1/metadata.yaml	2025-10-03 20:47:18.000000000 +0000
@@ -39,7 +39,7 @@
   enabled: false
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_first_seen_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_first_seen_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_first_seen_v1/metadata.yaml	2025-10-03 20:31:04.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_first_seen_v1/metadata.yaml	2025-10-03 20:47:18.000000000 +0000
@@ -49,7 +49,7 @@
   enabled: false
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_last_seen_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_last_seen_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_last_seen_v1/metadata.yaml	2025-10-03 20:31:04.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/baseline_clients_last_seen_v1/metadata.yaml	2025-10-03 20:47:18.000000000 +0000
@@ -48,7 +48,7 @@
   enabled: false
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/metrics_clients_daily_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/metrics_clients_daily_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/metrics_clients_daily_v1/metadata.yaml	2025-10-03 20:31:04.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/metrics_clients_daily_v1/metadata.yaml	2025-10-03 20:47:18.000000000 +0000
@@ -34,7 +34,7 @@
   enabled: false
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/metrics_clients_last_seen_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/metrics_clients_last_seen_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/metrics_clients_last_seen_v1/metadata.yaml	2025-10-03 20:31:04.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/burnham_derived/metrics_clients_last_seen_v1/metadata.yaml	2025-10-03 20:47:18.000000000 +0000
@@ -48,7 +48,7 @@
   enabled: false
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml	2025-10-03 20:31:04.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates/schema.yaml	2025-10-03 20:37:24.000000000 +0000
@@ -1,49 +1,49 @@
 fields:
-- mode: NULLABLE
-  name: submission_date
+- name: submission_date
   type: DATE
-- mode: NULLABLE
-  name: source
+  mode: NULLABLE
+- name: source
   type: STRING
-- mode: NULLABLE
-  name: event_type
+  mode: NULLABLE
+- name: event_type
   type: STRING
-- mode: NULLABLE
-  name: form_factor
+  mode: NULLABLE
+- name: form_factor
   type: STRING
-- mode: NULLABLE
-  name: country
+  mode: NULLABLE
+- name: country
   type: STRING
-- mode: NULLABLE
-  name: subdivision1
+  mode: NULLABLE
+- name: subdivision1
   type: STRING
-- mode: NULLABLE
-  name: advertiser
+  mode: NULLABLE
+- name: advertiser
   type: STRING
-- mode: NULLABLE
-  name: release_channel
+  mode: NULLABLE
+- name: release_channel
   type: STRING
-- mode: NULLABLE
-  name: position
+  mode: NULLABLE
+- name: position
   type: INTEGER
-- mode: NULLABLE
-  name: provider
+  mode: NULLABLE
+- name: provider
   type: STRING
-- mode: NULLABLE
-  name: match_type
+  mode: NULLABLE
+- name: match_type
   type: STRING
-- mode: NULLABLE
-  name: normalized_os
+  mode: NULLABLE
+- name: normalized_os
   type: STRING
-- mode: NULLABLE
-  name: suggest_data_sharing_enabled
+  mode: NULLABLE
+- name: suggest_data_sharing_enabled
   type: BOOLEAN
-- mode: NULLABLE
-  name: event_count
+  mode: NULLABLE
+- name: event_count
   type: INTEGER
-- mode: NULLABLE
-  name: user_count
+  mode: NULLABLE
+- name: user_count
   type: INTEGER
-- mode: NULLABLE
-  name: query_type
+  mode: NULLABLE
+- name: query_type
   type: STRING
+  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml	2025-10-03 20:31:04.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/contextual_services/event_aggregates_suggest/schema.yaml	2025-10-03 20:37:55.000000000 +0000
@@ -1,40 +1,40 @@
 fields:
-- mode: NULLABLE
-  name: submission_date
+- name: submission_date
   type: DATE
-- mode: NULLABLE
-  name: form_factor
+  mode: NULLABLE
+- name: form_factor
   type: STRING
-- mode: NULLABLE
-  name: country
+  mode: NULLABLE
+- name: country
   type: STRING
-- mode: NULLABLE
-  name: advertiser
+  mode: NULLABLE
+- name: advertiser
   type: STRING
-- mode: NULLABLE
-  name: normalized_os
+  mode: NULLABLE
+- name: normalized_os
   type: STRING
-- mode: NULLABLE
-  name: release_channel
+  mode: NULLABLE
+- name: release_channel
   type: STRING
-- mode: NULLABLE
-  name: position
+  mode: NULLABLE
+- name: position
   type: INTEGER
-- mode: NULLABLE
-  name: provider
+  mode: NULLABLE
+- name: provider
   type: STRING
-- mode: NULLABLE
-  name: match_type
+  mode: NULLABLE
+- name: match_type
   type: STRING
-- mode: NULLABLE
-  name: suggest_data_sharing_enabled
+  mode: NULLABLE
+- name: suggest_data_sharing_enabled
   type: BOOLEAN
-- mode: NULLABLE
-  name: impression_count
+  mode: NULLABLE
+- name: impression_count
   type: INTEGER
-- mode: NULLABLE
-  name: click_count
+  mode: NULLABLE
+- name: click_count
   type: INTEGER
-- mode: NULLABLE
-  name: query_type
+  mode: NULLABLE
+- name: query_type
   type: STRING
+  mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/baseline_clients_daily_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/baseline_clients_daily_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/baseline_clients_daily_v1/metadata.yaml	2025-10-03 20:31:04.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/baseline_clients_daily_v1/metadata.yaml	2025-10-03 20:47:18.000000000 +0000
@@ -39,7 +39,7 @@
   enabled: true
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/baseline_clients_first_seen_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/baseline_clients_first_seen_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/baseline_clients_first_seen_v1/metadata.yaml	2025-10-03 20:31:04.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/baseline_clients_first_seen_v1/metadata.yaml	2025-10-03 20:47:18.000000000 +0000
@@ -49,7 +49,7 @@
   enabled: true
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/baseline_clients_last_seen_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/baseline_clients_last_seen_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/baseline_clients_last_seen_v1/metadata.yaml	2025-10-03 20:31:04.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/baseline_clients_last_seen_v1/metadata.yaml	2025-10-03 20:47:18.000000000 +0000
@@ -51,7 +51,7 @@
   enabled: true
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/metrics_clients_daily_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/metrics_clients_daily_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/metrics_clients_daily_v1/metadata.yaml	2025-10-03 20:31:04.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/experimenter_cirrus_derived/metrics_clients_daily_v1/metadata.yaml	2025-10-03 20:47:18.000000000 +0000
@@ -34,7 +34,7 @@
   enabled: true
   collection: null
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/baseline_clients_city_seen/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/baseline_clients_city_seen/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/baseline_clients_city_seen/metadata.yaml	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/baseline_clients_city_seen/metadata.yaml	2025-10-03 20:47:18.000000000 +0000
@@ -0,0 +1,14 @@
+friendly_name: Baseline Clients City Seen
+description: |-
+  Please provide a description for the query
+owners: []
+labels: {}
+bigquery: null
+workgroup_access:
+- role: roles/bigquery.dataViewer
+  members:
+  - workgroup:mozilla-confidential
+references:
+  view.sql:
+  - moz-fx-data-shared-prod.fenix_derived.baseline_clients_city_seen_v1
+require_column_descriptions: false
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/baseline_clients_city_seen/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/baseline_clients_city_seen/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/baseline_clients_city_seen/schema.yaml	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/baseline_clients_city_seen/schema.yaml	2025-10-03 20:45:09.000000000 +0000
@@ -0,0 +1,58 @@
+fields:
+- name: app_id
+  type: STRING
+  mode: NULLABLE
+  description: ID of the browser application.
+- name: client_id
+  type: STRING
+  mode: NULLABLE
+  description: A UUID uniquely identifying the client.
+- name: sample_id
+  type: INTEGER
+  mode: NULLABLE
+  description: A number, 0-99, that samples by client_id and allows filtering data
+    for analysis.
+- name: first_seen_city_date
+  type: DATE
+  mode: NULLABLE
+  description: Date when the first seen city was captured.
+- name: first_seen_city
+  type: STRING
+  mode: NULLABLE
+  description: City captured on first_seen_city_date.
+- name: first_seen_subdivision1
+  type: STRING
+  mode: NULLABLE
+  description: Major country subdivision, typically a state, province, or county captured
+    on first_seen_city_date.
+- name: first_seen_subdivision2
+  type: STRING
+  mode: NULLABLE
+  description: Second major country subdivision; not applicable for most countries
+    captured on first_seen_city_date.
+- name: first_seen_country
+  type: STRING
+  mode: NULLABLE
+  description: An ISO 3166-1 alpha-2 country code captured on first_seen_city_date.
+- name: last_seen_city_date
+  type: DATE
+  mode: NULLABLE
+  description: Date when the last seen city was captured.
+- name: last_seen_city
+  type: STRING
+  mode: NULLABLE
+  description: City captured on last_seen_city_date.
+- name: last_seen_subdivision1
+  type: STRING
+  mode: NULLABLE
+  description: Major country subdivision, typically a state, province, or county captured
+    on last_seen_city_date.
+- name: last_seen_subdivision2
+  type: STRING
+  mode: NULLABLE
+  description: Second major country subdivision; not applicable for most countries
+    captured on last_seen_city_date.
+- name: last_seen_country
+  type: STRING
+  mode: NULLABLE
+  description: An ISO 3166-1 alpha-2 country code captured on last_seen_city_date.
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/baseline_clients_city_seen/view.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/baseline_clients_city_seen/view.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/baseline_clients_city_seen/view.sql	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/baseline_clients_city_seen/view.sql	2025-10-03 20:36:52.000000000 +0000
@@ -0,0 +1,8 @@
+--- User-facing view. Generated via sql_generators.baseline_clients_city_seen.
+CREATE OR REPLACE VIEW
+  `moz-fx-data-shared-prod.fenix.baseline_clients_city_seen`
+AS
+SELECT
+  *
+FROM
+  `moz-fx-data-shared-prod.fenix_derived.baseline_clients_city_seen_v1`
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/broken_site_report/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/broken_site_report/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/broken_site_report/metadata.yaml	2025-10-03 20:31:04.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/broken_site_report/metadata.yaml	2025-10-03 20:47:18.000000000 +0000
@@ -1,6 +1,10 @@
-friendly_name: Broken Site Report
+friendly_name: App-specific view for Glean ping "broken-site-report"
 description: |-
-  Please provide a description for the query
+  This a view that UNIONs the stable ping tables
+  across all channels of the Glean application "Firefox for Android"
+  (org_mozilla_firefox.broken_site_report, org_mozilla_firefox_beta.broken_site_report, org_mozilla_fenix.broken_site_report, org_mozilla_fenix_nightly.broken_site_report, org_mozilla_fennec_aurora.broken_site_report).
+
+  It is used by Looker.
 owners: []
 labels: {}
 bigquery: null
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/crash/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/crash/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/crash/metadata.yaml	2025-10-03 20:31:04.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/crash/metadata.yaml	2025-10-03 20:47:18.000000000 +0000
@@ -1,6 +1,10 @@
-friendly_name: Crash
+friendly_name: App-specific view for Glean ping "crash"
 description: |-
-  Please provide a description for the query
+  This a view that UNIONs the stable ping tables
+  across all channels of the Glean application "Firefox for Android"
+  (org_mozilla_firefox.crash, org_mozilla_firefox_beta.crash, org_mozilla_fenix.crash, org_mozilla_fenix_nightly.crash, org_mozilla_fennec_aurora.crash).
+
+  It is used by Looker.
 owners: []
 labels: {}
 bigquery: null
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml	2025-10-03 20:31:04.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_clients/schema.yaml	2025-10-03 20:38:11.000000000 +0000
@@ -26,6 +26,9 @@
 - name: adjust_network
   type: STRING
   mode: NULLABLE
+- name: install_source
+  type: STRING
+  mode: NULLABLE
 - name: retained_week_2
   type: BOOLEAN
   mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml	2025-10-03 20:31:04.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/funnel_retention_week_4/schema.yaml	2025-10-03 20:38:11.000000000 +0000
@@ -48,6 +48,9 @@
   description: 'The type of source of a client installation.
 
     '
+- name: install_source
+  type: STRING
+  mode: NULLABLE
 - name: new_profiles
   type: INTEGER
   mode: NULLABLE
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/use_counters/schema.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/use_counters/schema.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix/use_counters/schema.yaml	2025-10-03 20:31:04.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix/use_counters/schema.yaml	2025-10-03 20:41:27.000000000 +0000
@@ -1,11 +1,11 @@
 fields:
 - name: normalized_app_id
-  type: STRING
   mode: NULLABLE
+  type: STRING
   description: App ID of the channel data was received from
 - name: normalized_channel
-  type: STRING
   mode: NULLABLE
+  type: STRING
   description: Normalized channel name
 - name: additional_properties
   type: STRING
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/attribution_clients_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/attribution_clients_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/attribution_clients_v1/metadata.yaml	2025-10-03 20:31:04.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/attribution_clients_v1/metadata.yaml	2025-10-03 20:47:17.000000000 +0000
@@ -41,7 +41,7 @@
   enabled: true
   collection: Operational Checks
   partition_column: null
-  partition_column_set: true
+  partition_column_set: false
   freshness:
     blocking: false
   volume:
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/baseline_clients_city_seen_v1/metadata.yaml /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/baseline_clients_city_seen_v1/metadata.yaml
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/baseline_clients_city_seen_v1/metadata.yaml	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/baseline_clients_city_seen_v1/metadata.yaml	2025-10-03 20:47:17.000000000 +0000
@@ -0,0 +1,50 @@
+friendly_name: Baseline Clients City Seen
+description: |-
+  This table stores the first-seen and last-seen geo attributes for each client_id.
+  The table was initialized from stable tables (with ~2 years of retention),
+  so the initial dates reflect the earliest/latest observations within that window.
+  After initialization, the city fields in the stable tables are nullified and
+  the one-time initialization code is no longer relevant.  The table is updated daily using live tables,
+  updating last-seen fields for existing clients and adding new clients with first-seen geo attributes
+  that do not yet exist in the table.
+  This table should not be backfilled beyond 30 days which is the maximum retention of live tables.
+
+  JIRA ticket: https://mozilla-hub.atlassian.net/browse/DENG-9468
+owners:
+- [email protected]
+labels:
+  incremental: true
+  schedule: daily
+  table_type: client_level
+  dag: bqetl_baseline_clients_city_seen
+  owner1: wichan
+scheduling:
+  dag_name: bqetl_baseline_clients_city_seen
+  task_name: fenix_baseline_clients_city_seen_v1
+  depends_on_past: true
+  date_partition_parameter: null
+  parameters:
+  - submission_date:DATE:{{ds}}
+bigquery:
+  time_partitioning:
+    type: day
+    field: first_seen_city_date
+    require_partition_filter: false
+    expiration_days: null
+  range_partitioning: null
+  clustering:
+    fields:
+    - sample_id
+workgroup_access:
+- role: roles/bigquery.dataViewer
+  members:
+  - workgroup:mozilla-confidential
+references:
+  query.sql:
+  - moz-fx-data-shared-prod.fenix_derived.baseline_clients_city_seen_v1
+  - moz-fx-data-shared-prod.org_mozilla_fenix_live.baseline_v1
+  - moz-fx-data-shared-prod.org_mozilla_fenix_nightly_live.baseline_v1
+  - moz-fx-data-shared-prod.org_mozilla_fennec_aurora_live.baseline_v1
+  - moz-fx-data-shared-prod.org_mozilla_firefox_beta_live.baseline_v1
+  - moz-fx-data-shared-prod.org_mozilla_firefox_live.baseline_v1
+require_column_descriptions: false
diff -bur --no-dereference --new-file /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/baseline_clients_city_seen_v1/query.sql /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/baseline_clients_city_seen_v1/query.sql
--- /tmp/workspace/main-generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/baseline_clients_city_seen_v1/query.sql	1970-01-01 00:00:00.000000000 +0000
+++ /tmp/workspace/generated-sql/sql/moz-fx-data-shared-prod/fenix_derived/baseline_clients_city_seen_v1/query.sql	2025-10-03 20:36:52.000000000 +0000
@@ -0,0 +1,1243 @@
+ -- Query generated via sql_generators.baseline_clients_city_seen.
+ -- this mimics the logic used in baseline_clients_daily_v1.
+WITH base_org_mozilla_firefox AS (
+  -- Live table dedup logic mimics copy_deduplicate
+  SELECT
+    submission_timestamp,
+    DATE(submission_timestamp) AS submission_date,
+    LOWER(client_info.client_id) AS client_id,
+    sample_id,
+    mozfun.glean.parse_datetime(ping_info.end_time) AS parsed_end_time,
+    `moz-fx-data-shared-prod.udf.glean_timespan_seconds`(
+      metrics.timespan.glean_baseline_duration
+    ) AS duration,
+    metadata.geo.city AS city,
+    metadata.geo.subdivision1 AS subdivision1,
+    metadata.geo.subdivision2 AS subdivision2,
+    metadata.geo.country AS country
+  FROM
+    {% if is_init() %}
+      `moz-fx-data-shared-prod.org_mozilla_firefox_stable.baseline_v1`
+    {% else %}
+      `moz-fx-data-shared-prod.org_mozilla_firefox_live.baseline_v1`
+    {% endif %}
+  WHERE
+    client_info.client_id IS NOT NULL
+    {% if is_init() %}
+      AND sample_id = @sample_id
+      AND DATE(submission_timestamp) <= CURRENT_DATE()
+    {% else %}
+      AND DATE(submission_timestamp) = @submission_date
+      AND 'automation' NOT IN (
+        SELECT
+          TRIM(t)
+        FROM
+          UNNEST(SPLIT(metadata.header.x_source_tags, ',')) t
+      )
+      QUALIFY
+        ROW_NUMBER() OVER (PARTITION BY document_id ORDER BY submission_timestamp) = 1
+    {% endif %}
+),
+with_dates_org_mozilla_firefox AS (
+  SELECT
+    *,
+    -- For explanation of session start time calculation, see Glean docs:
+    -- https://mozilla.github.io/glean/book/user/pings/baseline.html#contents
+    DATE(SAFE.TIMESTAMP_SUB(parsed_end_time, INTERVAL duration SECOND)) AS session_start_date,
+    DATE(parsed_end_time) AS session_end_date
+  FROM
+    base_org_mozilla_firefox
+),
+with_date_offsets_org_mozilla_firefox AS (
+  SELECT
+    *,
+    DATE_DIFF(submission_date, session_start_date, DAY) AS session_start_date_offset,
+    DATE_DIFF(submission_date, session_end_date, DAY) AS session_end_date_offset,
+  FROM
+    with_dates_org_mozilla_firefox
+),
+overactive_org_mozilla_firefox AS (
+  -- Find client_ids with over 150 000 pings in a day,
+  -- which could cause errors in the next step due to aggregation overflows.
+  SELECT
+    submission_date,
+    client_id
+  FROM
+    with_date_offsets_org_mozilla_firefox
+  WHERE
+    {% if is_init() %}
+      submission_date >= '2018-01-01'
+    {% else %}
+      submission_date = @submission_date
+    {% endif %}
+  GROUP BY
+    submission_date,
+    client_id
+  HAVING
+    COUNT(*) > 150000
+),
+clients_daily_org_mozilla_firefox AS (
+  SELECT
+    "org_mozilla_firefox" AS app_id,
+    submission_date,
+    client_id,
+    sample_id,
+    `moz-fx-data-shared-prod.udf.mode_last`(
+      ARRAY_AGG(STRUCT(city, subdivision1, subdivision2, country) ORDER BY submission_timestamp)
+    ) AS geo
+  FROM
+    with_date_offsets_org_mozilla_firefox
+  LEFT JOIN
+    overactive_org_mozilla_firefox
+    USING (submission_date, client_id)
+  WHERE
+    overactive_org_mozilla_firefox.client_id IS NULL
+    -- `mode_last` can result in struct with all null values if it’s most frequent (or latest among ties).
+    -- This exclude structs with all null values so there will always be one non-NULL field.
+    AND COALESCE(city, subdivision1, subdivision2, country) IS NOT NULL
+    {% if is_init() %}
+      AND submission_date >= '2018-01-01'
+    {% else %}
+      AND submission_date = @submission_date
+    {% endif %}
+  GROUP BY
+    submission_date,
+    client_id,
+    sample_id
+),
+{% if is_init() %}
+  clients_city_first_seen_org_mozilla_firefox AS (
+    SELECT
+      app_id,
+      client_id,
+      sample_id,
+      submission_date AS first_seen_city_date,
+      geo.city AS first_seen_city,
+      geo.subdivision1 AS first_seen_subdivision1,
+      geo.subdivision2 AS first_seen_subdivision2,
+      geo.country AS first_seen_country
+    FROM
+      clients_daily_org_mozilla_firefox
+    WHERE
+      geo.city IS NOT NULL
+    QUALIFY
+      ROW_NUMBER() OVER (PARTITION BY client_id, sample_id ORDER BY submission_date) = 1
+  ),
+  clients_city_last_seen_org_mozilla_firefox AS (
+    SELECT
+      app_id,
+      client_id,
+      sample_id,
+      submission_date AS last_seen_city_date,
+      geo.city AS last_seen_city,
+      geo.subdivision1 AS last_seen_subdivision1,
+      geo.subdivision2 AS last_seen_subdivision2,
+      geo.country AS last_seen_country
+    FROM
+      clients_daily_org_mozilla_firefox
+    WHERE
+      geo.city IS NOT NULL
+    QUALIFY
+      ROW_NUMBER() OVER (PARTITION BY client_id, sample_id ORDER BY submission_date DESC) = 1
+  ),
+{% else %}
+  _previous_org_mozilla_firefox AS (
+    SELECT
+      *
+    FROM
+      `moz-fx-data-shared-prod.fenix_derived.baseline_clients_city_seen_v1`
+    WHERE
+      app_id = "org_mozilla_firefox"
+  ),
+  _current_org_mozilla_firefox AS (
+    SELECT
+      app_id,
+      client_id,
+      sample_id,
+      submission_date AS first_seen_city_date,
+      geo.city AS first_seen_city,
+      geo.subdivision1 AS first_seen_subdivision1,
+      geo.subdivision2 AS first_seen_subdivision2,
+      geo.country AS first_seen_country,
+      submission_date AS last_seen_city_date,
+      geo.city AS last_seen_city,
+      geo.subdivision1 AS last_seen_subdivision1,
+      geo.subdivision2 AS last_seen_subdivision2,
+      geo.country AS last_seen_country
+    FROM
+      clients_daily_org_mozilla_firefox
+    WHERE
+      geo.city IS NOT NULL
+  ),
+{% endif %}
+base_org_mozilla_firefox_beta AS (
+  -- Live table dedup logic mimics copy_deduplicate
+  SELECT
+    submission_timestamp,
+    DATE(submission_timestamp) AS submission_date,
+    LOWER(client_info.client_id) AS client_id,
+    sample_id,
+    mozfun.glean.parse_datetime(ping_info.end_time) AS parsed_end_time,
+    `moz-fx-data-shared-prod.udf.glean_timespan_seconds`(
+      metrics.timespan.glean_baseline_duration
+    ) AS duration,
+    metadata.geo.city AS city,
+    metadata.geo.subdivision1 AS subdivision1,
+    metadata.geo.subdivision2 AS subdivision2,
+    metadata.geo.country AS country
+  FROM
+    {% if is_init() %}
+      `moz-fx-data-shared-prod.org_mozilla_firefox_beta_stable.baseline_v1`
+    {% else %}
+      `moz-fx-data-shared-prod.org_mozilla_firefox_beta_live.baseline_v1`
+    {% endif %}
+  WHERE
+    client_info.client_id IS NOT NULL
+    {% if is_init() %}
+      AND sample_id = @sample_id
+      AND DATE(submission_timestamp) <= CURRENT_DATE()
+    {% else %}
+      AND DATE(submission_timestamp) = @submission_date
+      AND 'automation' NOT IN (
+        SELECT
+          TRIM(t)
+        FROM
+          UNNEST(SPLIT(metadata.header.x_source_tags, ',')) t
+      )
+      QUALIFY
+        ROW_NUMBER() OVER (PARTITION BY document_id ORDER BY submission_timestamp) = 1
+    {% endif %}
+),
+with_dates_org_mozilla_firefox_beta AS (
+  SELECT
+    *,
+    -- For explanation of session start time calculation, see Glean docs:
+    -- https://mozilla.github.io/glean/book/user/pings/baseline.html#contents
+    DATE(SAFE.TIMESTAMP_SUB(parsed_end_time, INTERVAL duration SECOND)) AS session_start_date,
+    DATE(parsed_end_time) AS session_end_date
+  FROM
+    base_org_mozilla_firefox_beta
+),
+with_date_offsets_org_mozilla_firefox_beta AS (
+  SELECT
+    *,
+    DATE_DIFF(submission_date, session_start_date, DAY) AS session_start_date_offset,
+    DATE_DIFF(submission_date, session_end_date, DAY) AS session_end_date_offset,
+  FROM
+    with_dates_org_mozilla_firefox_beta
+),
+overactive_org_mozilla_firefox_beta AS (
+  -- Find client_ids with over 150 000 pings in a day,
+  -- which could cause errors in the next step due to aggregation overflows.
+  SELECT
+    submission_date,
+    client_id
+  FROM
+    with_date_offsets_org_mozilla_firefox_beta
+  WHERE
+    {% if is_init() %}
+      submission_date >= '2018-01-01'
+    {% else %}
+      submission_date = @submission_date
+    {% endif %}
+  GROUP BY
+    submission_date,
+    client_id
+  HAVING
+    COUNT(*) > 150000
+),
+clients_daily_org_mozilla_firefox_beta AS (
+  SELECT
+    "org_mozilla_firefox_beta" AS app_id,
+    submission_date,
+    client_id,
+    sample_id,
+    `moz-fx-data-shared-prod.udf.mode_last`(
+      ARRAY_AGG(STRUCT(city, subdivision1, subdivision2, country) ORDER BY submission_timestamp)
+    ) AS geo
+  FROM
+    with_date_offsets_org_mozilla_firefox_beta
+  LEFT JOIN
+    overactive_org_mozilla_firefox_beta
+    USING (submission_date, client_id)
+  WHERE
+    overactive_org_mozilla_firefox_beta.client_id IS NULL
+    -- `mode_last` can result in struct with all null values if it’s most frequent (or latest among ties).
+    -- This exclude structs with all null values so there will always be one non-NULL field.
+    AND COALESCE(city, subdivision1, subdivision2, country) IS NOT NULL
+    {% if is_init() %}
+      AND submission_date >= '2018-01-01'
+    {% else %}
+      AND submission_date = @submission_date
+    {% endif %}
+  GROUP BY
+    submission_date,
+    client_id,
+    sample_id
+),
+{% if is_init() %}
+  clients_city_first_seen_org_mozilla_firefox_beta AS (
+    SELECT
+      app_id,
+      client_id,
+      sample_id,
+      submission_date AS first_seen_city_date,
+      geo.city AS first_seen_city,
+      geo.subdivision1 AS first_seen_subdivision1,
+      geo.subdivision2 AS first_seen_subdivision2,
+      geo.country AS first_seen_country
+    FROM
+      clients_daily_org_mozilla_firefox_beta
+    WHERE
+      geo.city IS NOT NULL
+    QUALIFY
+      ROW_NUMBER() OVER (PARTITION BY client_id, sample_id ORDER BY submission_date) = 1
+  ),
+  clients_city_last_seen_org_mozilla_firefox_beta AS (
+    SELECT
+      app_id,
+      client_id,
+      sample_id,
+      submission_date AS last_seen_city_date,
+      geo.city AS last_seen_city,
+      geo.subdivision1 AS last_seen_subdivision1,
+      geo.subdivision2 AS last_seen_subdivision2,
+      geo.country AS last_seen_country
+    FROM
+      clients_daily_org_mozilla_firefox_beta
+    WHERE
+      geo.city IS NOT NULL
+    QUALIFY
+      ROW_NUMBER() OVER (PARTITION BY client_id, sample_id ORDER BY submission_date DESC) = 1
+  ),
+{% else %}
+  _previous_org_mozilla_firefox_beta AS (
+    SELECT
+      *
+    FROM
+      `moz-fx-data-shared-prod.fenix_derived.baseline_clients_city_seen_v1`
+    WHERE
+      app_id = "org_mozilla_firefox_beta"
+  ),
+  _current_org_mozilla_firefox_beta AS (
+    SELECT
+      app_id,
+      client_id,
+      sample_id,
+      submission_date AS first_seen_city_date,
+      geo.city AS first_seen_city,
+      geo.subdivision1 AS first_seen_subdivision1,
+      geo.subdivision2 AS first_seen_subdivision2,
+      geo.country AS first_seen_country,
+      submission_date AS last_seen_city_date,
+      geo.city AS last_seen_city,
+      geo.subdivision1 AS last_seen_subdivision1,
+      geo.subdivision2 AS last_seen_subdivision2,
+      geo.country AS last_seen_country
+    FROM
+      clients_daily_org_mozilla_firefox_beta
+    WHERE
+      geo.city IS NOT NULL
+  ),
+{% endif %}
+base_org_mozilla_fenix AS (
+  -- Live table dedup logic mimics copy_deduplicate
+  SELECT
+    submission_timestamp,
+    DATE(submission_timestamp) AS submission_date,
+    LOWER(client_info.client_id) AS client_id,
+    sample_id,
+    mozfun.glean.parse_datetime(ping_info.end_time) AS parsed_end_time,
+    `moz-fx-data-shared-prod.udf.glean_timespan_seconds`(
+      metrics.timespan.glean_baseline_duration
+    ) AS duration,
+    metadata.geo.city AS city,
+    metadata.geo.subdivision1 AS subdivision1,
+    metadata.geo.subdivision2 AS subdivision2,
+    metadata.geo.country AS country
+  FROM
+    {% if is_init() %}
+      `moz-fx-data-shared-prod.org_mozilla_fenix_stable.baseline_v1`
+    {% else %}
+      `moz-fx-data-shared-prod.org_mozilla_fenix_live.baseline_v1`
+    {% endif %}
+  WHERE
+    client_info.client_id IS NOT NULL
+    {% if is_init() %}
+      AND sample_id = @sample_id
+      AND DATE(submission_timestamp) <= CURRENT_DATE()
+    {% else %}
+      AND DATE(submission_timestamp) = @submission_date
+      AND 'automation' NOT IN (
+        SELECT
+          TRIM(t)
+        FROM
+          UNNEST(SPLIT(metadata.header.x_source_tags, ',')) t
+      )
+      QUALIFY
+        ROW_NUMBER() OVER (PARTITION BY document_id ORDER BY submission_timestamp) = 1
+    {% endif %}
+),
+with_dates_org_mozilla_fenix AS (
+  SELECT
+    *,
+    -- For explanation of session start time calculation, see Glean docs:
+    -- https://mozilla.github.io/glean/book/user/pings/baseline.html#contents
+    DATE(SAFE.TIMESTAMP_SUB(parsed_end_time, INTERVAL duration SECOND)) AS session_start_date,
+    DATE(parsed_end_time) AS session_end_date
+  FROM
+    base_org_mozilla_fenix
+),
+with_date_offsets_org_mozilla_fenix AS (
+  SELECT
+    *,
+    DATE_DIFF(submission_date, session_start_date, DAY) AS session_start_date_offset,
+    DATE_DIFF(submission_date, session_end_date, DAY) AS session_end_date_offset,
+  FROM
+    with_dates_org_mozilla_fenix
+),
+overactive_org_mozilla_fenix AS (
+  -- Find client_ids with over 150 000 pings in a day,
+  -- which could cause errors in the next step due to aggregation overflows.
+  SELECT
+    submission_date,
+    client_id
+  FROM
+    with_date_offsets_org_mozilla_fenix
+  WHERE
+    {% if is_init() %}
+      submission_date >= '2018-01-01'
+    {% else %}
+      submission_date = @submission_date
+    {% endif %}
+  GROUP BY
+    submission_date,
+    client_id
+  HAVING
+    COUNT(*) > 150000
+),
+clients_daily_org_mozilla_fenix AS (
+  SELECT
+    "org_mozilla_fenix" AS app_id,
+    submission_date,
+    client_id,
+    sample_id,
+    `moz-fx-data-shared-prod.udf.mode_last`(
+      ARRAY_AGG(STRUCT(city, subdivision1, subdivision2, country) ORDER BY submission_timestamp)
+    ) AS geo
+  FROM
+    with_date_offsets_org_mozilla_fenix
+  LEFT JOIN
+    overactive_org_mozilla_fenix
+    USING (submission_date, client_id)
+  WHERE
+    overactive_org_mozilla_fenix.client_id IS NULL
+    -- `mode_last` can result in struct with all null values if it’s most frequent (or latest among ties).
+    -- This exclude structs with all null values so there will always be one non-NULL field.
+    AND COALESCE(city, subdivision1, subdivision2, country) IS NOT NULL
+    {% if is_init() %}
+      AND submission_date >= '2018-01-01'
+    {% else %}
+      AND submission_date = @submission_date
+    {% endif %}
+  GROUP BY
+    submission_date,
+    client_id,
+    sample_id
+),
+{% if is_init() %}
+  clients_city_first_seen_org_mozilla_fenix AS (
+    SELECT
+      app_id,
+      client_id,
+      sample_id,
+      submission_date AS first_seen_city_date,
+      geo.city AS first_seen_city,
+      geo.subdivision1 AS first_seen_subdivision1,
+      geo.subdivision2 AS first_seen_subdivision2,
+      geo.country AS first_seen_country
+    FROM
+      clients_daily_org_mozilla_fenix
+    WHERE
+      geo.city IS NOT NULL
+    QUALIFY
+      ROW_NUMBER() OVER (PARTITION BY client_id, sample_id ORDER BY submission_date) = 1
+  ),
+  clients_city_last_seen_org_mozilla_fenix AS (
+    SELECT
+      app_id,
+      client_id,
+      sample_id,
+      submission_date AS last_seen_city_date,
+      geo.city AS last_seen_city,
+      geo.subdivision1 AS last_seen_subdivision1,
+      geo.subdivision2 AS last_seen_subdivision2,
+      geo.country AS last_seen_country
+    FROM
+      clients_daily_org_mozilla_fenix
+    WHERE
+      geo.city IS NOT NULL
+    QUALIFY
+      ROW_NUMBER() OVER (PARTITION BY client_id, sample_id ORDER BY submission_date DESC) = 1
+  ),
+{% else %}
+  _previous_org_mozilla_fenix AS (
+    SELECT
+      *
+    FROM
+      `moz-fx-data-shared-prod.fenix_derived.baseline_clients_city_seen_v1`
+    WHERE
+      app_id = "org_mozilla_fenix"
+  ),
+  _current_org_mozilla_fenix AS (
+    SELECT
+      app_id,
+      client_id,
+      sample_id,
+      submission_date AS first_seen_city_date,
+      geo.city AS first_seen_city,
+      geo.subdivision1 AS first_seen_subdivision1,
+      geo.subdivision2 AS first_seen_subdivision2,
+      geo.country AS first_seen_country,
+      submission_date AS last_seen_city_date,
+      geo.city AS last_seen_city,
+      geo.subdivision1 AS last_seen_subdivision1,
+      geo.subdivision2 AS last_seen_subdivision2,
+      geo.country AS last_seen_country
+    FROM
+      clients_daily_org_mozilla_fenix
+    WHERE
+      geo.city IS NOT NULL
+  ),
+{% endif %}
+base_org_mozilla_fenix_nightly AS (
+  -- Live table dedup logic mimics copy_deduplicate
+  SELECT
+    submission_timestamp,
+    DATE(submission_timestamp) AS submission_date,
+    LOWER(client_info.client_id) AS client_id,
+    sample_id,
+    mozfun.glean.parse_datetime(ping_info.end_time) AS parsed_end_time,
+    `moz-fx-data-shared-prod.udf.glean_timespan_seconds`(
+      metrics.timespan.glean_baseline_duration
+    ) AS duration,
+    metadata.geo.city AS city,
+    metadata.geo.subdivision1 AS subdivision1,
+    metadata.geo.subdivision2 AS subdivision2,
+    metadata.geo.country AS country
+  FROM
+    {% if is_init() %}
+      `moz-fx-data-shared-prod.org_mozilla_fenix_nightly_stable.baseline_v1`
+    {% else %}
+      `moz-fx-data-shared-prod.org_mozilla_fenix_nightly_live.baseline_v1`
+    {% endif %}
+  WHERE
+    client_info.client_id IS NOT NULL
+    {% if is_init() %}
+      AND sample_id = @sample_id
+      AND DATE(submission_timestamp) <= CURRENT_DATE()
+    {% else %}
+      AND DATE(submission_timestamp) = @submission_date
+      AND 'automation' NOT IN (
+        SELECT
+          TRIM(t)
+        FROM
+          UNNEST(SPLIT(metadata.header.x_source_tags, ',')) t
+      )
+      QUALIFY
+        ROW_NUMBER() OVER (PARTITION BY document_id ORDER BY submission_timestamp) = 1
+    {% endif %}
+),
+with_dates_org_mozilla_fenix_nightly AS (
+  SELECT
+    *,
+    -- For explanation of session start time calculation, see Glean docs:
+    -- https://mozilla.github.io/glean/book/user/pings/baseline.html#contents
+    DATE(SAFE.TIMESTAMP_SUB(parsed_end_time, INTERVAL duration SECOND)) AS session_start_date,
+    DATE(parsed_end_time) AS session_end_date
+  FROM
+    base_org_mozilla_fenix_nightly
+),
+with_date_offsets_org_mozilla_fenix_nightly AS (
+  SELECT
+    *,
+    DATE_DIFF(submission_date, session_start_date, DAY) AS session_start_date_offset,
+    DATE_DIFF(submission_date, session_end_date, DAY) AS session_end_date_offset,
+  FROM
+    with_dates_org_mozilla_fenix_nightly
+),
+overactive_org_mozilla_fenix_nightly AS (
+  -- Find client_ids with over 150 000 pings in a day,
+  -- which could cause errors in the next step due to aggregation overflows.
+  SELECT
+    submission_date,
+    client_id
+  FROM
+    with_date_offsets_org_mozilla_fenix_nightly
+  WHERE
+    {% if is_init() %}
+      submission_date >= '2018-01-01'
+    {% else %}
+      submission_date = @submission_date
+    {% endif %}
+  GROUP BY
+    submission_date,
+    client_id
+  HAVING
+    COUNT(*) > 150000
+),
+clients_daily_org_mozilla_fenix_nightly AS (
+  SELECT
+    "org_mozilla_fenix_nightly" AS app_id,
+    submission_date,
+    client_id,
+    sample_id,
+    `moz-fx-data-shared-prod.udf.mode_last`(
+      ARRAY_AGG(STRUCT(city, subdivision1, subdivision2, country) ORDER BY submission_timestamp)
+    ) AS geo
+  FROM
+    with_date_offsets_org_mozilla_fenix_nightly
+  LEFT JOIN
+    overactive_org_mozilla_fenix_nightly
+    USING (submission_date, client_id)
+  WHERE
+    overactive_org_mozilla_fenix_nightly.client_id IS NULL
+    -- `mode_last` can result in struct with all null values if it’s most frequent (or latest among ties).
+    -- This exclude structs with all null values so there will always be one non-NULL field.
+    AND COALESCE(city, subdivision1, subdivision2, country) IS NOT NULL
+    {% if is_init() %}
+      AND submission_date >= '2018-01-01'
+    {% else %}
+      AND submission_date = @submission_date
+    {% endif %}
+  GROUP BY
+    submission_date,
+    client_id,
+    sample_id
+),
+{% if is_init() %}
+  clients_city_first_seen_org_mozilla_fenix_nightly AS (
+    SELECT
+      app_id,
+      client_id,
+      sample_id,
+      submission_date AS first_seen_city_date,
+      geo.city AS first_seen_city,
+      geo.subdivision1 AS first_seen_subdivision1,
+      geo.subdivision2 AS first_seen_subdivision2,
+      geo.country AS first_seen_country
+    FROM
+      clients_daily_org_mozilla_fenix_nightly
+    WHERE
+      geo.city IS NOT NULL
+    QUALIFY
+      ROW_NUMBER() OVER (PARTITION BY client_id, sample_id ORDER BY submission_date) = 1
+  ),
+  clients_city_last_seen_org_mozilla_fenix_nightly AS (
+    SELECT
+      app_id,
+      client_id,
+      sample_id,
+      submission_date AS last_seen_city_date,
+      geo.city AS last_seen_city,
+      geo.subdivision1 AS last_seen_subdivision1,
+      geo.subdivision2 AS last_seen_subdivision2,
+      geo.country AS last_seen_country
+    FROM
+      clients_daily_org_mozilla_fenix_nightly
+    WHERE
+      geo.city IS NOT NULL
+    QUALIFY
+      ROW_NUMBER() OVER (PARTITION BY client_id, sample_id ORDER BY submission_date DESC) = 1
+  ),
+{% else %}
+  _previous_org_mozilla_fenix_nightly AS (
+    SELECT
+      *
+    FROM
+      `moz-fx-data-shared-prod.fenix_derived.baseline_clients_city_seen_v1`
+    WHERE
+      app_id = "org_mozilla_fenix_nightly"
+  ),
+  _current_org_mozilla_fenix_nightly AS (
+    SELECT
+      app_id,
+      client_id,
+      sample_id,
+      submission_date AS first_seen_city_date,
+      geo.city AS first_seen_city,
+      geo.subdivision1 AS first_seen_subdivision1,
+      geo.subdivision2 AS first_seen_subdivision2,
+      geo.country AS first_seen_country,
+      submission_date AS last_seen_city_date,
+      geo.city AS last_seen_city,
+      geo.subdivision1 AS last_seen_subdivision1,
+      geo.subdivision2 AS last_seen_subdivision2,
+      geo.country AS last_seen_country
+    FROM
+      clients_daily_org_mozilla_fenix_nightly
+    WHERE
+      geo.city IS NOT NULL
+  ),
+{% endif %}
+base_org_mozilla_fennec_aurora AS (
+  -- Live table dedup logic mimics copy_deduplicate
+  SELECT
+    submission_timestamp,
+    DATE(submission_timestamp) AS submission_date,
+    LOWER(client_info.client_id) AS client_id,
+    sample_id,
+    mozfun.glean.parse_datetime(ping_info.end_time) AS parsed_end_time,
+    `moz-fx-data-shared-prod.udf.glean_timespan_seconds`(
+      metrics.timespan.glean_baseline_duration
+    ) AS duration,
+    metadata.geo.city AS city,
+    metadata.geo.subdivision1 AS subdivision1,
+    metadata.geo.subdivision2 AS subdivision2,
+    metadata.geo.country AS country
+  FROM
+    {% if is_init() %}
+      `moz-fx-data-shared-prod.org_mozilla_fennec_aurora_stable.baseline_v1`
+    {% else %}
+      `moz-fx-data-shared-prod.org_mozilla_fennec_aurora_live.baseline_v1`
+    {% endif %}
+  WHERE
+    client_info.client_id IS NOT NULL
+    {% if is_init() %}
+      AND sample_id = @sample_id
+      AND DATE(submission_timestamp) <= CURRENT_DATE()
+    {% else %}
+      AND DATE(submission_timestamp) = @submission_date
+      AND 'automation' NOT IN (
+        SELECT
+          TRIM(t)
+        FROM
+          UNNEST(SPLIT(metadata.header.x_source_tags, ',')) t
+      )
+      QUALIFY
+        ROW_NUMBER() OVER (PARTITION BY document_id ORDER BY submission_timestamp) = 1
+    {% endif %}
+),
+with_dates_org_mozilla_fennec_aurora AS (
+  SELECT
+    *,
+    -- For explanation of session start time calculation, see Glean docs:
+    -- https://mozilla.github.io/glean/book/user/pings/baseline.html#contents
+    DATE(SAFE.TIMESTAMP_SUB(parsed_end_time, INTERVAL duration SECOND)) AS session_start_date,
+    DATE(parsed_end_time) AS session_end_date
+  FROM
+    base_org_mozilla_fennec_aurora
+),
+with_date_offsets_org_mozilla_fennec_aurora AS (
+  SELECT
+    *,
+    DATE_DIFF(submission_date, session_start_date, DAY) AS session_start_date_offset,
+    DATE_DIFF(submission_date, session_end_date, DAY) AS session_end_date_offset,
+  FROM
+    with_dates_org_mozilla_fennec_aurora
+),
+overactive_org_mozilla_fennec_aurora AS (
+  -- Find client_ids with over 150 000 pings in a day,
+  -- which could cause errors in the next step due to aggregation overflows.
+  SELECT
+    submission_date,
+    client_id
+  FROM
+    with_date_offsets_org_mozilla_fennec_aurora
+  WHERE
+    {% if is_init() %}
+      submission_date >= '2018-01-01'
+    {% else %}
+      submission_date = @submission_date
+    {% endif %}
+  GROUP BY
+    submission_date,
+    client_id
+  HAVING
+    COUNT(*) > 150000
+),
+clients_daily_org_mozilla_fennec_aurora AS (
+  SELECT
+    "org_mozilla_fennec_aurora" AS app_id,
+    submission_date,
+    client_id,
+    sample_id,
+    `moz-fx-data-shared-prod.udf.mode_last`(
+      ARRAY_AGG(STRUCT(city, subdivision1, subdivision2, country) ORDER BY submission_timestamp)
+    ) AS geo
+  FROM
+    with_date_offsets_org_mozilla_fennec_aurora
+  LEFT JOIN
+    overactive_org_mozilla_fennec_aurora
+    USING (submission_date, client_id)
+  WHERE
+    overactive_org_mozilla_fennec_aurora.client_id IS NULL
+    -- `mode_last` can result in struct with all null values if it’s most frequent (or latest among ties).
+    -- This exclude structs with all null values so there will always be one non-NULL field.
+    AND COALESCE(city, subdivision1, subdivision2, country) IS NOT NULL
+    {% if is_init() %}
+      AND submission_date >= '2018-01-01'
+    {% else %}
+      AND submission_date = @submission_date
+    {% endif %}
+  GROUP BY
+    submission_date,
+    client_id,
+    sample_id
+),
+{% if is_init() %}
+  clients_city_first_seen_org_mozilla_fennec_aurora AS (
+    SELECT
+      app_id,
+      client_id,
+      sample_id,
+      submission_date AS first_seen_city_date,
+      geo.city AS first_seen_city,
+      geo.subdivision1 AS first_seen_subdivision1,
+      geo.subdivision2 AS first_seen_subdivision2,
+      geo.country AS first_seen_country
+    FROM
+      clients_daily_org_mozilla_fennec_aurora
+    WHERE
+      geo.city IS NOT NULL
+    QUALIFY
+      ROW_NUMBER() OVER (PARTITION BY client_id, sample_id ORDER BY submission_date) = 1
+  ),
+  clients_city_last_seen_org_mozilla_fennec_aurora AS (
+    SELECT
+      app_id,
+      client_id,
+      sample_id,
+      submission_date AS last_seen_city_date,
+      geo.city AS last_seen_city,
+      geo.subdivision1 AS last_seen_subdivision1,
+      geo.subdivision2 AS last_seen_subdivision2,
+      geo.country AS last_seen_country
+    FROM
+      clients_daily_org_mozilla_fennec_aurora
+    WHERE
+      geo.city IS NOT NULL
+    QUALIFY
+      ROW_NUMBER() OVER (PARTITION BY client_id, sample_id ORDER BY submission_date DESC) = 1
+  )
+{% else %}
+  _previous_org_mozilla_fennec_aurora AS (
+    SELECT
+      *
+    FROM
+      `moz-fx-data-shared-prod.fenix_derived.baseline_clients_city_seen_v1`
+    WHERE
+      app_id = "org_mozilla_fennec_aurora"
+  ),
+  _current_org_mozilla_fennec_aurora AS (
+    SELECT
+      app_id,
+      client_id,
+      sample_id,
+      submission_date AS first_seen_city_date,
+      geo.city AS first_seen_city,
+      geo.subdivision1 AS first_seen_subdivision1,
+      geo.subdivision2 AS first_seen_subdivision2,
+      geo.country AS first_seen_country,
+      submission_date AS last_seen_city_date,
+      geo.city AS last_seen_city,
+      geo.subdivision1 AS last_seen_subdivision1,
+      geo.subdivision2 AS last_seen_subdivision2,
+      geo.country AS last_seen_country
+    FROM
+      clients_daily_org_mozilla_fennec_aurora
+    WHERE
+      geo.city IS NOT NULL
+  )
+{% endif %}
+{% if is_init() %}
+  SELECT
+    app_id,
+    client_id,
+    sample_id,
+    first_seen_city_date,
+    first_seen_city,
+    first_seen_subdivision1,
+    first_seen_subdivision2,
+    first_seen_country,
+    last_seen_city_date,
+    last_seen_city,
+    last_seen_subdivision1,
+    last_seen_subdivision2,
+    last_seen_country
+  FROM
+    clients_city_first_seen_org_mozilla_firefox cfs
+  FULL OUTER JOIN
+    clients_city_last_seen_org_mozilla_firefox cls
+    USING (client_id, sample_id, app_id)
+  UNION ALL
+  SELECT
+    app_id,
+    client_id,
+    sample_id,
+    first_seen_city_date,
+    first_seen_city,
+    first_seen_subdivision1,
+    first_seen_subdivision2,
+    first_seen_country,
+    last_seen_city_date,
+    last_seen_city,
+    last_seen_subdivision1,
+    last_seen_subdivision2,
+    last_seen_country
+  FROM
+    clients_city_first_seen_org_mozilla_firefox_beta cfs
+  FULL OUTER JOIN
+    clients_city_last_seen_org_mozilla_firefox_beta cls
+    USING (client_id, sample_id, app_id)
+  UNION ALL
+  SELECT
+    app_id,
+    client_id,
+    sample_id,
+    first_seen_city_date,
+    first_seen_city,
+    first_seen_subdivision1,
+    first_seen_subdivision2,
+    first_seen_country,
+    last_seen_city_date,
+    last_seen_city,
+    last_seen_subdivision1,
+    last_seen_subdivision2,
+    last_seen_country
+  FROM
+    clients_city_first_seen_org_mozilla_fenix cfs
+  FULL OUTER JOIN
+    clients_city_last_seen_org_mozilla_fenix cls
+    USING (client_id, sample_id, app_id)
+  UNION ALL
+  SELECT
+    app_id,
+    client_id,
+    sample_id,
+    first_seen_city_date,
+    first_seen_city,
+    first_seen_subdivision1,
+    first_seen_subdivision2,
+    first_seen_country,
+    last_seen_city_date,
+    last_seen_city,
+    last_seen_subdivision1,
+    last_seen_subdivision2,
+    last_seen_country
+  FROM
+    clients_city_first_seen_org_mozilla_fenix_nightly cfs
+  FULL OUTER JOIN
+    clients_city_last_seen_org_mozilla_fenix_nightly cls
+    USING (client_id, sample_id, app_id)
+  UNION ALL
+  SELECT
+    app_id,
+    client_id,
+    sample_id,
+    first_seen_city_date,
+    first_seen_city,
+    first_seen_subdivision1,
+    first_seen_subdivision2,
+    first_seen_country,
+    last_seen_city_date,
+    last_seen_city,
+    last_seen_subdivision1,
+    last_seen_subdivision2,
+    last_seen_country
+  FROM
+    clients_city_first_seen_org_mozilla_fennec_aurora cfs
+  FULL OUTER JOIN
+    clients_city_last_seen_org_mozilla_fennec_aurora cls
+    USING (client_id, sample_id, app_id)
+{% else %}
+  SELECT
+-- _p.* fields are NULL for clients that are not yet captured in the baseline_city_seen derived table.
+    IF(_p.app_id IS NULL, _c.app_id, _p.app_id) AS app_id,
+    IF(_p.client_id IS NULL, _c.client_id, _p.client_id) AS client_id,
+    IF(_p.sample_id IS NULL, _c.sample_id, _p.sample_id) AS sample_id,
+    IF(
+      _p.client_id IS NULL,
+      _c.first_seen_city_date,
+      _p.first_seen_city_date
+    ) AS first_seen_city_date,
+    IF(_p.client_id IS NULL, _c.first_seen_city, _p.first_seen_city) AS first_seen_city,
+    IF(
+      _p.client_id IS NULL,
+      _c.first_seen_subdivision1,
+      _p.first_seen_subdivision1
+    ) AS first_seen_subdivision1,
+    IF(
+      _p.client_id IS NULL,
+      _c.first_seen_subdivision2,
+      _p.first_seen_subdivision2
+    ) AS first_seen_subdivision2,
+    IF(_p.client_id IS NULL, _c.first_seen_country, _p.first_seen_country) AS first_seen_country,
+    IF(
+      _p.client_id IS NULL
+      OR _p.last_seen_city_date < _c.last_seen_city_date,
+      _c.last_seen_city_date,
+      _p.last_seen_city_date
+    ) AS last_seen_city_date,
+    IF(
+      _p.client_id IS NULL
+      OR _p.last_seen_city_date < _c.last_seen_city_date,
+      _c.last_seen_city,
+      _p.last_seen_city
+    ) AS last_seen_city,
+    IF(
+      _p.client_id IS NULL
+      OR _p.last_seen_city_date < _c.last_seen_city_date,
+      _c.last_seen_subdivision1,
+      _p.last_seen_subdivision1
+    ) AS last_seen_subdivision1,
+    IF(
+      _p.client_id IS NULL
+      OR _p.last_seen_city_date < _c.last_seen_city_date,
+      _c.last_seen_subdivision2,
+      _p.last_seen_subdivision2
+    ) AS last_seen_subdivision2,
+    IF(
+      _p.client_id IS NULL
+      OR _p.last_seen_city_date < _c.last_seen_city_date,
+      _c.last_seen_country,
+      _p.last_seen_country
+    ) AS last_seen_country
+  FROM
+    _current_org_mozilla_firefox AS _c
+  FULL JOIN
+    _previous_org_mozilla_firefox AS _p
+    USING (client_id, sample_id, app_id)
+  UNION ALL
+  SELECT
+-- _p.* fields are NULL for clients that are not yet captured in the baseline_city_seen derived table.
+    IF(_p.app_id IS NULL, _c.app_id, _p.app_id) AS app_id,
+    IF(_p.client_id IS NULL, _c.client_id, _p.client_id) AS client_id,
+    IF(_p.sample_id IS NULL, _c.sample_id, _p.sample_id) AS sample_id,
+    IF(
+      _p.client_id IS NULL,
+      _c.first_seen_city_date,
+      _p.first_seen_city_date
+    ) AS first_seen_city_date,
+    IF(_p.client_id IS NULL, _c.first_seen_city, _p.first_seen_city) AS first_seen_city,
+    IF(
+      _p.client_id IS NULL,
+      _c.first_seen_subdivision1,
+      _p.first_seen_subdivision1
+    ) AS first_seen_subdivision1,
+    IF(
+      _p.client_id IS NULL,
+      _c.first_seen_subdivision2,
+      _p.first_seen_subdivision2
+    ) AS first_seen_subdivision2,
+    IF(_p.client_id IS NULL, _c.first_seen_country, _p.first_seen_country) AS first_seen_country,
+    IF(
+      _p.client_id IS NULL
+      OR _p.last_seen_city_date < _c.last_seen_city_date,
+      _c.last_seen_city_date,
+      _p.last_seen_city_date
+    ) AS last_seen_city_date,
+    IF(
+      _p.client_id IS NULL
+      OR _p.last_seen_city_date < _c.last_seen_city_date,
+      _c.last_seen_city,
+      _p.last_seen_city
+    ) AS last_seen_city,
+    IF(
+      _p.client_id IS NULL
+      OR _p.last_seen_city_date < _c.last_seen_city_date,
+      _c.last_seen_subdivision1,
+      _p.last_seen_subdivision1
+    ) AS last_seen_subdivision1,
+    IF(
+      _p.client_id IS NULL
+      OR _p.last_seen_city_date < _c.last_seen_city_date,
+      _c.last_seen_subdivision2,
+      _p.last_seen_subdivision2
+    ) AS last_seen_subdivision2,
+    IF(
+      _p.client_id IS NULL
+      OR _p.last_seen_city_date < _c.last_seen_city_date,
+      _c.last_seen_country,
+      _p.last_seen_country
+    ) AS last_seen_country
+  FROM
+    _current_org_mozilla_firefox_beta AS _c
+  FULL JOIN
+    _previous_org_mozilla_firefox_beta AS _p
+    USING (client_id, sample_id, app_id)
+  UNION ALL
+  SELECT
+-- _p.* fields are NULL for clients that are not yet captured in the baseline_city_seen derived table.
+    IF(_p.app_id IS NULL, _c.app_i

⚠️ Only part of the diff is displayed.

Link to full diff

[wwyc]

@BenWu @soGaussian could you do a final review/approval on this PR? I’d like to merge early next week.

@BenWu is there a time of day you recommend for the merge so initialization doesn’t affect artifact deployments? Thanks!

[BenWu]

I'll take a closer look on Monday.

Running it towards the end of the day should be fine so it doesn't block any fixes that need to go out. I didn't see your comment on how long it would take but it should be fine to do the initialization. I don't think running in parallel would make it much faster since the query is large enough to use up most of the slots. I just did a test and a 1% query uses up all 2k slots in the backfill reservation for basically the whole duration of the query.

One thing that we should do is make the initialization run in the backfill reservation instead of analysis-and-etl so there isn't slot contention with etl. I can make that change on Monday.

Also Kiran is planning on doing a large backfill of events_first_seen next week so make sure to coordinate with her so the backfills don't end up fighting for slots

[wwyc]

I'll take a closer look on Monday.

Running it towards the end of the day should be fine so it doesn't block any fixes that need to go out. I didn't see your comment on how long it would take but it should be fine to do the initialization. I don't think running in parallel would make it much faster since the query is large enough to use up most of the slots. I just did a test and a 1% query uses up all 2k slots in the backfill reservation for basically the whole duration of the query.

One thing that we should do is make the initialization run in the backfill reservation instead of analysis-and-etl so there isn't slot contention with etl. I can make that change on Monday.

Also Kiran is planning on doing a large backfill of events_first_seen next week so make sure to coordinate with her so the backfills don't end up fighting for slots

Thank you! I will check with kiran. The time it takes to run 2% sample: firefox_desktop: ~319s, fenix: ~111s.