Skip to content

restore: keep existing UDF, no DROP FUNCTION, restore functions before tables - #1657

Merged
Slach merged 2 commits into
masterfrom
restore_functions_1636
Oct 30, 2026
Merged

Slach merged 2 commits into
masterfrom
restore_functions_1636

Conversation

@Slach

@Slach Slach commented Oct 30, 2026

Copy link
Copy Markdown
Collaborator

Fix #1636

  • remove explicit DROP FUNCTION IF EXISTS before create, no more window when function doesn't exist
  • read system.functions before restore, for each function from backup:
    • missing -> CREATE FUNCTION IF NOT EXISTS [ON CLUSTER]
    • identical (after CREATE -> CREATE OR REPLACE normalization) -> skip without any query, so CREATE FUNCTION / DROP FUNCTION grants not required
    • different -> keep existing with warning, --rm overwrites it with atomic CREATE OR REPLACE FUNCTION, same as tables and RBAC
  • functions restored in restorePrologue before RestoreSchema, materialized views and tables which use UDF can be created on clean server (applies to restore, restore_remote and restore_remote --streaming)
  • restore --dry-run / restore_remote --dry-run show functions map with create, replace, keep_existing, skip_identical

Tests

  • unit TestPlanFunctionsRestore
  • integration TestRestoreFunctionsKeepExisting: MV uses UDF, function changed and another dropped after backup, check restore_remote --dry-run, restore without --rm keeps changed function and creates missing one before MV, --rm replaces, restore by user without CREATE FUNCTION, DROP FUNCTION grants works when functions are identical

Known limitation: comparison uses system.functions of local node only, with restore_schema_on_cluster and without user_defined_zookeeper_path function identical locally but missing on other node is not created there

🤖 Generated with Claude Code

Slach and others added 2 commits October 30, 2026 13:28
restore / restore_remote ran DROP FUNCTION IF EXISTS + CREATE OR REPLACE
FUNCTION for every function from backup, even with --tables for one
table, function was unavailable between DROP and CREATE and definitions
changed after backup were silently rolled back.

Now existing functions are read from system.functions first: missing
ones are created with CREATE FUNCTION IF NOT EXISTS, identical ones are
skipped without any query (no CREATE/DROP FUNCTION grants needed),
changed ones are kept with warning and replaced by atomic CREATE OR
REPLACE only with --rm, same as tables and RBAC. Functions are restored
before tables, so materialized views and tables which use UDF can be
created on a clean server. --dry-run shows the plan per function.

Fix #1636

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Download closes clickhouse connection on return, so functions plan in
restore_remote --dry-run failed with "connection is closed"; fix
integration test check of materialized view rows.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coveralls

Copy link
Copy Markdown

Coverage Report for CI Build 41036149125

Coverage increased (+0.05%) to 70.533%

Details

  • Coverage increased (+0.05%) from the base build.
  • Patch coverage: 21 uncovered changes across 4 files (79 of 100 lines covered, 79.0%).
  • 50 coverage regressions across 10 files.

Uncovered Changes

File Changed Covered %
pkg/backup/restore_functions.go 62 54 87.1%
pkg/backup/restore_remote.go 16 9 56.25%
pkg/backup/dry_run_restore.go 14 10 71.43%
pkg/backup/restore.go 4 2 50.0%
Total (6 files) 100 79 79.0%

Coverage Regressions

50 previously-covered lines in 10 files lost coverage.

File Lines Losing Coverage Coverage
pkg/storage/gcs.go 15 54.94%
pkg/backup/upload.go 9 73.85%
pkg/backup/watch_schedule.go 8 75.11%
pkg/backup/delete.go 4 74.2%
pkg/clickhouse/clickhouse.go 3 81.29%
pkg/storage/general.go 3 70.52%
pkg/backup/list.go 2 70.26%
pkg/storage/structs.go 2 81.82%
pkg/backup/create_remote.go 2 66.02%
pkg/backup/backuper.go 2 79.57%

Coverage Stats

Coverage Status
Relevant Lines: 29793
Covered Lines: 21014
Line Coverage: 70.53%
Coverage Strength: 34415.39 hits per line

💛 - Coveralls

@Slach
Slach merged commit 651a9b2 into master Oct 30, 2026
30 checks passed
@Slach Slach added this to the 2.8.3 milestone Oct 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

restore / restore_remote always drops and recreates all user-defined SQL functions, even when restoring a single table

2 participants