Skip to content

feat(enable-banking): safe pending transaction merge with sync re-import prevention#1709

Merged
jjmata merged 9 commits intowe-promise:mainfrom
CrossDrain:eb-merge-pending-transactions
May 9, 2026
Merged

feat(enable-banking): safe pending transaction merge with sync re-import prevention#1709
jjmata merged 9 commits intowe-promise:mainfrom
CrossDrain:eb-merge-pending-transactions

Conversation

@CrossDrain
Copy link
Copy Markdown
Contributor

@CrossDrain CrossDrain commented May 8, 2026

Overview

Enable Banking syncs pending transactions before they are settled by the bank. Once the bank posts the settled version, users are left with two entries for the same transaction. This PR gives users the ability to manually merge a pending entry into its posted counterpart and ensures subsequent syncs never re-import the already-resolved pending entry. It also corrects a systemic gap where enable_banking was silently excluded from every provider-aware pending scope in Entry, making stale-entry cleanup and duplicate-suggestion detection inoperative for this provider.


The Problem

Context:

Enable Banking accounts accumulate duplicate entry pairs — one pending, one posted. The merge UI (added in #1088) exists, but the backend it calls was a thin placeholder that was neither atomic nor durable. Stale and re-imported pending entries continued to bloat the transaction list and corrupt balance calculations even after a user performed a merge.

Root Cause / Requirement:

  1. The existing merge action called a placeholder entry.destroy! — non-atomic, no locking, no sync suppression. The next sync immediately re-imported the destroyed pending entry, undoing the user's action.
  2. Entry.pending, Entry.excluding_pending, and reconcile_pending_duplicates hardcoded only simplefin, plaid, and lunchflowenable_banking entries were invisible to stale cleanup and the fuzzy-match suggestion engine.
  3. The external_id construction formula was duplicated across two processors, creating a silent divergence risk.

Linked Issues:


Key Changes

Features

  • app/models/transaction.rbmerge_with_duplicate!

    • Complete rewrite: wraps the full merge in ApplicationRecord.transaction(requires_new: true) with row-level SELECT FOR UPDATE locks on both entries
    • Cross-account guard: rejects merges where posted_entry.account_id != pending_entry.account_id
    • Writes durable "manual_merge" metadata onto posted_tx.extra (keys: merged_from_entry_id, merged_from_external_id, merged_at, source) so the sync engine can detect and skip re-import on subsequent runs
    • Attribute inheritance: date and category are propagated from pending → posted when the posted entry is not user_modified; name and merchant are intentionally preserved from the canonical posted record
    • Calls posted_entry.mark_user_modified! after merge to protect the posted entry from future sync overwrites
  • app/models/transaction.rb — new model API backing the pre-existing merge UI

    • has_potential_duplicate? — returns true when a non-dismissed potential_posted_match is present
    • potential_duplicate_entry — resolves the suggested posted Entry from match metadata
    • potential_duplicate_confidence — returns match confidence ("low" / "medium" / "high")
    • potential_duplicate_posted_amount — exposes the posted amount for UI comparison
    • low_confidence_duplicate? — convenience predicate
    • dismiss_duplicate_suggestion! — marks dismissed: true on the stored suggestion, blocking further merge attempts
    • clear_duplicate_suggestion! — removes the potential_posted_match key entirely
    • pending_duplicate_candidates(limit:, offset:) — returns paginated posted entries from the same account and currency eligible for manual merge
  • app/models/enable_banking_account/transactions/processor.rb

    • Pre-fetches all merged_from_external_id values into an in-memory Set (one query per sync, O(1) per transaction lookup) before the main processing loop
    • Skips re-import for any transaction whose external_id appears in the merged set; records the skip under a dedicated skipped_count counter
    • Adds :skipped key to the result hash so callers can distinguish new imports from no-op skips

Refactors

  • app/models/enable_banking_entry/processor.rb

    • Extracted self.compute_external_id(raw_transaction_data) as a public class method — single authoritative source for the "enable_banking_#{id}" formula
    • Private instance external_id delegates to the class method; callers outside the class use the class method directly, eliminating duplication
  • app/models/entry.rbscope :pending, scope :excluding_pending

    • Replaced hardcoded 3-provider SQL strings with dynamic generation from Transaction::PENDING_PROVIDERS, making enable_banking (and any future provider) automatically included
  • app/models/entry.rbreconcile_pending_duplicates

    • Extracted not_pending_sql variable built from Transaction::PENDING_PROVIDERS; replaces two identical hardcoded WHERE blocks for exact-match and fuzzy-match candidate queries
    • Stored potential_posted_match hash now includes "confidence": "medium" and "dismissed": false for structural consistency with the merge reader
  • app/models/transaction.rbmerge_with_duplicate!

    • Consolidated two sequential UPDATE statements on posted_tx (metadata write + category write) into a single update!(tx_attrs) call, halving after_save callback fires and DB round-trips
    • Removed redundant || {} guard on posted_tx.extra (transactions.extra is null: false, default: {})

Bug Fixes

  • app/models/transaction.rbmerge_with_duplicate!

    • Added return false unless pending? as first guard — prevents non-pending transactions with stale potential_posted_match data from entering the merge flow
    • Both pending_entry.lock! and posted_entry.lock! are now individually guarded: pending_entry gone → idempotent return true; posted_entry gone → raise ActiveRecord::Rollback → returns false (previously: unhandled 500)
    • merge_succeeded flag replaces the hardcoded true return so rollback paths correctly return false
  • app/models/transaction.rbpending?

    • Narrowed bare rescue to rescue StandardError to avoid swallowing SignalException, Interrupt, and NoMemoryError
  • app/controllers/transactions_controller.rbmerge_duplicate

    • Expanded rescue to include ActiveRecord::RecordNotFound, ActiveRecord::Deadlocked, ActiveRecord::LockWaitTimeout in addition to pre-existing RecordInvalid / RecordNotDestroyed
  • app/controllers/pending_duplicate_merges_controller.rbcreate

    • Added first-ever rescue clause for RecordInvalid, RecordNotDestroyed, Deadlocked, LockWaitTimeout — previously any database exception from merge_with_duplicate! produced an unhandled 500

Expected Impact

  • Merge is now durable — syncs no longer re-import a pending entry that was manually merged
  • enable_banking pending entries are fully visible — stale cleanup and fuzzy-match suggestions now work for this provider
  • Concurrent merges are safe — row-level locks replace the previous unguarded destroy!; both race paths return gracefully instead of 500ing
  • Sync query is indexed — the excluded_ids pre-fetch uses a partial functional index instead of a sequential scan; per-merge DB writes reduced from 2 → 1

Technical Implementation

Architectural Decisions

Savepoint-based atomic merge (requires_new: true)

merge_with_duplicate! wraps all writes inside ApplicationRecord.transaction(requires_new: true), creating a PostgreSQL savepoint. If the outer request is already inside a transaction (e.g., a controller callback), the savepoint rolls back independently without aborting the outer transaction. This is critical for correctness in the Deadlocked / LockWaitTimeout paths.

Consistent lock ordering to prevent deadlocks

Entries are always locked in the same order: pending_entry first, posted_entry second. Any concurrent merge of the same pair will block on the first lock and re-evaluate state once released, rather than deadlocking. This is the standard two-phase locking protocol applied to polymorphic AR records.

Durable merge marker as the sync skip signal

Rather than relying on the pending entry's absence (which is fragile — a bug or re-sync could recreate it), the merge writes merged_from_external_id onto the surviving posted transaction's extra JSONB field. The Processor reads this at sync time and skips re-import. This is a write-once audit trail, not a soft-delete flag.

compute_external_id as single source of truth

The ID formula (transaction_id || entry_reference"enable_banking_#{id}") must be identical in both the sync skip check and the entry processor. Extracting it to EnableBankingEntry::Processor.compute_external_id eliminates the class of bug where one site is updated and the other isn't — silent double-imports being the failure mode.

Dynamic PENDING_PROVIDERS SQL generation

Entry.pending previously hardcoded provider names. Any new provider added to Transaction::PENDING_PROVIDERS now propagates automatically to all scopes — zero-touch extensibility for future integrations.

Data Flow (Merge Path)

User clicks "Merge" (TransactionsController or PendingDuplicateMergesController)
  → Authorization check (require_account_permission!)
  → transaction.merge_with_duplicate!
      → Guard: pending? && has_potential_duplicate? && same account
      → BEGIN SAVEPOINT
          → LOCK pending_entry (SELECT … FOR UPDATE)    # idempotent on RecordNotFound
          → LOCK posted_entry  (SELECT … FOR UPDATE)    # rollback on RecordNotFound
          → Accumulate tx_attrs: { extra: merge_metadata [, category_id] }
          → posted_tx.update!(tx_attrs)                 # 1 round-trip
          → posted_entry.update!(date:) if applicable   # 1 round-trip (Entry row)
          → posted_entry.mark_user_modified!
          → pending_entry.destroy!                      # cascades → Transaction destroyed
          → merge_succeeded = true
      → RELEASE SAVEPOINT
  → Controller: flash + redirect
Next sync (EnableBankingAccount::Transactions::Processor)
  → excluded_ids = Transaction.where(account).pluck(manual_merge ext_id).to_set  # 1 query
  → for each raw transaction:
      → ext_id = EnableBankingEntry::Processor.compute_external_id(raw_data)
      → if excluded_ids.include?(ext_id) → skip (skipped_count++)
      → else → EnableBankingEntry::Processor.new(…).process
  → return { imported:, skipped:, failed:, … }

Testing & Validation

Test Coverage

  • Unit Teststest/models/transaction/merge_with_duplicate_test.rb33 cases

    • Happy path: entry destroyed, Transaction cascade confirmed (Transaction.count diff), metadata written, date/category inherited, name preserved, user_modified set
    • Guard paths: no duplicate, posted entry missing, cross-account rejection, non-pending caller, dismissed suggestion
    • Concurrency: pending_entry already destroyed (idempotent true); posted_entry deleted between check and lock (returns false, no entries destroyed)
    • user_modified protection: date and category not overwritten when posted entry is already protected; metadata still written
    • No external_id path: metadata block skipped cleanly
    • dismiss_duplicate_suggestion!: dismissed flag set, has_potential_duplicate? returns false, merge blocked
    • clear_duplicate_suggestion!: key removed, returns false when already absent
    • pending_duplicate_candidates: same-account posted entries returned; self excluded; other-account entries excluded; returns empty when not pending
  • Unit Teststest/models/enable_banking_account/transactions/processor_test.rb5 cases

    • Already-merged pending transaction not re-imported
    • New (unmerged) pending transaction is imported normally
    • Mixed batch: excluded + new in same payload → only new entry created
    • Empty payload → graceful { success: true, total: 0, … }
    • Stats: skipped: 1, imported: 0, failed: 0 for a skipped-only batch
  • Integration — Full suite (bin/rails test, 3,471 tests) — all pre-existing failures confirmed pre-existing and in unrelated areas (subscriptions, market data, AI chat)

Test Results / Logs

Running 51 tests in parallel using 8 processes
Run options: --seed 19539

...................................................

Finished in 2.793730s, 18.2552 runs/s, 34.7206 assertions/s.
51 runs, 97 assertions, 0 failures, 0 errors, 0 skips

Rubocop: 9 files inspected, no offenses detected

Summary by CodeRabbit

  • Bug Fixes

    • Enhanced error handling for duplicate transaction merge operations with user-friendly failure alerts.
  • New Features

    • Prevents re-importing transactions that were already manually merged into existing records.
    • Improved duplicate transaction detection and merging with better metadata preservation and concurrent operation safety.

Review Change Stack

@brin-security-scanner brin-security-scanner Bot added the contributor:verified Contributor passed trust analysis. label May 8, 2026
@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 8, 2026

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 355a0930-ec2a-477f-b519-7b1ccbff4ebd

📥 Commits

Reviewing files that changed from the base of the PR and between 0b11883 and 748e36c.

📒 Files selected for processing (2)
  • app/models/enable_banking_account/transactions/processor.rb
  • db/schema.rb
🚧 Files skipped from review as they are similar to previous changes (1)
  • app/models/enable_banking_account/transactions/processor.rb

📝 Walkthrough

Walkthrough

The PR adds a locked atomic merge flow for pending duplicates with metadata tracking, prevents re-importing manually-merged transactions by external_id, centralizes Enable Banking external_id computation, makes pending detection provider-agnostic via dynamic SQL, adds controller exception handling, and provides comprehensive test coverage for processor and merge behavior.

Changes

Pending Duplicate Merge with Re-import Prevention

Layer / File(s) Summary
Data Schema Version Update
db/schema.rb
Schema version bumped to 2026_05_08_120000; existing indexes and constraints reordered in securities, sophtron_accounts, and webauthn_credentials tables.
External ID Computation
app/models/enable_banking_entry/processor.rb
New public compute_external_id(raw_transaction_data) class method derives external_id from transaction_id or entry_reference. Process method caches a non-raising safe_id for logging; private external_id delegates to helper; log_invalid_currency uses computed safe_id.
Provider-Agnostic Pending Detection
app/models/transaction.rb, app/models/entry.rb
Transaction::PENDING_CHECK_SQL constant added for reusable SQL fragment. Transaction#pending? rescues StandardError on casting. Entry.pending, excluding_pending, and reconcile_pending_duplicates build SQL dynamically from PENDING_PROVIDERS instead of hardcoded provider names.
Atomic Merge with Locking
app/models/transaction.rb
merge_with_duplicate! rewritten with nested requires_new transaction, entry locking, account_id validation, manual_merge metadata merge (array + legacy hash handling), conditional date/category inheritance, and idempotent pending entry destruction.
Skip Previously-Merged Transactions
app/models/enable_banking_account/transactions/processor.rb
Pre-fetches merged_from_external_id values from existing transactions' manual_merge metadata. Skips importing raw pending transactions whose external_id matches; logs skip events and returns skipped_count in result hash alongside existing counters.
Controller Error Handling
app/controllers/pending_duplicate_merges_controller.rb, app/controllers/transactions_controller.rb
Both controllers now rescue ActiveRecord::RecordInvalid, RecordNotDestroyed, RecordNotFound, Deadlocked, and LockWaitTimeout exceptions, log the error, and redirect to transactions_path with a localized failure alert.
Tests: EnableBanking Processor
test/models/enable_banking_account/transactions/processor_test.rb
New test suite covering skip behavior (matched external_id not re-imported), batch imports (mixed matched/unmatched), multi-merge arrays, nil payloads, and counter reporting. Verifies skipped and imported counts reflect exclusion logic.
Tests: Transaction Merge Behavior
test/models/transaction/merge_with_duplicate_test.rb
Comprehensive test suite covering metadata persistence (manual_merge fields), date/category inheritance rules, array/legacy hash migration, idempotency under concurrency, suggestion lifecycle (dismiss/clear), pending_duplicate_candidates filtering, and account_id validation.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related issues

  • we-promise/sure#954: The main changes add a compute_external_id that maps Enable Banking's entry_reference (fallback from transaction_id) to external_id and skip-import logic based on those external IDs, directly addressing the Enable Banking missing entry_reference→external_id mapping that caused duplicates.

Possibly related PRs

  • we-promise/sure#731: Main PR's refactor to generalize pending-provider checks (PENDING_PROVIDERS, PENDING_CHECK_SQL, and Transaction#pending?) directly overlaps and builds on the same code area changed in the retrieved Lunchflow PR (which added lunchflow pending checks), so the changes are related.
  • we-promise/sure#1088: Both PRs touch the same pending-duplicate merge feature—modifying PendingDuplicateMergesController (create flow/error handling) and Transaction merge/pending logic (PENDING_PROVIDERS, pending?/merge_with_duplicate!), so they are related.
  • we-promise/sure#988: Both PRs address Enable Banking deduplication (processor/importer external_id and skipping duplicate transactions) and touch the EnableBanking processor flow and external_id handling, so they are related.

Suggested reviewers

  • jjmata
  • michelroegl-brunner

Poem

🐰 A locked merge dance in transactions deep,
External IDs now a promise we keep,
No duplicates born from once-merged folk,
Provider-agnostic checks—no hardcoded stroke,
Atomic and safe, with tests to attest,
The pending duplicates put to rest!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 12.50% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title accurately summarizes the main changes: implementing safe pending transaction merge with re-import prevention for Enable Banking, which are the core objectives of this PR.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f6f702e924

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread app/models/transaction.rb Outdated
@CrossDrain CrossDrain force-pushed the eb-merge-pending-transactions branch from f6f702e to cf4d2f0 Compare May 8, 2026 20:19
@brin-security-scanner brin-security-scanner Bot added pr:verified PR passed security analysis. and removed pr:verified PR passed security analysis. labels May 8, 2026
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
app/models/enable_banking_account/transactions/processor.rb (1)

9-11: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Keep the processor result shape consistent on the empty-payload path.

This early return omits :skipped, while the normal path always includes it. Any caller consuming the new API now has to special-case the zero-transaction case. Return skipped: 0 here too.

Suggested fix
-      return { success: true, total: 0, imported: 0, failed: 0, errors: [] }
+      return { success: true, total: 0, imported: 0, skipped: 0, failed: 0, errors: [] }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@app/models/enable_banking_account/transactions/processor.rb` around lines 9 -
11, The early return in EnableBankingAccount::Transactions::Processor inside the
unless enable_banking_account.raw_transactions_payload.present? branch returns a
result hash missing the :skipped key; update that return to include skipped: 0
so the processor result shape matches the normal path (i.e., return { success:
true, total: 0, imported: 0, failed: 0, skipped: 0, errors: [] }).
app/controllers/pending_duplicate_merges_controller.rb (1)

22-58: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Localize the new flash messages.

These new notices/alerts are hard-coded English strings, so they bypass the repo's i18n convention and won't be translatable. Please move them to locale keys and use t(...) here.

As per coding guidelines **/*.{erb,rb}: Always use t() helper for user-facing strings.

Also applies to: 66-68

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@app/controllers/pending_duplicate_merges_controller.rb` around lines 22 - 58,
Replace the hard-coded English flash strings in the manual merge flow with i18n
keys and use the t(...) helper: update the redirect_back_or_to calls that pass
"Please select a posted transaction to merge with", "Invalid transaction
selected for merge", "Pending transaction merged with posted transaction", and
"Could not merge transactions" to use t(...) (e.g.
t("transactions.merge_duplicate.select_posted"),
t("transactions.merge_duplicate.invalid_selection"),
t("transactions.merge_duplicate.success"),
t("transactions.merge_duplicate.failure_manual") respectively), keep the control
flow around merge_params, find_eligible_posted_entry, `@transaction.update`!(...),
and `@transaction.merge_with_duplicate`! unchanged, and add corresponding locale
entries in the locale yml so the strings are translatable (also update the
similar messages mentioned at lines 66-68 to use t()).
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@app/models/enable_banking_entry/processor.rb`:
- Around line 13-17: compute_external_id currently raises when both
transaction_id and entry_reference are missing, and the process method re-logs
external_id inside rescue blocks causing exceptions during error handling;
change compute_external_id (or add a new safe_external_id helper) to return a
non-raising diagnostic string (e.g. "enable_banking_unknown" or "unknown")
instead of raising/nil, and update process to cache that safe identifier at the
start (use the cached safe id in all rescue/logging paths) so logging never
calls the potentially-raising external_id method; update references mentioned
around compute_external_id and process (and the rescue lines ~72-75) to use the
cached safe id.

In `@app/models/transaction.rb`:
- Around line 200-249: The merge currently only locks entries but then
reads/modifies the associated Transaction (posted_tx) which can be concurrently
updated; after assigning posted_tx = posted_entry.entryable (and before reading
posted_tx.extra or changing category_id), acquire a row lock on the transaction
(e.g. call posted_tx.lock! and similarly lock pending_transaction if you read
its DB state) and handle ActiveRecord::RecordNotFound the same way you did for
entries; this ensures the posted_tx row is DB-fresh and prevents lost updates
before building tx_attrs and calling posted_tx.update!.

In `@db/migrate/20260508120000_add_manual_merge_ext_id_index.rb`:
- Around line 2-6: The migration currently uses add_index on the transactions
table for Arel.sql("(extra->'manual_merge'->>'merged_from_external_id')") with
name "idx_transactions_manual_merge_ext_id" and a where clause, which can block
writes; update the migration to run the index build concurrently by adding
disable_ddl_transaction! at the top of the migration and passing algorithm:
:concurrently to add_index (keep the same Arel.sql expression, name and where
clause), so the final change method disables the DDL transaction and calls
add_index(..., algorithm: :concurrently, name:
"idx_transactions_manual_merge_ext_id", where: "(extra ? 'manual_merge')").

In `@test/models/transaction/merge_with_duplicate_test.rb`:
- Around line 190-202: The test uses `@posted_entry.dup` which creates a
non-persisted record (id: nil) and can trigger persisted?/id guards before lock!
is called; replace the dup-based ghost with an object that preserves the real
id/account_id and any other attributes read by merge_with_duplicate! (e.g., use
an OpenStruct or a mocha stub) and stub its lock! to raise
ActiveRecord::RecordNotFound so execution reaches the lock! rescue path; ensure
you stub the transaction.potential_duplicate_entry to return this constructed
ghost (referencing merge_with_duplicate!, potential_duplicate_entry, lock!, and
`@posted_entry`) and include any attributes the method inspects so the guard
checks pass.

---

Outside diff comments:
In `@app/controllers/pending_duplicate_merges_controller.rb`:
- Around line 22-58: Replace the hard-coded English flash strings in the manual
merge flow with i18n keys and use the t(...) helper: update the
redirect_back_or_to calls that pass "Please select a posted transaction to merge
with", "Invalid transaction selected for merge", "Pending transaction merged
with posted transaction", and "Could not merge transactions" to use t(...) (e.g.
t("transactions.merge_duplicate.select_posted"),
t("transactions.merge_duplicate.invalid_selection"),
t("transactions.merge_duplicate.success"),
t("transactions.merge_duplicate.failure_manual") respectively), keep the control
flow around merge_params, find_eligible_posted_entry, `@transaction.update`!(...),
and `@transaction.merge_with_duplicate`! unchanged, and add corresponding locale
entries in the locale yml so the strings are translatable (also update the
similar messages mentioned at lines 66-68 to use t()).

In `@app/models/enable_banking_account/transactions/processor.rb`:
- Around line 9-11: The early return in
EnableBankingAccount::Transactions::Processor inside the unless
enable_banking_account.raw_transactions_payload.present? branch returns a result
hash missing the :skipped key; update that return to include skipped: 0 so the
processor result shape matches the normal path (i.e., return { success: true,
total: 0, imported: 0, failed: 0, skipped: 0, errors: [] }).
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: b2da8667-090d-4a92-aa39-3cb598a85f1e

📥 Commits

Reviewing files that changed from the base of the PR and between 81cdccb and f6f702e.

📒 Files selected for processing (10)
  • app/controllers/pending_duplicate_merges_controller.rb
  • app/controllers/transactions_controller.rb
  • app/models/enable_banking_account/transactions/processor.rb
  • app/models/enable_banking_entry/processor.rb
  • app/models/entry.rb
  • app/models/transaction.rb
  • db/migrate/20260508120000_add_manual_merge_ext_id_index.rb
  • db/schema.rb
  • test/models/enable_banking_account/transactions/processor_test.rb
  • test/models/transaction/merge_with_duplicate_test.rb

Comment thread app/models/enable_banking_entry/processor.rb
Comment thread app/models/transaction.rb
Comment thread db/migrate/20260508120000_add_manual_merge_ext_id_index.rb Outdated
Comment thread test/models/transaction/merge_with_duplicate_test.rb
@brin-security-scanner brin-security-scanner Bot added the pr:verified PR passed security analysis. label May 8, 2026
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

♻️ Duplicate comments (1)
test/models/transaction/merge_with_duplicate_test.rb (1)

230-236: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

Use a persisted-like mock instead of dup for the lock-race test path.

Line 234 uses @posted_entry.dup, which can produce a non-persisted object (id=nil) and short-circuit guards before lock! is reached, so this test may not actually exercise the lock-failure rescue path.

Proposed fix
-    ghost = `@posted_entry.dup.tap` { |e| e.stubs(:lock!).raises(ActiveRecord::RecordNotFound) }
+    ghost = OpenStruct.new(
+      id: `@posted_entry.id`,
+      account_id: `@posted_entry.account_id`,
+      date: `@posted_entry.date`,
+      name: `@posted_entry.name`
+    )
+    ghost.stubs(:persisted?).returns(true)
+    ghost.stubs(:lock!).raises(ActiveRecord::RecordNotFound)
     `@pending_entry.transaction.stubs`(:potential_duplicate_entry).returns(ghost)
#!/bin/bash
# Verify this file no longer uses dup-based ghost objects in the concurrency test.
rg -nP --type=rb -C2 '@posted_entry\.dup\.tap|OpenStruct\.new' test/models/transaction/merge_with_duplicate_test.rb

As per coding guidelines, "Always prefer OpenStruct when creating mock instances, or in complex cases, a mock class."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/models/transaction/merge_with_duplicate_test.rb` around lines 230 - 236,
The test uses `@posted_entry.dup` to build ghost which can be non-persisted
(id=nil) and cause early exits instead of exercising the lock-failure path;
replace the dup-based ghost with a persisted-like mock (e.g., an OpenStruct or a
stubbed object with an id and any required attributes) that responds to lock!
and whose lock! raises ActiveRecord::RecordNotFound, then stub
`@pending_entry.transaction.potential_duplicate_entry` to return that mock so
merge_with_duplicate! goes through the same code path as a real persisted
posted_entry and triggers the rescue handling.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Duplicate comments:
In `@test/models/transaction/merge_with_duplicate_test.rb`:
- Around line 230-236: The test uses `@posted_entry.dup` to build ghost which can
be non-persisted (id=nil) and cause early exits instead of exercising the
lock-failure path; replace the dup-based ghost with a persisted-like mock (e.g.,
an OpenStruct or a stubbed object with an id and any required attributes) that
responds to lock! and whose lock! raises ActiveRecord::RecordNotFound, then stub
`@pending_entry.transaction.potential_duplicate_entry` to return that mock so
merge_with_duplicate! goes through the same code path as a real persisted
posted_entry and triggers the rescue handling.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 40b3520f-b24c-4983-b8e9-4274a1851a69

📥 Commits

Reviewing files that changed from the base of the PR and between cf4d2f0 and 6cf2426.

📒 Files selected for processing (4)
  • app/models/enable_banking_account/transactions/processor.rb
  • app/models/transaction.rb
  • test/models/enable_banking_account/transactions/processor_test.rb
  • test/models/transaction/merge_with_duplicate_test.rb
🚧 Files skipped from review as they are similar to previous changes (3)
  • test/models/enable_banking_account/transactions/processor_test.rb
  • app/models/enable_banking_account/transactions/processor.rb
  • app/models/transaction.rb

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
app/models/enable_banking_entry/processor.rb (1)

232-234: ⚠️ Potential issue | 🟠 Major | ⚡ Quick win

log_invalid_currency still calls the potentially-raising external_id — the fix in process was incomplete.

This is the same class of defect the PR's safe_id introduction was designed to eliminate. log_invalid_currency is called via parse_currencycurrency → inside process's begin/rescue. When both transaction_id and entry_reference are absent, the call to external_id on line 233 raises ArgumentError before the warning is logged, meaning the diagnostic message is silently swallowed and the raise propagates as an ArgumentError (which the outer rescue re-raises as a StandardError). The fix is the same pattern already used in process: use safe_id or compute a non-raising identifier inline.

Because safe_id is scoped to process, the cleanest fix is to move the safe fallback into a memoized private helper:

🐛 Proposed fix
+    def safe_external_id
+      self.class.compute_external_id(data) || "unknown"
+    end
+
     def log_invalid_currency(currency_value)
-      Rails.logger.warn("Invalid currency code '#{currency_value}' in Enable Banking transaction #{external_id}, falling back to account currency")
+      Rails.logger.warn("Invalid currency code '#{currency_value}' in Enable Banking transaction #{safe_external_id}, falling back to account currency")
     end

And in process, replace the inline computation with the shared helper:

-    safe_id = self.class.compute_external_id(`@enable_banking_transaction`) || "unknown"
+    safe_id = safe_external_id

Based on learnings: the project standard is to override log_invalid_currency in each provider processor to include provider-specific context — this override correctly does so, but it must not call a method that can raise.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@app/models/enable_banking_entry/processor.rb` around lines 232 - 234,
log_invalid_currency currently calls external_id which can raise when both
transaction_id and entry_reference are missing; create a private memoized helper
(e.g., safe_identifier or safe_id_helper) that returns transaction_id ||
entry_reference || "(no id)" without raising, use that helper inside
log_invalid_currency (and in parse_currency/currency if they still call
external_id), and remove any direct external_id calls from log_invalid_currency
so the warning never triggers an ArgumentError before logging.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@app/models/enable_banking_entry/processor.rb`:
- Around line 232-234: log_invalid_currency currently calls external_id which
can raise when both transaction_id and entry_reference are missing; create a
private memoized helper (e.g., safe_identifier or safe_id_helper) that returns
transaction_id || entry_reference || "(no id)" without raising, use that helper
inside log_invalid_currency (and in parse_currency/currency if they still call
external_id), and remove any direct external_id calls from log_invalid_currency
so the warning never triggers an ArgumentError before logging.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: a53d2d83-594f-4a9e-a2b5-78d29838f66f

📥 Commits

Reviewing files that changed from the base of the PR and between 6cf2426 and 0c769ea.

📒 Files selected for processing (4)
  • app/models/enable_banking_entry/processor.rb
  • app/models/transaction.rb
  • db/migrate/20260508120000_add_manual_merge_ext_id_index.rb
  • test/models/transaction/merge_with_duplicate_test.rb
🚧 Files skipped from review as they are similar to previous changes (3)
  • db/migrate/20260508120000_add_manual_merge_ext_id_index.rb
  • test/models/transaction/merge_with_duplicate_test.rb
  • app/models/transaction.rb

@CrossDrain
Copy link
Copy Markdown
Contributor Author

Three follow-up fixes after review:

  • 6cf24267manual_merge was a single object, so merging a second pending entry into the same posted transaction overwrote the first merged_from_external_id, causing re-imports on the next sync. Changed to an append-only array; legacy single-object records are migrated transparently.

  • 0c769ea4 — Three hardening changes: lock the posted_tx Transaction row (not just the Entry) to prevent concurrent merge races; cache safe_id in process so rescue/logging paths never call the raising external_id method; add CONCURRENTLY to the index migration so it doesn't block the transactions table in production.

  • 65b12f6alog_invalid_currency also called external_id directly and would raise before logging if both ID fields were absent. Replaced with compute_external_id || "unknown".

@CrossDrain
Copy link
Copy Markdown
Contributor Author

@coderabbitai resume

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented May 8, 2026

✅ Actions performed

Reviews resumed.

CrossDrain added 2 commits May 9, 2026 00:06
Move the SQL fragment used to identify pending transactions from the `Entry` model to a constant in the `Transaction` model. This improves maintainability and ensures that the logic for determining if a transaction is pending is defined in a single location.
Copy link
Copy Markdown
Collaborator

jjmata commented May 9, 2026

Really solid work on the atomic merge and lock ordering. Two things on the index and the excluded-IDs query:

1. The functional index doesn't cover the new Array format

The migration adds:

add_index :transactions,
  Arel.sql("(extra->'manual_merge'->>'merged_from_external_id')"),
  where: "(extra ? 'manual_merge')"

The ->> operator extracts a scalar from a JSON object. When manual_merge is a JSON array (the format introduced in this PR), extra->'manual_merge'->>'merged_from_external_id' returns NULL for every row — so every new merge record indexes as NULL and the index provides no benefit for them. The index only covers the legacy single-Hash records.

Additionally, the sync pre-fetch in EnableBankingAccount::Transactions::Processor doesn't use this index at all — it does:

.where("transactions.extra ? 'manual_merge'")
.pluck(:extra)

That ? 'manual_merge' key-existence check hits the GIN index on extra (already present), and then filters in Ruby. The new functional index is never consulted by this query path.

If you want to index for the array case, a GIN index with jsonb_path_ops on the manual_merge array element, or a generated column, would be needed. As-is the index is mostly dead weight for new records.

2. pluck(:extra) loads the entire JSONB column

Transaction.joins(:entry)
           .where(entries: { account_id: ... })
           .where("transactions.extra ? 'manual_merge'")
           .pluck(:extra)
           .flat_map { |extra| ... }

This pulls the full extra blob for every matched row into Ruby before extracting the one nested field. For accounts with many merged transactions (or large extra payloads), that's unnecessary memory pressure. A tighter query that extracts only what's needed in SQL would be more efficient — something like using jsonb_array_elements in a lateral join to pluck just the merged_from_external_id strings directly.

Not a blocker, but something to revisit if this becomes a performance concern in production.


Generated by Claude Code


Generated by Claude Code

@CrossDrain
Copy link
Copy Markdown
Contributor Author

  1. Dropped the functional index — it was intended as a performance aid for large databases, but the gain is negligible: the ->> expression returns NULL for every array-format record, and the sync pre-fetch never consulted it anyway (it hits the existing GIN index via extra ? 'manual_merge'). Removed the migration and schema entry.

  2. Replaced pluck(:extra) + Ruby extraction with a CROSS JOIN LATERAL jsonb_array_elements(...) join. The query now extracts only the merged_from_external_id strings in SQL via jsonb_typeof to handle both the current Array and legacy Hash formats. No full extra blobs shipped to Ruby.

@jjmata jjmata merged commit 96b1d28 into we-promise:main May 9, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contributor:verified Contributor passed trust analysis. pr:verified PR passed security analysis.

Development

Successfully merging this pull request may close these issues.

2 participants