feat(api): add import preflight validation

[JSONbored]

Supersedes #1633. The original PR head was on the old JSONbored/sure fork network; this replacement branch lives on JSONbored/sure-upstream, the current fork of we-promise/sure.

This recreation uses the latest old-fork branch head (cc97a12), which includes the shared API UUID validation and public CSV size documentation fixes requested on the old PR.

---

Summary

• Adds a non-mutating import preflight endpoint for CSV and Sure NDJSON content.
• Returns counts, headers, validation errors, and warnings without creating imports, rows, attachments, or jobs.

What changed

• Adds POST /api/v1/imports/preflight.
• Reuses existing import type configuration, CSV parsing, Sure NDJSON size/type constraints, and SureImport count helpers.
• Adds Minitest coverage plus docs-only rswag/OpenAPI updates.

Why

API clients and self-hosted operators need a safe way to inspect import content before starting a state-changing import workflow.

Screenshots

Not applicable: this PR adds a public API/OpenAPI import preflight endpoint and does not add or change web UI. The reviewer-facing surface is the generated OpenAPI contract and the JSON response shape covered by controller/model tests.

Validation

Run through the packaged devcontainer stack for this worktree:

• devcontainer up --workspace-folder . --mount-git-worktree-common-dir --include-configuration
• devcontainer exec --workspace-folder . bash -lc 'cd /workspace && bin/rails test test/controllers/api/v1/imports_controller_test.rb test/models/mint_import_test.rb test/models/sure_import_test.rb'
  • 73 runs, 358 assertions, 0 failures, 0 errors, 0 skips
• devcontainer exec --workspace-folder . bash -lc 'cd /workspace && RAILS_ENV=test bundle exec rake rswag:specs:swaggerize'
  • 282 examples, 0 failures; regenerated docs/api/openapi.yaml
• devcontainer exec --workspace-folder . bash -lc 'cd /workspace && bin/rubocop app/controllers/api/v1/base_controller.rb app/controllers/api/v1/imports_controller.rb app/models/import.rb app/models/import/preflight.rb app/models/mint_import.rb app/models/sure_import.rb spec/requests/api/v1/imports_spec.rb spec/swagger_helper.rb test/controllers/api/v1/imports_controller_test.rb test/models/mint_import_test.rb test/models/sure_import_test.rb'
  • 11 files inspected, no offenses detected
• git diff --check

Summary by CodeRabbit

• New Features

  • Added a preflight endpoint to validate CSV/NDJSON import content and return structured validation results without creating imports.

• Improvements

  • More robust CSV and NDJSON validation, clearer size-limit enforcement, improved error handling and detailed warnings/statistics.
  • Centralized default CSV column mappings for Mint imports.

• Documentation

  • OpenAPI and API docs updated to describe the preflight endpoint and clarify import size/field requirements.

• Tests

  • Extensive tests added for preflight and import validation scenarios.

[coderabbitai]

📝 Walkthrough

Walkthrough

Adds an Import::Preflight service and POST /api/v1/imports/preflight endpoint to validate CSV and NDJSON content without creating imports; exposes class-level size/limit accessors, centralizes MintImport mappings, updates OpenAPI docs, and adds comprehensive tests.

Changes

Import Preflight Feature

Layer / File(s)	Summary
Model Constants & Limits app/models/import.rb, app/models/sure_import.rb, app/models/mint_import.rb, app/controllers/api/v1/base_controller.rb	Import and SureImport expose size/row limits via class methods; SureImport adds IMPORTABLE_NDJSON_TYPES and parsing safeguards; MintImport centralizes DEFAULT_COLUMN_MAPPINGS; BaseController adds class-level valid_uuid?.
Preflight Service app/models/import/preflight.rb	New Import::Preflight service validates import content without persistence. Implements type dispatch (SureImport vs CSV), content source handling, NDJSON line-by-line parsing with structure validation, CSV header/model validation, and structured error/warning aggregation.
Route & Controller config/routes.rb, app/controllers/api/v1/imports_controller.rb	Adds POST /api/v1/imports/preflight route; implements preflight action with read-scope authorization, preflight_params sanitization, and error handling (RecordNotFound, MalformedCSVError, generic fallback); refactors create action size checks to use Import.max_csv_size and SureImport.max_ndjson_size class methods.
API Documentation docs/api/openapi.yaml, spec/requests/api/v1/imports_spec.rb, spec/swagger_helper.rb	Adds five new OpenAPI component schemas (ImportPreflightContent, ImportPreflightError, ImportPreflightStats, ImportPreflight, ImportPreflightResponse); documents POST /api/v1/imports/preflight endpoint with request/response examples; updates POST /api/v1/imports documentation to clarify 10MB CSV limit and SureImport requirements.
Tests test/controllers/api/v1/imports_controller_test.rb, test/models/mint_import_test.rb, test/models/sure_import_test.rb	Adds 400+ lines of preflight endpoint tests covering CSV/NDJSON success/failure scenarios, refactors size-limit tests to stub class methods instead of mutating constants, and adds model tests for default mappings and dry-run totals.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~75 minutes

Possibly related PRs

• we-promise/sure#1599: Related changes to SureImport and import parsing that interact with preflight type handling.
• we-promise/sure#1601: Related imports controller and SureImport NDJSON handling changes.
• we-promise/sure#1632: Adds use of Api::V1::BaseController.valid_uuid? in other controllers alongside this PR's validator addition.

Suggested reviewers

• jjmata

Poem

🐰 I sniff the headers, count each line,
I hop through rows to make them fine,
NDJSON types and CSV spells,
I leave no broken header tells,
Safe to import — then you’ll shine.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 3.45% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title 'feat(api): add import preflight validation' clearly and concisely summarizes the main change: adding a new import preflight validation feature to the API.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 95e4dfbc9d

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[coderabbitai]

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@app/controllers/api/v1/imports_controller.rb`:
- Around line 153-160: The 500 response in ImportsController#preflight returns a
generic message; change the render JSON to match API v1 convention by using
message: "Error: #{e.message}". Keep the existing Rails.logger.error calls and
backtrace logging, but replace the payload's message value so the rescue
StandardError => e block renders { error: "internal_server_error", message:
"Error: #{e.message}" }, status: :internal_server_error.

In `@app/models/import/preflight.rb`:
- Around line 297-300: The apply_import_defaults method currently
unconditionally assigns MintImport.default_column_mappings and overwrites any
client-provided values; change it so defaults are only applied for attributes
that are not already set on the import (i.e., do not clobber explicit
*_col_label, date_format, signage_convention). In apply_import_defaults(import)
(and when dealing with MintImport), merge defaults into the import by assigning
each key from MintImport.default_column_mappings only if the corresponding
attribute on import is nil/blank/unset (or use a reverse_merge-style behavior),
preserving any values already provided by import_config_params.

In `@app/models/sure_import.rb`:
- Around line 42-45: The current counting (in SureImport around the
JSON.parse(line) branch) increments counts for any JSON object that has a "type"
even if it lacks "data"; change the condition so you only increment counts for
fully-formed NDJSON records that are a Hash and have both a "type" and a "data"
key (e.g. check record.is_a?(Hash) && record["type"] && record.key?("data") or
an equivalent presence check), so malformed lines like { "type": "Transaction" }
are not counted by counts[...] and thus dry_run/rows_count/publishable? reflect
only valid records.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 64a81506-cd21-415b-bf87-f9a9171bddf1

📥 Commits

Reviewing files that changed from the base of the PR and between 36960fe and 95e4dfb.

📒 Files selected for processing (13)

• app/controllers/api/v1/base_controller.rb
• app/controllers/api/v1/imports_controller.rb
• app/models/import.rb
• app/models/import/preflight.rb
• app/models/mint_import.rb
• app/models/sure_import.rb
• config/routes.rb
• docs/api/openapi.yaml
• spec/requests/api/v1/imports_spec.rb
• spec/swagger_helper.rb
• test/controllers/api/v1/imports_controller_test.rb
• test/models/mint_import_test.rb
• test/models/sure_import_test.rb