Skip to content

Task 1.3.1: Create odf_data_quality_dashboard Module #8

@bosd

Description

@bosd

Task 1.3.1: Create the odf_data_quality_dashboard Module

Mission Context:
This task is based on section 4.2 of the Project Constitution. The goal is to improve the data import workflow by decoupling data validation from the import process itself. Instead of blocking an entire import due to validation failures (e.g., an invalid VAT number), we will import the data quickly and then run validations separately. This module will create a "Data Quality Dashboard" that serves as a work queue for functional users to identify and correct problematic records after the fact, ensuring the import process is fast and non-blocking.

Core Objective:
Create a new, installable Odoo module named odf_data_quality_dashboard that implements a non-blocking, post-import data validation system and a user interface for managing data quality issues.

Desired Outcome:
A self-contained Odoo version 18.0 module is created. When installed, it adds a "Data Quality Dashboard" menu item. A nightly scheduled action will automatically scan for common data issues (e.g., invalid VAT numbers on partners). Instead of raising errors, this action will create records in a new odf.data.quality.issue model. The dashboard will present these issues in a Kanban view, allowing users to easily navigate to the problematic records and correct them.

Required inputs:

  • Project Constitution: Project Constitution_ odoo-data-flow Next-Generation v3.md

Visual Workflow (Mermaid):
This diagram illustrates the backend logic for the scheduled action that finds and flags data quality issues.

```mermaid
graph TD
    A[Start: Scheduled Action Triggers] --> B{Query for records modified recently};
    B --> C[Loop through each record];
    C --> D{Perform validation checks, e.g., is VAT valid?};
    D -- "Yes" --> E[Next Record];
    D -- "No" --> F[Create 'odf.data.quality.issue' record];
    F --> G[Link issue to problematic record via Reference field];
    G --> E;
    E --> C;
    C -- "End of Loop" --> H[End];
```

The Process / Workflow:

  1. Module Scaffolding: Create the directory structure for a new Odoo module named odf_data_quality_dashboard. This includes creating the __init__.py and __manifest__.py files. The manifest should declare dependencies on the base Odoo module.
  2. Model Definition: In models/data_quality_issue.py, define the odf.data.quality.issue model. It must have the following fields:
    • name: fields.Char, a descriptive title for the issue.
    • issue_type: fields.Char, a category for the issue (e.g., 'Invalid VAT').
    • related_record: fields.Reference, with a selection of models to link to (e.g., [('res.partner', 'Partner'), ('product.product', 'Product')]). This is critical for linking to any problematic record.
    • status: fields.Selection with states: [('new', 'New'), ('in_progress', 'In Progress'), ('resolved', 'Resolved')]. Default to 'new'.
    • notes: fields.Text for user comments.
  3. Business Logic: In the same models/data_quality_issue.py file, create a method responsible for running the validations.
    • This method should query for records (e.g., res.partner) created or updated in the last 24 hours to keep the checks efficient.
    • For each record that fails validation, use self.env['odf.data.quality.issue'].create({...}) to create a new issue record. Do not use raise ValidationError.
  4. Scheduled Action: In data/scheduled_actions.xml, create an ir.cron record that calls the validation method you created. Configure it to run once daily.
  5. User Interface: In views/data_quality_issue_views.xml, create the following:
    • A Form view for the odf.data.quality.issue model showing all fields. Make the related_record field a clickable link.
    • A Kanban view for the same model. The Kanban cards should be grouped by the status field. Each card must clearly display the name, issue_type, and the related_record.
    • A Tree (List) view as a standard alternative view.
    • An ir.actions.act_window to tie these views together.
  6. Menu Item: In views/menus.xml, create a menu item named "Data Quality Dashboard" that opens the action window created in the previous step. Place it under a suitable top-level menu (e.g., a new "Data Flow" menu).
  7. Security: In security/ir.model.access.csv, provide Read, Write, Create, and Unlink access for the base.group_user group to the odf.data.quality.issue model.

Anticipated Pitfalls:

  • Blocking Errors: The most common mistake would be to use raise ValidationError inside the scheduled action. This is explicitly forbidden as it violates the non-blocking principle of the task.
  • Reference Field Handling: The related_record Reference field is populated differently from a Many2one. It requires a string in the format 'model_name,record_id'. Ensure the creation logic formats this correctly.
  • Performance: Failing to filter the records for the validation check (e.g., by date) could lead to performance degradation on systems with large databases. The logic must target recent records.
  • Security: Forgetting to include the ir.model.access.csv file will result in permission errors for non-admin users, making the feature unusable.

Acceptance Criteria / Verification Steps:

  1. The odf_data_quality_dashboard module can be installed in a standard Odoo database without any errors.
  2. After installation, a "Data Quality Dashboard" menu is visible in the UI.
  3. Create a new Partner record with an obviously invalid VAT number (e.g., "123").
  4. Manually trigger the "Data Validations" Scheduled Action from the developer menu.
  5. Verify that the action completes successfully without raising any Python exceptions or Odoo errors.
  6. Navigate to the Data Quality Dashboard; a new issue record for the invalid VAT should be present in the "New" column of the Kanban view.
  7. The issue card correctly identifies the partner record.
  8. Clicking the link on the issue's form view successfully navigates to the form view of the specific partner record with the invalid VAT.

Strict Constraints / Rules to Follow:

  • You are absolutely forbidden from using raise statements for validation failures. The sole mechanism for reporting an issue must be the creation of an odf.data.quality.issue record.
  • The related_record field must be implemented using the fields.Reference type.
  • The module must be self-contained. Do not add fields to or modify the logic of existing Odoo models like res.partner.
  • All code must comply with Odoo development guidelines and standard Python conventions.

Context and Reference Files:

  • Project Constitution_ odoo-data-flow Next-Generation v3.md (specifically section 4.2)

Proceed with the task.

Metadata

Metadata

Assignees

No one assigned

    Labels

    JulesGoogle Jules

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions