`book_transformations` silently drops repeated transforms due to dict overwrites

[VibezCoder]

In data_generation.py, book_transformations is defined with duplicate keys like:

book_transformations = {
    'book2': {
        'ensure_300ppi': {'target_dpi': 150},
        'remove_bleed_dual_layer': {},
        'denoise_image': {'method': 'bilateral'},
        'denoise_image': {'method': 'nlm'},
    },
    ...
}

In Python, later dict entries overwrite earlier ones, so only the last denoise_image config is kept. This means you cannot actually apply denoise_image multiple times with different methods, even though the config suggests that.

This dict is passed into process_multiple_books / process_book_with_transformations in data_utils.py, where transform_order = list(transform_config.keys()) and transform_config[transform_name] are used. Because keys are unique, each transform runs at most once per page.

Suggested fix: change book_transformations to an ordered list of operations, for example:

book_transformations = {
    'book2': [
        ('ensure_300ppi', {'target_dpi': 150}),
        ('remove_bleed_dual_layer', {}),
        ('denoise_image', {'method': 'bilateral'}),
        ('denoise_image', {'method': 'nlm'}),
    ],
}

and update process_book_with_transformations to iterate over that list instead of transform_config.keys().

[swagat-mishra28]

Hi @VibezCoder,

Thanks for raising this issue. I’ve submitted a PR that addresses the duplicate key overwrite problem in book_transformations.

As noted in the issue, defining repeated transformations (e.g. multiple denoise_image steps) inside a dictionary causes earlier entries to be silently overwritten because Python dictionaries do not allow duplicate keys.

Changes introduced :

• Updated process_book_with_transformations() in data_utils.py to support an ordered transformation pipeline using a tuple-based structure.

• Transformations can now be defined as:

[
('denoise_image', {'method': 'bilateral'}),
('denoise_image', {'method': 'nlm'})
]

This ensures that repeated transformations are preserved and executed sequentially.

• Added logic to handle different transformation formats while maintaining backward compatibility with:

  • dictionary-based configurations
  • simple transformation lists

• Updated occurrences of book_transformations in:

  • data_generation_pipeline.py
  • experimentation scripts

so that configurations using repeated transformations follow the ordered tuple format.

This allows the preprocessing pipeline to correctly apply multiple passes of operations like denoising.

PR: #80

Please let me know if any adjustments or improvements would be helpful.