Resolving nested structured config of Lists with interpolation raises ValidationError #1005

soldni · 2022-09-12T21:42:55Z

Describe the bug
This is a follow up to #847; OmegaConf raises a ValidationError when trying to use an interpolation inside a structured ListConfig.

To Reproduce

from omegaconf import OmegaConf, MISSING, II
from dataclasses import dataclass
from typing import List


@dataclass
class User:
    name: str = MISSING


@dataclass
class Config:
    user: User = User("John")
    admins: List[User] = "[${user}]"


cfg: Config = OmegaConf.structured(Config)
OmegaConf.resolve(cfg) # this  raises ValidationError

Expected behavior
I expect this resolution to not fail.

Additional context

OmegaConf version: 2.2.3
Python version: 3.9
Operating system: macOS 12.5.1
Please provide a minimal repro

binkjakub · 2023-11-06T11:00:58Z

Are there any updates on whether this is planned to be fixed? I think it is important to handle nested interpolations.

Jasha10 · 2023-11-07T05:30:45Z

I believe there is as typo in the original repro.
The interpolation should be ["${user}"], not "[${user}]".
Here's the updated repro (also using dataclasses.field for python3.11 compatibility):

from dataclasses import dataclass, field
from typing import List

from omegaconf import MISSING, OmegaConf


@dataclass
class User:
    name: str = MISSING


@dataclass
class Config:
    user: User = User("John")
    admins: List[User] = field(default_factory=lambda: ["${user}"])


cfg: Config = OmegaConf.structured(Config)
OmegaConf.resolve(cfg)  # this  raises ValidationError

omegaconf.errors.ValidationError: Invalid type assigned: str is not a subclass of User. value: ${user}
    full_key: admins[0]
    reference_type=List[User]
    object_type=list

Here's the full backtrace:

$ python repro.py
Traceback (most recent call last):
  File "/home/homestar/dev/omegaconf/repro.py", line 18, in <module>
    cfg: Config = OmegaConf.structured(Config)
  File "/home/homestar/dev/omegaconf/omegaconf/omegaconf.py", line 124, in structured
    return OmegaConf.create(obj, parent, flags)
  File "/home/homestar/dev/omegaconf/omegaconf/omegaconf.py", line 177, in create
    return OmegaConf._create_impl(
  File "/home/homestar/dev/omegaconf/omegaconf/omegaconf.py", line 917, in _create_impl
    format_and_raise(node=None, key=None, value=None, msg=str(e), cause=e)
  File "/home/homestar/dev/omegaconf/omegaconf/_utils.py", line 813, in format_and_raise
    _raise(ex, cause)
  File "/home/homestar/dev/omegaconf/omegaconf/_utils.py", line 791, in _raise
    raise ex.with_traceback(sys.exc_info()[2])  # set env var OC_CAUSE=1 for full trace
  File "/home/homestar/dev/omegaconf/omegaconf/omegaconf.py", line 878, in _create_impl
    return DictConfig(
  File "/home/homestar/dev/omegaconf/omegaconf/dictconfig.py", line 110, in __init__
    format_and_raise(node=None, key=key, value=None, cause=ex, msg=str(ex))
  File "/home/homestar/dev/omegaconf/omegaconf/_utils.py", line 813, in format_and_raise
    _raise(ex, cause)
  File "/home/homestar/dev/omegaconf/omegaconf/_utils.py", line 791, in _raise
    raise ex.with_traceback(sys.exc_info()[2])  # set env var OC_CAUSE=1 for full trace
  File "/home/homestar/dev/omegaconf/omegaconf/dictconfig.py", line 93, in __init__
    self._set_value(content, flags=flags)
  File "/home/homestar/dev/omegaconf/omegaconf/dictconfig.py", line 645, in _set_value
    raise e
  File "/home/homestar/dev/omegaconf/omegaconf/dictconfig.py", line 642, in _set_value
    self._set_value_impl(value, flags)
  File "/home/homestar/dev/omegaconf/omegaconf/dictconfig.py", line 672, in _set_value_impl
    data = get_structured_config_data(value, allow_objects=ao)
  File "/home/homestar/dev/omegaconf/omegaconf/_utils.py", line 499, in get_structured_config_data
    return get_dataclass_data(obj, allow_objects=allow_objects)
  File "/home/homestar/dev/omegaconf/omegaconf/_utils.py", line 423, in get_dataclass_data
    format_and_raise(
  File "/home/homestar/dev/omegaconf/omegaconf/_utils.py", line 813, in format_and_raise
    _raise(ex, cause)
  File "/home/homestar/dev/omegaconf/omegaconf/_utils.py", line 791, in _raise
    raise ex.with_traceback(sys.exc_info()[2])  # set env var OC_CAUSE=1 for full trace
  File "/home/homestar/dev/omegaconf/omegaconf/_utils.py", line 415, in get_dataclass_data
    d[name] = _maybe_wrap(
  File "/home/homestar/dev/omegaconf/omegaconf/omegaconf.py", line 1122, in _maybe_wrap
    return _node_wrap(
  File "/home/homestar/dev/omegaconf/omegaconf/omegaconf.py", line 1034, in _node_wrap
    node = ListConfig(
  File "/home/homestar/dev/omegaconf/omegaconf/listconfig.py", line 81, in __init__
    format_and_raise(node=None, key=key, value=None, cause=ex, msg=str(ex))
  File "/home/homestar/dev/omegaconf/omegaconf/_utils.py", line 813, in format_and_raise
    _raise(ex, cause)
  File "/home/homestar/dev/omegaconf/omegaconf/_utils.py", line 791, in _raise
    raise ex.with_traceback(sys.exc_info()[2])  # set env var OC_CAUSE=1 for full trace
  File "/home/homestar/dev/omegaconf/omegaconf/listconfig.py", line 79, in __init__
    self._set_value(value=content, flags=flags)
  File "/home/homestar/dev/omegaconf/omegaconf/listconfig.py", line 616, in _set_value
    raise e
  File "/home/homestar/dev/omegaconf/omegaconf/listconfig.py", line 612, in _set_value
    self._set_value_impl(value, flags)
  File "/home/homestar/dev/omegaconf/omegaconf/listconfig.py", line 657, in _set_value_impl
    self.append(item)
  File "/home/homestar/dev/omegaconf/omegaconf/listconfig.py", line 300, in append
    self._format_and_raise(key=index, value=item, cause=e)
  File "/home/homestar/dev/omegaconf/omegaconf/base.py", line 230, in _format_and_raise
    format_and_raise(
  File "/home/homestar/dev/omegaconf/omegaconf/_utils.py", line 893, in format_and_raise
    _raise(ex, cause)
  File "/home/homestar/dev/omegaconf/omegaconf/_utils.py", line 791, in _raise
    raise ex.with_traceback(sys.exc_info()[2])  # set env var OC_CAUSE=1 for full trace
  File "/home/homestar/dev/omegaconf/omegaconf/listconfig.py", line 297, in append
    self._set_item_impl(index, item)
  File "/home/homestar/dev/omegaconf/omegaconf/basecontainer.py", line 574, in _set_item_impl
    self._validate_set(key, value)
  File "/home/homestar/dev/omegaconf/omegaconf/listconfig.py", line 122, in _validate_set
    raise ValidationError(msg)
omegaconf.errors.ValidationError: Invalid type assigned: str is not a subclass of User. value: ${user}
    full_key: admins[0]
    reference_type=List[User]
    object_type=list

Jasha10 · 2023-11-07T05:37:25Z

@binkjakub the OmegaConf team is currently volunteer-run and lacking in bandwidth. I cannot make a promise as to when this will be fixed.

Edit: I've submitted PR #1139 to fix this.

This PR closes omry#1005, fixing a bug where a validation error was raised when an interpolation points to a structured config from within a typed list. ```python from dataclasses import dataclass, field from omegaconf import MISSING, OmegaConf @DataClass class User: name: str = MISSING @DataClass class Config: user: User = User("John") users: list[User] = field(default_factory=lambda: ["${user}"]) print(OmegaConf.structured(Config)) ``` BEFORE ------ ```text $ python repro.py $ python repro.py Traceback (most recent call last): ... omegaconf.errors.ValidationError: Invalid type assigned: str is not a subclass of User. value: ${user} full_key: users[0] reference_type=List[User] object_type=list ``` AFTER ----- ```text $ python repro.py {'user': {'name': 'John'}, 'users': [{'name': 'John'}]} ```

This PR closes omry#1005, fixing a bug where a validation error was raised when an interpolation points to a structured config from within a typed list. ```python from dataclasses import dataclass, field from omegaconf import MISSING, OmegaConf @DataClass class User: name: str = MISSING @DataClass class Config: user: User = User("John") users: list[User] = field(default_factory=lambda: ["${user}"]) cfg = OmegaConf.structured(Config) OmegaConf.resolve(cfg) print(cfg) ``` BEFORE ------ ```text $ python repro.py $ python repro.py Traceback (most recent call last): ... omegaconf.errors.ValidationError: Invalid type assigned: str is not a subclass of User. value: ${user} full_key: users[0] reference_type=List[User] object_type=list ``` AFTER ----- ```text $ python repro.py {'user': {'name': 'John'}, 'users': [{'name': 'John'}]} ```

soldni added the bug Something isn't working label Sep 12, 2022

Jasha10 mentioned this issue Nov 7, 2023

Fix interpolation to structured config from within typed list #1139

Merged

Jasha10 closed this as completed in #1139 Nov 15, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Resolving nested structured config of Lists with interpolation raises ValidationError #1005

Resolving nested structured config of Lists with interpolation raises ValidationError #1005

soldni commented Sep 12, 2022 •

edited

Loading

binkjakub commented Nov 6, 2023

Jasha10 commented Nov 7, 2023 •

edited

Loading

Jasha10 commented Nov 7, 2023 •

edited

Loading

Resolving nested structured config of Lists with interpolation raises ValidationError #1005

Resolving nested structured config of Lists with interpolation raises ValidationError #1005

Comments

soldni commented Sep 12, 2022 • edited Loading

binkjakub commented Nov 6, 2023

Jasha10 commented Nov 7, 2023 • edited Loading

Jasha10 commented Nov 7, 2023 • edited Loading

soldni commented Sep 12, 2022 •

edited

Loading

Jasha10 commented Nov 7, 2023 •

edited

Loading

Jasha10 commented Nov 7, 2023 •

edited

Loading