[SL-3389] [Feature] MetricFlow doesn't support `DBT_PROJECT_DIR` environment variable #1591

mercedes-wu-wandb · 2024-12-30T16:43:30Z

Is this a new bug in dbt-core?

I believe this is a new bug in dbt-core
I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

commands from dbt project level:
mf health-checks

output:

AttributeError: 'Namespace' object has no attribute 'PROFILES_DIR'

Expected Behavior

✔ Health checks completed.
• ✅ SqlEngine.BIGQUERY - SELECT 1: Success!

Steps To Reproduce

Set up a dbt project with a valid profiles.yml.
Set the DBT_PROJECT_DIR environment variable to point to the project directory:
export DBT_PROJECT_DIR=/path/to/dbt/project
Set the DBT_PROFILES_DIR environment variable to point to the profiles
export DBT_PROFILES_DIR=/path/to/profiles/directory
Run the following command from the dbt directory:
mf health-checks

Relevant log output

No response

Environment

- OS:Darwin Kernel Version 23.6.0
- Python: 3.12.3
- dbt: 1.8.9

Which database adapter are you using with dbt?

bigquery

Additional Context

I am trying to run metricflow on dbt-core

_SL-3389

The text was updated successfully, but these errors were encountered:

github-actions · 2024-12-30T16:43:39Z

Thank you for your bug report! Our team is will be out of the office for Christmas and our Global Week of Rest, from December 25, 2024, through January 3, 2025.

We will review your issue as soon as possible after returning.
Thank you for your understanding, and happy holidays! 🎄🎉

If you are a customer of dbt Cloud, please contact our Customer Support team via the dbt Cloud web interface or email [email protected].

dbeatty10 · 2025-01-10T16:49:22Z

Thanks for reaching out @mercedes-wu-wandb!

I'm going to transfer this issue to the metricflow repository for follow-up by another member of our team.

dbeatty10 · 2025-01-10T16:50:04Z

Transferred from https://github.com/dbt-labs/dbt-core/issues/11179 to #1591.

plypaul · 2025-02-23T20:14:42Z

@mercedes-wu-wandb To clarify, mf works for you when run it from the dbt project directory?

mercedes-wu-wandb · 2025-02-27T21:46:31Z

Yes mf works in the dbt project directory but even with my environment variable set to DBT_PROJECT_DIR it does not seem to recognize it

analytics-py3.12➜  analytics git:(semantic-layer) export DBT_PROJECT_DIR=dbt/
analytics-py3.12➜  analytics git:(semantic-layer) mf health-checks           
❌ Unable to locate 'dbt_project.yml' in the current directory
In order to run the MetricFlow CLI, you must be running in the root directory of a working dbt project.
Please check out `https://docs.getdbt.com/reference/commands/init` if you want to get started on building a dbt project.
analytics-py3.12➜  analytics git:(semantic-layer) cd dbt
analytics-py3.12➜  dbt git:(semantic-layer) mf health-checks
ERROR:root:Logging exception
Traceback (most recent call last):
  File "/Users/mercedeswu/wb-dev/analytics/.venv/lib/python3.12/site-packages/dbt_metricflow/cli/utils.py", line 116, in wrapper
    func(*args, **kwargs)
  File "/Users/mercedeswu/wb-dev/analytics/.venv/lib/python3.12/site-packages/metricflow/telemetry/reporter.py", line 150, in wrapped
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mercedeswu/wb-dev/analytics/.venv/lib/python3.12/site-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mercedeswu/wb-dev/analytics/.venv/lib/python3.12/site-packages/dbt_metricflow/cli/utils.py", line 170, in new_func
    return ctx.invoke(func, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mercedeswu/wb-dev/analytics/.venv/lib/python3.12/site-packages/click/core.py", line 788, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mercedeswu/wb-dev/analytics/.venv/lib/python3.12/site-packages/dbt_metricflow/cli/main.py", line 384, in health_checks
    cfg.setup()
  File "/Users/mercedeswu/wb-dev/analytics/.venv/lib/python3.12/site-packages/dbt_metricflow/cli/cli_configuration.py", line 54, in setup
    raise e
  File "/Users/mercedeswu/wb-dev/analytics/.venv/lib/python3.12/site-packages/dbt_metricflow/cli/cli_configuration.py", line 40, in setup
    self._dbt_project_metadata = dbtProjectMetadata.load_from_project_path(dbt_project_path)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mercedeswu/wb-dev/analytics/.venv/lib/python3.12/site-packages/dbt_metricflow/cli/dbt_connectors/dbt_config_accessor.py", line 46, in load_from_project_path
    profile = load_profile(str(project_path), {})
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mercedeswu/wb-dev/analytics/.venv/lib/python3.12/site-packages/dbt/config/runtime.py", line 79, in load_profile
    profile = Profile.render(
              ^^^^^^^^^^^^^^^
  File "/Users/mercedeswu/wb-dev/analytics/.venv/lib/python3.12/site-packages/dbt/config/profile.py", line 400, in render
    raw_profiles = read_profile(flags.PROFILES_DIR)
                                ^^^^^^^^^^^^^^^^^^
AttributeError: 'Namespace' object has no attribute 'PROFILES_DIR'
Traceback (most recent call last):
  File "/Users/mercedeswu/wb-dev/analytics/.venv/lib/python3.12/site-packages/dbt_metricflow/cli/utils.py", line 116, in wrapper
    func(*args, **kwargs)
  File "/Users/mercedeswu/wb-dev/analytics/.venv/lib/python3.12/site-packages/metricflow/telemetry/reporter.py", line 150, in wrapped
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mercedeswu/wb-dev/analytics/.venv/lib/python3.12/site-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mercedeswu/wb-dev/analytics/.venv/lib/python3.12/site-packages/dbt_metricflow/cli/utils.py", line 170, in new_func
    return ctx.invoke(func, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mercedeswu/wb-dev/analytics/.venv/lib/python3.12/site-packages/click/core.py", line 788, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mercedeswu/wb-dev/analytics/.venv/lib/python3.12/site-packages/dbt_metricflow/cli/main.py", line 384, in health_checks
    cfg.setup()
  File "/Users/mercedeswu/wb-dev/analytics/.venv/lib/python3.12/site-packages/dbt_metricflow/cli/cli_configuration.py", line 54, in setup
    raise e
  File "/Users/mercedeswu/wb-dev/analytics/.venv/lib/python3.12/site-packages/dbt_metricflow/cli/cli_configuration.py", line 40, in setup
    self._dbt_project_metadata = dbtProjectMetadata.load_from_project_path(dbt_project_path)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mercedeswu/wb-dev/analytics/.venv/lib/python3.12/site-packages/dbt_metricflow/cli/dbt_connectors/dbt_config_accessor.py", line 46, in load_from_project_path
    profile = load_profile(str(project_path), {})
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mercedeswu/wb-dev/analytics/.venv/lib/python3.12/site-packages/dbt/config/runtime.py", line 79, in load_profile
    profile = Profile.render(
              ^^^^^^^^^^^^^^^
  File "/Users/mercedeswu/wb-dev/analytics/.venv/lib/python3.12/site-packages/dbt/config/profile.py", line 400, in render
    raw_profiles = read_profile(flags.PROFILES_DIR)
                                ^^^^^^^^^^^^^^^^^^
AttributeError: 'Namespace' object has no attribute 'PROFILES_DIR'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/mercedeswu/wb-dev/analytics/.venv/bin/mf", line 8, in <module>
    sys.exit(cli())
             ^^^^^
  File "/Users/mercedeswu/wb-dev/analytics/.venv/lib/python3.12/site-packages/click/core.py", line 1161, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mercedeswu/wb-dev/analytics/.venv/lib/python3.12/site-packages/click/core.py", line 1082, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/Users/mercedeswu/wb-dev/analytics/.venv/lib/python3.12/site-packages/click/core.py", line 1697, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mercedeswu/wb-dev/analytics/.venv/lib/python3.12/site-packages/click/core.py", line 1443, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mercedeswu/wb-dev/analytics/.venv/lib/python3.12/site-packages/click/core.py", line 788, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mercedeswu/wb-dev/analytics/.venv/lib/python3.12/site-packages/click/decorators.py", line 92, in new_func
    return ctx.invoke(f, obj, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mercedeswu/wb-dev/analytics/.venv/lib/python3.12/site-packages/click/core.py", line 788, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mercedeswu/wb-dev/analytics/.venv/lib/python3.12/site-packages/dbt_metricflow/cli/utils.py", line 123, in wrapper
    click.echo(f"\nERROR: {str(e)}\nLog file: {cli_context.log_file_path}")
                                               ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mercedeswu/wb-dev/analytics/.venv/lib/python3.12/site-packages/dbt_metricflow/cli/cli_configuration.py", line 112, in log_file_path
    return pathlib.Path(self.dbt_project_metadata.project.log_path, "metricflow.log")
                        ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mercedeswu/wb-dev/analytics/.venv/lib/python3.12/site-packages/dbt_metricflow/cli/cli_configuration.py", line 97, in dbt_project_metadata
    return self._get_dbt_project_metadata()
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/mercedeswu/wb-dev/analytics/.venv/lib/python3.12/site-packages/dbt_metricflow/cli/cli_configuration.py", line 58, in _get_dbt_project_metadata
    raise RuntimeError(
RuntimeError: CLIConfiguration.setup() should have been called before accessing the configuration.

plypaul · 2025-02-28T16:03:40Z

It should be straightforward to add support for DBT_PROJECT_DIR, but did you see documentation that allows for that environment variable for the mf CLI? Looking to double check the classification of this as a bug vs. feature.

mercedes-wu-wandb · 2025-02-28T16:13:13Z

Hey @plypaul, so far i have not so I'm happy to change this to issue to a feature if that's helpful!

plypaul · 2025-02-28T20:21:38Z

Yes, that would be great. Thanks!

mercedes-wu-wandb · 2025-02-28T21:00:23Z

Changed to "[SL-3389] [Feature] MetricFlow doesn't support DBT_PROJECT_DIR environment variable"! Do you know when this would get worked on by chance?

plypaul · 2025-03-03T18:29:48Z

There was some relevant prior work that has already been done, and I'm aiming to get a PR out this week.

mercedes-wu-wandb · 2025-03-03T18:56:03Z

awesome, thanks a lot for the updates and looking forward to the PR!

mercedes-wu-wandb · 2025-03-21T17:22:13Z

Hi @plypaul ! Just wondering if there were any updates on this feature, thanks a ton for all the help so far!

plypaul · 2025-03-24T20:45:19Z

To give an update, getting this out for release required a large number of changes to the project's CLI test framework as we want to have good test coverage for this feature. In addition, as I was getting ramped up on the CLI implementation, I saw there were a few CLI bugs (e.g. display of numeric types) that would be beneficial to have in the release. Right now, the feature is in the review pipeline but at the bottom of a stack of changes:

The PR for this feature is relatively small, but partly due to the work that was included by prior PRs in the stack.

We're looking to get these reviewed and merged, and then I can cut a release. While there a number of changes, I think we can get it out this week.

mercedes-wu-wandb · 2025-03-25T15:13:06Z

thanks a ton for the detailed update! really appreciate the help and looking forward to the release!

mercedes-wu-wandb added bug Something isn't working triage Tasks that need to be triaged labels Dec 30, 2024

dbeatty10 changed the title ~~[Bug] MetricFlow doesn't support DBT_PROJECT_DIR~~ [Bug] MetricFlow doesn't support DBT_PROJECT_DIR environment variable Jan 10, 2025

dbeatty10 transferred this issue from dbt-labs/dbt-core Jan 10, 2025

Jstein77 added the linear label Jan 30, 2025

Jstein77 changed the title ~~[Bug] MetricFlow doesn't support DBT_PROJECT_DIR environment variable~~ [SL-3389] [Bug] MetricFlow doesn't support DBT_PROJECT_DIR environment variable Jan 30, 2025

mercedes-wu-wandb changed the title ~~[SL-3389] [Bug] MetricFlow doesn't support DBT_PROJECT_DIR environment variable~~ [SL-3389] [Feature] MetricFlow doesn't support DBT_PROJECT_DIR environment variable Feb 28, 2025

plypaul mentioned this issue Mar 28, 2025

Add support for DBT_PROFILES_DIR / DBT_PROJECT_DIR in the CLI #1702

Merged

plypaul closed this as completed in #1702 Mar 29, 2025

plypaul closed this as completed in b1c1c2a Mar 29, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[SL-3389] [Feature] MetricFlow doesn't support `DBT_PROJECT_DIR` environment variable #1591

[SL-3389] [Feature] MetricFlow doesn't support `DBT_PROJECT_DIR` environment variable #1591

mercedes-wu-wandb commented Dec 30, 2024 •

edited by Jstein77

Loading

github-actions bot commented Dec 30, 2024

dbeatty10 commented Jan 10, 2025

dbeatty10 commented Jan 10, 2025

plypaul commented Feb 23, 2025

mercedes-wu-wandb commented Feb 27, 2025 •

edited

Loading

plypaul commented Feb 28, 2025

mercedes-wu-wandb commented Feb 28, 2025

plypaul commented Feb 28, 2025

mercedes-wu-wandb commented Feb 28, 2025

plypaul commented Mar 3, 2025

mercedes-wu-wandb commented Mar 3, 2025

mercedes-wu-wandb commented Mar 21, 2025

plypaul commented Mar 24, 2025

mercedes-wu-wandb commented Mar 25, 2025

[SL-3389] [Feature] MetricFlow doesn't support DBT_PROJECT_DIR environment variable #1591

[SL-3389] [Feature] MetricFlow doesn't support DBT_PROJECT_DIR environment variable #1591

Comments

mercedes-wu-wandb commented Dec 30, 2024 • edited by Jstein77 Loading

Is this a new bug in dbt-core?

Current Behavior

Expected Behavior

Steps To Reproduce

Relevant log output

Environment

Which database adapter are you using with dbt?

Additional Context

github-actions bot commented Dec 30, 2024

dbeatty10 commented Jan 10, 2025

dbeatty10 commented Jan 10, 2025

plypaul commented Feb 23, 2025

mercedes-wu-wandb commented Feb 27, 2025 • edited Loading

plypaul commented Feb 28, 2025

mercedes-wu-wandb commented Feb 28, 2025

plypaul commented Feb 28, 2025

mercedes-wu-wandb commented Feb 28, 2025

plypaul commented Mar 3, 2025

mercedes-wu-wandb commented Mar 3, 2025

mercedes-wu-wandb commented Mar 21, 2025

plypaul commented Mar 24, 2025

mercedes-wu-wandb commented Mar 25, 2025

[SL-3389] [Feature] MetricFlow doesn't support `DBT_PROJECT_DIR` environment variable #1591

[SL-3389] [Feature] MetricFlow doesn't support `DBT_PROJECT_DIR` environment variable #1591

mercedes-wu-wandb commented Dec 30, 2024 •

edited by Jstein77

Loading

mercedes-wu-wandb commented Feb 27, 2025 •

edited

Loading