Error on export when keyword preservation is enabled

[jcerjak]

Checklist

• I have looked into the README and have not found a suitable solution or answer.
• I have looked into the documentation and have not found a suitable solution or answer.
• I have searched the issues and have not found a suitable solution or answer.
• I have upgraded to the latest version of this tool and the issue still persists.
• I have searched the Auth0 Community forums and have not found a suitable solution or answer.
• I agree to the terms within the Auth0 Code of Conduct.

Description

I have existing Auth0 config which was exported with the 7.15.1 version of this tool. I'm now trying to run the export again with the latest version (7.17.0), to try out the awesome new keyword preservation feature implemented in #688.

But if I set "AUTH0_PRESERVE_KEYWORDS": true in my config.json and run:

npx a0deploy export -c config.json -f yaml -o config/ --debug

I get an error:

2023-03-10T11:03:47.407Z - debug: Loading YAML from <MY PROJECT PATH>/config/tenant.yaml
2023-03-10T11:03:47.440Z - error: Problem running command export
2023-03-10T11:03:47.440Z - error: Problem loading tenant data from Auth0 TypeError: Cannot read properties of undefined (reading 'replace')
2023-03-10T11:03:47.440Z - debug: Error: Problem loading tenant data from Auth0 TypeError: Cannot read properties of undefined (reading 'replace')
    at YAMLContext.<anonymous> (<MY PROJECT PATH>/node_modules/auth0-deploy-cli/lib/context/yaml/index.js:143:23)
    at Generator.next (<anonymous>)
    at fulfilled (<MY PROJECT PATH>/node_modules/auth0-deploy-cli/lib/context/yaml/index.js:5:58)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

I have all keyword mappings defined with AUTH0_KEYWORD_REPLACE_MAPPINGS (in env variable). The mappings should be correct, since I previously successfully ran import with this config.

If I remove the AUTH0_PRESERVE_KEYWORDS setting from my config, export finishes successfully (but without the desired keyword preservation).

Expectation

I would like export to finish successfully, when AUTH0_PRESERVE_KEYWORDS is enabled.

Reproduction

See Description above, issue can be reproduced consistently.

Deploy CLI version

7.17.0

Node version

v18.12.1

[willvedd]

Appreciate you reporting this. Can you report which resource types you have keyword replacement enabled for? If I had to guess which I'd say at least hooks or actions. Looking at the code it appears that there could be certain theoretical scenarios the Cannot read properties of undefined (reading 'replace') error for either of those two types.

Could you perhaps provide an example of how you're performing these replacements in your configuration so I can better reproduce and patch? Thanks.

[jcerjak]

Thanks for a quick reply @willvedd , these are the fields where I use keyword replacements:

pages[].url
resourceServers[].identifier
clients[].allowed_logout_urls
clients[].callbacks
clients[].initiate_login_uri
clients[].web_origins
connections[].options.email.from
emailProvider.credentials.smtp_user
emailProvider.credentials.smtp_pass
emailProvider.default_from_address
clientGrants[].audience
customDomains[].domain

Example config in tenant.yaml for pages:

pages:
  - name: error_page
    html: ./pages/error_page.html
    url: https://##MY_DOMAIN##/error

Then setting up the mappings with:

export AUTH0_KEYWORD_REPLACE_MAPPINGS='{"MY_DOMAIN": "example.com"}'

Will also investigate on my end to see if issue happens only with specific resources.

[jcerjak]

Looks like the issue is not related to a resource, but to a specific keyword. If I replace that keyword in tenant.yaml (which is used in several resources) with a static value, then the export goes through.

However, none of the other keywords are preserved, except smtp_pass: ##SMTP_PASS##, which after the export becomes smtp_pass: '##SMTP_PASS##' (notice the single quotes). Though the SMTP_PASS behavior is probably related to the special handling of this secret (see #689).

[jcerjak]

Additional investigation:

• Removed local config and did a fresh export with latest version (7.17.0) of cli
• Removed all other keyword mappings except the problematic one (##MY_DOMAIN##)
• Moved the keyword mapping to config.json
• Replaced relevant domain in tenant.yaml with the ##MY_DOMAIN## keyword
• Re-ran the export and got the same error

[jcerjak]

Seems the error happens in keywordArrayReplace function, because input is undefined.

[willvedd]

@jcerjak Really appreciate your help with this one. I have finally been able to reproduce under certain conditions and can confirm that the issue stems from input being undefined. Though my suspicion is that the true origin of this problem exists in a previous stage. I'll need to investigate a bit more but I'm certainly interested in getting this fixed for you over the coming days.

[willvedd]

@jcerjak I'm not certain, but I suspect that this issue is the same as #770 . The issue there is that keywords are being used in a resource's identifier field (de facto id in absence of literal id). I've released a fix to enable this type of preservation with v7.17.2. Can you update and see if it fixes your issue?

[willvedd]

I believe this is fixed in the aforementioned release, so going to close. However, feel empowered to open a new ticket if you're continuing to have issues with preservation.