Address LanguageWorkerOptions null in FunctionMetadataManager

[jviau]

Issue describing the changes in this PR

resolves #9859

Pull request checklist

IMPORTANT: Currently, changes must be backported to the in-proc branch to be included in Core Tools and non-Flex deployments.

• Backporting to the in-proc branch is not required
  • Otherwise: Link to backporting PR
• My changes do not require documentation changes
  • Otherwise: Documentation issue linked to PR
• My changes should not be added to the release notes for the next release
  • Otherwise: I've added my notes to release_notes.md
• My changes do not need to be backported to a previous version
  • Otherwise: Backport tracked by issue/PR #issue_or_pr
• My changes do not require diagnostic events changes
  • Otherwise: I have added/updated all related diagnostic events and their documentation (Documentation issue linked to PR)
• I have added all required tests (Unit tests, E2E tests)

Additional information

This PR reverts a portion of #9264 and uses a different approach to solve stale function metadata. This alternative approach also addresses the null-ref from #9859.

Background

#9264 set out to solve function metadata being calculated and cached with a stale set of worker configs. To address this, the PR allowed for worker configs to be explicitly supplied to the GetFunctionMetadata method. When configs were not explicitly supplied, it would get 'fallback' configs from the current service provider. However, that provider is the active host. If this was called too early in the hosts lifetime, the active host would be null (leading to the null-ref).

Examining the issue it is a cache invalidation problem. GetFunctionMetadata will get the current value of LanguageWorkerOptions, then build and cache the function metadata results. That caching is the problem, as language worker options changes over the lifetime of the functions host.

Fix

To address the null ref and after taking a closer look at the root issue, a new approach has been taken. Instead of explicitly supplying the RPC configs (which all 2 instances of that were from the current value of LanguageWorkerOptions), we go back to the original approach of getting the current value of those options. With one major difference: we now subscribe to the OnChange of those options and will invalidate the cache whenever they change. The next call to GetFunctionMetadata will then re-build and cache the function metadata set.

Additional Information

#9264 would get the LanguageWorkerOptions from the job host. This PR now gets them from the script host (parent of job host). These options are identical between the two hosts. PR #10369 even sets out to make the job hosts copy of these options a direct forwarding of the script hosts copy.

[jviau]

Was testing this further and I am seeing an actual discrepancy between the worker configs between WebHost and ScriptHost, will need to investigate that further.

[fabiocav]

Blocked on #10369

[kshyju]

Blocked on #10369
#10369 is merged now.

[jviau]

@fabiocav @kshyju I have updated this PR now that the blocker is checked in. It is ready for review. Thanks!