Use configurable credentials in reconcile in place of db secret scope

[m-abulazm]

Changes

What does this PR do?

• Move away from hardcoded secrets in reconcile
• use credential manager which enables local, env and databricks

Relevant implementation details

• add load_credentials to DataSource which takes care of loading the credentials

Caveats/things to watch out for when reviewing:

Linked issues

Progresses #1008, #2123, #2157

Functionality

• added relevant user documentation
• added new CLI command
• modified existing command: databricks labs lakebridge reconcile
• ... +add your own

Tests

• manually tested
• added unit tests
• added integration tests

[codecov]

Codecov Report

❌ Patch coverage is 71.73913% with 26 lines in your changes missing coverage. Please review.
✅ Project coverage is 65.14%. Comparing base (c284f50) to head (bf928ff).

Files with missing lines	Patch %	Lines
.../labs/lakebridge/reconcile/connectors/snowflake.py	66.66%	6 Missing and 3 partials ⚠️
src/databricks/labs/lakebridge/reconcile/utils.py	0.00%	5 Missing ⚠️
...cks/labs/lakebridge/reconcile/connectors/oracle.py	80.00%	3 Missing and 1 partial ⚠️
...ricks/labs/lakebridge/reconcile/connectors/tsql.py	76.47%	3 Missing and 1 partial ⚠️
...abs/lakebridge/reconcile/connectors/data_source.py	77.77%	2 Missing ⚠️
...labs/lakebridge/reconcile/connectors/databricks.py	80.00%	1 Missing ⚠️
.../lakebridge/reconcile/connectors/source_adapter.py	75.00%	1 Missing ⚠️

Additional details and impacted files

@@                    Coverage Diff                     @@
##           refactor/creds-manager    #2159      +/-   ##
==========================================================
- Coverage                   65.32%   65.14%   -0.18%     
==========================================================
  Files                         100       98       -2     
  Lines                        8539     8539              
  Branches                      878      884       +6     
==========================================================
- Hits                         5578     5563      -15     
- Misses                       2773     2784      +11     
- Partials                      188      192       +4     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[github-actions]

✅ 51/51 passed, 10 flaky, 3m39s total

Flaky tests:

• 🤪 test_validate_mixed_checks (157ms)
• 🤪 test_validate_invalid_source_tech (170ms)
• 🤪 test_validate_invalid_schema_path (1ms)
• 🤪 test_validate_table_not_found (1ms)
• 🤪 test_validate_non_empty_tables (8ms)
• 🤪 test_transpiles_informatica_to_sparksql_non_interactive[True] (19.946s)
• 🤪 test_transpiles_informatica_to_sparksql (21.232s)
• 🤪 test_transpile_teradata_sql (22.597s)
• 🤪 test_transpile_teradata_sql_non_interactive[False] (6.553s)
• 🤪 test_transpile_teradata_sql_non_interactive[True] (7.141s)

_{Running from acceptance #2992}