Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

adding Gitlab support #174

Merged
merged 21 commits into from
Nov 20, 2024
Merged

adding Gitlab support #174

merged 21 commits into from
Nov 20, 2024

Conversation

valx
Copy link
Contributor

@valx valx commented Oct 18, 2024

Main changes:

  • added support for Gitlab. The Gitlab pipeline files are in the folder template/{{.input_root_dir}}/.gitlab/ . Changes include:
    • Modified /mlops-stacks/databricks_template_schema.json prompt metadata by adding gitlab option
    • Searched all occurrences of input_cicd_platform and added a gitlab option
  • in databricks bundle file, added mode: development to dev target

@valx valx marked this pull request as draft October 18, 2024 01:02
@valx valx marked this pull request as ready for review October 19, 2024 01:13
Copy link
Collaborator

@arpitjasa-db arpitjasa-db left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR @valx ! Left a few comments, would you mind also generating a couple of Stacks projects from this PR and linking them in the PR? Would be good to verify this works with GitLab as expected, and the docs all look ok, as well as other providers remain unaffected.

README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
databricks_template_schema.json Outdated Show resolved Hide resolved
databricks_template_schema.json Show resolved Hide resolved
@valx

This comment was marked as resolved.

@arpitjasa-db
Copy link
Collaborator

@valx any update on this PR? Looks like the tests are still failing

@valx

This comment was marked as resolved.

tests/utils.py Outdated Show resolved Hide resolved
tests/utils.py Outdated Show resolved Hide resolved
tests/test_gitlab.py Outdated Show resolved Hide resolved
@arpitjasa-db arpitjasa-db merged commit 11b9db4 into databricks:main Nov 20, 2024
1 check passed
@@ -27,6 +27,7 @@ include:
# Deployment Target specific values for workspace
targets:
dev: {{ if (eq .input_include_models_in_unity_catalog `yes`)}} # UC Catalog Name {{ end }}
mode: development
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@arpitjasa-db Was this intentional?

We have an integration test that confirms successful template initialization and noticed that the job names now are prefixed with the username (DAB default for mode: development), so checking.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I intentionally added it as I thought it would be a best practice and most environment I've been working with would benefit from it (as multiple developers could be deploying the same project into dev).
But if that has any negative impact the change could be reverted

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is, no need to revert! It seemed like a piggybacked change, so want to make sure there isn't any docs that need to be updated, or otherwise other downstream changes that need to be made.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup intentional! Didn't realize this would break the naming downstream, thanks for flagging. For docs though, we don't really mention the file names for the dev target so we should be good on that

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for confirming.

github-merge-queue bot pushed a commit to databricks/cli that referenced this pull request Nov 21, 2024
## Changes
The ML production team modified mlops-stack to use `mode: development`
for their development target here:
databricks/mlops-stacks#174

This PR makes the integration test assertion agnostic of the prefix to
make it pass again.

## Tests
The test passes now
github-merge-queue bot pushed a commit to databricks/cli that referenced this pull request Nov 21, 2024
## Changes
The ML production team modified mlops-stack to use `mode: development`
for their development target here:
databricks/mlops-stacks#174

This PR makes the integration test assertion agnostic of the prefix to
make it pass again.

## Tests
The test passes now
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants