v1.0.0
v1.0.0 (2025-10-20)
Bug Fixes
-
Add default value to env vars on dbt models (
eef59ff) -
Add missing curl and jq dependencies (
99ce192) -
Add missing curl dependency (
0d4609f) -
Add missing env vars (
a5e153d) -
Add missing graph load command, add spacing and comment sections (
f9150bc) -
Add universe apt repo (
ce5e18a) -
Change to correct context path (
4534a27) -
Correct command to ml server (
29dfbf7) -
Destroy services was not switching to the appropriate docker context (
3aee07e) -
Enable prevent_destroy so we can tune cores and memory without risking destruction in the future (
ae7947d) -
Expired password by default and dhcp hostname broadcast (
2e5bc08) -
Force build to ensure image is rebuilt when required (
d84cc77) -
Incorrect create user script path (
22e6e0f) -
Indentation (
c7f1b17) -
Mark env var secrets as sensitive (
c52f940) -
Mark gitlab token as sensitive (
3832bb1) -
Missing add-apt-repository (
2e2cf53) -
Missing backslash (
3ab832d) -
Overcommit gitlab resources with 4 cores and 8 GiB without ballooning, for stability (
48622fe) -
Portainer would collide with minio when running locally (
fcc0093) -
Preinstall missing jq (
4c80525) -
Pushing large images would fail without checksum_disabled on the registry storage (
eae3d6f) -
Re-enable prevent_destroy (
94a5d69) -
Script executable permissions (
17bfd3b) -
Set env file to the ci project dir (
2642e03) -
Set options that reconfigure couldn't using gitlab-rails console (
d4e88cd) -
Should be secret_key, not a duplicate access_key (
e10b44e) -
Should be the node name, not the endpoint (
09816d4) -
Tfstate backups have a timestamp before the backup extension (
74261fe) -
Use env var instead of input, return empty string on null (
f7ea5db) -
Use env var to expose external kafka advertised listener (
355c787) -
Variables were not being expanded (
5c97801) -
Wrong env filename (
0d26f03)
Chores
-
Add gitlab bucket to default buckets (
f56252c) -
Add mlflow bucket to default foundation layer minio buckets (
63a6ab9) -
Bump up mlflow from 3.2.0 to 3.4.0 (
cd3760d) -
Enable prevent_destroy for docker and gitlab vms (
0b1e914) -
Ignore terraform state files and tfvars (
4e3bc78) -
Increase cores and memory for gitlab, reducing memory for docker-apps (
b65292e) -
Increase the number of concurrent jobs (
602b85a) -
Minimal s3 config example (
22bddc6) -
Normalize tags to use layer number and name (
ea5ef9d) -
Postgres root password config (
34bfb06) -
deps: Add http provider (
0f90970) -
deps: Update uv.lock with the latest datalab version (
5f3c233)
Continuous Integration
-
Add explicit stage to postgres template (
840c4f8) -
Add explicit stages block (
b7ee333) -
Add explicit stages to kafka and ollama templates (
b56bd57) -
Add missing provision stage (
f2091c6) -
Add missing stage (
abe9df5) -
Automated apps deployment (
39facd0) -
Change entry point to force trigger (
9a86ada) -
Change entry point to force trigger (
093a8a9) -
Create_db now returns a credentials.env file that expires in 15 min (
ed66f69) -
Deploy job only runs when services are changed (
cc857df) -
Docker compose stack update (
6785af2) -
Drop intermediary job (
d3649c4) -
Ensure both topics are created (
6668b22) -
Fix changes rule with matches to files (
08c6d1d) -
Fix syntax for manual triggering conditions (
33d667b) -
Implement kafka topic and topic consumer group creation (
cd3f8ea) -
Implement ollama model pull (
2956c09) -
Improve logging messages (
30f9b31) -
Kafka and ollama provisioner templates to use on external projects (
e1acd62) -
Manual trigger option for services and apps deployment (
685ecda) -
Missing variable name for update (
cadf82c) -
Posgres provisioning job template for including on other projects (
2c3008e) -
Postgres user and db creation, kafka initial setup (
1cd1ec2) -
Provision kafka topics and groups explicitly for the mlserver app (
fdaeaa0) -
Refactor so variables are on top (
bfb8847) -
Refactor with more general job names (
c04c149) -
Remove redundant when manual that was blocking the job (
d4f7305) -
Remove unrequired sleep (
1bab0dc) -
Replace before_script with custom ubuntu image (
6271461) -
Reproduce rules from apps deploy job (
f40fde4) -
Rollback to production topic and group names (
2dc2e6b) -
Soft fail when db exists (
82415ba) -
Stub for ollama jobs (
17ee3de) -
Switch back to inline scripts for postgres (
59f7e81) -
Testing docker access (
92c35d1) -
Testing file list (
f6257bd) -
Testing topic and group provisioning (
92eba7a) -
Trigger on changes to the deploy template (
01e1048) -
Upsert behavior for PSQL_SECRETS (
efc00ba) -
fix: Missing backslash (
8627b5c) -
fix: Remove -it from docker command (
eadcdab) -
fix: Single line command (
024fc4a) -
fix: Try again (
eab2623) -
refactor: Clarify log message (
1e2a976) -
refactor: Improve description (
c2f5472) -
refactor: Rename job to init consumer (
bcaedb5)
Documentation
-
Update requirements and quick start (
b1ef974) -
Update with latest workflows and add missing documentation (
ba48b98) -
Add missing requirements.
-
Add missing ml/ and infra/ components.
-
Add missing shared/ modules.
-
Add PosgreSQL dotenv config.
-
Add missing dotenv configs: datalab, DuckLake, Kuzu, Ollama, MLflow, and Kafka.
-
Add docs for CLI test and docs commands.
-
Add docs for ML CLI commands.
-
Add just commands documentation.
Features
-
Add confirmation task, destroy services but never volumes (
43fa345) -
Add custom image for gitlab runner (
91c86ae) -
Add GITLAB_TOKEN to CI/CD variables (
2602d5b) -
Add missing docker configs to use the nvidia gpu (
a005a68) -
Add missing registry config and configure a remote docker gitlab runner (
81784cd) -
Add open webui and switch to plain http on portainer (
4c23e64) -
Add volume to open webui for persistence (
0d4c883) -
Basic Docker VM provisioning (untested) (
1f2f7fd) -
Basic gitlab-terraform project to handle CI/CD variables (
1e04c04) -
Basic Terraform project with stored state using S3 backend (
d14b5cb) -
Basic Terraform setup for provisioning an LXC running MinIO on Proxmox (
60cb6b2) -
Change to official open-webui image and preconfigure the ollama endpoint (
fd56c92) -
Configs now done via gitlab.rb directly, added container registry (
1a79081) -
Data lab infra config check tasks (
b861d95) -
Disable usage tracking and user creation, remove unused gitlab-env, and fix indentation (
4c2333a) -
Dockerized ml server (
544427f) -
Dotenv loading into gitlab ci/cd vars now working (
1dd5691) -
Env var configurable topics and consumer groups (
87d7b55) -
Extract MinIO environment variables from the install script into Terraform and produce random passwords (
d6ab46e) -
First gitlab working deployment, and docker and gitlab now split into separate tf files (
09a7ca0) -
Gpu passthrough for docker-shared VM (
12eecef) -
Improve error control and add credentials printing task (
406f5b8) -
Improve postgres workflow for credentials and db creation (
ac7c573) -
Optional NVIDIA driver install for Docker VMs (cloud-config now a template) (
8a2abfb) -
Overall task cleanup, add infra provisioning for services layer and destruction tasks (
f88158c) -
Postgres deployment (
f8f7db2) -
Preconfigure gitlab container registry as an insecure registry for all docker vms (
5520d35) -
Refactor docker-compose.yml into the services layer compose file, adding portainer and limiting minio to the dev profile (
fcb43e0)
BREAKING CHANGE: There no longer is a docker-compose.yml, as it will be integrated into infra/services/docker/compose.yml with MinIO available only under the dev project.
-
S3 config variables (required by gitlab) (
a544409) -
Set a fixed port for MinIO's console (
78741fe) -
Simplify showing credentials and add validation (
d0a7d36) -
Simplify the way the custom docker context is accessed (
57977f4) -
Tcp listening for remote access (
a0caf5a) -
Update PSQL_SECRETS env var (
0dbd79d)
Refactoring
-
Better defaults, less redundant title comments (
6c3f42b) -
Explicitly use true/false for masked and improve formatting (
4e63668) -
Extract scripts from templates (
cdf5283) -
Fix linting issues (
01bdae2) -
Make it clear that create database can safely fail (
9fff4c1) -
Move services docker files into its own directory (
2ae6fee) -
Normalize comment title formatting (
9b255d3) -
Remove explicit user (
9d2118a) -
Rename the container resource to minio (
3c41783) -
Rename to gitlab (
c2c383c) -
Rollback to inline scripts, move templates to root of dotci (
f0d12b8) -
Split docker vm passwords into multiple outputs again (
23b4cbe) -
Split script into multiple lines for psql_create_db (
10444ff) -
Switch to single rootfs volume and improve resource naming (
f773714)
Detailed Changes: v0.7.0...v1.0.0