Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[1/n] move db schema into its own crate #7880

Merged

Conversation

sunshowers
Copy link
Contributor

@sunshowers sunshowers commented Mar 27, 2025

While working on Diesel-related code, I noticed that schema generation is both
very large (~450k lines of code currently), and also can be separated out
into its own crate. Spent a couple of hours doing so and I think I quite like
the result! The most important thing here is that the schema code is completely
independent of the rest of Omicron, so it can be compiled in parallel with
crates like omicron-common.

For me, a full cargo check --all-targets in Omicron goes from 2m04s to 1m51s,
which is a really nice speedup. Incremental builds should also be faster due to
the parallelization and reduction of crate size. (But this doesn't really touch
nexus-db-queries, which is the worst offender at the moment.)

Created using spr 1.3.6-beta.1
};
}

define_enums! {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Enum definitions are here.

@@ -6,6 +6,8 @@
//!
//! NOTE: Should be kept up-to-date with dbinit.sql.

use diesel::{allow_tables_to_appear_in_same_query, joinable, table};
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the schema -- it's moved with some minor changes.

#[derive(SqlType, Debug)]
#[diesel(postgres_type(name = "dns_group", schema = "public"))]
pub struct DnsGroupEnum;
DnsGroupEnum:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Example of a conversion.

Comment on lines 300 to +302
macro_rules! impl_enum_type {
(
$(#[$enum_meta:meta])*
pub struct $diesel_type:ident;
$diesel_type:ident:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These macros change slightly.

Copy link
Collaborator

@smklein smklein left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I know you're still working through this, but I like the direction. It makes sense to me that "if the queries change, but the schemas don't, we should leave the schema-compiling code alone"

Created using spr 1.3.6-beta.1
@sunshowers sunshowers requested a review from smklein March 28, 2025 21:21
@sunshowers sunshowers changed the title [RFC] move db schema into its own crate move db schema into its own crate Mar 28, 2025
@sunshowers sunshowers marked this pull request as draft March 28, 2025 22:21
@sunshowers
Copy link
Contributor Author

(need to fix up tests still)

Created using spr 1.3.6-beta.1
Created using spr 1.3.6-beta.1
@sunshowers sunshowers marked this pull request as ready for review March 29, 2025 01:39
@@ -86,3 +95,11 @@ define_enums! {
VpcRouterKindEnum => "vpc_router_kind",
ZoneTypeEnum => "zone_type",
}

define_enums_non_static! {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm assuming this is just 'moving' the definitions from before, but is there a reason we have these enums as "non-static" and the others as static?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm curious, but I haven't investigated in depth -- wanted to keep this PR without any functional changes.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#7897 is a followup which makes that change.

@sunshowers sunshowers changed the title move db schema into its own crate [1/n] move db schema into its own crate Mar 31, 2025
Created using spr 1.3.6-beta.1
@sunshowers sunshowers enabled auto-merge (squash) April 1, 2025 00:06
@sunshowers sunshowers merged commit aa9d990 into main Apr 1, 2025
18 checks passed
@sunshowers sunshowers deleted the sunshowers/spr/rfc-move-db-schema-into-its-own-crate branch April 1, 2025 08:55
sunshowers added a commit that referenced this pull request Apr 1, 2025
…7897)

This is meant to be false if and only if types don't match 1:1 to
queries, e.g. with CTEs. These types do match 1:1 to queries, so using
`HAS_STATIC_QUERY_ID` should be fine.

There's a small possibility some code was inadvertently depending on
`HAS_STATIC_QUERY_ID` being false, but tests should hopefully catch
that.

Depends on #7880.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants