Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding ability to configure extensions for Sqlite #428

Merged
merged 16 commits into from
Feb 18, 2025
Merged

Conversation

aaronpowell
Copy link
Member

@aaronpowell aaronpowell commented Jan 29, 2025

Adding the ability to specify SQLite extensions that you want to load on the database. Extensions can be shipped as NuGet packages (https://www.nuget.org/packages/mod_spatialite is an example) or you can specify a path to the extension binary (an absolute path).

This is done in the app host and the info is projected as part of the connection string for the client.

In the client library, it'll read the extensions from the connection string (then delete that value so it is a valid connection string) and load the extensions using the SqliteConnection. Since SQLite needs the extension to be in PATH (see this doc) there is a bit of code to ensure the path for the binary is loaded correctly.

I've added some tests for observing that the extension loads, but it's a hard one to write tests for as you have to have a bunch of different binaries checked into the repo (I'd use mod_spatialite but it only ships Windows binaries in the nuget package). Because if this, I've marked the feature as experimental.

This doesn't support the EF SQLite integration as there's no way I can find using the EF API in which you can add extensions, you have to do it manually when the DbContext is created.

@aaronpowell aaronpowell requested a review from Copilot January 29, 2025 23:56
Copy link
Contributor

@Copilot Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot reviewed 5 out of 9 changed files in this pull request and generated 2 comments.

Files not reviewed (4)
  • src/CommunityToolkit.Aspire.Hosting.Sqlite/PublicAPI.Unshipped.txt: Language not supported
  • src/CommunityToolkit.Aspire.Microsoft.Data.Sqlite/PublicAPI.Unshipped.txt: Language not supported
  • src/CommunityToolkit.Aspire.Hosting.Sqlite/SqliteResource.cs: Evaluated as low risk
  • src/CommunityToolkit.Aspire.Hosting.Sqlite/SqliteResourceBuilderExtensions.cs: Evaluated as low risk

Copy link
Member

@Alirexaa Alirexaa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@aaronpowell, I understand that writing functional tests for this situation is challenging, but I believe we need to proceed with it. For now, we could create tests that run only on Windows, but we should definitely include some functional tests. Additionally, we might want to update the example project to utilize the one extension, which I could then attempt to test locally.

@aaronpowell
Copy link
Member Author

I'll have to think about a functional test because yes, it needs to run on Windows only (which means I can't actually run the tests myself).

@aaronpowell
Copy link
Member Author

@Alirexaa I've created a test that runs on Windows only to verify the extension loading, but as you can see, it doesn't pass on GitHub Actions.

Running locally, the test passes:

Passing test

Can you run the test on your machine and see if it passes? I'm not sure why it's failing, looking at the logging I've added it's finding the right location to load the extension from (it comes from D:\a\Aspire\Aspire\tests\CommunityToolkit.Aspire.Hosting.Sqlite.Tests\bin\Release\net8.0\runtimes\win-x64\native), so I wonder if there's something more with how the test runner is running and doing isolation, and we're going to either have to do a lot of workarounds or drop this test (could have it run locally but not in CI).

@Alirexaa
Copy link
Member

Can you run the test on your machine and see if it passes?

I'll try.

@aaronpowell
Copy link
Member Author

Apparently you can load extensions from an absolute path, so I'll try that instead as it might simplify the code and maybe fix our issue.

@aaronpowell aaronpowell marked this pull request as draft February 11, 2025 23:44
@aaronpowell
Copy link
Member Author

Going to put this back to draft, as the package that I'm using seems to be really out of date and it's not meant to be used the way that I was thinking.

I actually am more interested in https://github.com/asg017/sqlite-vec but we depend on asg017/sqlite-vec#193

@Alirexaa
Copy link
Member

@aaronpowell, test passed locally. I think there is a limitation in the runner.

@aaronpowell
Copy link
Member Author

@aaronpowell, test passed locally. I think there is a limitation in the runner.

That's my guess. Are you ok if we merge without tests, as I'm going to be looking into the sqlite-vec plugin and shipping it via NuGet which I'd then migrate to instead.

Copy link
Member

@Alirexaa Alirexaa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's keep this test and skip it for now. Overall, it looks good to me.

@aaronpowell aaronpowell marked this pull request as ready for review February 18, 2025 05:52
@aaronpowell aaronpowell merged commit a8d1717 into main Feb 18, 2025
5 checks passed
@aaronpowell aaronpowell deleted the sqlite-extensions branch February 18, 2025 05:52
Copy link

Code Coverage

Package Line Rate Branch Rate Complexity Health
CommunityToolkit.Aspire.EventStore 100% 100% 46
CommunityToolkit.Aspire.GoFeatureFlag 100% 97% 74
CommunityToolkit.Aspire.Hosting.ActiveMQ 70% 27% 144
CommunityToolkit.Aspire.Hosting.ActiveMQ.MassTransit 1% 0% 14
CommunityToolkit.Aspire.Hosting.Azure.DataApiBuilder 100% 100% 22
CommunityToolkit.Aspire.Hosting.Azure.StaticWebApps 100% 100% 28
CommunityToolkit.Aspire.Hosting.Bun 81% 71% 54
CommunityToolkit.Aspire.Hosting.Dapr 60% 51% 560
CommunityToolkit.Aspire.Hosting.Dapr.AzureRedis 93% 71% 54
CommunityToolkit.Aspire.Hosting.DbGate 63% 11% 62
CommunityToolkit.Aspire.Hosting.Deno 84% 75% 72
CommunityToolkit.Aspire.Hosting.EventStore 90% 71% 62
CommunityToolkit.Aspire.Hosting.GoFeatureFlag 59% 11% 62
CommunityToolkit.Aspire.Hosting.Golang 94% 50% 16
CommunityToolkit.Aspire.Hosting.Java 69% 72% 120
CommunityToolkit.Aspire.Hosting.Meilisearch 61% 27% 94
CommunityToolkit.Aspire.Hosting.MongoDB.Extensions 65% 38% 80
CommunityToolkit.Aspire.Hosting.Ngrok 52% 35% 82
CommunityToolkit.Aspire.Hosting.NodeJS.Extensions 90% 68% 92
CommunityToolkit.Aspire.Hosting.Ollama 65% 64% 198
CommunityToolkit.Aspire.Hosting.PapercutSmtp 92% 50% 10
CommunityToolkit.Aspire.Hosting.PostgreSQL.Extensions 70% 45% 88
CommunityToolkit.Aspire.Hosting.Python.Extensions 69% 50% 86
CommunityToolkit.Aspire.Hosting.RavenDB 64% 49% 176
CommunityToolkit.Aspire.Hosting.Redis.Extensions 68% 38% 80
CommunityToolkit.Aspire.Hosting.Rust 94% 83% 16
CommunityToolkit.Aspire.Hosting.SqlDatabaseProjects 74% 61% 114
CommunityToolkit.Aspire.Hosting.Sqlite 96% 96% 50
CommunityToolkit.Aspire.MassTransit.RabbitMQ 100% 100% 30
CommunityToolkit.Aspire.Meilisearch 97% 92% 68
CommunityToolkit.Aspire.Microsoft.Data.Sqlite 37% 27% 194
CommunityToolkit.Aspire.Microsoft.EntityFrameworkCore.Sqlite 51% 50% 92
CommunityToolkit.Aspire.OllamaSharp 72% 69% 98
CommunityToolkit.Aspire.RavenDB.Client 60% 53% 237
Summary 70% (4819 / 6901) 53% (1468 / 2786) 3275

Minimum allowed line rate is 60%

@krwq
Copy link

krwq commented Feb 19, 2025

@aaronpowell I believe EnsureLoadableFromNuGet won't be needed after dotnet/efcore#35617 is merged

@aaronpowell
Copy link
Member Author

@aaronpowell I believe EnsureLoadableFromNuGet won't be needed after dotnet/efcore#35617 is merged

Nice - I'll keep an eye on that being released.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants