Merge pull request #202 from rails/change-database-yml

Change database yml directly and double down on separate DB

Author: dhh

README.md

@@ -4,76 +4,15 @@ Solid Cache is a database-backed Active Support cache store that let's you keep
 
 ## Installation
 
-Solid Cache is configured by default in new Rails 8 applications. But if you're running an earlier version, you can add it manually using `bundle add solid_cache`.
+Solid Cache is configured by default in new Rails 8 applications. But if you're running an earlier version, you can add it manually following these steps:
 
-#### Cache database configuration
+1. `bundle add solid_cache`
+2. `bin/rails solid_cache:install`
+3. `bin/rails db:migrate`
 
-The default installation of Solid Cache expects a database named `cache` in `database.yml`. It should
-have it's own connection pool to avoid mixing cache queries in other transactions.
+This will configure Solid Queue as the production cache store, create `config/solid_cache.yml`, and alter `config/database.yml` to include the new cache database.
 
-You can use the primary database for your cache like this:
-
-```yaml
-# config/database.yml
-production:
-  primary: &production_primary
-    ...
-  cache:
-    <<: *production_primary
-```
-
-Or a separate database like this:
-
-```yaml
-production:
-  primary:
-    ...
-  cache:
-    database: cache_development
-    host: 127.0.0.1
-    migrations_paths: "db/cache/migrate"
-```
-
-#### Install Solid Cache
-
-Now, you need to install the necessary migrations and configure the cache store. You can do both at once using the provided generator:
-
-```bash
-# If using the primary database
-$ bin/rails generate solid_cache:install
-
-# Or if using a dedicated database
-$ DATABASE=cache bin/rails generate solid_cache:install
-```
-
-This will set solid_cache as the cache store in production, and will copy the optional configuration file and the required migration over to your app.
-
-Alternatively, you can add only the migration to your app:
-
-```bash
-# If using the primary database
-$ bin/rails generate solid_cache:install:migrations
-
-# Or if using a dedicated database
-$ DATABASE=cache bin/rails generate solid_cache:install:migrations
-```
-
-And set Solid Cache as your application's cache store backend manually, in your environment config:
-
-```ruby
-# config/environments/production.rb
-config.cache_store = :solid_cache_store
-```
-
-#### Run migrations
-
-Finally, you need to run the migrations:
-
-```bash
-$ bin/rails db:migrate
-```
-
-### Configuration
+## Configuration
 
 Configuration will be read from `config/solid_cache.yml`. You can change the location of the config file by setting the `SOLID_CACHE_CONFIG` env variable.
 
@@ -101,7 +40,7 @@ production: &production
 
 For the full list of keys for `store_options` see [Cache configuration](#cache-configuration). Any options passed to the cache lookup will overwrite those specified here.
 
-#### Connection configuration
+### Connection configuration
 
 You can set one of `database`, `databases` and `connects_to` in the config file. They will be used to configure the cache databases in `SolidCache::Record#connects_to`.
 
@@ -119,10 +58,9 @@ SolidCache::Record.connects_to shards: { cache_db1: { writing: :cache_db1 },  ca
 
 If `connects_to` is set, it will be passed directly.
 
-If none of these are set, Solid Cache will use the `ActiveRecord::Base` connection pool. This means that cache reads and writes will be part of any wrapping
-database transaction.
+If none of these are set, Solid Cache will use the `ActiveRecord::Base` connection pool. This means that cache reads and writes will be part of any wrapping database transaction.
 
-#### Engine configuration
+### Engine configuration
 
 There are three options that can be set on the engine:
 
@@ -140,7 +78,7 @@ Rails.application.configure do
 end
 ```
 
-#### Cache configuration
+### Cache configuration
 
 Solid Cache supports these options in addition to the standard `ActiveSupport::Cache::Store` options:
 
@@ -160,7 +98,7 @@ Solid Cache supports these options in addition to the standard `ActiveSupport::C
 
 For more information on cache clusters, see [Sharding the cache](#sharding-the-cache)
 
-### Cache expiry
+## Cache expiry
 
 Solid Cache tracks writes to the cache. For every write it increments a counter by 1. Once the counter reaches 50% of the `expiry_batch_size` it adds a task to run on a background thread. That task will:
 
@@ -176,7 +114,7 @@ Only triggering expiry when we write means that if the cache is idle, the backgr
 
 If you want the cache expiry to be run in a background job instead of a thread, you can set `expiry_method` to `:job`. This will enqueue a `SolidCache::ExpiryJob`.
 
-### Sharding the cache
+## Sharding the cache
 
 Solid Cache uses the [Maglev](https://static.googleusercontent.com/media/research.google.com/en//pubs/archive/44824.pdf) consistent hashing scheme to shard the cache across multiple databases.
 
@@ -207,7 +145,7 @@ production:
   databases: [cache_shard1, cache_shard2, cache_shard3]
 ```
 
-### Enabling encryption
+## Enabling encryption
 
 To encrypt the cache values, you can add set the encrypt property.
 
@@ -245,7 +183,7 @@ config.solid_cache.encryption_context_properties = {
 Encryption currently does not work for PostgreSQL, as Rails does not yet support encrypting binary columns for it.
 See https://github.com/rails/rails/pull/52650.
 
-### Index size limits
+## Index size limits
 The Solid Cache migrations try to create an index with 1024 byte entries. If that is too big for your database, you should:
 
 1. Edit the index size in the migration.

db/migrate/20240820123641_create_solid_cache_entries.rb

@@ -3,22 +3,70 @@
 class SolidCache::InstallGenerator < Rails::Generators::Base
   source_root File.expand_path("templates", __dir__)
 
-  class_option :skip_migrations,    type: :boolean, default: nil,
-                                    desc: "Skip migrations"
-
   def add_rails_cache
-    if (env_config = Pathname(destination_root).join("config/environments/production.rb")).exist?
-      gsub_file env_config, /(# )?config\.cache_store = (:.*)/, "config.cache_store = :solid_cache_store"
-    end
+    gsub_file app_root.join("config/environments/production.rb"),
+      /(# )?config\.cache_store = (:.*)/, "config.cache_store = :solid_cache_store"
   end
 
   def create_config_solid_cache_yml
     template "config/solid_cache.yml"
   end
 
-  def create_migrations
-    unless options[:skip_migrations]
-      rails_command "railties:install:migrations FROM=solid_cache", inline: true
+  def add_cache_db_to_database_yml
+    if app_is_using_sqlite?
+      gsub_file database_yml, /production:\s*<<: \*default.*/m, sqlite_database_config_with_cache
+    else
+      gsub_file database_yml, /production:\s*<<: \*default.*/m, generic_database_config_with_cache
     end
   end
+
+  def add_solid_cache_db_schema
+    template "db/cache_schema.rb"
+  end
+
+  private
+    def app_root
+      Pathname.new(destination_root)
+    end
+
+    def database_yml
+      app_root.join("config/database.yml")
+    end
+
+    def app_is_using_sqlite?
+      database_yml.read.match?(/production:.*sqlite3/m)
+    end
+
+    def sqlite_database_config_with_cache
+      <<~YAML
+        production:
+          primary:
+            <<: *default
+            database: storage/production.sqlite3
+          cache:
+            <<: *default
+            database: storage/production_cache.sqlite3
+            migrations_paths: db/cache_migrate
+      YAML
+    end
+
+    def app_name_from_production_database_name
+      database_yml.read.scan(/database: (\w+)_production/).flatten.first
+    end
+
+    def generic_database_config_with_cache
+      app_name = app_name_from_production_database_name
+
+      <<~YAML
+production:
+  primary: &production_primary
+    <<: *default
+    database: #{app_name}_production
+    username: #{app_name}
+    password: <%= ENV["#{app_name.upcase}_DATABASE_PASSWORD"] %>
+  cache:
+    <<: *production_primary
+    database: #{app_name}_production_cache
+      YAML
+    end
 end

lib/generators/solid_cache/install/install_generator.rb

@@ -0,0 +1,14 @@
+# frozen_string_literal: true
+
+ActiveRecord::Schema[8.0].define(version: 1) do
+  create_table "solid_cache_entries", force: :cascade do |t|
+    t.binary "key", limit: 1024, null: false
+    t.binary "value", limit: 536870912, null: false
+    t.datetime "created_at", null: false
+    t.integer "key_hash", limit: 8, null: false
+    t.integer "byte_size", limit: 4, null: false
+    t.index ["byte_size"], name: "index_solid_cache_entries_on_byte_size"
+    t.index ["key_hash", "byte_size"], name: "index_solid_cache_entries_on_key_hash_and_byte_size"
+    t.index ["key_hash"], name: "index_solid_cache_entries_on_key_hash", unique: true
+  end
+end

lib/generators/solid_cache/install/templates/db/cache_schema.rb

@@ -0,0 +1,59 @@
+# MySQL. Versions 5.5.8 and up are supported.
+#
+# Install the MySQL driver
+#   gem install mysql2
+#
+# Ensure the MySQL gem is defined in your Gemfile
+#   gem "mysql2"
+#
+# And be sure to use new-style password hashing:
+#   https://dev.mysql.com/doc/refman/5.7/en/password-hashing.html
+#
+default: &default
+  adapter: mysql2
+  encoding: utf8mb4
+  pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
+  username: root
+  password:
+  host: <%= ENV.fetch("DB_HOST") { "127.0.0.1" } %>
+
+development:
+  <<: *default
+  database: bongo_development
+
+# Warning: The database defined as "test" will be erased and
+# re-generated from your development database when you run "rake".
+# Do not set this db to the same as development or production.
+test:
+  <<: *default
+  database: bongo_test
+
+# As with config/credentials.yml, you never want to store sensitive information,
+# like your database password, in your source code. If your source code is
+# ever seen by anyone, they now have access to your database.
+#
+# Instead, provide the password or a full connection URL as an environment
+# variable when you boot the app. For example:
+#
+#   DATABASE_URL="mysql2://myuser:mypass@localhost/somedatabase"
+#
+# If the connection URL is provided in the special DATABASE_URL environment
+# variable, Rails will automatically merge its configuration values on top of
+# the values provided in this file. Alternatively, you can specify a connection
+# URL environment variable explicitly:
+#
+#   production:
+#     url: <%= ENV["MY_APP_DATABASE_URL"] %>
+#
+# Read https://guides.rubyonrails.org/configuring.html#configuring-a-database
+# for a full overview on how database connection configuration can be specified.
+#
+production:
+  primary: &production_primary
+    <<: *default
+    database: bongo_production
+    username: bongo
+    password: <%= ENV["BONGO_DATABASE_PASSWORD"] %>
+  cache:
+    <<: *production_primary
+    database: bongo_production_cache

test/configs/mysql-database.yml

@@ -0,0 +1,28 @@
+# SQLite. Versions 3.8.0 and up are supported.
+#   gem install sqlite3
+#
+#   Ensure the SQLite 3 gem is defined in your Gemfile
+#   gem "sqlite3"
+#
+default: &default
+  adapter: sqlite3
+  pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
+  timeout: 5000
+
+development:
+  <<: *default
+  database: storage/development.sqlite3
+
+# Warning: The database defined as "test" will be erased and
+# re-generated from your development database when you run "rake".
+# Do not set this db to the same as development or production.
+test:
+  <<: *default
+  database: storage/test.sqlite3
+
+
+# Store production database in the storage/ directory, which by default
+# is mounted as a persistent Docker volume in config/deploy.yml.
+production:
+  <<: *default
+  database: storage/production.sqlite3