diff --git a/README.md b/README.md index 7f3b028..33851dc 100644 --- a/README.md +++ b/README.md @@ -1,7 +1,7 @@ # goseeder #### Motivation -While golang is a great language and getting better, there are still a lot of pieces missing for developing fast and in accurate way to avoid repetitions. +Golang is a great language and getting better as community and frameworks, but there are still a lot of pieces missing for developing fast, accurate way and avoiding repetitions. I was searching for a go seeder similar to the one that Laravel/Lumen provides and could not find one. @@ -18,6 +18,11 @@ For now the library supports only MySql as a database driver for its utility fun - [Installation](#installation) - [Usage](#usage) + - [1. Change Your Main Function](#1-change-your-main-function) + - [2. Registering Your Seeds](#2-registering-your-seeds) + - [3. Run Seeds Only For Specific Env](#3-run-seeds-only-for-specific-env) + - [4. Run Seeds By Name](#4-run-seeds-by-name) +- [Summary Of Cli Args](#summary-of-cli-args) - [License](#license) ## Installation @@ -28,9 +33,255 @@ go get github.com/kristijorgji/goseeder ## Usage -To use goseeder you just need to wrap your main function with `WithSeeder` and specify your seeds under a local package named `seeds` in one folder with structure `db/seeds` (it is important if you want to use json or other data to follow this structure) +[Please check examples/simpleshop](examples/simpleshop) for a full working separate go project that uses the seeder -// TODO full example usage will be uploaded later in this repo +Below I will explain once more all the steps needed to have goseeder up and running for your project. + +### 1. Change Your Main Function + +In order to give your executable seeding abilities and support for its command line arguments, the first thing we have to do is to wrap our main function +with the provided `goseeder.WithSeeder` + +```go +func WithSeeder(conProvider func() *sql.DB, clientMain func()) +``` + +The function requires as argument +1. one function that returns a db connection necessary to seed +2. your original main function, which will get executed if no seed is requested + +One such main file can look like below: + +```go +// main.go +package main + +import ( + "database/sql" + "fmt" + _ "github.com/go-sql-driver/mysql" + "github.com/joho/godotenv" + "github.com/kristijorgji/goseeder" + "log" + "net/url" + "os" + _ "simpleshop/db/seeds" +) + +func main() { + err := godotenv.Load() + if err != nil { + log.Panic("Error loading .env file") + } + + goseeder.WithSeeder(connectToDbOrDie, func() { + myMain() + }) +} + +func myMain() { + fmt.Println("Here you will execute whatever you were doing before using github.com/kristijorgji/goseeder like start your webserver etc") +} + +func connectToDbOrDie() *sql.DB { + dbDriver := os.Getenv("DB_DRIVER") + dbHost := os.Getenv("DB_HOST") + dbPort := os.Getenv("DB_PORT") + dbName := os.Getenv("DB_DATABASE") + dbUser := os.Getenv("DB_USERNAME") + dbPassword := os.Getenv("DB_PASSWORD") + + dbSource := fmt.Sprintf( + "%s:%s@tcp(%s:%s)/%s?parseTime=true", + dbUser, + url.QueryEscape(dbPassword), + dbHost, + dbPort, + dbName, + ) + con, err := sql.Open(dbDriver, dbSource) + if err != nil { + log.Fatalf("Error opening DB: %v", err) + } + + return con +} + +``` + +### 2. Registering Your Seeds + +Great! After step 1 our executable is able to run in seed mode or default mode. + +Now we want to know how to register our custom seeds. + +If you look at the imports in main file from step one, we might notice that we import +`_ "simpleshop/db/seeds"` even though we do not use them directly. + +This is mandatory because our seeds will get registered during package initialisation as we will see later. + +The recommended project folder structure to work properly with `goseeder` is to have the following path for the seeders `db/seeds` and the package name to be `seeds` + +Inside the folder you can add your seeders, for example lets seed some data into the `categories` table from one json file located at `db/seeds/data/categories.json`. + +To do that we create our `categories.go` file at `db/seeds` folder: + +```go +// db/seeds/categories.go +package seeds + +import ( + "github.com/kristijorgji/goseeder" +) + +func categoriesSeeder(s goseeder.Seeder) { + goseeder.FromJson(s, "categories") +} + +``` + +To use this seed, the last step is to register it. + +Seeds can be registered as: +1. `common` seeds that run for all environments +2. for a specific environment like `test`, `yourcustomenv` (more in step 3) + +We are going to create below a seed that runs for all environments, so we will not specify any env while registering it. + +To do that we create in the `db/seeds` folder the file `common.go` that will register seeds that get always executed regardless of the environment: +```go +// db/seeds/common.go +package seeds + +import "github.com/kristijorgji/goseeder" + +func init() { + goseeder.Register(categoriesSeeder) +} +``` + +We used `goseeder.Register` to register our seed function to run for all environments. + +That is all for the basic usage of goseeder!!! + +Our function in categories.go file is now registered and ready to be used. + +Now you can run +``` +go run main.go --gseed +``` + +and it will run all your seeds against the provided db connection. + +The framework will look for `categories.json` file in the path `db/seeds/data`, and insert all the entries there in a table named `categories` (inferred from the file name) + +### 3. Run Seeds Only For Specific Env + +Many times we want to have seeds only for `test` environment, test purpose and want to avoid having thousand of randomly generated rows inserted into production database by mistake! + +Or we just want granular control, to have separate data to populate our app/web in different way for `staging` `prod` `yourcustomenv` and so on. + +goseeder is designed to take care of this by using one of the following methods: + +- `goseeder.RegisterForTest(seeder func(s Seeder)` - registers the specified seed for the env named `test` +- `goseeder.RegisterForEnv(env string, seeder func(s Seeder))` - will register your seeder to be executed only for the custom specified env + +Let's add to our previous categories.go seeder one seed function specific only for test env! +The file now will look like: + +```go +package seeds + +import ( + "fmt" + "github.com/kristijorgji/goseeder" + "simpleshop/util" +) + +func categoriesSeeder(s goseeder.Seeder) { + goseeder.FromJson(s, "categories") +} + +func testCategoriesSeeder(s goseeder.Seeder) { + for i := 0; i < 100; i++ { + stmt, _ := s.DB.Prepare(`INSERT INTO categories(id, name) VALUES (?,?)`) + _, err := stmt.Exec(util.RandomInt(1, int64(^uint16(0))), []byte(fmt.Sprintf(`{"en": "%s"}`, util.RandomString(7)))) + if err != nil { + panic(err) + } + } +} + +``` + +Finally, lets create our registrator file for all test seeds same way as we did with `common.go`, we will create `test.go` now with content as below: + +```go +// db/seeds/test.go +package seeds + +import "github.com/kristijorgji/goseeder" + +func init() { + goseeder.RegisterForTest(testCategoriesSeeder) +} + +``` + +That is all! + +Now if you run your app without specifying test env, only the common env seeders will run and you cannot mess by mistake production or other environments! + +To run the test seeder above you have to run: + +```bash +go run main.go --gseed --gsenv=test +``` + +### 4. Run Seeds By Name + +When we register a seed like shown in step 2, the seed name is the same as the function name, so our seed is called `categoriesSeeder` because that is the name of the function we register below + +```go +func init() { + goseeder.Register(categoriesSeeder) +} +``` + +This is important because we are totally flexible and can do cool things like execute only the specific seed functions that we want! + +Let's assume that we have 100 seed functions, and want to execute only one of them which is named categoriesSeeder (that we registered above) and ignore all the other seeds. + +Easy as this, just run: + +```bash +go run main.go --gseeder --gsnames=categoriesSeeder +``` + +If you want to execute multiple seeds by specifying their names, just use comma separated value like `--gsnames=categoriesSeeder,someOtherSeeder` + +## Summary Of Cli Args + +You can always run +```bash +run go run main.go --help +```` + +to see all the available arguments and their descriptions. + +For the current version the result is: + +```bash +INR00009:simpleshop kristi.jorgji$ go run main.go --help +Usage of /var/folders/rd/2bkszcpx6xgcddpn7f3bhczn1m9fb7/T/go-build810543742/b001/exe/main: + -gseed + goseeder - if set will seed + -gsenv string + goseeder - env for which seeds to execute + -gsnames string + goseeder - comma separated seeder names to run specific ones +INR00009:simpleshop kristi.jorgji$ +``` ## License diff --git a/examples/simpleshop/.env.dist b/examples/simpleshop/.env.dist new file mode 100644 index 0000000..9626282 --- /dev/null +++ b/examples/simpleshop/.env.dist @@ -0,0 +1,8 @@ +#LOCAL DB + +DB_DRIVER=mysql +DB_HOST=127.0.0.1 +DB_PORT=3306 +DB_DATABASE=goseeder_simpleshop +DB_USERNAME=root +DB_PASSWORD=verySecretPassword \ No newline at end of file diff --git a/examples/simpleshop/.gitignore b/examples/simpleshop/.gitignore new file mode 100644 index 0000000..08e51fc --- /dev/null +++ b/examples/simpleshop/.gitignore @@ -0,0 +1,3 @@ +/.idea + +.env diff --git a/examples/simpleshop/README.md b/examples/simpleshop/README.md new file mode 100644 index 0000000..ff82820 --- /dev/null +++ b/examples/simpleshop/README.md @@ -0,0 +1,25 @@ +# simpleshop example + +This is one project example of using github.com/kristijorgji/goseeder + +To have this example project up and running follow these steps: +1. you need to only create the database, just by executing the content of +`db/migrations/000001_init_schema.up.sql` against your database server +2. Then you need to create one .env file by copying the example .env.dist, and inside update the db credentials + + +Then in order to seed against run +```bash +go run main.go --gseed +``` + +and you will run all seeds in this example project. + +If you do not want to seed, but run the original main logic of the project run without the gseed argument like: + + ```bash + go run main.go + ``` + + +For full documentation of usage check the main documentation of github.com/kristijorgji/goseeder \ No newline at end of file diff --git a/examples/simpleshop/db/migrations/000001_init_schema.up.sql b/examples/simpleshop/db/migrations/000001_init_schema.up.sql new file mode 100644 index 0000000..8984f72 --- /dev/null +++ b/examples/simpleshop/db/migrations/000001_init_schema.up.sql @@ -0,0 +1,19 @@ +create schema if not exists goseeder_simpleshop collate utf8_general_ci; + +create table categories +( + id smallint unsigned auto_increment + primary key, + name json not null, + created_at timestamp default CURRENT_TIMESTAMP not null, + updated_at timestamp default CURRENT_TIMESTAMP not null on update CURRENT_TIMESTAMP +); + +create table products +( + id char(36) not null + primary key, + name json not null, + created_at timestamp default CURRENT_TIMESTAMP not null, + updated_at timestamp default CURRENT_TIMESTAMP not null on update CURRENT_TIMESTAMP +); diff --git a/examples/simpleshop/db/seeds/categories.go b/examples/simpleshop/db/seeds/categories.go new file mode 100644 index 0000000..83f29a0 --- /dev/null +++ b/examples/simpleshop/db/seeds/categories.go @@ -0,0 +1,21 @@ +package seeds + +import ( + "fmt" + "github.com/kristijorgji/goseeder" + "simpleshop/util" +) + +func categoriesSeeder(s goseeder.Seeder) { + goseeder.FromJson(s, "categories") +} + +func testCategoriesSeeder(s goseeder.Seeder) { + for i := 0; i < 100; i++ { + stmt, _ := s.DB.Prepare(`INSERT INTO categories(id, name) VALUES (?,?)`) + _, err := stmt.Exec(util.RandomInt(1, int64(^uint16(0))), []byte(fmt.Sprintf(`{"en": "%s"}`, util.RandomString(7)))) + if err != nil { + panic(err) + } + } +} diff --git a/examples/simpleshop/db/seeds/common.go b/examples/simpleshop/db/seeds/common.go new file mode 100644 index 0000000..5833934 --- /dev/null +++ b/examples/simpleshop/db/seeds/common.go @@ -0,0 +1,8 @@ +package seeds + +import "github.com/kristijorgji/goseeder" + +func init() { + goseeder.Register(categoriesSeeder) + goseeder.Register(productsSeeder) +} diff --git a/examples/simpleshop/db/seeds/data/categories.json b/examples/simpleshop/db/seeds/data/categories.json new file mode 100644 index 0000000..08dbc78 --- /dev/null +++ b/examples/simpleshop/db/seeds/data/categories.json @@ -0,0 +1,10 @@ +[ + { + "id": "1", + "name": "{\"en\": \"Push\"}" + }, + { + "id": "2", + "name": "{\"en\": \"Pull\"}" + } +] \ No newline at end of file diff --git a/examples/simpleshop/db/seeds/data/products.json b/examples/simpleshop/db/seeds/data/products.json new file mode 100644 index 0000000..ceffa47 --- /dev/null +++ b/examples/simpleshop/db/seeds/data/products.json @@ -0,0 +1,10 @@ +[ + { + "id": "0055bd0b-8629-490d-8552-906d04dfca84", + "name": "{\"en\": \"Super Smartphone\"}" + }, + { + "id": "0162e545-2678-4cde-a291-665dcf6251a4", + "name": "{\"en\": \"Fitness Tracker Fitbit Charge 4, black\"}" + } +] \ No newline at end of file diff --git a/examples/simpleshop/db/seeds/products.go b/examples/simpleshop/db/seeds/products.go new file mode 100644 index 0000000..8292984 --- /dev/null +++ b/examples/simpleshop/db/seeds/products.go @@ -0,0 +1,25 @@ +package seeds + +import ( + "simpleshop/util" + "fmt" + "github.com/google/uuid" + "github.com/kristijorgji/goseeder" +) + +func productsSeeder(s goseeder.Seeder) { + goseeder.FromJson(s, "products") +} + +func testExercisesSeeder(s goseeder.Seeder) { + for i := 0; i < 100; i++ { + stmt, _ := s.DB.Prepare(`INSERT INTO products(id, name) VALUES (?, ?)`) + _, err := stmt.Exec( + uuid.New().String(), + []byte(fmt.Sprintf(`{"en": "%s"}`, util.RandomString(7))), + ) + if err != nil { + panic(err) + } + } +} diff --git a/examples/simpleshop/db/seeds/test.go b/examples/simpleshop/db/seeds/test.go new file mode 100644 index 0000000..b324097 --- /dev/null +++ b/examples/simpleshop/db/seeds/test.go @@ -0,0 +1,8 @@ +package seeds + +import "github.com/kristijorgji/goseeder" + +func init() { + goseeder.RegisterForTest(testCategoriesSeeder) + goseeder.RegisterForTest(testExercisesSeeder) +} diff --git a/examples/simpleshop/go.mod b/examples/simpleshop/go.mod new file mode 100644 index 0000000..57d38a9 --- /dev/null +++ b/examples/simpleshop/go.mod @@ -0,0 +1,10 @@ +module simpleshop + +go 1.14 + +require ( + github.com/go-sql-driver/mysql v1.5.0 + github.com/google/uuid v1.1.2 + github.com/joho/godotenv v1.3.0 + github.com/kristijorgji/goseeder v1.0.0 +) diff --git a/examples/simpleshop/go.sum b/examples/simpleshop/go.sum new file mode 100644 index 0000000..0640432 --- /dev/null +++ b/examples/simpleshop/go.sum @@ -0,0 +1,19 @@ +github.com/davecgh/go-spew v1.1.0 h1:ZDRjVQ15GmhC3fiQ8ni8+OwkZQO4DARzQgrnXU1Liz8= +github.com/davecgh/go-spew v1.1.0/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38= +github.com/go-sql-driver/mysql v1.5.0 h1:ozyZYNQW3x3HtqT1jira07DN2PArx2v7/mN66gGcHOs= +github.com/go-sql-driver/mysql v1.5.0/go.mod h1:DCzpHaOWr8IXmIStZouvnhqoel9Qv2LBy8hT2VhHyBg= +github.com/google/uuid v1.1.2 h1:EVhdT+1Kseyi1/pUmXKaFxYsDNy9RQYkMWRH68J/W7Y= +github.com/google/uuid v1.1.2/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo= +github.com/joho/godotenv v1.3.0 h1:Zjp+RcGpHhGlrMbJzXTrZZPrWj+1vfm90La1wgB6Bhc= +github.com/joho/godotenv v1.3.0/go.mod h1:7hK45KPybAkOC6peb+G5yklZfMxEjkZhHbwpqxOKXbg= +github.com/kristijorgji/goseeder v1.0.0 h1:u9Ds+xCuBiD269w2M4BtD8/yR3d4hMXRQrm6A4Jien4= +github.com/kristijorgji/goseeder v1.0.0/go.mod h1:G8kIXQODvaUXQ+EL+3oE+OqtEx1L1D39ZSHUzqKQ/MU= +github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM= +github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4= +github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME= +github.com/stretchr/testify v1.6.1 h1:hDPOHmpOpP40lSULcqw7IrRb/u7w6RpDC9399XyoNd0= +github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405 h1:yhCVgyC4o1eVCa2tZl7eS0r+SDo693bJlVdllGtEeKM= +gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0= +gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c h1:dUUwHk2QECo/6vqA44rthZ8ie2QXMNeKRTHCNY2nXvo= +gopkg.in/yaml.v3 v3.0.0-20200313102051-9f266ea9e77c/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM= diff --git a/examples/simpleshop/main.go b/examples/simpleshop/main.go new file mode 100644 index 0000000..3f614fa --- /dev/null +++ b/examples/simpleshop/main.go @@ -0,0 +1,52 @@ +package main + +import ( + "database/sql" + "fmt" + _ "github.com/go-sql-driver/mysql" + "github.com/joho/godotenv" + "github.com/kristijorgji/goseeder" + "log" + "net/url" + "os" + _ "simpleshop/db/seeds" +) + +func main() { + err := godotenv.Load() + if err != nil { + log.Panic("Error loading .env file") + } + + goseeder.WithSeeder(connectToDbOrDie, func() { + myMain() + }) +} + +func myMain() { + fmt.Println("Here you will execute whatever you were doing before using github.com/kristijorgji/goseeder like start your webserver etc") +} + +func connectToDbOrDie() *sql.DB { + dbDriver := os.Getenv("DB_DRIVER") + dbHost := os.Getenv("DB_HOST") + dbPort := os.Getenv("DB_PORT") + dbName := os.Getenv("DB_DATABASE") + dbUser := os.Getenv("DB_USERNAME") + dbPassword := os.Getenv("DB_PASSWORD") + + dbSource := fmt.Sprintf( + "%s:%s@tcp(%s:%s)/%s?parseTime=true", + dbUser, + url.QueryEscape(dbPassword), + dbHost, + dbPort, + dbName, + ) + con, err := sql.Open(dbDriver, dbSource) + if err != nil { + log.Fatalf("Error opening DB: %v", err) + } + + return con +} diff --git a/examples/simpleshop/util/random.go b/examples/simpleshop/util/random.go new file mode 100644 index 0000000..f27a04a --- /dev/null +++ b/examples/simpleshop/util/random.go @@ -0,0 +1,31 @@ +package util + +import ( + "math/rand" + "strings" + "time" +) + +const alphabet = "abcdefghijklmnopqrstuvwxyz" + +func init() { + rand.Seed(time.Now().UnixNano()) +} + +// RandomInt generates a random integer between min and max +func RandomInt(min, max int64) int64 { + return min + rand.Int63n(max-min+1) +} + +// RandomString generates a random string of length n +func RandomString(n int) string { + var sb strings.Builder + k := len(alphabet) + + for i := 0; i < n; i++ { + c := alphabet[rand.Intn(k)] + sb.WriteByte(c) + } + + return sb.String() +}