Skip to content

Commit

Permalink
add usage documentation and one example project of using goseeder
Browse files Browse the repository at this point in the history
  • Loading branch information
kristijorgji committed Oct 22, 2020
1 parent 4566f4c commit dac5610
Show file tree
Hide file tree
Showing 15 changed files with 503 additions and 3 deletions.
257 changes: 254 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# goseeder

#### Motivation
While golang is a great language and getting better, there are still a lot of pieces missing for developing fast and in accurate way to avoid repetitions.
Golang is a great language and getting better as community and frameworks, but there are still a lot of pieces missing for developing fast, accurate way and avoiding repetitions.

I was searching for a go seeder similar to the one that Laravel/Lumen provides and could not find one.

Expand All @@ -18,6 +18,11 @@ For now the library supports only MySql as a database driver for its utility fun

- [Installation](#installation)
- [Usage](#usage)
- [1. Change Your Main Function](#1-change-your-main-function)
- [2. Registering Your Seeds](#2-registering-your-seeds)
- [3. Run Seeds Only For Specific Env](#3-run-seeds-only-for-specific-env)
- [4. Run Seeds By Name](#4-run-seeds-by-name)
- [Summary Of Cli Args](#summary-of-cli-args)
- [License](#license)

## Installation
Expand All @@ -28,9 +33,255 @@ go get github.com/kristijorgji/goseeder

## Usage

To use goseeder you just need to wrap your main function with `WithSeeder` and specify your seeds under a local package named `seeds` in one folder with structure `db/seeds` (it is important if you want to use json or other data to follow this structure)
[Please check examples/simpleshop](examples/simpleshop) for a full working separate go project that uses the seeder

// TODO full example usage will be uploaded later in this repo
Below I will explain once more all the steps needed to have goseeder up and running for your project.

### 1. Change Your Main Function

In order to give your executable seeding abilities and support for its command line arguments, the first thing we have to do is to wrap our main function
with the provided `goseeder.WithSeeder`

```go
func WithSeeder(conProvider func() *sql.DB, clientMain func())
```

The function requires as argument
1. one function that returns a db connection necessary to seed
2. your original main function, which will get executed if no seed is requested

One such main file can look like below:

```go
// main.go
package main

import (
"database/sql"
"fmt"
_ "github.com/go-sql-driver/mysql"
"github.com/joho/godotenv"
"github.com/kristijorgji/goseeder"
"log"
"net/url"
"os"
_ "simpleshop/db/seeds"
)

func main() {
err := godotenv.Load()
if err != nil {
log.Panic("Error loading .env file")
}

goseeder.WithSeeder(connectToDbOrDie, func() {
myMain()
})
}

func myMain() {
fmt.Println("Here you will execute whatever you were doing before using github.com/kristijorgji/goseeder like start your webserver etc")
}

func connectToDbOrDie() *sql.DB {
dbDriver := os.Getenv("DB_DRIVER")
dbHost := os.Getenv("DB_HOST")
dbPort := os.Getenv("DB_PORT")
dbName := os.Getenv("DB_DATABASE")
dbUser := os.Getenv("DB_USERNAME")
dbPassword := os.Getenv("DB_PASSWORD")

dbSource := fmt.Sprintf(
"%s:%s@tcp(%s:%s)/%s?parseTime=true",
dbUser,
url.QueryEscape(dbPassword),
dbHost,
dbPort,
dbName,
)
con, err := sql.Open(dbDriver, dbSource)
if err != nil {
log.Fatalf("Error opening DB: %v", err)
}

return con
}

```

### 2. Registering Your Seeds

Great! After step 1 our executable is able to run in seed mode or default mode.

Now we want to know how to register our custom seeds.

If you look at the imports in main file from step one, we might notice that we import
`_ "simpleshop/db/seeds"` even though we do not use them directly.

This is mandatory because our seeds will get registered during package initialisation as we will see later.

The recommended project folder structure to work properly with `goseeder` is to have the following path for the seeders `db/seeds` and the package name to be `seeds`

Inside the folder you can add your seeders, for example lets seed some data into the `categories` table from one json file located at `db/seeds/data/categories.json`.

To do that we create our `categories.go` file at `db/seeds` folder:

```go
// db/seeds/categories.go
package seeds

import (
"github.com/kristijorgji/goseeder"
)

func categoriesSeeder(s goseeder.Seeder) {
goseeder.FromJson(s, "categories")
}

```

To use this seed, the last step is to register it.

Seeds can be registered as:
1. `common` seeds that run for all environments
2. for a specific environment like `test`, `yourcustomenv` (more in step 3)

We are going to create below a seed that runs for all environments, so we will not specify any env while registering it.

To do that we create in the `db/seeds` folder the file `common.go` that will register seeds that get always executed regardless of the environment:
```go
// db/seeds/common.go
package seeds

import "github.com/kristijorgji/goseeder"

func init() {
goseeder.Register(categoriesSeeder)
}
```

We used `goseeder.Register` to register our seed function to run for all environments.

That is all for the basic usage of goseeder!!!

Our function in categories.go file is now registered and ready to be used.

Now you can run
```
go run main.go --gseed
```

and it will run all your seeds against the provided db connection.

The framework will look for `categories.json` file in the path `db/seeds/data`, and insert all the entries there in a table named `categories` (inferred from the file name)

### 3. Run Seeds Only For Specific Env

Many times we want to have seeds only for `test` environment, test purpose and want to avoid having thousand of randomly generated rows inserted into production database by mistake!

Or we just want granular control, to have separate data to populate our app/web in different way for `staging` `prod` `yourcustomenv` and so on.

goseeder is designed to take care of this by using one of the following methods:

- `goseeder.RegisterForTest(seeder func(s Seeder)` - registers the specified seed for the env named `test`
- `goseeder.RegisterForEnv(env string, seeder func(s Seeder))` - will register your seeder to be executed only for the custom specified env

Let's add to our previous categories.go seeder one seed function specific only for test env!
The file now will look like:

```go
package seeds

import (
"fmt"
"github.com/kristijorgji/goseeder"
"simpleshop/util"
)

func categoriesSeeder(s goseeder.Seeder) {
goseeder.FromJson(s, "categories")
}

func testCategoriesSeeder(s goseeder.Seeder) {
for i := 0; i < 100; i++ {
stmt, _ := s.DB.Prepare(`INSERT INTO categories(id, name) VALUES (?,?)`)
_, err := stmt.Exec(util.RandomInt(1, int64(^uint16(0))), []byte(fmt.Sprintf(`{"en": "%s"}`, util.RandomString(7))))
if err != nil {
panic(err)
}
}
}

```

Finally, lets create our registrator file for all test seeds same way as we did with `common.go`, we will create `test.go` now with content as below:

```go
// db/seeds/test.go
package seeds

import "github.com/kristijorgji/goseeder"

func init() {
goseeder.RegisterForTest(testCategoriesSeeder)
}

```

That is all!

Now if you run your app without specifying test env, only the common env seeders will run and you cannot mess by mistake production or other environments!

To run the test seeder above you have to run:

```bash
go run main.go --gseed --gsenv=test
```

### 4. Run Seeds By Name

When we register a seed like shown in step 2, the seed name is the same as the function name, so our seed is called `categoriesSeeder` because that is the name of the function we register below

```go
func init() {
goseeder.Register(categoriesSeeder)
}
```

This is important because we are totally flexible and can do cool things like execute only the specific seed functions that we want!

Let's assume that we have 100 seed functions, and want to execute only one of them which is named categoriesSeeder (that we registered above) and ignore all the other seeds.

Easy as this, just run:

```bash
go run main.go --gseeder --gsnames=categoriesSeeder
```

If you want to execute multiple seeds by specifying their names, just use comma separated value like `--gsnames=categoriesSeeder,someOtherSeeder`

## Summary Of Cli Args

You can always run
```bash
run go run main.go --help
````

to see all the available arguments and their descriptions.

For the current version the result is:

```bash
INR00009:simpleshop kristi.jorgji$ go run main.go --help
Usage of /var/folders/rd/2bkszcpx6xgcddpn7f3bhczn1m9fb7/T/go-build810543742/b001/exe/main:
-gseed
goseeder - if set will seed
-gsenv string
goseeder - env for which seeds to execute
-gsnames string
goseeder - comma separated seeder names to run specific ones
INR00009:simpleshop kristi.jorgji$
```
## License
Expand Down
8 changes: 8 additions & 0 deletions examples/simpleshop/.env.dist
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
#LOCAL DB

DB_DRIVER=mysql
DB_HOST=127.0.0.1
DB_PORT=3306
DB_DATABASE=goseeder_simpleshop
DB_USERNAME=root
DB_PASSWORD=verySecretPassword
3 changes: 3 additions & 0 deletions examples/simpleshop/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
/.idea

.env
25 changes: 25 additions & 0 deletions examples/simpleshop/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
# simpleshop example

This is one project example of using github.com/kristijorgji/goseeder

To have this example project up and running follow these steps:
1. you need to only create the database, just by executing the content of
`db/migrations/000001_init_schema.up.sql` against your database server
2. Then you need to create one .env file by copying the example .env.dist, and inside update the db credentials


Then in order to seed against run
```bash
go run main.go --gseed
```

and you will run all seeds in this example project.

If you do not want to seed, but run the original main logic of the project run without the gseed argument like:

```bash
go run main.go
```


For full documentation of usage check the main documentation of github.com/kristijorgji/goseeder
19 changes: 19 additions & 0 deletions examples/simpleshop/db/migrations/000001_init_schema.up.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
create schema if not exists goseeder_simpleshop collate utf8_general_ci;

create table categories
(
id smallint unsigned auto_increment
primary key,
name json not null,
created_at timestamp default CURRENT_TIMESTAMP not null,
updated_at timestamp default CURRENT_TIMESTAMP not null on update CURRENT_TIMESTAMP
);

create table products
(
id char(36) not null
primary key,
name json not null,
created_at timestamp default CURRENT_TIMESTAMP not null,
updated_at timestamp default CURRENT_TIMESTAMP not null on update CURRENT_TIMESTAMP
);
21 changes: 21 additions & 0 deletions examples/simpleshop/db/seeds/categories.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
package seeds

import (
"fmt"
"github.com/kristijorgji/goseeder"
"simpleshop/util"
)

func categoriesSeeder(s goseeder.Seeder) {
goseeder.FromJson(s, "categories")
}

func testCategoriesSeeder(s goseeder.Seeder) {
for i := 0; i < 100; i++ {
stmt, _ := s.DB.Prepare(`INSERT INTO categories(id, name) VALUES (?,?)`)
_, err := stmt.Exec(util.RandomInt(1, int64(^uint16(0))), []byte(fmt.Sprintf(`{"en": "%s"}`, util.RandomString(7))))
if err != nil {
panic(err)
}
}
}
8 changes: 8 additions & 0 deletions examples/simpleshop/db/seeds/common.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
package seeds

import "github.com/kristijorgji/goseeder"

func init() {
goseeder.Register(categoriesSeeder)
goseeder.Register(productsSeeder)
}
10 changes: 10 additions & 0 deletions examples/simpleshop/db/seeds/data/categories.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
[
{
"id": "1",
"name": "{\"en\": \"Push\"}"
},
{
"id": "2",
"name": "{\"en\": \"Pull\"}"
}
]
10 changes: 10 additions & 0 deletions examples/simpleshop/db/seeds/data/products.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
[
{
"id": "0055bd0b-8629-490d-8552-906d04dfca84",
"name": "{\"en\": \"Super Smartphone\"}"
},
{
"id": "0162e545-2678-4cde-a291-665dcf6251a4",
"name": "{\"en\": \"Fitness Tracker Fitbit Charge 4, black\"}"
}
]
Loading

0 comments on commit dac5610

Please sign in to comment.