Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

V2 #1

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

V2 #1

Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
150 changes: 62 additions & 88 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,116 +9,90 @@

### Goroutines lifecycle manager :bug: :butterfly: :coffin:

Rungroup was created to manage multiple goroutines which may or may not interrupt other goroutines on error.
RunGroup is a Go package that is designed to effectively manage concurrent tasks within a group, allowing you to track and store their results using a result map. It includes features for setting up contexts, running interrupting concurrent tasks, and retrieving results.

Rationale:
Whilst hacking with golang, some or the other day you will encounter a situation where you have to manage multiple goroutines which can interrupt other goroutines if an error occurs.
Table of contents
=================

A rungroup is essentially a composition of:
- goroutines
- optional concurrent(thread safe) ErrorMap to track errors from different goroutines
- context cancel func, when cancelled can stop other goroutines
- [Installation](#installation-floppy_disk)
- [Usage](#usage)
- [API Reference](#api-reference)
- [Examples](#examples)
- [Contributing](#contributing)
- [License](#license)

A goroutine in rungroup is essentially composition of:
- a user(programmer) defined function which returns error
- an identifier (string) which may help you to track goroutine error.

Installation:floppy_disk:
=================

### Installation :floppy_disk::
```shell
go get -u github.com/bharat-rajani/rungroup
go get -u github.com/bharat-rajani/rungroup/v2
```

### Example :keyboard::
Usage
=================

#### A quick and simple (and maybe dirty) example, where we need to call 3 REST Endpoints concurrently.
RunGroup is designed to help you manage concurrent tasks and their errors efficiently. Here's how you can use it:

Three gorutines:
- F_API, interrupter
- S_API
- T_API, interrupter

Let's say we don't care about the response from second REST API Endpoint, hence that routine cannot interrupt other routines (F_API, T_API).
Now as soon as there is an error in F_API (or in T_API) then all other goroutines will be stopped.
## Setting up a Group with Context and Result Map
You can create a new concurrent task group with a specific context and result map using the provided functions:

```go
package main

import (
"fmt"
"context"
"net/http"
"github.com/bharat-rajani/rungroup"
"github.com/bharat-rajani/rungroup/pkg/concurrent"
)

func main() {
g, ctx := rungroup.WithContextErrorMap(context.Background(),concurrent.NewRWMutexMap())
// g, ctx := rungroup.WithContextErrorMap(context.Background(),new(sync.Map)) //refer Benchmarks for performance difference

var fResp, sResp, tResp *http.Response
g.GoWithFunc(func(ctx context.Context) error {
// error placeholder
var tErr error
fResp, tErr = http.Get("F_API_URL")
if tErr != nil {
return tErr
}
return nil
}, ctx, true, "F_API")

g.GoWithFunc(func(ctx context.Context) error {
// error placeholder
var tErr error
sResp, tErr = http.Get("S_API_URL")
if tErr != nil {
return tErr
}
return nil

}, ctx, false, "S_API")

g.GoWithFunc(func(ctx context.Context) error {
// error placeholder
var tErr error
tResp, tErr = http.Get("T_API_URL")
if tErr != nil {
return tErr
}
return nil
}, ctx, true, "T_API")

// returns first error from interrupter routine
err := g.Wait()
if err != nil {
fmt.Println(err)
}
}
group, ctx := concurrent.WithContextResultMap[TaskIdentifierType, TaskOutputType](parentContext, resultMap)
```

OR

#### What if error occurs in "S_API" routine ? How can I retrieve its error?
```go
group, ctx := concurrent.WithContext[TaskIdentifierType, TaskOutputType](parentContext)
```

Since "S_API" is a non interrupter goroutine hence the only way to track its error is by:
The first option (WithContextResultMap) initializes the group with a specific result map for storing task results, while the second option (WithContext) sets up the group without a result map.

```golang
err, ok := g.GetErrorByID("S_API")
if ok && err!=nil{
fmt.Println(err)
}
## Running Concurrent Tasks
To run concurrent tasks within the group, use the `GoWithFunc` method. You can associate each task with a unique identifier for tracking purposes:

```go
ctx := context.Background()
group.GoWithFunc(func() (V, error) {
// Your task logic here
return result, err
},ctx, interrupter, taskID)
```

#### I don't want to concurrently Read or Write errors.

Ok, I heard you, using concurrent maps comes with performance tradeoff.
If you don't want to track errors of all gorutines and you are happy with first occurring error, then just use rungroup WithContext:

```golang
g, ctx := rungroup.WithContext(context.Background())
...
...
- func() (V, error): The function to execute concurrently.
- interrupter: A boolean flag indicating whether the task can be interrupted upon error.
- taskID: A unique identifier associated with the task.


## Retrieving Task Results
You can retrieve task results using the GetResultByID method, which takes the task's unique identifier as a parameter:

```go
result, err, found := group.GetResultByID(taskID)
```

> Note: When you use rungroup.WithContext (no error tracking) then calling g.GetErrorByID() will yield you a nice uninitialized map error and ok = false.
- result: The result value.
- err: An error, if any.
- found: A boolean indicating whether the result was found.

API Reference
=================

GoDoc
For detailed information about available functions and types, refer to the GoDoc documentation.

Examples
=================

Explore the [examples](examples/) directory for usage examples and sample code.

### Contributing
Contributions to this project are welcome. Please open an issue or submit a pull request for any improvements or bug fixes.

### License
This project is licensed under the MIT License. See the LICENSE file for details.

> Rungroup is inspired by [errorgroup]( https://github.com/golang/sync/blob/master/errgroup/errgroup.go).
140 changes: 140 additions & 0 deletions examples/example.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,140 @@
package main

import (
"context"
"encoding/json"
"fmt"
"github.com/bharat-rajani/rungroup"
"github.com/bharat-rajani/rungroup/pkg/concurrent"
"net/http"
"strconv"
)

type JsonResp map[string]any

const userID = 1

func main() {

// they key of the map (our goroutine identifier) will be string, and the value will be response from goroutine
g, ctx := rungroup.WithContextResultMap[string, JsonResp](context.Background(), concurrent.NewRWMutexMap())
// g, ctx := rungroup.WithContextAndMaps(context.Background(),new(sync.Map)) //refer Benchmarks for performance difference

/*
Contrived example where we need to fetch a user details, their posts and their albums.
We have a user with id as 1
we need to get the user details
concurrently we need to get the posts belonging to a particular user
concurrently we need to get the albums belonging to a particular user
optionally we need to login the user
*/

g.GoWithFunc(FetchUser, ctx, true, "fetch_user")

g.GoWithFunc(FetchPosts, ctx, true, "fetch_posts")

g.GoWithFunc(FetchAlbums, ctx, true, "fetch_albums")

// returns first error from interrupter routine
err := g.Wait()
if err != nil {
fmt.Println(err)
}

prettyPrint(g.GetResultByID("fetch_user"))
prettyPrint(g.GetResultByID("fetch_posts"))
prettyPrint(g.GetResultByID("fetch_albums"))
}

func FetchUser(ctx context.Context) (JsonResp, error) {

req, err := http.NewRequest(http.MethodGet, fmt.Sprintf("https://jsonplaceholder.typicode.com/users/%d", userID), nil)
if err != nil {
return nil, err
}
req = req.WithContext(ctx)
resp, err := http.DefaultClient.Do(req)
if err != nil {
return nil, err
}

var tResp JsonResp
if err := json.NewDecoder(resp.Body).Decode(&tResp); err != nil {
return nil, err
}
return tResp, nil
}

func FetchPosts(ctx context.Context) (JsonResp, error) {

req, err := http.NewRequest(http.MethodGet, "https://jsonplaceholder.typicode.com/posts", nil)
if err != nil {
return nil, err
}
req = req.WithContext(ctx)

resp, err := http.DefaultClient.Do(req)
if err != nil {
return nil, err
}

var tResp []JsonResp
if err := json.NewDecoder(resp.Body).Decode(&tResp); err != nil {
return nil, err
}
posts := make(JsonResp)
userIDStr := strconv.Itoa(userID)
for _, t := range tResp {
if id := t["userId"]; id.(float64) == float64(userID) {
if posts[userIDStr] == nil {
posts[userIDStr] = make([]JsonResp, 0)
}
delete(t, "userId")
posts[userIDStr] = append(posts[userIDStr].([]JsonResp), t)
}
}
return posts, nil
}

func FetchAlbums(ctx context.Context) (JsonResp, error) {

req, err := http.NewRequest(http.MethodGet, "https://jsonplaceholder.typicode.com/albums", nil)
if err != nil {
return nil, err
}
req = req.WithContext(ctx)

resp, err := http.DefaultClient.Do(req)
if err != nil {
return nil, err
}

var tResp []JsonResp
if err := json.NewDecoder(resp.Body).Decode(&tResp); err != nil {
return nil, err
}

albums := make(JsonResp)
userIDStr := strconv.Itoa(userID)
for _, t := range tResp {
if id := t["userId"]; id.(float64) == float64(userID) {
if albums[userIDStr] == nil {
albums[userIDStr] = make([]JsonResp, 0)
}
delete(t, "userId")
albums[userIDStr] = append(albums[userIDStr].([]JsonResp), t)
}
}
return albums, nil
}

func prettyPrint(V any, err error, ok bool) {
if err != nil || !ok {
return
}
d, err := json.MarshalIndent(V, "", " ")
if err != nil {
return
}
fmt.Println(string(d))
}
2 changes: 1 addition & 1 deletion go.mod
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
module github.com/bharat-rajani/rungroup

go 1.16
go 1.21
Loading
Loading