Simple techniques to minimize goroutine leaks and channel misuse in Go
Golang has effective built-in primitives designed for concurrent work. Concurrent algorithms can be implemented in Golang using simpler and more readable code in comparison to other languages. In contrast, misusing these primitives will make the code complex and prone to errors when changing it. In the following paragraphs, I’ll describe some techniques to prevent misusing the concurrency primitives that I found useful in my day-to-day work.
There is a great book that I read a while ago about working with goroutines and channels, called Concurrency in Go. To the day of this writing, I haven’t found a better book describing concurrency patterns in Go. I learned most of the techniques from this book.
- Isolate concurrent code from sequential code
Don’t mix code that does concurrent work with the code that does sequential work. Isolate the concurrent code in a separate function (or a struct with methods if needed) that takes care of declaring and closing channels, launching and waiting for goroutines, etc. This function should be easy to use, without worrying about concurrency.
type Task struct {
ID int
}
func main() {
taskList := []Task{{ID: 1}, {ID: 2}, {ID: 3}}
doWork(taskList...)
}
func doWork(taskList ...Task) {
// Sequential code.
doConcurrentWork(taskList...)
// Sequential code.
}
// Isolated concurrent code.
func doConcurrentWork(taskList ...Task) {
wg := sync.WaitGroup{}
wg.Add(len(taskList))
for _, task := range taskList {
go func(task Task) {
defer wg.Done()
time.Sleep(1 * time.Second)
fmt.Printf("Task %d is done.\n", task.ID)
}(task)
}
wg.Wait()
}2. Output channels should be returned, not passed as an argument
The function that creates the output channel should also be responsible for closing it. By doing this, the caller of the function doesn’t have to worry about when and where to close the output channel. It only has to consume the output values. Another advantage of this is that you can simply range over the values sent on the channel.
To return the output channel, you have to wrap the work in another go routine and defer the instruction that closes the channel. When the work is finished and the wrapping goroutine ends, the defer is called and the channel is closed.
type Task struct {
ID int
}
type Result struct {
Msg string
}
func main() {
taskList := []Task{{ID: 1}, {ID: 2}, {ID: 3}}
doWork(taskList...)
}
func doWork(taskList ...Task) {
for result := range doConcurrentWork(taskList...) {
fmt.Println(result.Msg)
}
}
func doConcurrentWork(taskList ...Task) chan Result {
outputCh := make(chan Result)
// Wrap the work in a goroutine.
go func() {
// Defer closing the output channel. Going forward, it doesn't matter
// when you return in the goroutine, the channel will always be closed. (*)
defer close(outputCh)
wg := sync.WaitGroup{}
wg.Add(len(taskList))
for _, task := range taskList {
go func(task Task) {
defer wg.Done()
time.Sleep(1 * time.Second)
r := Result{
Msg: fmt.Sprintf("Task %d is done.", task.ID),
}
outputCh <- r
}(task)
}
wg.Wait()
}()
return outputCh
}(*) There can be cases when deferred function calls are not executed. For example, if os.Exit(0) is called the application will stop without executing the deferred calls.
3. Enforce the direction of channels
An outcome of the previous technique is that the channel can be enforced to be read-only. The compiler won’t process your code if it detects an instruction that sends a value on a read-only channel.
type Task struct {
ID int
}
type Result struct {
Msg string
}
func main() {
taskList := []Task{{ID: 1}, {ID: 2}, {ID: 3}}
doWork(taskList...)
}
func doWork(taskList ...Task) {
for result := range doConcurrentWork(taskList...) {
fmt.Println(result.Msg)
}
}
// A read-only channel is returned for the caller to read the output.
func doConcurrentWork(taskList ...Task) <-chan Result {
outputCh := make(chan Result)
go func() {
defer close(outputCh)
wg := sync.WaitGroup{}
wg.Add(len(taskList))
for _, task := range taskList {
go func(task Task) {
defer wg.Done()
time.Sleep(1 * time.Second)
r := Result{
Msg: fmt.Sprintf("Task %d is done.", task.ID),
}
outputCh <- r
}(task)
}
wg.Wait()
}()
return outputCh
}4. Pass errors back to the caller
Most of the time, the caller function has more information based on which the best approach for handling the error can be decided. Although it should be common sense when it comes to Go error handling, it might not be so obvious when working with goroutines.
type Task struct {
ID int
}
type Result struct {
Message string
Error error
}
func main() {
ctx := context.Background()
taskList := []Task{{ID: 1}, {ID: 2}, {ID: 3}}
doWork(ctx, taskList...)
}
func doWork(ctx context.Context, taskList ...Task) {
ctx, cancel := context.WithCancel(ctx)
defer cancel()
// The caller can decide what to do with the results.
for result := range doConcurrentWork(ctx, taskList...) {
if result.Error != nil {
fmt.Println(result.Error)
return
}
fmt.Println(result.Message)
}
}
func doConcurrentWork(ctx context.Context, taskList ...Task) <-chan Result {
outputCh := make(chan Result)
go func() {
defer close(outputCh)
wg := sync.WaitGroup{}
wg.Add(len(taskList))
for _, task := range taskList {
go func(ctx context.Context, task Task) {
defer wg.Done()
// The error is returned to the caller.
msg, err := solveTask(ctx, task)
r := Result{
Message: msg,
Error: err,
}
outputCh <- r
}(ctx, task)
}
wg.Wait()
}()
return outputCh
}
func solveTask(ctx context.Context, t Task) (string, error) {
select {
case <-ctx.Done():
return "", ctx.Err()
default:
}
if t.ID%2 == 0 {
return "", fmt.Errorf("solving task %d failed", t.ID)
}
return fmt.Sprintf("solving task %d done", t.ID), nil
}5. Don’t use buffered channels without a solid reason
Buffered channels might hide possible deadlocks because they block only when full. Start with unbuffered channels and change later accordingly after thinking carefully about the optimization that you are trying to do.
There is a great style on Go written by Uber. I always go back to it and check it out when in doubt. I highly recommend it.
These techniques helped me write cleaner and more reliable code. Use them when it is appropriate, don’t apply them without consideration, and always think about the context and the problem that you are trying to solve first.
