Pipelines
Pipeline stages should close outbound channels, respect cancellation, and avoid blocked senders.
Canonical guidance
- each stage receives values, transforms them, and emits downstream
- stages should close outbound channels when done sending
- downstream abandonment must not strand upstream goroutines
- cancellation must be designed into the pipeline
Use when
- streaming multi-stage work
- fan-out / fan-in designs
- CPU or I/O pipelines
Avoid
- pipeline stages with no cancellation story
- leaving senders blocked forever
- unbounded buffering as a substitute for control flow
Preferred pattern
func sq(ctx context.Context, in <-chan int) <-chan int {
out := make(chan int)
go func() {
defer close(out)
for n := range in {
select {
case <-ctx.Done():
return
case out <- n * n:
}
}
}()
return out
}
Anti-pattern
- stages that assume the downstream consumer will always drain all results
Explanation: This anti-pattern is common because downstream consumers often drain everything in demos, but real callers frequently stop early and expose blocked senders.
Why
- pipeline bugs are often goroutine-leak bugs in disguise
Related pages
Sources
- Go Concurrency Patterns: Pipelines and cancellation - Sameer Ajmani
- Advanced Go Concurrency Patterns - Sameer Ajmani