Concurrency: The Art of Goroutines, Channels, and Context

March 12, 2026 Programming Learning Go, Concurrency, Goroutine, Channel, Context, Learning Notes Programming Language Learning Series 6460 words 31 min read

🔊

Concurrency is the core feature that distinguishes Go from other languages. Unlike C++/Java’s thread model, and unlike JavaScript’s single-threaded event loop, Go provides a set of concurrency primitives — goroutines, channels, and context — that let you write high-performance concurrent programs with relatively straightforward code.

This article covers the technical details of Go’s concurrency model: the underlying scheduling mechanism, the implementation of channels and context, common concurrency patterns, and performance optimization and best practices.

goroutines: Deep Dive into Lightweight Threads

The goroutine is the foundation of Go concurrency. Its design philosophy reflects the Go team’s profound understanding of concurrency. To truly master goroutines, we need to understand their execution mechanism, scheduling strategy, and performance characteristics.

The Essence of Goroutines

Technically, a goroutine is a lightweight user thread managed by the Go runtime, distinct from operating system kernel threads. Each goroutine has its own stack space (initial size only 2KB, dynamically growing), program counter, state information, etc., but the management of these resources is completely controlled by the runtime, not the operating system kernel.

The advantage of this design is: the cost of creating and destroying goroutines is extremely low (typically at the nanosecond level), and the switching overhead is far less than thread switching (completed entirely in user mode). This makes creating thousands or even tens of thousands of goroutines commonplace without rapidly exhausting system resources like threads do.

Detailed GMP Scheduling Model

The Go runtime uses an M:N scheduling model, mapping M goroutines to N operating system threads for execution. This model consists of three core components:

Component	Meaning	Responsibility
G (Goroutine)	goroutine	Execution unit, contains stack, instruction pointer, etc.
M (Machine)	System thread	The actual carrier executing goroutines
P (Processor)	Logical processor	Maintains local run queue, M binds to P for execution

Scheduling Workflow:

Local Queue Priority: Each P maintains a local run queue (LRQ) containing goroutines waiting to execute. After M binds to P, it prioritizes fetching tasks from the LRQ.
Global Queue Assistance: To ensure fairness, every 61 goroutines scheduled, the scheduler checks the global queue (GRQ) and fetches half the tasks to the local queue in batches.
Work Stealing: When the local queue is empty, P attempts to “steal” tasks from other P’s queues. The algorithm randomly traverses other Ps, prioritizing stealing from the tail to reduce contention. This process attempts at most 4 times to avoid excessive scheduling overhead.
Network Polling: If there are no runnable goroutines, the scheduler checks whether any network I/O events are ready. Go uses non-blocking I/O (epoll/kqueue), so network operations do not block threads.
System Calls: When a goroutine executes a blocking system call, M unbinds from the current P, and this P can schedule other goroutines. After the system call completes, M attempts to rebind to a P or enter the idle list.

Work Stealing Implementation Details (based on Go source code):

go
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
func stealWork(now int64) (gp *g, inheritTime bool, rnow, pollUntil int64, newWork bool) {
    pp := getg().m.p.ptr()
    
    const stealTries = 4
    for i := 0; i < stealTries; i++ {
        // Randomly traverse other Ps (excluding self)
        for enum := stealOrder.start(cheaprand()); !enum.done(); enum.next() {
            p2 := allp[enum.position()]
            if pp == p2 {
                continue
            }
            
            // Attempt to steal goroutine from non-idle P
            if !idlepMask.read(enum.position()) {
                if gp := runqsteal(pp, p2, false); gp != nil {
                    return gp, false, now, pollUntil, false
                }
            }
        }
    }
    
    return nil, false, now, pollUntil, false
}

Advantages of this scheduling strategy:

Fairness: By periodically checking the global queue and work stealing, it avoids certain goroutines starving
Locality: Local queues reduce lock contention and improve cache hit rates
Scalability: Automatically adapts to CPU core count (adjusted via GOMAXPROCS)
Responsiveness: Network I/O and system calls do not block the entire scheduler

Deep Comparison: Goroutine vs Thread

Understanding the difference between goroutines and operating system threads is crucial:

Feature	goroutine	Operating System Thread
Creation Cost	Nanosecond level (~2KB stack)	Microsecond level (MB-level stack)
Scheduling Level	User-mode scheduling	Kernel-mode scheduling
Switching Cost	Tens of nanoseconds (user mode)	Microsecond level (kernel mode + context save)
Stack Management	Dynamic segmented stack (initial 2KB, auto-expand)	Fixed size (typically 1-8MB)
Quantity Limit	High theoretical limit (memory-limited)	System limit (thousands)
Scheduling Strategy	Work stealing + time slice round-robin	OS priority scheduling
Blocking Behavior	When goroutine blocks, M can switch	Thread blocking wastes CPU resources

Practical Impact: In a web server scenario handling 10,000 concurrent connections:

Traditional thread model: Need 10,000 threads × 2MB stack = 20GB memory, unrealistic
Go goroutine: 10,000 goroutines × 2KB initial stack = 20MB memory, easily achievable

Best Practices for Starting Goroutines

You can start a goroutine using the go keyword, but in actual projects, you need to pay attention to the following points:

go
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
package main

import (
    "fmt"
    "time"
)

func sayHello(name string) {
    for i := 0; i < 5; i++ {
        fmt.Printf("%s says hello %d\n", name, i)
        time.Sleep(100 * time.Millisecond)
    }
}

func main() {
    // Correct: Use function literal to avoid loop variable capture issues
    for i := 0; i < 3; i++ {
        go func(id int) {
            sayHello(fmt.Sprintf("Worker-%d", id))
        }(i)  // Execute immediately, capturing current i value
    }
    
    // Wrong: Causes all goroutines to use the same i value
    // for i := 0; i < 3; i++ {
    //     go func() {
    //         sayHello(fmt.Sprintf("Worker-%d", i))  // i will loop to 3
    //     }()
    // }
    
    // Main goroutine needs to wait, otherwise program exits immediately
    time.Sleep(600 * time.Millisecond)
}

Note: In actual projects, never use time.Sleep to wait for goroutines; you should use sync.WaitGroup or channels. This is just demo code.

Channels: Implementation Principles of “Communication as Synchronization”

Go’s concurrency philosophy is: “Don’t communicate by sharing memory; instead, share memory by communicating.” Channels are not only data transmission channels but also a synchronization primitive. Their implementation embodies the exquisite design of concurrent programming.

Channel Underlying Structure

From a source code perspective, a channel is a circular buffer containing the following fields (based on the hchan structure in Go 1.21+):

go
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
type hchan struct {
    qcount   uint           // Number of elements in buffer
    dataqsiz uint           // Buffer capacity
    buf      unsafe.Pointer // Buffer pointer (circular array)
    elemsize uint16         // Element size
    closed   uint32         // Close flag
    elemtype *_type         // Element type information
    sendx    uint           // Send index
    recvx    uint           // Receive index
    recvq    waitq          // Receive wait queue
    sendq    waitq          // Send wait queue
    lock     mutex          // Mutex protecting all fields
}

Key Design:

Circular Buffer: Avoids frequent memory allocation, fixed-size buffer is reused
Dual Queues: sendq and recvq separately manage goroutines blocked on send and receive
Single Mutex: All channel operations are protected by one lock, avoiding deadlock risk
Type Safety: Compile-time type checking prevents type mixing

Synchronization Semantics of Channel Operations

Understanding the happens-before relationships of channels is crucial for writing correct concurrent programs:

Synchronization Semantics of Unbuffered Channels:

go
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
var c = make(chan int)
var a string

func f() {
    a = "hello, world"  // (1)
    c <- 0             // (2) Send operation
}

func main() {
    go f()
    <-c                 // (3) Receive operation
    print(a)            // (4)
}

In this example, there is a happens-before relationship: (1) → (2) → (3) → (4). This means the assignment to a definitely executes before print(a), ensuring output “hello, world”. Unbuffered channels force synchronization: both sender and receiver must be ready simultaneously.

Asynchronous Semantics of Buffered Channels:

go
1
2
3
ch := make(chan int, 10)
ch <- 1  // Returns immediately if buffer not full
x := <-ch  // Returns immediately if buffer not empty

Buffered channels provide some decoupling capability when the buffer is not full, but still block when full or empty.

Deep Application of Unbuffered Channels

Unbuffered channels are synchronization primitives, commonly used in the following scenarios:

go
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
package main

import (
    "fmt"
    "sync"
    "time"
)

func worker(id int, jobs <-chan int, results chan<- int) {
    for j := range jobs {
        fmt.Printf("Worker %d processing job %d\n", id, j)
        time.Sleep(200 * time.Millisecond)
        results <- j * 2
    }
}

func main() {
    jobs := make(chan int)       // Unbuffered channel
    results := make(chan int, 5)  // Buffered channel, decouples producer speed

    var wg sync.WaitGroup
    
    // Start 3 worker goroutines
    for w := 1; w <= 3; w++ {
        wg.Add(1)
        go func(id int) {
            defer wg.Done()
            worker(id, jobs, results)
        }(w)
    }

    // Send jobs
    go func() {
        for j := 1; j <= 5; j++ {
            jobs <- j
        }
        close(jobs)
    }()

    // Wait for all workers to complete
    go func() {
        wg.Wait()
        close(results)
    }()

    // Collect results
    for result := range results {
        fmt.Printf("Result: %d\n", result)
    }
}

Key Points:

jobs is an unbuffered channel, ensuring send and receive are synchronized
results is a buffered channel, allowing workers to continue processing tasks without blocking
Use close(jobs) to notify workers that there are no more tasks
range jobs automatically detects channel closure

Performance Considerations for Buffered Channels

The choice of buffered channel capacity directly affects program performance:

go
1
2
3
4
5
6
7
8
// Create a buffered channel with capacity 3
ch := make(chan int, 3)

ch <- 1  // Doesn't block, channel not full
ch <- 2  // Doesn't block
ch <- 3  // Doesn't block, channel full

// ch <- 4  // Would block until a goroutine receives data

Buffer Capacity Selection Guide:

Scenario	Recommended Capacity	Reason
Producer-Consumer, small speed difference	1-10	Reduce memory usage, avoid delay accumulation
Producer-Consumer, large speed difference	100-1000	Allow producer bursts, smooth processing
Rate limiting	GOMAXPROCS or task count	Control concurrency
Batch processing	Batch size	Support batch operations

Performance Pitfalls: Oversized buffers may cause:

Memory pressure (each buffered element occupies memory)
Delay accumulation (messages stay longer in buffer)
Backpressure propagation failure (producer cannot perceive consumer pressure)

Channel Direction and Type Safety

Go allows restricting the read/write direction of channels, which is particularly useful in function signatures:

go
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
// Send-only channel (unidirectional channel)
func producer(ch chan<- int) {
    for i := 0; i < 10; i++ {
        ch <- i
    }
    close(ch)  // Only sender can close channel
}

// Receive-only channel
func consumer(ch <-chan int) {
    for v := range ch {
        fmt.Println("Received:", v)
    }
    // close(ch)  // Compilation error: cannot close read-only channel
}

// Bidirectional channel can be converted to unidirectional
func main() {
    ch := make(chan int)
    
    go producer(ch)  // Bidirectional channel implicitly converted to unidirectional
    consumer(ch)
}

Advantages of Type Constraints:

Compile-time checking: Prevents sending or receiving in wrong places
Clear interface: Function signatures directly express communication intent
Avoid errors: Prevents incorrectly closing channels (only sender should close)

Best Practices for Closing Channels:

Only sender should close the channel
Never close receive channels
Don’t close channels repeatedly (causes panic)
Sending to a closed channel causes panic
Receiving from a closed channel returns zero value + false

select: The Ingenuity of Multi-Channel Monitoring

The select statement is one of the most powerful tools in Go concurrent programming. It allows you to simultaneously monitor multiple channel operations, implementing complex concurrent control logic. But its behavioral details and best practices require deep understanding.

Randomness and Fairness of select

When multiple cases are ready simultaneously, select will randomly choose one to execute, rather than choosing in order. This design seems simple but solves the starvation problem in concurrent programming:

go
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
package main

import (
    "fmt"
)

func main() {
    ch1 := make(chan string, 1)
    ch2 := make(chan string, 1)

    ch1 <- "from ch1"
    ch2 <- "from ch2"

    for i := 0; i < 10; i++ {
        select {
        case msg1 := <-ch1:
            fmt.Println("Received from ch1:", msg1)
            ch1 <- msg1  // Put it back
        case msg2 := <-ch2:
            fmt.Println("Received from ch2:", msg2)
            ch2 <- msg2  // Put it back
        }
    }
}

Output Example (may differ each run):

1
2
3
4
5
Received from ch2: from ch2
Received from ch1: from ch1
Received from ch2: from ch2
Received from ch1: from ch1
...

If select always chose the first case, ch2 might never be selected, causing starvation. Random selection ensures fairness.

Blocking Semantics of select

When no case is ready, select blocks. This is the foundation for implementing timeouts and cancellation:

go
1
2
3
4
5
6
7
8
select {
case msg := <-ch:
    fmt.Println("Received:", msg)
case <-time.After(2 * time.Second):
    fmt.Println("Timeout after 2 seconds")
case <-ctx.Done():
    fmt.Println("Context canceled:", ctx.Err())
}

Note: time.After creates a new timer each time it’s called. In frequently called scenarios, you should reuse timers or use time.NewTimer:

go
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
// Inefficient: Creates new timer for each select
for {
    select {
    case msg := <-ch:
        process(msg)
    case <-time.After(100 * time.Millisecond):
        checkTimeout()
    }
}

// Efficient: Reuse timer
timer := time.NewTimer(100 * time.Millisecond)
for {
    timer.Reset(100 * time.Millisecond)
    select {
    case msg := <-ch:
        process(msg)
    case <-timer.C:
        checkTimeout()
    }
}

Clever Use of default Case

select’s default case makes it a non-blocking operation, often used for polling or checking state:

go
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
// Non-blocking send
select {
case ch <- msg:
    fmt.Println("Sent successfully")
default:
    fmt.Println("Channel full, message dropped")
}

// Non-blocking receive
select {
case msg := <-ch:
    fmt.Println("Received:", msg)
default:
    fmt.Println("No message available")
}

Practical Application: Implementing a rate limiter

go
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
type RateLimiter struct {
    ticker   *time.Ticker
    capacity int
    tokens   int
    mu       sync.Mutex
}

func NewRateLimiter(rate int, capacity int) *RateLimiter {
    rl := &RateLimiter{
        ticker:   time.NewTicker(time.Second / time.Duration(rate)),
        capacity: capacity,
        tokens:   capacity,
    }
    go rl.refill()
    return rl
}

func (rl *RateLimiter) refill() {
    for range rl.ticker.C {
        rl.mu.Lock()
        if rl.tokens < rl.capacity {
            rl.tokens++
        }
        rl.mu.Unlock()
    }
}

func (rl *RateLimiter) Allow() bool {
    rl.mu.Lock()
    defer rl.mu.Unlock()
    
    if rl.tokens > 0 {
        rl.tokens--
        return true
    }
    return false
}

func (rl *RateLimiter) Wait(ctx context.Context) error {
    for !rl.Allow() {
        select {
        case <-ctx.Done():
            return ctx.Err()
        case <-rl.ticker.C:
            // Wait for token refill
        }
    }
    return nil
}

Performance Optimization Tips for select

Avoid empty select: An empty select {} blocks forever unless used for special purposes.
Prioritize checking if channel is closed: Use the v, ok := <-ch pattern:

go
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
for {
    select {
    case v, ok := <-ch:
        if !ok {
            fmt.Println("Channel closed")
            return
        }
        process(v)
    case <-ctx.Done():
        return ctx.Err()
    }
}

Use range instead of infinite loop: For monitoring only one channel, range is more concise:

go
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
// Recommended
for msg := range ch {
    process(msg)
}

// Not recommended (unless multiple channels needed)
for {
    select {
    case msg := <-ch:
        process(msg)
    }
}

sync Package: The Art of Traditional Synchronization Primitives

While Go advocates communication through channels, in certain scenarios, traditional synchronization primitives are still necessary. The sync package provides Mutex, RWMutex, WaitGroup, Once, Cond, Pool, and other tools, each with its unique applicable scenarios.

Correct Use of Mutex

Mutex is the most basic synchronization primitive, but improper use can lead to deadlocks or performance issues.

go
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
package main

import (
    "fmt"
    "sync"
)

type Counter struct {
    mu    sync.Mutex
    value int
}

func (c *Counter) Increment() {
    c.mu.Lock()
    defer c.mu.Unlock()
    c.value++
}

func (c *Counter) Value() int {
    c.mu.Lock()
    defer c.mu.Unlock()
    return c.value
}

func main() {
    var wg sync.WaitGroup
    counter := &Counter{}

    for i := 0; i < 1000; i++ {
        wg.Add(1)
        go func() {
            defer wg.Done()
            counter.Increment()
        }()
    }

    wg.Wait()
    fmt.Println("Final counter value:", counter.Value())
}

Performance Characteristics of Mutex:

Mutex is very fast when uncontended (about 20-50 nanoseconds)
With contention, it triggers OS scheduling, causing severe performance degradation
Frequent lock/unlock adds overhead

Best Practices:

Use defer to ensure unlocking, even when panicking
Minimize critical section scope, only protecting code that truly needs synchronization
Avoid calling potentially blocking functions while holding locks
Consider using sync.RWMutex to optimize read-heavy scenarios

Performance Advantages of RWMutex

For read-heavy scenarios, RWMutex can significantly improve performance:

go
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
type Cache struct {
    mu    sync.RWMutex
    data  map[string]string
}

func (c *Cache) Get(key string) (string, bool) {
    c.mu.RLock()  // Read lock: allows multiple readers
    defer c.mu.RUnlock()
    value, ok := c.data[key]
    return value, ok
}

func (c *Cache) Set(key, value string) {
    c.mu.Lock()  // Write lock: exclusive access
    defer c.mu.Unlock()
    c.data[key] = value
}

Performance Trade-off of RWMutex:

Scenario	Mutex	RWMutex
Read-heavy, write-light	Slow	Fast
Write-heavy, read-light	Fast	Slow
Balanced read/write	Comparable	Comparable

Note: RWMutex’s internal implementation is more complex than Mutex and may actually be slower when reads and writes are balanced. You should verify with benchmarks before actual use.

WaitGroup: An Elegant Way to Wait for Multiple Goroutines

WaitGroup is the standard way to wait for a group of goroutines to complete:

go
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
var wg sync.WaitGroup

// Correct: Add and Done appear in pairs
for i := 0; i < 5; i++ {
    wg.Add(1)
    go func(id int) {
        defer wg.Done()  // Ensure counter decrements
        fmt.Printf("Worker %d starting\n", id)
        time.Sleep(100 * time.Millisecond)
        fmt.Printf("Worker %d done\n", id)
    }(i)
}

wg.Wait()
fmt.Println("All workers completed")

Common Mistakes:

go
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
// Wrong: Add called inside goroutine
for i := 0; i < 5; i++ {
    go func() {
        wg.Add(1)  // May cause Wait to complete first
        defer wg.Done()
        doWork()
    }()
}

// Correct: Add called before starting goroutine
for i := 0; i < 5; i++ {
    wg.Add(1)
    go func() {
        defer wg.Done()
        doWork()
    }()
}

Once: Guarantee of Single Execution

sync.Once ensures that an operation executes only once, making it very suitable for singleton patterns and lazy initialization:

go
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
var once sync.Once
var instance *Resource

func GetInstance() *Resource {
    once.Do(func() {
        instance = &Resource{}  // Only executes once
    })
    return instance
}

// Do method supports returning errors
var initErr error
var once sync.Once

func Init() error {
    once.Do(func() {
        initErr = initializeExpensiveResource()
    })
    return initErr
}

Implementation Principle: Once uses atomic operations and a flag internally to guarantee that the function in Do is executed only once. Even if multiple goroutines call Do simultaneously, it will execute only once.

Pool: Object Pool for Reducing GC Pressure

sync.Pool can reuse temporary objects, reducing GC pressure:

go
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
var bufferPool = sync.Pool{
    New: func() interface{} {
        return make([]byte, 1024)
    },
}

func processData() {
    // Get object from pool
    buf := bufferPool.Get().([]byte)
    defer bufferPool.Put(buf)  // Put back in pool
    
    // Use buf
    n := copy(buf, sourceData)
    process(buf[:n])
}

Applicable Scenarios:

Temporary objects (e.g., buffers, parse results)
Frequently created/destroyed objects
Objects with high creation cost (e.g., database connections)

Inapplicable Scenarios:

Long-held objects
Objects that need state cleanup
Objects where concurrent safety is not an issue

Note: Objects in Pool may be cleared during GC, so you cannot rely on their persistence.

context: The Art of Context Management

The context package provides a mechanism for passing request-scoped cancellation signals, deadlines, and values across API boundaries. It is the key to building cancelable and timeout concurrent programs and is a widely used design pattern in the Go ecosystem.

Design Philosophy of context

The core idea of context is: cancellation propagates in a tree. When a context is canceled, all contexts derived from it are automatically canceled. This design makes cancellation across multiple goroutines and function calls simple and elegant.

Context Interface:

go
1
2
3
4
5
6
type Context interface {
    Deadline() (deadline time.Time, ok bool)
    Done() <-chan struct{}
    Err() error
    Value(key interface{}) interface{}
}

Deadline(): Returns whether a deadline is set
Done(): Returns a channel that closes when the context is canceled
Err(): Returns the reason the context was canceled
Value(): Gets key-value pairs from the context

Deep Application of Cancellation Propagation

Cancellation propagation is context’s most powerful feature, applicable to scenarios like HTTP requests, database operations, file I/O, etc.:

go
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
package main

import (
    "context"
    "fmt"
    "time"
)

func worker(ctx context.Context, id int) {
    for {
        select {
        case <-ctx.Done():
            fmt.Printf("Worker %d stopping: %v\n", id, ctx.Err())
            // Perform cleanup
            cleanup(id)
            return
        default:
            fmt.Printf("Worker %d working\n", id)
            time.Sleep(500 * time.Millisecond)
        }
    }
}

func cleanup(id int) {
    fmt.Printf("Cleaning up worker %d\n", id)
}

func main() {
    ctx, cancel := context.WithCancel(context.Background())

    // Start multiple workers
    for i := 1; i <= 3; i++ {
        go worker(ctx, i)
    }

    // Simulate external condition triggering cancellation
    time.Sleep(2 * time.Second)
    cancel()  // Cancel all workers
    time.Sleep(1 * time.Second)
}

Key Points:

Every goroutine checking ctx.Done() responds to cancellation
Can perform cleanup before cancellation
Cancellation operation is asynchronous and does not block the caller

Multiple Ways of Timeout Control

context provides three timeout control methods:

Fixed Timeout:

go
1
2
3
4
5
6
7
8
9
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
defer cancel()  // Avoid resource leak

select {
case result := <-ch:
    fmt.Println("Operation succeeded:", result)
case <-ctx.Done():
    fmt.Println("Operation timed out:", ctx.Err())
}

Deadline:

go
1
2
3
4
5
deadline := time.Now().Add(24 * time.Hour)
ctx, cancel := context.WithDeadline(context.Background(), deadline)
defer cancel()

doLongRunningTask(ctx)

Cancelable:

go
1
2
3
4
5
6
ctx, cancel := context.WithCancel(context.Background())

// Actively cancel based on business logic
if shouldCancel {
    cancel()
}

Best Practices:

Always call defer cancel(), even if you don’t actively cancel
Pass context as the first parameter of functions
Don’t store context in structs
When deriving contexts, prioritize using existing contexts

Elegant Pattern of Value Passing

context.WithValue can pass request-scoped values in the call chain, such as user ID, request ID, etc.:

go
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
type ctxKey string

const (
    userIDKey    ctxKey = "userID"
    requestIDKey ctxKey = "requestID"
)

func handler(ctx context.Context) {
    userID := ctx.Value(userIDKey).(string)
    requestID := ctx.Value(requestIDKey).(string)
    fmt.Printf("User ID: %s, Request ID: %s\n", userID, requestID)
}

func main() {
    ctx := context.Background()
    ctx = context.WithValue(ctx, userIDKey, "user123")
    ctx = context.WithValue(ctx, requestIDKey, "req-456")
    
    handler(ctx)
}

Note:

Use custom types as keys to avoid key conflicts
context.Value returns interface{}, requiring type assertion
Don’t use context to pass optional parameters; use function parameters instead
Value passing is thread-safe, but should only pass immutable data

Practical Application Scenarios of context

HTTP Request Handling:

go
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
    ctx := r.Context()
    
    // Set timeout
    ctx, cancel := context.WithTimeout(ctx, 5*time.Second)
    defer cancel()
    
    // Pass to downstream functions
    data, err := h.service.GetData(ctx)
    if err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }
    
    json.NewEncoder(w).Encode(data)
}

Database Query:

go
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
func (db *DB) Query(ctx context.Context, query string, args ...interface{}) (*sql.Rows, error) {
    return db.db.QueryContext(ctx, query, args...)
}

func (s *Service) GetUser(ctx context.Context, userID string) (*User, error) {
    rows, err := s.db.Query(ctx, "SELECT * FROM users WHERE id = ?", userID)
    if err != nil {
        return nil, err
    }
    defer rows.Close()
    
    // Process results...
}

Concurrency Patterns

The Go community has accumulated many classic concurrency patterns. Mastering these patterns allows you to write more elegant and efficient code.

Worker Pool: Essential for Production

Worker pool pattern limits concurrency count, avoiding resource exhaustion. It’s the foundation for building high-performance services:

go
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
package main

import (
    "fmt"
    "sync"
    "time"
)

type WorkerPool struct {
    numWorkers int
    jobQueue   chan Job
    wg         sync.WaitGroup
}

type Job struct {
    ID   int
    Data interface{}
}

type Result struct {
    JobID  int
    Output interface{}
    Err    error
}

func NewWorkerPool(numWorkers int, queueSize int) *WorkerPool {
    return &WorkerPool{
        numWorkers: numWorkers,
        jobQueue:   make(chan Job, queueSize),
    }
}

func (wp *WorkerPool) Start(results chan<- Result) {
    for i := 1; i <= wp.numWorkers; i++ {
        wp.wg.Add(1)
        go wp.worker(i, results)
    }
}

func (wp *WorkerPool) worker(id int, results chan<- Result) {
    defer wp.wg.Done()
    for job := range wp.jobQueue {
        fmt.Printf("Worker %d processing job %d\n", id, job.ID)
        
        // Simulate processing
        time.Sleep(100 * time.Millisecond)
        
        result := Result{
            JobID:  job.ID,
            Output: fmt.Sprintf("Processed by worker %d", id),
        }
        results <- result
    }
}

func (wp *WorkerPool) Submit(job Job) {
    wp.jobQueue <- job
}

func (wp *WorkerPool) Stop() {
    close(wp.jobQueue)
    wp.wg.Wait()
}

func main() {
    const numWorkers = 3
    const numJobs = 10

    pool := NewWorkerPool(numWorkers, numJobs)
    results := make(chan Result, numJobs)

    pool.Start(results)

    // Submit jobs
    for j := 1; j <= numJobs; j++ {
        pool.Submit(Job{ID: j, Data: fmt.Sprintf("Data-%d", j)})
    }

    // Wait for all jobs to complete
    go func() {
        pool.Stop()
        close(results)
    }()

    // Collect results
    for result := range results {
        fmt.Printf("Result: %+v\n", result)
    }
}

Advantages:

Control concurrency count, avoid resource exhaustion
Reuse goroutines, reduce creation/destruction overhead
Unified error handling and result collection
Easy monitoring and tuning

Fan-out / Fan-in: Standard Pattern for Concurrent Processing

Fan-out distributes tasks to multiple goroutines, Fan-in aggregates results:

go
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
func fanOut(in <-chan int, out chan<- int, worker func(int) int) {
    go func() {
        for num := range in {
            out <- worker(num)
        }
        close(out)
    }()
}

func fanIn(channels ...<-chan int) <-chan int {
    out := make(chan int)
    var wg sync.WaitGroup

    for _, ch := range channels {
        wg.Add(1)
        go func(c <-chan int) {
            defer wg.Done()
            for v := range c {
                out <- v
            }
        }(ch)
    }

    go func() {
        wg.Wait()
        close(out)
    }()

    return out
}

// Practical application: Concurrently process HTTP requests
func fetchURL(url string) string {
    // Simulate HTTP request
    time.Sleep(100 * time.Millisecond)
    return fmt.Sprintf("Content from %s", url)
}

func main() {
    urls := []string{
        "https://example.com/1",
        "https://example.com/2",
        "https://example.com/3",
    }

    // Fan-out: Create multiple goroutines to process different URLs
    urlCh := make(chan string, len(urls))
    for _, url := range urls {
        urlCh <- url
    }
    close(urlCh)

    // Create multiple result channels
    resultChs := make([]chan string, 3)
    for i := 0; i < 3; i++ {
        resultChs[i] = make(chan string, 1)
        fanOut(urlCh, resultChs[i], func(url string) string {
            return fetchURL(url)
        })
    }

    // Fan-in: Aggregate all results
    results := fanIn(resultChs...)

    for result := range results {
        fmt.Println(result)
    }
}

Pipeline: Elegant Solution for Stream Processing

Pipeline pattern processes data streams by connecting multiple stages:

go
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
func gen(nums ...int) <-chan int {
    out := make(chan int)
    go func() {
        for _, n := range nums {
            out <- n
        }
        close(out)
    }()
    return out
}

func sq(in <-chan int) <-chan int {
    out := make(chan int)
    go func() {
        for n := range in {
            out <- n * n
        }
        close(out)
    }()
    return out
}

func filter(in <-chan int, predicate func(int) bool) <-chan int {
    out := make(chan int)
    go func() {
        for n := range in {
            if predicate(n) {
                out <- n
            }
        }
        close(out)
    }()
    return out
}

func main() {
    // Generate -> Square -> Filter -> Square -> Output
    pipeline := sq(filter(sq(gen(2, 3, 4, 5)), func(n int) bool {
        return n > 10
    }))

    for n := range pipeline {
        fmt.Println(n)  // Output squares of numbers greater than 10
    }
}

Practical Application: Log processing pipeline

go
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
// Log parsing pipeline
func logReader(filePath string) <-chan string {
    out := make(chan string)
    go func() {
        defer close(out)
        // Read log file
        file, err := os.Open(filePath)
        if err != nil {
            return
        }
        defer file.Close()

        scanner := bufio.NewScanner(file)
        for scanner.Scan() {
            out <- scanner.Text()
        }
    }()
    return out
}

func logParser(in <-chan string) <-chan LogEntry {
    out := make(chan LogEntry)
    go func() {
        defer close(out)
        for line := range in {
            if entry, err := parseLogLine(line); err == nil {
                out <- entry
            }
        }
    }()
    return out
}

func logFilter(in <-chan LogEntry, level LogLevel) <-chan LogEntry {
    out := make(chan LogEntry)
    go func() {
        defer close(out)
        for entry := range in {
            if entry.Level >= level {
                out <- entry
            }
        }
    }()
    return out
}

func logAggregator(in <-chan LogEntry) map[string]int {
    counts := make(map[string]int)
    for entry := range in {
        counts[entry.Message]++
    }
    return counts
}

func main() {
    // Build processing pipeline
    pipeline := logAggregator(
        logFilter(
            logParser(
                logReader("app.log"),
            ),
            LogLevelError,
        ),
    )

    counts := pipeline
    for msg, count := range counts {
        fmt.Printf("%s: %d\n", msg, count)
    }
}

Concurrent Safety

Deep Understanding of Data Races

Data races are the most common problem in concurrent programming. The following code has a data race:

go
1
2
3
4
5
6
7
var counter int

for i := 0; i < 1000; i++ {
    go func() {
        counter++  // Data race! Multiple goroutines reading/writing simultaneously
    }()
}

Why do data races occur? counter++ is not an atomic operation, it contains:

Read the value of counter
Add 1
Write back to counter

Multiple goroutines may execute these steps simultaneously, causing indeterminate results.

Using the Race Detector

Go provides a built-in data race detector. Simply add the -race flag when compiling:

bash
1
2
3
go run -race main.go
go test -race ./...
go build -race

The detector will report all detected data races:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
==================
WARNING: DATA RACE
Write at 0x00c0000a4008 by goroutine 8:
  main.main.func1()
      /path/to/main.go:10 +0x44

Previous write at 0x00c0000a4008 by goroutine 7:
  main.main.func1()
      /path/to/main.go:10 +0x44
==================

Performance Impact: The race detector significantly reduces program performance (5-10 times), so it should only be used during development and testing.

Strategies for Fixing Data Races

Use Mutex or atomic operations to fix data races:

go
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
// Method 1: Use Mutex
var mu sync.Mutex
var counter int

go func() {
    mu.Lock()
    counter++
    mu.Unlock()
}()

// Method 2: Use atomic
import "sync/atomic"

var counter int64

go func() {
    atomic.AddInt64(&counter, 1)
}()

// Method 3: Use channel (recommended)
var counter int
ch := make(chan int, 1)

go func() {
    ch <- 1
}()

counter = <-ch + counter

Selection Strategy:

channel: First choice, aligns with Go’s concurrency philosophy
atomic: Simple counters and similar scenarios, best performance
Mutex: When protecting complex critical sections

Best Practices for Concurrent Testing

When writing concurrent tests, you need to pay special attention to test reliability and reproducibility:

go
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
func TestCounterConcurrent(t *testing.T) {
    var counter int64
    var wg sync.WaitGroup

    // Concurrent increment
    for i := 0; i < 1000; i++ {
        wg.Add(1)
        go func() {
            defer wg.Done()
            atomic.AddInt64(&counter, 1)
        }()
    }

    wg.Wait()
    
    if counter != 1000 {
        t.Errorf("Expected 1000, got %d", counter)
    }
}

// Test channel close
func TestChannelClose(t *testing.T) {
    ch := make(chan int, 1)
    ch <- 1
    close(ch)

    _, ok := <-ch
    if ok {
        t.Error("Channel should be closed")
    }

    // Reading from a closed channel should not block
    select {
    case v, ok := <-ch:
        if ok {
            t.Error("Expected false for ok")
        }
        if v != 0 {
            t.Error("Expected zero value")
        }
    case <-time.After(100 * time.Millisecond):
        t.Error("Read should not block on closed channel")
    }
}

// Test context cancellation
func TestContextCancel(t *testing.T) {
    ctx, cancel := context.WithCancel(context.Background())
    
    done := make(chan struct{})
    go func() {
        <-ctx.Done()
        close(done)
    }()

    cancel()
    
    select {
    case <-done:
        // Context was canceled as expected
    case <-time.After(100 * time.Millisecond):
        t.Error("Context cancel should be immediate")
    }
}

Performance Optimization and Best Practices

Avoid Common Performance Pitfalls

Excessive Goroutine Creation:

go
1
2
3
4
5
6
7
8
9
// Wrong: Create goroutine for each request
for _, request := range requests {
    go processRequest(request)  // May lead to tens of thousands of goroutines
}

// Correct: Use worker pool
for _, request := range requests {
    jobQueue <- request
}

Improper Channel Capacity Selection:

go
1
2
3
4
5
6
7
8
// Wrong: Capacity too small, frequent blocking
ch := make(chan int, 1)

// Wrong: Capacity too large, wasting memory
ch := make(chan int, 1000000)

// Correct: Choose based on actual needs
ch := make(chan int, 100)  // Moderate buffer

Excessive Lock Granularity:

go
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
// Wrong: Lock the entire function
func (c *Cache) Process(key string) {
    c.mu.Lock()
    defer c.mu.Unlock()
    
    value := c.data[key]        // Needs lock
    processed := heavyWork(value)  // Doesn't need lock
    c.data[key] = processed     // Needs lock
}

// Correct: Only lock necessary parts
func (c *Cache) Process(key string) {
    c.mu.Lock()
    value := c.data[key]
    c.mu.Unlock()
    
    processed := heavyWork(value)
    
    c.mu.Lock()
    c.data[key] = processed
    c.mu.Unlock()
}

pprof Performance Analysis

Use pprof to analyze performance issues in concurrent programs:

go
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
import (
    _ "net/http/pprof"
    "net/http"
)

func main() {
    go func() {
        http.ListenAndServe("localhost:6060", nil)
    }()
    
    // Application logic
}

Common Commands:

bash
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
# View goroutine status
go tool pprof http://localhost:6060/debug/pprof/goroutine

# View CPU usage
go tool pprof http://localhost:6060/debug/pprof/profile?seconds=30

# View memory allocation
go tool pprof http://localhost:6060/debug/pprof/heap

# View lock contention
go tool pprof http://localhost:6060/debug/pprof/mutex

Goroutine Leak Detection

Goroutine leaks are one of the most common concurrency problems:

go
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
// Wrong: Goroutine leak
func leakyFunction() {
    ch := make(chan int)
    go func() {
        <-ch  // Blocks forever
    }()
    // Forgot to close ch or send data
}

// Correct: Ensure goroutine can exit
func correctFunction() {
    ch := make(chan int)
    done := make(chan struct{})
    
    go func() {
        defer close(done)
        select {
        case <-ch:
            // Process data
        case <-time.After(5 * time.Second):
            // Timeout exit
        }
    }()
    
    // Send data or close channel
    // ch <- 42
    close(ch)
    
    // Wait for goroutine to complete
    <-done
}

Detecting Goroutine Leaks:

go
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
import (
    "runtime"
    "testing"
    "time"
)

func TestNoGoroutineLeak(t *testing.T) {
    initial := runtime.NumGoroutine()
    
    // Execute test function
    testFunction()
    
    // Wait for goroutine cleanup
    time.Sleep(100 * time.Millisecond)
    
    final := runtime.NumGoroutine()
    if final > initial {
        t.Errorf("Goroutine leak detected: %d -> %d", initial, final)
    }
}

Summary

The core of Go’s concurrency model is decomposing complex problems into a few primitives:

goroutines provide lightweight concurrent execution units, scheduled by the GMP scheduler
channels implement “communication as synchronization,” guaranteeing correctness through happens-before relationships
select supports multiplexing, with random selection ensuring fairness
context provides cancellation and timeout mechanisms, with tree propagation simplifying concurrent control
sync package provides traditional synchronization primitives when needed, each with its applicable scenarios

These primitives combine into worker pools, pipelines, fan-out/fan-in, and rate-limiting patterns. This is one reason Go is widely adopted in cloud-native systems.

A few practical principles for concurrent programming:

Correctness First: Use race detector to catch data races
Simplicity First: Prefer channels over complex lock logic
Control Concurrency: Use worker pools to bound resource usage, avoid goroutine leaks
Cancelable: Use context for cancellation and timeouts, avoid resource leaks
Data-Driven Optimization: Use pprof to locate bottlenecks before optimizing

Part of series: Programming Language Learning Series

← Previous Error Handling: Go's Multi-Return Value Approach Next → Standard Library, Testing, and Toolchain: A Panoramic View of Go Engineering Practices