Why is goroutine allocation slower on multiple cores?

go, multithreading

Solution

When running on a single core, goroutine allocation and switching is just a matter of internal accounting. Goroutines are never preempted, so the switching logic is extremely simple and very fast. And more importantly in this case, your main routine does not yield at all, so the goroutines never even begin execution before they're terminated. You allocate the structure and then delete it, and that's that. (edit This may not be true with newer versions of go, but it is certainly more orderly with only 1 process)

But when you map routines over multiple threads, then you suddenly get os-level context switching involved, which is orders of magnitude slower and more complex. And even if you're on multiple cores, there's a lot more work that has to be done. Plus now your gouroutines may actually be running before the program gets terminated.

Try `strace`ing the program under both conditions and see how its behavior differs.

Problem

I was doing some experiments in Go and I found something really odd. When I run the following code on my computer it executes in ~0.5 seconds. ``` package main import ( "fmt" "runtime" "time" ) func waitAround(die chan bool) { <- die } func main() { var startMemory runtime.MemStats runtime.ReadMemStats(&startMemory) start := time.Now() cpus := runtime.NumCPU() runtime.GOMAXPROCS(cpus) die := make(chan bool) count := 100000 for i := 0; i < count; i++ { go waitAround(die) } elapsed := time.Since(start) var endMemory runtime.MemStats runtime.ReadMemStats(&endMemory) fmt.Printf("Started %d goroutines\n%d CPUs\n%f seconds\n", count, cpus, elapsed.Seconds()) fmt.Printf("Memory before %d\nmemory after %d\n", startMemory.Alloc, endMemory.Alloc) fmt.Printf("%d goroutines running\n", runtime.NumGoroutine()) fmt.Printf("%d bytes per goroutine\n", (endMemory.Alloc - startMemory.Alloc)/uint64(runtime.NumGoroutine())) close(die) } ``` However, when I execute it using `runtime.GOMAXPROCS(1)` it executes much faster (~0.15 seconds). Can anybody explain to me why running many goroutines would be slower using more cores? Is there any significant overhead to multiplexing the goroutines onto multiple cores? I realize the goroutines aren't doing anything and it would probably be a different story if I had to wait for the routines to actually do something.

Original source