Linux pthread mutex and kernel scheduler
linux, mutex, pthreads, scheduling
Solution
a. I think that during a `pthread_mutex_lock`, the thread actively waits.
Yes, glibc's NPTL `pthread_mutex_lock` have active wait (spinning), BUT the spinning is used only for very short amount of time and only for some types of mutexes. After this amount, `pthread_mutex_lock` will go to sleep, by calling linux syscall `futex` with WAIT argument.
Only mutexes with type PTHREAD_MUTEX_ADAPTIVE_NP will spin, and default is `PTHREAD_MUTEX_TIMED_NP` (normal mutex) without spinning. Check `MAX_ADAPTIVE_COUNT` in `__pthread_mutex_lock` sources).
If you want to do infinite spinning (active waiting), use `pthread_spin_lock` function with `pthread_spinlock_t`-types locks.
I'll consider the rest of your question as if you are using `pthread_spin_lock`:
Then, another thread is scheduled which potentially free the locked field, etc. So this means that the scheduler waits for the thread to complete its schedule time before switching to the next one, no matter what the thread is doing.
Yes, if there is contention for CPU cores, the your thread with active spinning may block other thread from execute, even if the other thread is the one who will unlock the mutex (spinlock) which is needed by your thread.
But if there is no contention (no thread oversubscribing), and threads are scheduled on different cores (by coincidence, or by manual setting of cpu affinity with `sched_setaffinity` or `pthread_setaffinity_np`), spinning will enable you to proceed faster, then using OS-based futex.
b. My friend thinks that the waiting thread somehow tells the kernel "Hey, I'm asleep, don't wait for me at all". In this case, the kernel would schedule the next thread right away, without waiting for the current thread to complete...
Yes, he is right.
`futex` is the modern way to say OS that this thread is waiting for some value in memory (for opening some `mutex`); and in current implementation `futex` also puts our thread to sleep. It is not needed to wake it to do spinning, if kernel knows when to wake up this thread. How it knows? The lock owner, when doing `pthread_mutex_unlock`, will check, is there any other threads, sleeping on this mutex. If there is any, lock owner will call `futex` with `FUTEX_WAKE`, telling OS to wake some thread, registered as sleeper on this mutex.
There is no need to spin, if thread registers itself as waiter in OS.
Problem
With a friend of mine, we disagree on how synchronization is handled at userspace level (in the pthread library). a. I think that during a pthread_mutex_lock, the thread actively waits. Meaning the linux scheduler rises this thread, let it execute his code, which should looks like: ``` while (mutex_resource->locked); ``` Then, another thread is scheduled which potentially free the `locked` field, etc. So this means that the scheduler waits for the thread to complete its schedule time before switching to the next one, no matter what the thread is doing. b. My friend thinks that the waiting thread somehow tells the kernel "Hey, I'm asleep, don't wait for me at all". In this case, the kernel would schedule the next thread right away, without waiting for the current thread to complete its schedule time, being aware this thread is sleeping. From what I see in the code of pthread, it seems there is loop handling the lock. But maybe I missed something. In embedded systems, it could make sense to prevent the kernel from waiting. So he may be right (but I hope he does not :D). Thanks!