When does the App Engine scheduler use a new thread vs. a new instance?
google-app-engine, python
Solution
The following set of rules are currently used to determine if a given instance can accept a new request:
if processing more than N concurrent requests (today N=10): false
elif exceeding the soft memory limit: false
elif exceeding the instance class CPU limit: false
elif warming up: false
else true
The following of total CPU/core limits currently apply to each instance classes:
CLASS 1: 600MHz 1 core
CLASS 2: 1.2GHz 1 core
CLASS 4: 2.4GHz 1 core
CLASS 8: 4.8GHz 2 core
So only a `B8` instance can process up to 2 fully CPU bound requests in parallel.
Setting `threadsafe: true` (Python) or `<threadsafe>true</threadsafe>` (Java) for instances classes < 8 would not allow more than one CPU bound requests to be processed in parallel on a single instance.
If you are not fully CPU bound or doing I/O, the Python and Java runtime will spawn new threads for handling new request up to 10 concurrent requests with `threadsafe: true`
Also note that even though the Go runtime is single threaded, it does support concurrent requests: It will spawn 1 goroutine per requests and yield control between goroutines while they are performing I/O.
Problem
If I set `threadsafe: true` in my `app.yaml` file, what are the rules that govern when a new instance will be created to serve a request, versus when a new thread will be created on an existing instance? If I have an app which performs something computationally intensive on each request, does multi-threading buy me anything? In other words, is an instance a multi-core instance or a single core? Or, are new threads only spun up when existing threads are waiting on IO?