How do CUDA threads work
cuda
Solution
Your card has compute capability `3.0`, see here.
From Table 12 of the CUDA C Programming Guide, the number of `2048` threads you are mentionining for your compute capability refers to maximum number of resident threads per multiprocessor. This does not mean that you cannot launch more than `2048` threads overall. For example, from a few rows above of that table you can read that the maximum maximum `x`-dimension of a grid of thread blocks is `2^31-1`. This means that it is perfectly legal to launch, for example, a `1d` thread grid of, for example, `8192` threads. The reason is that the card will perform a context switching between thread warps as indicated in this post: What is the context switching mechanism in GPU?.
Concerning the second part of your question, your implementation of the `add` function is conceptually wrong. You are using the index `i` both as thread index and as a `for` loop index. A more correct implementation is the following
__global__ void add(int* a,int* b,int* c)
{
int i = threadIdx.x;
c[i] = a[i] + b[i];
}
The above writing means the following: each thread will execute the two assignments, namely
int i = threadIdx.x;
c[i] = a[i] + b[i];
Now, for example, for thread `#3` the value of the `threadIdx.x` variable will be `3`. Thus, thread `#3` will deal with a local variable `i`, private to its memory space, whose value will be assigned to `3`. Furthermore, it will load `a[3]` and `b[3]` from global memory, add them up, assign the result to `c[3]` and then store the final result to global memory. Accordingly, when you launch the grid you cannot of course fill the whole array of `100` elements by only `64` threads and you will need `100` threads.
Note that the above explanation is oversimplified. I recommend you read some basic textbook as the famous CUDA By Example.
Problem
I have a lot of doubts about the way threads are formed and executed. Firstly, the documentation describes GPU threads as lightweight threads. Suppose I wish to multiply two `100*100` matrices. This would require `100*100` threads if each element were calculated by a different thread. However, my GPU (NVIDIA GT 640M LE) specifications show two SM's each of which can support only 2048 threads. How is it two possible to calculate the rest of the elements parallel y given that my GPU can't support so many threads. Also consider the basic vector add code. Suppose I invoke a kernel with 1 block and 64 threads to add two arrays of 100 elements each as follows: ``` __global__ void add(int* a,int* b,int* c) { int i = threadIdx.x; for(i<100) { c[i] = a[i] + b[i]; { } ``` Since only 64 threads were initialized I assume 64 elements are added in parallel. - How are the remaining elements added? - How does the warp scheduler decide which threads to assign to add the last 36 elements? My main problem is: I don't understand how a thread knows which elements to operate on.