Does accessing via a shared_ptr pollute the cache line more than raw pointer?
c++, optimization, performance, shared-ptr
Solution
There are several things to consider, but in summary one can say: It doesn't really matter.
First, there is no guarantee (or rather, no requirement) that there is a reference counter at all. It is merely required which way `std::shared_ptr` has to behave. Using a reference counter is one way of achieving this, circular-linked-list being another. Practically, all implementations (all that I know of, at least) do use a reference counter.
Second, the reference counter can be allocated separately via `operator new` or in the same location and using the same allocation as the managed object (which is created via placement `new`) if you use `make_shared`. The latter is, again, not strictly guaranteed. The standard states "Implementations are encouraged, but not required, to perform no more than one memory allocation", explicitly allowing something different
If the reference counter is separately allocated, then it will most likely live in a different cache line, and so the system will consume two cache lines rather than one when accessing the object. That, however, is in some situations an advantage, not a disadvantage (that is, whenever you copy the smart pointer, see below). In addition to the data cache, there exists also the TLB, which is much smaller (usually less than 64 entries). Given that there is a certain likelihood that two separately allocated objects will not only be in different cache lines but in different memory pages, this may be another thing that will cost a few extra cycles.
If the reference counter is allocated in the same location, it will very likely be in the same cache line as the beginning of the object (but it might be in the previous cache line as well). This looks like an advantage, but it is not necessarily one.
Whenever the `shared_ptr` is copied or one smart pointer referencing the same object goes out of scope, the reference counter must be modified. This is a write to the cache line the counter lives in (or rather, since it's an atomic operation, not a write to the cache line, but the net effect to the outside world is the same). The cache line is invalidated and must be fetched again by everybody else wanting to access it. There exists a common problem known as "false sharing" where naive parallel processing runs much slower than everybody would expect, which occurs for that exact same reason.
Now if the reference counter is allocated together with the object, this means that whenever the shared pointer is copied (or goes out of scope) the next access to the object is a guaranteed cache miss (since the cache line containing the start of the object will be purged from cache).
tl;dr
Yes, there are cache effects, however, you should not worry too much. Cache misses happen all the time, and regularly (the same goes for the TLB). As long as you have at least a somewhat coherent access pattern, this will not matter at all. After the next context switch (that is, every few milliseconds, or after the next interrupt or syscall) your cache and TLB will likely be gone anyway. That's something everyone has to live with, and it's not a problem at all.
You do not use a `shared_ptr` for fun, but because it provides a valuable functionality that you need. Copying the pointer might be 3-4 cycles slower than copying a raw pointer, and it might cause an occasional extra cache miss when you copy it, but you don't make a hundred thousand copies per second.
The safety and overall utility benefits by far outweight the disadvantages.
Problem
I understand a good compiler can perform optimization so that accessing, say an `int*` via a `std::shared_ptr` is done using the same assembly as if a raw `int*` pointer was used. My question is: would the cache line containing the optimized smart pointer be polluted with other data members from the smart pointer, like the reference counter? So that although the assembly generated would be identical with a real pointer, the cache performance could be worse because not as much of the cache line is being used efficiently? EDIT: This performance effect could be more-noticeable if we were iterating through a structure like `std::vector<std::shared_ptr<int>>` and using the ints.