How does Q_FOREACH (= foreach) macro work and why is it that complex?

c++, foreach, macros, qt

Solution

The GCC version

The GCC one is really quite simple. First of all it is used like this:

Q_FOREACH(x, cont)
{
    // do stuff
}

And that will be expanded to

for (QForeachContainer<__typeof__(cont)> _container_(cont); !_container_.brk && _container_.i != _container_.e; __extension__  ({ ++_container_.brk; ++_container_.i; }))
    for (x = *_container_.i;; __extension__ ({--_container_.brk; break;}))
    {
        // do stuff
    }

So first of all:

for (QForeachContainer<__typeof__(cont)> _container_(cont); !_container_.brk && _container_.i != _container_.e; __extension__  ({ ++_container_.brk; ++_container_.i; }))

This is the actual `for` loop. It sets up a `QForeachContainer` to help with the iteration. The `brk` variable is intitialised to 0. Then the condition is tested:

!_container_.brk && _container_.i != _container_.e

`brk` is zero so `!brk` is true, and presumably if the container has any elements `i` (the current element) doesn't equal `e` (the last element) yet.

Then the body of that outer `for` is entered, which is:

for (variable = *_container_.i;; __extension__ ({--_container_.brk; break;}))
{
    // do stuff
}

So `x` is set to `*_container_.i` which is the current element the iteration is on, and there is no condition so presumably this loop will continue forever. Then the body of the loop is entered, which is our code, and it's just a comment so it doesn't do anything.

Then the increment part of the inner loop is entered, which is interesting:

__extension__ ({--_container_.brk; break;})

It decrements `brk` so that's now -1, and breaks out of the loop (with `__extension__` which makes GCC not emit warnings for using GCC extensions, like you now know).

Then the increment part of the outer loop is entered:

__extension__  ({ ++_container_.brk; ++_container_.i; })

which increments `brk` again and makes it 0 again, and then `i` is incremented so we get to the next element. The condition is checked, and since `brk` is now 0 and `i` presumably doesn't equal `e` yet (if we have more elements) the process is repeated.

Why did we decrement and then increment `brk` like that? The reason is because the increment part of the inner loop will not be executed if we used `break` in the body of our code, like this:

Q_FOREACH(x, cont)
{
    break;
}

Then `brk` would still be 0 when it breaks out of the inner loop, and then the increment part of the outer loop would be entered and increment it to 1, then `!brk` would be false and the outer loop's condition would evaluate to false, and the foreach would stop.

The trick is to realise that there are two `for` loops; the outer one's lifetime is the whole foreach, but the inner one only lasts for one element. It would be an infinite loop since it doesn't have a condition, but it is either `break`ed out of by it's increment part, or by a `break` in the code you provide it. That's why `x` looks like it is assigned to "only once" but actually it's assigned to on every iteration of the outer loop.

The VS version

The VS version is a little more complicated because it has to work around the lack of the GCC extension `__typeof__` and block-expressions, and the version of VS it was written for (6) didn't have `auto` or other fancy C++11 features.

Let's look at an example expansion for what we used earlier:

if(0){}else
    for (const QForeachContainerBase &_container_ = qForeachContainerNew(cont); qForeachContainer(&_container_, true ? 0 : qForeachPointer(cont))->condition(); ++qForeachContainer(&_container_, true ? 0 : qForeachPointer(cont))->i)
        for (x = *qForeachContainer(&_container_, true ? 0 : qForeachPointer(cont))->i; qForeachContainer(&_container_, true ? 0 : qForeachPointer(cont))->brk; --qForeachContainer(&_container_, true ? 0 : qForeachPointer(cont))->brk)
        {
            // stuff
        }

The `if(0){}else` is because VC++ 6 did the scoping of `for` variables wrong and a variable declared in the initialisation part of a `for` loop could be used outside the loop. So it's a workaround for a VS bug. The reason they did `if(0){}else` instead of just `if(0){...}` is so that you can't add an `else` after the loop, like

Q_FOREACH(x, cont)
{
    // do stuff
} else {
    // This code is never called
}

Second, let's look at the initialisation of the outer `for`:

const QForeachContainerBase &_container_ = qForeachContainerNew(cont)

The definition of `QForeachContainerBase` is:

struct QForeachContainerBase {};

And the definition of `qForeachContainerNew` is

template <typename T>
inline QForeachContainer<T>
qForeachContainerNew(const T& t) {
    return QForeachContainer<T>(t);
}

And the definition of `QForeachContainer` is

template <typename T>
class QForeachContainer : public QForeachContainerBase {
public:
    inline QForeachContainer(const T& t): c(t), brk(0), i(c.begin()), e(c.end()){};
    const T c;
    mutable int brk;
    mutable typename T::const_iterator i, e;
    inline bool condition() const { return (!brk++ && i != e); }
};

So to make up for the lack of `__typeof__` (which analogous to the `decltype` of C++11) we have to use polymorphism. The `qForeachContainerNew` function returns a `QForeachContainer<T>` by value but due to lifetime extension of temporaries, if we store it in a `const QForeachContainer&`, we can prolong it's lifetime till the end of the outer `for` (actually the `if` because of VC6's bug). We can store a `QForeachContainer<T>` in a `QForeachContainerBase` because the former is a subclass of the latter, and we have to make it a reference like `QForeachContainerBase&` instead of a value like `QForeachContainerBase` to avoid slicing.

Then for the condition of the outer `for`:

qForeachContainer(&_container_, true ? 0 : qForeachPointer(cont))->condition(); 

The definition of `qForeachContainer` is

inline const QForeachContainer<T> *qForeachContainer(const QForeachContainerBase *base, const T *) {
    return static_cast<const QForeachContainer<T> *>(base);
}

And the definition of `qForeachPointer` is

template <typename T>
inline T *qForeachPointer(const T &) {
    return 0;
}

This is where you might not be aware of what's going on since these functions seem kind of pointless. Well here's how they work and why you need them:

We have a `QForeachContainer<T>` stored in a reference to a `QForeachContainerBase` with no way to get it back out (that we can see). We have to cast it to the proper type somehow, and that's where the two functions come in. But how do we know what type to cast it to?

A rule of the ternary operator `x ? y : z` is that `y` and `z` must be of the same type. We need to know the type of the container, so we use the `qForeachPointer` function to do that:

qForeachPointer(cont)

The return type of `qForeachPointer` is `T*`, so we use template type deduction to deduce the type of the container.

The `true ? 0 : qForeachPointer(cont)` is to be able to pass a `NULL` pointer of the right type to `qForeachContainer` so it will know what type to cast the pointer we give it to. Why do we use the ternary operator for this instead of just doing `qForeachContainer(&_container_, qForeachPointer(cont))`? It's to avoid evaluating `cont` many times. The second (actually third) operand to `?:` is not evaluated unless the condition is `false`, and since the condition is `true` itself, we can get the right type of `cont` without evaluating it.

So that solves that, and we use `qForeachContainer` to cast `_container_` to the right type. The call is:

qForeachContainer(&_container_, true ? 0 : qForeachPointer(cont))

And again the definition is

inline const QForeachContainer<T> *qForeachContainer(const QForeachContainerBase *base, const T *) {
    return static_cast<const QForeachContainer<T> *>(base);
}

The second parameter will always be `NULL` because we do `true ? 0` which always evaluates to `0`, and we use qForeachPointer to deduce the type `T`, and use that to cast the first argument to a `QForeachContainer<T>*` so we can use its member functions/variables with the condition (still in the outer `for`):

qForeachContainer(&_container_, true ? 0 : qForeachPointer(cont))->condition()

And `condition` returns:

(!brk++ && i != e)

which is the same as the GCC version above except that it increments `brk` after evaluating it. So `!brk++` evaluates to `true` and then `brk` is incremented to 1.

Then we enter the inner `for` and begin with the initialisation:

x = *qForeachContainer(&_container_, true ? 0 : qForeachPointer(cont))->i

Which just sets the variable to what the iterator `i` is pointing to.

Then the condition:

qForeachContainer(&_container_, true ? 0 : qForeachPointer(cont))->brk

Since `brk` is 1, the body of the loop is entered, which is our comment:

// stuff

Then the increment is entered:

--qForeachContainer(&_container_, true ? 0 : qForeachPointer(cont))->brk

That decrements `brk` back to 0. Then the condition is checked again:

qForeachContainer(&_container_, true ? 0 : qForeachPointer(cont))->brk

And `brk` is 0 which is `false` and the loop is exited. We come to the increment part of the outer `for`:

++qForeachContainer(&_container_, true ? 0 : qForeachPointer(cont))->i

And that increments `i` to the next element. Then we get to the condition:

qForeachContainer(&_container_, true ? 0 : qForeachPointer(cont))->condition()

Which checks that `brk` is 0 (which it is) and increments it to 1 again, and the process is repeated if `i != e`.

This handles `break` in client code only a little differently than the GCC version, since `brk` will not be decremented if we use `break` in our code and it will still be 1, and the `condition()` will be false for the outer loop and the outer loop will `break`.

And as GManNickG stated in the comments, this macro is a lot like Boost's `BOOST_FOREACH` which you can read about here.

Problem

Original source

Related problems

• Why lifetime of temporary doesn't extend till lifetime of enclosing object?