Is Session.run(fetches) guaranteed to execute its "fetches" arguments in-order?
machine-learning, python, tensorflow
Solution
No. By default, Tensorflow is free to evaluate operators in any order. Because of concurrency, that order may even change between runs. This is usually a good thing because it means that Tensorflow may make optimal use of the available hardware. It can be problematic if your code mutates state such as Variables.
However, if for some reason you do wish to control the order of evaluation, in general you can use control dependencies to enforce an order between operators. Control dependencies are documented here:
https://www.tensorflow.org/api_docs/python/tf/Graph#control_dependencies
Hope that helps!
Problem
Is `Session.run(fetches, feed_dict)` guaranteed to execute its `fetches` arguments in-order? The documentation doesn't seem to mention it. For example, if you run ``` sess.run([accuracy, train_op], feed_dict=feed_dict) ``` the order of execution matters: `train_op` will update parameters affecting `accuracy`.