Update: The Tensorflow 2.0 beta is out, and it uses Eager Execution by default. Much of the advice in this article is only relevant for 1.x versions of Tensorflow. I recommend moving to 2.0! :)

One of my favorite videos from the Tensorflow 2018 Dev Summit is the one where Alex Passos introduces Tensorflow’s new Eager Execution mode. It promises an alternative to the inflexible graph system, where we can simply start our file with tf.enable_eager_execution() , and then actually print out the values of tensors as we create them, without having to create a session.

I tried out Eager on my last project, a fun little RNN to generate new Pokemon names. While building it, I discovered that there was more to using Eager than tf.enable_eager_execution() , especially for those of us who have already gotten used to the traditional Tensorflow way of doing things. In general, “action at a distance” approaches don’t really work with Eager — if you want a value to change, you need to explicitly set it. There are also several important Tensorflow libraries that are not yet compatible with Eager, and may never be, where you will need to use an Eager-friendly alternative.

Ultimately, I found eager execution to be much more intuitive. Learning to use it is mostly a process of unlearning the idiosyncrasies of working with sessions.

#1. Imports and Setup

Before you start, be sure to import the Eager contrib library and enable eager execution. It is important that the enabling occur before any other Tensorflow code.

import tensorflow as tf

import tensorflow.contrib.eager as tfe tf.enable_eager_execution()

#2. Printing Tensors

One of the most beautiful things about Eager mode is that you can see tensor values at each line without running it in a session. This makes it much easier to understand what is happening in your code, and to debug.

x = tf.constant([[1.,2.,3.],[4.,5.,6.]])

print(x.numpy())

=> [[1. 2. 3.]

[4. 5. 6.]]

print(x.shape)

=> (2,3)

print(x.dtype)

=> <dtype: ‘float32’>

#3. Placeholders

Placeholders are not supported in eager mode. If you try to create an instance of tf.placeholder you will get an error. This makes sense, because placeholders are a core part of the sessions workflow, and the whole point of Eager is to get rid of sessions. Instead, just pass data into functions as an argument. This is much more intuitive anyway.

#4. Variables

tf.Variable is not supported in eager mode, and if you try to use it you will get an error. Instead, use tfe.Variable , and pass in the initial value you want for that variable. Then you are done! No additional initialization of variables is required. You will still be able to update your variables from an Optimizer .

initial_value = tf.random_normal([2,3], stddev=0.2)

w = tfe.Variable(initial_value, name=‘weights’)

#5. Iterating Over Data

Datasets are still used in Eager, but iterating over them is a bit simpler. Instead of having to create an iterator and call iterator.get_next() , you can simply iterate over the dataset itself.

words = tf.constant([‘cat’, ‘dog’, ‘house’, ‘car’])

dataset = tf.data.Dataset.from_tensor_slices(words)

for x in dataset:

print x

#6. Computing Gradients

Fortunately, like the rest of Tensorflow, eager execution supports automatic differentiation. Since tf.gradients will not work in eager mode, it is best to use gradient tape, which flexibly tracks the operations within its block, and constructs a computation graph from them. Here’s an example of how you might use gradient tape in training:

variables = [w1, b1, w2, b2]

optimizer = tf.train.AdamOptimizer() with tf.GradientTape() as tape:

y_pred = model.predict(x, variables)

loss = model.compute_loss(y_pred, y)

grads = tape.gradient(loss, variables)

optimizer.apply_gradients(zip(grads, variables))

#7. Logging

To log training details for viewing in Tensorboard, they first need to be saved to an event file. Unfortunately, the tf.summary module that normally handles this is not supported in Eager. You can use tf.contrib.summary instead. Here’s an example:

summary_writer = tf.contrib.summary.create_file_writer(‘logs’, flush_millis=10000)

summary_writer.set_as_default()

global_step = tf.train.get_or_create_global_step() def log_loss(loss):

with tf.contrib.summary.always_record_summaries:

tf.contrib.summary.scalar(‘loss’, loss) # In your training loop

global_step.assign_add(1)

loss = some value….

log_loss(loss)

#8. Saving

To save a checkpoint during training, you must use tfe.Saver instead of tf.train.Saver . Instead of saving an entire session, the eager saver will simply checkpoint the current values of the variables, which you can load back at a later time.

variables = [w1, b1, w2, b2]

saver = tfe.Saver(variables)

# do some training…

saver.save(‘checkpoints/checkpoint.ckpt’, global_step=step)

To restore from a checkpoint, you can load the variables list back from the checkpoint files like so:

checkpoint_path = tf.train.latest_checkpoint(‘checkpoints’)

saver.restore(checkpoint_path)

The variables will now have their saved values.