How does multi_gpu_model work with multiple outputs with different losses?

Beginning Keras 2.0.9, there is support for using multi-GPUs, which is a welcome addition for the library to stay on par with counterparts. I encourage you to follow Chollet on twitter, for latest updates (and also well worded philosophical points).

Just to be clear, this is a kind of embaressing parallelism stemming from the Tensorflow no longer requiring a lock on the GPU. For a nice tutorial on how to leverage this, I recommend here. However, this is so straightforward to do, that I recommend a quick glance at the method that makes this functionality possible directly in the Keras git repo: https://github.com/fchollet/keras/blob/master/keras/utils/training_utils.py

What is not clear is, what to do if your basemodel ‘model’ has multiple outputs with different losses. Under the hood, is to split the input data using the Lambda layer. So your model, called “Model” is wrapped like this:

Layer (type) Output Shape Param # Connected to 
====================================================================
input_image (InputLayer) (None, 320, 320, 3) 0 
____________________________________________________________________
input_addParams (InputLayer (None, 7) 0 
____________________________________________________________________
lambda_1 (Lambda) (None, 320, 320, 3) 0 input_image[0][0] 
____________________________________________________________________
lambda_2 (Lambda) (None, 7) 0 input_addParams[0][0] 
____________________________________________________________________
lambda_3 (Lambda) (None, 320, 320, 3) 0 input_image[0][0] 
____________________________________________________________________
lambda_4 (Lambda) (None, 7) 0 input_addParams[0][0] 
____________________________________________________________________
model_2 (Model) [(None, 20, 20, 4), 27948069 
 lambda_1[0][0] 
 lambda_2[0][0] 
 lambda_3[0][0] 
 lambda_4[0][0] 
____________________________________________________________________
concatenate_1 (Concatenate) (None, 20, 20, 4) 0 model_2[1][0] 
 model_2[2][0] 
____________________________________________________________________
concatenate_2 (Concatenate) (None, 1) 0 model_2[1][1] 
 model_2[2][1] 
====================================================================

So, I now need to pass the losses for each output layer to the corresponding concatenate layers as follows:

parallel_model.compile(losses={'concatenate_1': loss1, 'concatenate_2': loss2})

Thats all!

There is another breaking change in keras 2.0.9 in the ImageGenerator and the corresponding Iterator in preprocessing/image.py. This is related to calling the function `_get_batches_of_transformed_samples(self, index_array)`

Basically, the Iterator’s next() now only returns `index_array`. This is annoying for anyone who wrote there custom ImageGenerator, such as SegImageGenerator for the Keras-FCN code thats now in keras-extra I believe. Its also a simple fix, just make sure you have that function, since its called from the fit_generator function.

Ping me, if you want me to elaborate, happy to do it!