Dynamic-Net: Tuning the Objective Without Re-training

Technion – Israel Institute of Technology

arXiv 2018 [paper]               Code [GitHub]

Dynamic-Net: We propose an approach that enables traversing the “objective-space”, spanned by two different objectives, at test-time, without re-training, as illustrated by the blue dot moving along the blue curve in the plot. This is different from the common practice of training a separate network for each objective, represented by X’s on the plot. Using a single Dynamic-Net we can tune the level of stylization of an image, monitor completion quality per image, or control facial attributes, all interactively at test-time, without re-training.

# Abstract

One of the key ingredients for successful optimization of modern CNNs is identifying a suitable objective. To date, the objective is fixed a-priori at training time, and any variation to it requires re-training a new network. In this paper we present a first attempt at alleviating the need for re-training. Rather than fixing the network at training time, we train a “Dynamic-Net” that can be modified at inference time. Our approach considers an “objective-space” as the space of all linear combinations of two objectives, and the Dynamic-Net can traverse this objective-space at test-time, without any further training. We show that this upgrades pre-trained networks by providing an out-of-learning extension, while maintaining the performance quality. The solution we propose is fast and allows a user to interactively modify the network, in real-time, in order to obtain the result he/she desires. We show the benefits of such an approach via several different applications.

Proposed framework: Our training has two steps: (i) First the “main” network $$\theta$$ (green blocks) is trained to minimize $${\cal O}_0$$. (ii) Then $$\theta$$ is fixed, one or more tuning-blocks $$\psi$$ are added (orange block), and trained to minimize $${\cal O}_1$$. The output $$\hat y_1$$ approximates the output $$y_1$$ one would get from training the main network $$\theta$$ with objective $${\cal O}_1$$. At test-time, we can emulate results equivalent to a network trained with objective $${\cal O}_m$$ by tuning the parameter $$\alpha_m$$ (in blue) that determines the latent representation $$z_m$$. Our method can be applied as (a) a single-block framework or as (b) multi-block framework.

# Applications

Dynamic Style Transfer

Control over Stylization level

Interpolation between two Styles

Dynamic DC-GAN: Controlled Image Generation

The proposed method allow us to generate faces with control over the facial attributes e.g gender or hair color.

Image Completion

Input $$\alpha=0$$ $$\alpha=0.4$$ $$\alpha=1$$     Input $$\alpha=0$$ $$\alpha=0.5$$ $$\alpha=1$$

Original result Improved result     Original result Improved result

Dynamic-Net allows the user to select the best working point for each image, improving results of networks that were trained with sub-optimal objectives.

# Dynamic-Net: Tuning the Objective Without Re-training

Paper [pdf]

Supplementary [pdf]

# Try Our Code

Code and demo of the the experiments described in our paper is available in [GitHub]

Dynamic style transfer demo