Deep Declarative Networks
ECCV 2020 Tutorial, 23-28 August, Glasgow, UK
Conventional deep learning architectures involve composition of simple and explicitly defined feedforward processing functions (inner products, convolutions, elementwise non-linear transforms and pooling operations). Over the past several years researchers have been exploring deep learning models with embedded differentiable optimization problems (Agrawal et al., 2019; Amos and Kotler, 2017; Gould et al., 2016), and recently these models have been applied to solving problems in computer vision (e.g., Fernando and Gould, 2016; Cherian et al., 2017, Santa Cruz et al., 2018, Lee et al., 2019, Wang et al., 2019) and other areas of machine learning.
Since these networks define the behavior rather than algorithmic implementation of individual processing layers they are called deep declarative networks (DDNs), borrowing nomenclature from the programming languages community (Gould et al., 2019). Importantly, the gradient of the solution to the optimization problem with respect to inputs and parameters can be calculated even without prior knowledge of the algorithm used for solving the optimization problem in the first place, allowing for efficient backpropagation and end-to-end learning.
This tutorial will introduce deep declarative networks and their variants. We will discuss the theory behind differentiable optimization, technical issues that need to be overcome in developing such models and applications of these models to computer vision problems. Additionally, the tutorial will provide hands-on experience in designing and implementing a custom declarative node. Topics include:
- Declarative end-to-end learnable processing nodes.
- Differentiable convex optimization problems.
- Declarative nodes for computer vision applications.
- Implementation techniques and gotchas.