Deep Neural Networks (DNNs) now represent a fundamental building block of many machine perception methods. The reason is simple—these models achieve exceptional performance. DNNs represent the state-of-the-art for core competencies such as image classification, object detection and semantic segmentation as well as for integrated approaches to higher level tasks including environment mapping and video understanding. While their usefulness is clear, our understanding of how best to design these models remains far from complete.

The goal of this workshop was to bring together researchers to discuss questions and ideas relating to various aspects of the structure and design of DNNs. We considered in particular the following the two questions: (1) What we have learned as a community from our experience of designing these models? (2) Which research directions are most promising for improving existing architectures?

As part of the workshop, we are collecting suggestions (ongoing) for DNN-design related idea that "everyone should know about", which will be edited into a summary document that we hope will prove useful for members of the field. Add your suggestion for DNN-design related idea that "everyone should know about" here.

A short report of the day itself can be found here. A YouTube playlist of most of the invited talks can be found here.


This workshop took place on Monday 28th October 2019 in 308BC of the COEX Convention Centre, Seoul.

9:20 - 9:30 Welcome (video, slides)
9:30 - 10:00 Neural Architecture Search and Beyond
(video, slides)
Barret Zoph (Google)
10:00 - 11:10 Poster presentations and coffee Posters (in 308BC).
11:10 - 11:40 Towards General Vision Architectures: Attentive Single-Tasking of Multiple Tasks (video, slides) Iasonas Kokkinos (UCL/Ariel AI)
11:40 - 12:10 Deep Nets: What have they ever done for Vision? (video, slides) Alan Yuille (Johns Hopkins)
12:10 - 2:00 Lunch
2:05 - 2:20 (Oral) Understanding the Effects of Pre-training for Object Detectors via Eigenspectrum (slides) Yosuke Shinya, Edgar Simo-Serra, Taiji Suzuki
2:20 - 2:35 (Oral) GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyond (slides) Yue Cao, Jiarui Xu, Stephen Lin, Fangyun Wei, Han Hu
2:35 - 2:50 (Oral) 4-Connected Shift Residual Networks (slides) Andrew Brown, Pascal Mettes, Marcel Worring
2:50 - 2:55 Best Paper Award
3:00 - 3:30 Capsule Architectures (slides) Sara Sabour (Google)
3:30 - 3:50 Tea and coffee
3:50 - 4:20 From Architecting Networks to Architecting Network Generators (video, slides) Ross Girshick (FAIR)
4:25 - 5:20 Panel discussion
5:20 - 5:25 Closing Remarks

Accepted papers

Oral Presentations

Understanding the Effects of Pre-training for Object Detectors via Eigenspectrum Yosuke Shinya (DENSO CORPORATION), Edgar Simo-Serra (Waseda University), Taiji Suzuki (The University of Tokyo / RIKEN)
GCNet: Non-local Networks Meet Squeeze-Excitation Networks and BeyondYue Cao (Tsinghua University), Jiarui Xu (Hong Kong University of Science and Technology (HKUST)), Stephen Lin (Microsoft Research), Fangyun Wei (Microsoft Research Asia), Han Hu (Microsoft Research Asia)
4-Connected Shift Residual NetworksAndrew Brown (University of Amsterdam), Pascal Mettes (University of Amsterdam), Marcel Worring (University of Amsterdam)

Long Papers

EENA: Efficient Evolution of Neural ArchitectureHui Zhu (Institute of Computing Technology, Chinese Academy of Sciences and University of Chinese Academy of Sciences), Zhulin An (ICT, CAS), Chuanguang Yang (ICT, CAS), Kaiqiang Xu (ICT, CAS), Erhu Zhao (ICT, CAS), Yongjun Xu (ICT, CAS)
Sequentially Aggregated Convolutional NetworksYiwen Huang (Huazhong University of Science and Technology Wenhua College), Rihui Wu (University of Sydney), Pinglai Ou (Virginia Tech), Ziyong Feng (DeepGlint)
Resource Efficient 3D Convolutional Neural NetworksOkan Köpüklü (Technical University of Munich), Neslihan Kose Cihangir (Intel Deutschland GmbH), Ahmet Gündüz (Technical University of Munich), Gerhard Rigoll (Institute for Human-Machine Communication, TU Munich, Germany)
Cross-Granularity Attention Network for Semantic SegmentationLingyu Zhu (Tampere University), Tinghuai Wang (Nokia Technologies), Emre Aksu (Nokia Technologies), Joni-Kristian Kamarainen (Tampere University)
HM-NAS: Efficient Neural Architecture Search via Hierarchical MaskingShen Yan (Michigan State University), Biyi Fang (Michigan State University), Faen Zhang (AInnovation), Yu Zheng (Michigan State University), Xiao Zeng (Michigan State University), Hui Xu (AInnovation), Mi Zhang (Michigan State University)
BinaryDenseNet: Developing an Architecture for Binary Neural NetworksJoseph Bethge (Hasso Plattner Institute), Haojin Yang (Hasso Plattner Institute), Marvin Bornstein (Hasso Plattner Institute), ChristophMeinel (Hasso Plattner Institute)
Efficient Structured Pruning and Architecture Searching for Group ConvolutionRuizhe Zhao (Imperial College London), Wayne Luk (Imperial College London)
Attention routing between capsulesJaewoong Choi (Seoul National University), Hyun Seo (Seoul National University), Suii Im (Seoul National University), Myungjoo Kang (Seoul National University)
Adaptive Convolutional KernelsJulio c Zamora (Intel), Jesus Adan Cruz Vargas (intel), Paulo Lopez Meyer (Intel), Omesh Tickoo (Intel)
Adaptive Activation Functions Using Fractional CalculusJulio c Zamora (Intel), Jesus Adan Cruz Vargas (intel), Rodrigo Camacho (Intel), Paulo Lopez Meyer (Intel), Hector Cordourier (Intel), Omesh Tickoo (Intel)
SqueezeNAS: Fast neural architecture search for faster semantic segmentationAlbert E Shaw (DeepScale), Daniel Hunter (DeepScale), Forrest Iandola (DeepScale), Sammy Sidhu (DeepScale)

Short Papers

Matrix Nets: A New Deep Architecture for Object DetectionAbdullah Rashwan (University of Waterloo), Agastya Kalra (University of Waterloo), Pascal Poupart (University of Waterloo)
Localizing Occluders with Compositional Convolutional NetworksAdam Kortylewski (Johns Hopkins University), Qing Liu (Johns Hopkins University), Huiyu Wang (Johns Hopkins University), Zhishuai Zhang (Johns Hopkins University), Alan Yuille (Johns Hopkins University)
MSNet: Structural Wired Neural Architecture Search for Internet of ThingsHsin-Pai Cheng (Duke University), Tunhou Zhang (Duke University), Yukun Yang (Duke University), Feng Yan (University of Nevada, Reno), Harris Teague (Qualcomm AI Research), Yiran Chen (Duke University), Hai Li (Duke University)
HoloGAN: Unsupervised learning of 3D representations from natural imagesThu H Nguyen-Phuoc (University of Bath), Chuan Li (Lambda Labs), Lucas Theis (Twitter), Christian Richardt (University of Bath), Yongliang Yang (University of Bath)
Searching for Accurate Binary Neural ArchitecturesMingzhu Shen (Beihang University), Kai Han (Huawei Noah's Ark Lab), Chunjing Xu (Huawei Noah's Ark Lab), Yunhe Wang (Huawei Noah's Ark Lab)
AdvGAN++ : Harnessing latent features for adversary generationPuneet Mangla (IIT Hyderabad)*, Surgan Jandial (IIT Hyderabad), Vineeth N Balasubramanian (Indian Institute of Technology, Hyderabad), Sakshi Varshney (IIT Hyderabad)
Layer-Wise Invertibility for Extreme Memory Cost Reduction of CNN TrainingTristan Hascoet (Kobe University), Quentin Febvre (Sicara), Weihao Zhuang (Kobe University), Yasuo Ariki (Kobe University), Tetusya Takiguchi (Kobe University)
Investigating Convolutional Neural Networks using Spatial OrdernessRohan Ghosh (National University of Singapore), Anupam Gupta (National University of Singapore)
Why Does Data-Driven Beat Theory-Driven Computer Vision?John Tsotsos (York University), Iuliia Kotseruba (York University), Alexander Andreopoulos (IBM Research Almaden), Yulong Wu (Aion Foundation)
Posters should follow the ICCV 2019 format - the dimensions of the poster panels are 1950mm (width) x 950mm (height). Oral presentations should be no more than 10 minutes long (a further two minutes will be available for questions). Papers are made available via the Computer Vision Foundation.

Program Committee

We are deeply indebted to the following who generously gave of their time and expertise to assist in paper selection:

A. S. Koepke, Albert Pumarola, Bichen Wu, Chao Peng, Chuhan Zhang, Dan Xu, Gang Sun, Hakan Bilen, Jack Turner, Jack Valmadre, James Thewlis, Ji Liang, Jiankang Deng, Kai Han, Liu Leqi, Lukas Neumann, Oriane Simeoni, Peter Vajda, Qiang Guo, Robert McCraith, Ruth C. Fong, Sergi Caelles, Shangzhe Wu, Song Bai, Tengda Han, Tom Gunter, Vedanuj Goswami, Vicky Kalogeiton, Weidi Xie, Weilin Huang, Xia Li, Xiaoliang Dai, Xu Ji, Yang Liu, Yuki Asano, Zihang Lai

All reviews were of a high standard and we would additionally like to highlight the outstanding reviews contributed by:
Jack Valmadre, Sergi Caelles, Jack Turner and Vicky Kalogeiton.



    This workshop takes place on Monday 28th October in 308BC of the COEX Convention Centre, Seoul. More details about the main conference are provided on the ICCV website. The workshop has been assigned the poster boards 67 - 83 (inclusive). There is no predefined ordering for these boards, so please feel free to put up your poster whenever you arrive on the first available board within this range.


Contact email for any queries relating to the workshop: albanie[AT]