Deep Neural Networks (DNNs) now represent a fundamental building block of many machine perception methods. The reason is simple—these models achieve exceptional performance. DNNs represent the state-of-the-art for core competencies such as image classification, object detection and semantic segmentation as well as for integrated approaches to higher level tasks including environment mapping and video understanding. While their usefulness is clear, our understanding of how best to design these models remains far from complete.
The goal of this workshop was to bring together researchers to discuss questions and ideas relating to various aspects of the structure and design of DNNs. We considered in particular the following the two questions: (1) What we have learned as a community from our experience of designing these models? (2) Which research directions are most promising for improving existing architectures?
As part of the workshop, we are collecting suggestions (ongoing) for DNN-design related idea that "everyone should know about", which will be edited into a summary document that we hope will prove useful for members of the field. Add your suggestion for DNN-design related idea that "everyone should know about" here.
A short report of the day itself can be found here. A YouTube playlist of most of the invited talks can be found here.
Schedule
9:20 - 9:30 | Welcome (video, slides) | ||
9:30 - 10:00 | Neural Architecture Search and Beyond (video, slides) |
Barret Zoph (Google) |
|
10:00 - 11:10 | Poster presentations and coffee | Posters (in 308BC). | |
11:10 - 11:40 | Towards General Vision Architectures: Attentive Single-Tasking of Multiple Tasks (video, slides) | Iasonas Kokkinos (UCL/Ariel AI) |
|
11:40 - 12:10 | Deep Nets: What have they ever done for Vision? (video, slides) | Alan Yuille (Johns Hopkins) |
|
12:10 - 2:00 | Lunch | ||
2:05 - 2:20 | (Oral) Understanding the Effects of Pre-training for Object Detectors via Eigenspectrum (slides) | Yosuke Shinya, Edgar Simo-Serra, Taiji Suzuki | |
2:20 - 2:35 | (Oral) GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyond (slides) | Yue Cao, Jiarui Xu, Stephen Lin, Fangyun Wei, Han Hu | |
2:35 - 2:50 | (Oral) 4-Connected Shift Residual Networks (slides) | Andrew Brown, Pascal Mettes, Marcel Worring | |
2:50 - 2:55 | Best Paper Award | ||
3:00 - 3:30 | Capsule Architectures (slides) | Sara Sabour (Google) |
3:30 - 3:50 | Tea and coffee |
3:50 - 4:20 | From Architecting Networks to Architecting Network Generators (video, slides) | Ross Girshick (FAIR) |
4:25 - 5:20 | Panel discussion | 5:20 - 5:25 | Closing Remarks |
Accepted papers
Oral Presentations
Understanding the Effects of Pre-training for Object Detectors via Eigenspectrum | Yosuke Shinya (DENSO CORPORATION), Edgar Simo-Serra (Waseda University), Taiji Suzuki (The University of Tokyo / RIKEN) |
GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyond | Yue Cao (Tsinghua University), Jiarui Xu (Hong Kong University of Science and Technology (HKUST)), Stephen Lin (Microsoft Research), Fangyun Wei (Microsoft Research Asia), Han Hu (Microsoft Research Asia) |
4-Connected Shift Residual Networks | Andrew Brown (University of Amsterdam), Pascal Mettes (University of Amsterdam), Marcel Worring (University of Amsterdam) |
Long Papers
EENA: Efficient Evolution of Neural Architecture | Hui Zhu (Institute of Computing Technology, Chinese Academy of Sciences and University of Chinese Academy of Sciences), Zhulin An (ICT, CAS), Chuanguang Yang (ICT, CAS), Kaiqiang Xu (ICT, CAS), Erhu Zhao (ICT, CAS), Yongjun Xu (ICT, CAS) |
Sequentially Aggregated Convolutional Networks | Yiwen Huang (Huazhong University of Science and Technology Wenhua College), Rihui Wu (University of Sydney), Pinglai Ou (Virginia Tech), Ziyong Feng (DeepGlint) |
Resource Efficient 3D Convolutional Neural Networks | Okan Köpüklü (Technical University of Munich), Neslihan Kose Cihangir (Intel Deutschland GmbH), Ahmet Gündüz (Technical University of Munich), Gerhard Rigoll (Institute for Human-Machine Communication, TU Munich, Germany) |
Cross-Granularity Attention Network for Semantic Segmentation | Lingyu Zhu (Tampere University), Tinghuai Wang (Nokia Technologies), Emre Aksu (Nokia Technologies), Joni-Kristian Kamarainen (Tampere University) |
HM-NAS: Efficient Neural Architecture Search via Hierarchical Masking | Shen Yan (Michigan State University), Biyi Fang (Michigan State University), Faen Zhang (AInnovation), Yu Zheng (Michigan State University), Xiao Zeng (Michigan State University), Hui Xu (AInnovation), Mi Zhang (Michigan State University) |
BinaryDenseNet: Developing an Architecture for Binary Neural Networks | Joseph Bethge (Hasso Plattner Institute), Haojin Yang (Hasso Plattner Institute), Marvin Bornstein (Hasso Plattner Institute), ChristophMeinel (Hasso Plattner Institute) |
Efficient Structured Pruning and Architecture Searching for Group Convolution | Ruizhe Zhao (Imperial College London), Wayne Luk (Imperial College London) |
Attention routing between capsules | Jaewoong Choi (Seoul National University), Hyun Seo (Seoul National University), Suii Im (Seoul National University), Myungjoo Kang (Seoul National University) |
Adaptive Convolutional Kernels | Julio c Zamora (Intel), Jesus Adan Cruz Vargas (intel), Paulo Lopez Meyer (Intel), Omesh Tickoo (Intel) |
Adaptive Activation Functions Using Fractional Calculus | Julio c Zamora (Intel), Jesus Adan Cruz Vargas (intel), Rodrigo Camacho (Intel), Paulo Lopez Meyer (Intel), Hector Cordourier (Intel), Omesh Tickoo (Intel) |
SqueezeNAS: Fast neural architecture search for faster semantic segmentation | Albert E Shaw (DeepScale), Daniel Hunter (DeepScale), Forrest Iandola (DeepScale), Sammy Sidhu (DeepScale) |
Short Papers
Matrix Nets: A New Deep Architecture for Object Detection | Abdullah Rashwan (University of Waterloo), Agastya Kalra (University of Waterloo), Pascal Poupart (University of Waterloo) |
Localizing Occluders with Compositional Convolutional Networks | Adam Kortylewski (Johns Hopkins University), Qing Liu (Johns Hopkins University), Huiyu Wang (Johns Hopkins University), Zhishuai Zhang (Johns Hopkins University), Alan Yuille (Johns Hopkins University) |
MSNet: Structural Wired Neural Architecture Search for Internet of Things | Hsin-Pai Cheng (Duke University), Tunhou Zhang (Duke University), Yukun Yang (Duke University), Feng Yan (University of Nevada, Reno), Harris Teague (Qualcomm AI Research), Yiran Chen (Duke University), Hai Li (Duke University) |
HoloGAN: Unsupervised learning of 3D representations from natural images | Thu H Nguyen-Phuoc (University of Bath), Chuan Li (Lambda Labs), Lucas Theis (Twitter), Christian Richardt (University of Bath), Yongliang Yang (University of Bath) |
Searching for Accurate Binary Neural Architectures | Mingzhu Shen (Beihang University), Kai Han (Huawei Noah's Ark Lab), Chunjing Xu (Huawei Noah's Ark Lab), Yunhe Wang (Huawei Noah's Ark Lab) |
AdvGAN++ : Harnessing latent features for adversary generation | Puneet Mangla (IIT Hyderabad)*, Surgan Jandial (IIT Hyderabad), Vineeth N Balasubramanian (Indian Institute of Technology, Hyderabad), Sakshi Varshney (IIT Hyderabad) |
Layer-Wise Invertibility for Extreme Memory Cost Reduction of CNN Training | Tristan Hascoet (Kobe University), Quentin Febvre (Sicara), Weihao Zhuang (Kobe University), Yasuo Ariki (Kobe University), Tetusya Takiguchi (Kobe University) |
Investigating Convolutional Neural Networks using Spatial Orderness | Rohan Ghosh (National University of Singapore), Anupam Gupta (National University of Singapore) |
Why Does Data-Driven Beat Theory-Driven Computer Vision? | John Tsotsos (York University), Iuliia Kotseruba (York University), Alexander Andreopoulos (IBM Research Almaden), Yulong Wu (Aion Foundation) |
Program Committee
We are deeply indebted to the following who generously gave of their time and expertise to assist in paper selection:
A. S. Koepke, Albert Pumarola, Bichen Wu, Chao Peng, Chuhan Zhang, Dan Xu, Gang Sun, Hakan Bilen, Jack Turner, Jack Valmadre, James Thewlis, Ji Liang, Jiankang Deng, Kai Han, Liu Leqi, Lukas Neumann, Oriane Simeoni, Peter Vajda, Qiang Guo, Robert McCraith, Ruth C. Fong, Sergi Caelles, Shangzhe Wu, Song Bai, Tengda Han, Tom Gunter, Vedanuj Goswami, Vicky Kalogeiton, Weidi Xie, Weilin Huang, Xia Li, Xiaoliang Dai, Xu Ji, Yang Liu, Yuki Asano, Zihang Lai
All reviews were of a high standard and we would additionally like to highlight the outstanding reviews contributed by:
Jack Valmadre, Sergi Caelles, Jack Turner and Vicky Kalogeiton.
Organisers
Sponsor
We are grateful to Momenta for providing sponshorship and support for this workshop.
Venue
This workshop takes place on Monday 28th October in 308BC of the COEX Convention Centre, Seoul. More details about the main conference are provided on the ICCV website. The workshop has been assigned the poster boards 67 - 83 (inclusive). There is no predefined ordering for these boards, so please feel free to put up your poster whenever you arrive on the first available board within this range.
Contact
Contact email for any queries relating to the workshop: albanie[AT]robots.ox.ac.uk