Automated and Disciplined ConvNet Architecture Exploration

Abstract

Convolutional neural networks (CNNs) are widely used for diverse tasks, such as image recognition and analysis. Recently, research aimed at finding CNN architectures that can be used in various contexts/applications and provide the latest performance has yielded fruitful results, resulting in numerous recommendation models tailored for more or less specific purposes. Finding the right CNN is a challenging issue - there are many possible architectures, hyperparameters, and frameworks that can be considered. From a software engineering perspective, having such diversity can be difficult to deal with when trying to maintain a system or trying to reason effectively (for example, consider choosing the best solution for deployment on a system with high potential impact on daily life). In this master thesis, we investigate how variability can be expressed to derive different CNN variants. We develop a generator on top of Keras for deriving variants of LeNet, ResNet, and DenseNet architectures. Our results show that we can reach accurate results on MNIST and CIFAR-10. The next step of our work is to improve the generator for other architectures (e.g. Xception, SqueezeNet,…) and find optimal ways to explore the configuration space.