Efficient Automation of Neural Network Design: A Survey on Differentiable Neural Architecture Search. (arXiv:2304.05405v1 [cs.LG])

In the past few years, Differentiable Neural Architecture Search (DNAS)
rapidly imposed itself as the trending approach to automate the discovery of
deep neural network architectures. This rise is mainly due to the popularity of
DARTS, one of the first major DNAS methods. In contrast with previous works
based on Reinforcement Learning or Evolutionary Algorithms, DNAS is faster by
several orders of magnitude and uses fewer computational resources. In this
comprehensive survey, we focus specifically on DNAS and review recent
approaches in this field. Furthermore, we propose a novel challenge-based
taxonomy to classify DNAS methods. We also discuss the contributions brought to
DNAS in the past few years and its impact on the global NAS field. Finally, we
conclude by giving some insights into future research directions for the DNAS
field.