Neural Architecture Search: SOTA till 2018

published on

The problem is to automatically find the optimal neural net architecture for your task, rather than explicitly code a nn.Module inherited model (say in Pytorch).

  1. 2016: The early methods were based on policy gradient methods (RL).
  2. 2018: DARTS was based on casting this problem as an alternating minimization problem and not resorting to RL.

Impression: This is a combinatorial problem (actually mixed-integer), so the heuristics that have been proposed in the literature seem very elementary.