AutoML Grows Up in 2026
NAS began as a technique for finding CNN architectures for image classification. The renaissance comes from applying it to the right problems: optimizing specific transformer components, not the full architecture in a vacuum.
Micro vs Macro Search
Modern NAS distinguishes between macro search — overall model structure — and micro search — the design of individual components like attention mechanisms, FFN ratios, or positional encoding schemes. Micro search is far more tractable computationally.
Efficient NAS Methods
Weight sharing approaches and differentiable NAS methods have reduced architecture search from thousands of GPU-hours to tens of GPU-hours for targeted optimization tasks — making NAS viable for mid-size organizations.
Frequently Asked Questions
What is neural architecture search?
Neural Architecture Search (NAS) uses automated algorithms to design neural network architectures, replacing manual trial-and-error. NAS searches possible architectures to maximise performance for a target task and hardware constraint.
Is NAS still used in 2026?
Yes, particularly hardware-aware NAS for edge and mobile deployment. NAS is commonly applied to sub-component design (attention variants, FFN designs) and efficient small-model discovery.
What are the main types of NAS?
Main NAS paradigms: reinforcement learning-based NAS (controller proposes architectures), evolutionary search (mutations and selection), and differentiable NAS like DARTS (continuous optimisation for efficiency).