POMONAG: Pareto-Optimal Many-Objective Neural Architecture Generator

Neural Architecture Search (NAS) automates neural network design, reducing dependence on human expertise. Whilst NAS methods are computationally intensive and dataset-specific, utilising auxiliary predictors to estimate architecture properties has proved advantageous, reducing the number of models requiring training and search time. This strategy is employed to generate architectures meeting multiple computational constraints. Transferable NAS generalised the search from dataset-dependent to task-dependent. In this domain, DiffusionNAG represents the state of the art. This diffusion-based method generates architectures optimised for accuracy on unseen datasets without requiring adaptation. However, by focusing solely on accuracy, DiffusionNAG neglects crucial objectives such as complexity, computational efficiency and latency essential factors for deployment in resource-constrained environments. This paper introduces POMONAG (Pareto-Optimal Many-Objective Neural Architecture Generator), extending DiffusionNAG through a many-objective diffusion process. POMONAG simultaneously considers accuracy, parameter count, multiply-accumulate operations and latency. It integrates Performance Predictor models to estimate secondary metrics and guide diffusion gradients. Optimisation is enhanced by expanding the training Meta-Dataset, applying Pareto Front Filtering and refining embeddings for conditional generation. These improvements enable POMONAG to generate Pareto-optimal architectures that surpass the state of the art in both performance and efficiency. Results were validated on two search spaces NASBench201 and MobileNetV3 and evaluated across 15 image classification datasets.