SMX: A RISC-V ISA Extension for Scale-Adaptive Quantized Softmax

The softmax function is a fundamental component of modern neural network inference, yet it represents a significant performance bottleneck on general-purpose processors due to its computational complexity and reduction patterns. As a result, softmax often limits inference throughput on scalar RISC-V cores used in embedded and clustered systems. This paper presents an instruction-level approach to accelerating INT8 softmax on scalar RISC-V processors through a tightly coupled hardware-software co-design. We first introduce an adaptive two-dimensional lookup-table (2D LUT) softmax algorithm that compensates for runtime input scale mismatch without requiring prior knowledge of quantization parameters. Building on this formulation, we define a compact RISC-V ISA extension that exposes the multi-phase softmax computation through a small set of custom instructions operating on packed INT8 data. The instructions are executed by a dedicated softmax functional unit integrated into the processor pipeline, enabling instruction-level acceleration without relying on floating-point datapaths or external accelerators. Experimental results show that the proposed adaptive algorithm significantly improves probability estimation fidelity under scale mismatch while preserving ranking behavior. Cycle-level evaluation on a RISC-V core demonstrates speedups of up to 984× over a software FP32 baseline and more than 25× over a software-only INT8 implementation.