YoseUe: “trimming” Random Forest’s training towards resource-constrained inference

Endowing artificial objects with intelligence is a longstanding computer science and engineering vision that recently converged under the umbrella of Artificial Intelligence of Things (AIoT). Nevertheless, AIoT's mission cannot be fulfilled if objects rely on the cloud for their "brain," at least concerning inference. Thanks to heterogeneous hardware, it is possible to bring Machine Learning (ML) inference on resource-constrained embedded devices, but this requires careful co-optimization between model training and its hardware acceleration. This work proposes YoseUe, a memory-centric hardware co-processor for Random Forests (RFs) inference, which significantly reduces the waste of memory resources by exploiting a novel train-acceleration co-optimization. YoseUe proposes a novel ML model, the Multi-Depth Random Forest Classifier (MDRFC), in which a set of RFs are trained at decreasing depths and then weighted, exploiting a Neural Network (NN) tailored to counteract potential accuracy losses w.r.t. classical RFs. With this modeling technique, first proposed in this paper, it becomes possible to accelerate the inference of RFs that count up to 2 orders of magnitude more Decision Trees (DTs) than those the current state-of-the-art architectures can fit on embedded devices. Furthermore, this is achieved without losing accuracy with respect to classical, full-depth RF in their most relevant configurations.