Deep Surrogate Models of Serverless Batch Processing Services

Serverless computing breaks down applications into workflows of stateless functions, where the output of each function serves as the input for the next. When these workflows are distributed across multiple processing nodes, managing the interactions between nodes is crucial to maintaining overall system performance. However, existing analytical performance models do not cope well with the dependencies involved in distributed workflows when the offloaded jobs arrive in batches. In this paper, we study the problem for two processing resources in tandem and develop a scalable surrogate modeling approach based on neural networks that can be used for serverless resource management purposes. We validate our performance model with both synthetic and real-world AI application traces, demonstrating that our surrogate model achieves a mean average percentage error of about 5%.