Processing financial market data at scale and in real-time poses a set of unique challenges to event-driven architectures due to the volume, variety, velocity, and veracity of the enclosed information on top of other constraints. Reproducible stress tests at scale using configurable benchmarks are key to building and tuning suitable processing systems. Available benchmarks, however, lack realistic and configurable workload models for market data scenarios. In previous work we already addressed this gap by describing the specific challenges of processing financial data at scale and by introducing a modular open-source benchmarking framework. This paper makes two contributions to the ongoing challenge of building realistic benchmarks for the financial data processing domain by outlining: (a) a detailed statistical analysis of real-world financial market data feeds processed on a global scale by Infront Financial Technology GmbH; and (b) a simple workload model built on this analysis to simulate high-volume market data feeds with their distinctive characteristics to be used in benchmarks. We evaluate our model using the DEBS 2022 Grand Challenge data set Trading Data.

Deriving a realistic workload model to simulate high-volume financial data feeds for performance benchmarking

Alessandro Margara
2022-01-01

Abstract

Processing financial market data at scale and in real-time poses a set of unique challenges to event-driven architectures due to the volume, variety, velocity, and veracity of the enclosed information on top of other constraints. Reproducible stress tests at scale using configurable benchmarks are key to building and tuning suitable processing systems. Available benchmarks, however, lack realistic and configurable workload models for market data scenarios. In previous work we already addressed this gap by describing the specific challenges of processing financial data at scale and by introducing a modular open-source benchmarking framework. This paper makes two contributions to the ongoing challenge of building realistic benchmarks for the financial data processing domain by outlining: (a) a detailed statistical analysis of real-world financial market data feeds processed on a global scale by Infront Financial Technology GmbH; and (b) a simple workload model built on this analysis to simulate high-volume market data feeds with their distinctive characteristics to be used in benchmarks. We evaluate our model using the DEBS 2022 Grand Challenge data set Trading Data.
2022
DEBS '22: Proceedings of the 16th ACM International Conference on Distributed and Event-Based Systems
9781450393089
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: https://hdl.handle.net/11311/1228552
Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 1
  • ???jsp.display-item.citation.isi??? ND
social impact