Scalable Policy-as-Code Decision Points for Data Products

Data products are emerging as architectural elements of data mesh with the aim of improving data management within organizations. Among the main aspects in the design of a data product is the need to define the policies that regulate its access. These policies concern not only security aspects but, more generally, compliance with regulations (e.g., GDPR, HIPAA) or organizational regulations. Taking advantage of similarities with microservice-based solutions, policy-as-code is also used in the data mesh to regulate access to data products. Following the proposed models for the service mesh which regulates a microservice-based environment, the performance of the Policy Decision Point (PDP) becomes crucial to ensure efficient access to data managed by data products. Since this may require a replication of PDPs, in order to achieve true scalability, each of them will have to take into account only the policies associated with the data products under their responsibilities. This work provides a solution to automate the replication of PDPs and the distribution of policies to be controlled according to an approach that balances policy consistency and system scalability. The work has been validated through an extension of the Kubernetes orchestrator that, based on the information collected from the monitoring system, it is able to define which is the right number of PDPs and which data product, and related policies, should be assigned to each of them. Notably, the policy management is based on a Open Policy Agent (OPA) implementation.