Training Encrypted Neural Networks on Encrypted Data with Fully Homomorphic Encryption

Training machine and deep learning models on encrypted data is the next challenge in the field of privacy-preserving Machine and Deep Learning. The related literature in this field is very limited, since most of the solutions focus only on inference on encrypted data (leaving the training to be carried out on plain data). In this paper we introduce a multi-class and non-linear family of neural networks based on the Torus Fully Homomorphic Encryption (TFHE) scheme (named TFHE-NNs), which can be entirely trained on encrypted data. The proposed learning procedure, implementing a TFHE-compliant version of the Direct-Feedback-Alignment algorithm, is combined with a novel Cross-Validation procedure able to operate on encrypted models and encrypted accuracy. The experimental results demonstrate the feasibility of the proposed solution. The proposed models and algorithms are made available to the scientific community as a public repository.