Exploring the Runtime Performance of Knowledge Graph Embedding Methods

In recent years, Knowledge Graphs (KGs) have become ubiquitous, powering recommendation systems, natural language processing, and query answering, among others. Moreover, representation learning on graphs has reached unprecedentedly effective graph mining. In particular, Knowledge Graph Embedding (KGE) methods have gained increasing attention due to their effectiveness in representing real-world structured information while preserving relevant properties. Current research mainly focuses on improving and comparing the effectiveness of new KGE models on different predictive tasks. However, the application of KGE techniques in the industrial scenario sets a series of requirements on the runtime performance of the employed models. For this reason, this work aims to enable an effortless characterization of the runtime performance of KGE methods in terms of memory footprint and execution time. To this extent, we propose KGE-Perf, a framework for evaluating available state-of-the-art implementations of KGE models against graphs with different properties, focusing on the efficacy of the adopted optimization strategies. Experimental evaluation of three representative KGE algorithms on open-access KGs shows that multi-threading on CPU is effective, but its benefits decrease as the number of threads grows. The usage of vectorized instruction shows encouraging results in speeding up the training of KGE models, but GPU proves, hands down, to be the best architecture for the given task. Moreover, experimental results show how the RAM usage strongly depends on the input KG, with only slight variations between different models or hardware configurations.