Leveraging LLMs to Automate Software Architecture Design from Informal Specifications

Designing a software architecture starting from specifications and requirements is a time-consuming and errorprone process that demands domain expertise. Automating this process has become a significant research focus in software engineering. Traditional approaches rely on rule-based mechanisms to translate manually derived, standardized requirements into the desired architecture. However, these methods struggle to identify implicit patterns without expert intervention. Recently, approaches leveraging Large Language Models (LLMs) have gained attention. This study evaluates the performance of LLMs in generating software architecture blueprints, specifically UML component diagrams, from informal natural-language specifications. We develop a formal characterization of component diagrams to derive quantitative metrics for analyzing LLM-generated diagrams, comparing them against expert-drawn ground truths associated with the specifications. Our findings indicate that while LLM-based approaches show promise in addressing the flaws of rule-based methods, they currently lack the accuracy needed for deployment in real-world scenarios.