Quality assurance for large-scale cyber-physical systems relies on sophisticated test activities using complex test environments investigated with the help of numerous types of simulators. As these systems grow, extensive resources are required to develop and maintain simulation models of hardware and software components, as well as physical environments. Meanwhile, recent advances in generative AI have led to tools that can produce executable test cases for software systems, offering potential benefits such as reducing manual efforts or increasing test coverage. However, the application of generative AI techniques to simulation-based testing of large-scale cyber-physical systems remains underexplored. To better understand this gap, this study captures practitioners’ perspectives on leveraging generative AI, based on a cross-company workshop with six organizations. Our contribution is twofold: (1) detailed, experience-based insights into challenges faced by engineers, and (2) a research agenda comprising three high-priority directions: (a) AI-generated scenarios and environment models, (b) simulators and AI in CI/CD pipelines, and (c) trustworthiness in generative AI for simulation. While participants acknowledged substantial potential, they also highlighted unresolved challenges. By detailing these issues, the paper aims to guide future academia-industry collaboration towards the responsible adoption of generative AI in simulation-based testing.

Conference CORE rank: B

Conference acceptance rate: 37% (The Research Papers track received 62 submissions, of which 23 papers were accepted as full research papers.)

Presentation

DOI

A popular research area in recent years has been mining software repositories of reliable free/open-source software projects to harness the crowd’s wisdom. Due to the growing number of available technologies, selecting an appropriate one has become a challenge for software architects. So, building a recommender system is a suitable solution for supporting architects in technology selection. For building these systems, researchers use a variety of machine learning methods. However, the quality and size of the project have not been examined. Additionally, their recommendations do not perform well enough considering the recent advancements in deep learning algorithms, and they have the cold start problem. In this study, two recommender systems are developed once quality data is extracted. The first is a deep learning-based recommender called DeepLibAide, while the second is a content-based recommender called ContentLibAide, and a hybrid recommender that solves the cold start problem. Furthermore, we compare the results of their implementation on different samples based on different levels of project quality. DeepLibAide shows significant improvement in the criteria of accuracy, recall, and normalized cumulative gain and training loss with respect to the baseline method. This is because it uses the linear activation function, removing feature transformation and summing the embedding vectors of different layers. A dataset containing only high-quality projects and a dataset containing a random sample of projects were compared for precision and recall. On average, DeepLibAide improves recall criteria by 7 percent. Throughout the pipeline architecture, raw input data is extracted or updated, samples are built, and models are trained. Several parameters are used in the pipeline, and recommendations are stored. After that, various diagrams are used to evaluate and compare them with the baseline method. It is possible to execute the entire program automatically and with negligible overhead.

Supervisor: Sadegh Aliakbary

Opponent: Hasan Haghighi