More Information

Background

AI assistants like GitHub Copilot are generating an increasing share of production code. As a result, the bottleneck in quality assurance is shifting from writing to testing: developers adopt generated code without fully understanding its edge cases and implicit assumptions. However, traditional testing methods—from unit and property-based testing to mutation testing and end-to-end testing—were developed for code written by humans. It remains unclear which of these methods actually uncover the typical error patterns found in AI-generated code. At the same time, agent-based tools open up the possibility of automating test selection itself: A test agent could analyze newly generated code, determine the necessary testing requirements, and generate appropriate test cases based on them.

Research Objective

The target of this work is to systematically evaluate existing testing methods in terms of their suitability for AI-generated code and to implement the findings in a prototype test agent. The agent is intended to derive test requirements from newly written or generated code, formalize verifiable test requirements, and generate test cases from them. Possible areas of focus include:

• Systematization: Review of current testing methods and development of a catalog of criteria for their suitability for AI-generated code
• Error Pattern Analysis: Identification of typical error types in AI-generated code and their assignment to suitable testing methods
• Agent Design: Design and prototypical implementation of a Copilot-based test agent (code extraction, derivation of test requirements, generation of test cases)
• Integration: Expansion to include end-to-end test execution, including automated evaluation of test results