Repository-Aware Metamorphic Relation Generation for Augmented Reality Applications using Large Language Models
Source
Dibyendu Brinto Bose, Jiawei Qin, Chris Brown
Author
Dibyendu Brinto Bose, Jiawei Qin, Chris Brown
Date
Key takeaways · AI-distilled
Metamorphic relations sidestep the AR test-oracle problem - dynamic interplay of virtual content, physical environment and code makes exact-output oracles impractical - but writing MRs by hand is the bottleneck this pipeline removes.
Repository-level hierarchical context windowThe maximum amount of text a model can consider at once — its working memory for the current conversation or task.Full definition → produced the broadest, least redundant generation: 7,004 candidate MRs across 142 mobile AR repositories and 5,167 class-method pairs.
Conflicting candidate MRs showed up in 79% of cases; an agentic deliberation step reconciles them and picks context-aware relations in 88.2% of outcomes.
Validation: a manual oracle study found refined relations concrete enough to translate directly into test assertions, and converted MRs caught non-equivalent mutations in real-world code.
Terms in this piece · Glossary
LLM — A large language model — the neural network behind tools like Claude and ChatGPT, trained on huge amounts of text to predict what comes next.
context window — The maximum amount of text a model can consider at once — its working memory for the current conversation or task.
Why it matters
If you're building test automation for AR/mobile apps where correctness oracles are hard to define, this shows a concrete method for using LLMs plus repository context to generate and validate metamorphic test relations that catch real code mutations.
Key quotes
“Metamorphic Testing (MT) provides a promising approach for testing software without defined test oracles by specifying expected relations between inputs and outputs, instead of relying on exact outputs.”
“An agentic deliberation process then reconciled conflicting candidates---observed in 79.0% of cases---reducing duplication and selecting context-aware relations in 88.2% of outcomes.”
articleWhen Knowledge Changes: Metamorphic Testing of RAG Systems with MutationsJinhan Kim, Samuele Pasini, Paolo Tonella
articleMetaSpace: Metamorphic Testing for Spatial Cognition in Embodied AgentsGengyang Xu, Dongwei Xiao, Yiteng Peng, Shuai Wang
articleTangent: An Empirical Study of Testing Practices for LLM-Based Agent ApplicationsRangeet Pan, Tyler Stennett, Divya Sankar, Bridget McGinn, Alessandro Orso, Raju Pavuluri, Saurabh Sinha, Maja Vukovic