QA4Agents

QA4Agents

Over the last years, advances in AI technologies have led to the widespread diffusion of conversational agentic systems, namely systems capable of interacting with users and external systems through natural language and executing tasks to answer given requests.

As these systems are becoming increasingly integrated into everyday life, issues such as incorrect outputs, security vulnerabilities, misinformation propagation, and poor robustness in understanding user requests can raise serious concerns.

Ensuring the quality of conversational agentic systems is an emerging research challenge. Their conversation-driven nature and non-deterministic behavior limit the straightforward application of traditional testing and validation systems, and introduce difficulties in establishing clear expected outputs and reliable oracle mechanisms.

QA4AGENTS is a workshop on quality assurance of conversational agentic systems, with a specific focus on the challenges introduced by their conversational nature. The workshop targets a broad spectrum of systems that use natural language to interact with users and external components, ranging from traditional NLP-based chatbots to LLM-based assistants and agentic systems. The goal is to assess the quality of these systems across multiple dimensions, including functional correctness, robustness to adversarial or unexpected inputs, safety and ethical compliance, security, and overall system reliability.

The workshop welcomes contributions from different research communities, including software engineering, software testing, AI safety, security, and dependable AI, with the goal of fostering interdisciplinary discussion and cross-community collaboration.

QA4AGENTS aims at bringing together researchers and practitioners to present research contributions, emerging ideas, and ongoing work on quality assurance of conversational agentic systems and to foster discussion on the key challenges in this area.