Workshop on the Verification 

of Autonomous Systems (VAS)

A Workshop at ICRA

 The 39th IEEE International Conference on Robotics and Automation, 

Philadelphia, PA, USA, 23 May 2022.

https://robotistry.org/vaswg/ICRA22_Workshop/

Registration:

Workshop registration (whatever in-person or virtual) should be completed through ICRA main website: https://www.icra2022.org/registration/registration-fee


Overview:

Organisers:

With the rising autonomy of devices, softwares, and systems, there is a clear need to provide convincing evidence of their safety, security, transparency, reliability, dependency, and resilience to users and other stakeholders. Consequently, verification, i.e. compelling evidence that autonomous systems satisfy their requirements, has become increasingly important, especially for building explainable technologies. As autonomous systems become more complex, with added intelligence and adaptive capabilities, the challenges for verification grow. Worldwide efforts to devise methodologies and develop tools related to the verification of autonomous systems are thus crucial. 


Our goal in this workshop is to bring together experts in robotics, system design, and verification, with industrial partners to discuss the main challenges in the verification of autonomous systems. We will explore the domain of verification of autonomous systems in terms of requirements, techniques, tools, architectures and successful applications. A major goal will be to find common ground to address the challenges relating to design and verification in embodied robotic systems and vehicles, to ensure trustworthy human-robot cooperation and collaboration. We aim to open new avenues for future interdisciplinary collaboration among participants and expect that this will significantly advance the field of verification of autonomous systems. 

Structure:

This full-day workshop comprises a modular set of invited oral presentations followed by Panel discussion with IEEE-RAS TC-VAS group members and other speakers from both Academia and Industry.


Note that this is the fourth in a series of workshops at major conferences and will follow the format established in the previous workshops. The previous workshops were at ICRA 2016, IROS 2017, and ICRA 2018.

Agenda:

Each session will consist of short panelist presentations followed by panel discussions. The detailed agenda is as follows (with times mentioned in the Philadelphia, PA, USA time zone: EDT, UTC/GMT-4).

8:30-9:00 EDT

Workshop Co-chairs, Welcome, Introduction of IEEE-RAS TC-VAS and the VAS workshop

9:00-10:30 EDT

Panel 1: Requirements and Metrics for the Verification of Autonomous Systems

Session Co-Chairs:


Joanna Isabelle Olszewska (University of the West of Scotland, UK)
Javier Ibanez-Guzman (Renault S. A., France)



Talks:


1.1 'Defining Requirements for Autonomous Systems'
by Alessandro Pinto (Raytheon Technologies, USA)


1.2 'Functional Specification of Autonomous Behavior and Tasking'
by Signe Redfield (Naval Research Laboratory, USA)


1.3 'Tasking Standards and Interfaces for Agile Behavior'
by Stephen Balakirsky (Georgia Institute of Technology, USA)


1.4 'Measuring Autonomous Systems'
by David Scheidt (Weather Gage Technologies, LLC, MD, USA)


1.5 'Reasonable Requirements on Data to Validate Safe Perception'
by Roland Meertens (Annotell, Germany)


Panel discussion moderated by session co-chairs:


What are the requirements and metrics for the verification of autonomous systems?

10:30-11:00 EDT

Coffee break (in-person coffee break & chat and/or virtual breakout sessions to socialize and chat)

11:00-12:30 EDT

Panel 2: Formal Approaches to and Standards for Verifying Autonomous Systems

Session Co-Chairs:


Dejanira Araiza-Illan (Johnson & Johnson, Singapore)
Alessandro Pinto (Raytheon Technologies, USA)



Talks:


2.1 'Formal Approaches, Ontologies, and Standards for the Verification of Autonomous Systems'
by Joanna Isabelle Olszewska (University of the West of Scotland, UK)


2.2 'Formal Verification of Real-Time Autonomous Robots: An Interdisciplinary Approach'
by Mohammed Foughali (Université de Paris/IRIF, France


2.3 'IEEE ECPAIS Ethics Certification Criteria: Laying the Framework for the Verification of Autonomous Intelligent Systems'
by Zvikomborero Murahwi (Gratia ICT Projects Advisory Ltd, South Africa)


2.4 'Judgment Proxies for Certification of Autonomous Systems'
by Donald Costello (United States Naval Academy, MD, USA)


2.5 'An ODD-based Hazard Identification Framework for Artificial Intelligence-based Systems'
by Guillaume Ollier (Commissariat à l’énergie atomique (CEA), France)


2.6 'Formal Methods and Autonomous Vehicles: Addressing Complexity'
by Javier Ibanez-Guzman (Renault S.A, France)


Panel discussion moderated by session co-chairs:


Which formal and semi-formal approaches should be used to verify autonomous systems?

12:30-13:00 EDT

Meal break

13:00-14:30 EDT

Panel 3: Safety for Learning-Enabled Autonomous Systems

Session Co-Chairs:


Srishti Dhamija (John Hopkins University, USA)
Michael Fisher (University of Manchester, UK)



Talks:


3.1 'Safety Verification of Autonomous Systems: a Multi-Fidelity Reinforcement Learning Approach'
by Ali Baheri (West Virginia University, USA)


3.2 'Execution Failures and Learning-Based Robots: (How) Can Robots Analyse Their Own Failures?'
by Alex Mitrevski (Hochschule Bonn-Rhein-Sieg, Germany)


3.3 'Interrogating Performance Characteristics in Learning-enabled Autonomous Systems'
by Subramanian Ramamoorthy (University of Edinburgh, UK)


3.4 'Measures of Generalization Capability for Verifying and Predicting the Performance of ANN Algorithms'
by Darryl Hond (Thales, UK)


3.5 'Safe Exploration and Learning of Nonlinear Dynamics: A Chance-Constraint Approach'
by Yashwanth Kumar Nakka (NASA Jet Propulsion Laboratory, USA)


3.6 'Learning that Avoids Known Dangers and Unknown States'
by Christian Ellis (University of Massachusetts at Dartmouth, USA)


Panel discussion moderated by session co-chairs:


How to ensure safety of learning-enabled autonomous systems?

14:30-15:00 EDT

Coffee break (in-person coffee break & chat and/or virtual breakout sessions to socialize and chat)

15:00-16:30 EDT

Panel 4: Trusting Autonomous Systems

Session Co-Chairs:


Signe Redfield (US Naval Research Laboratory, USA)
Tom Freund (Dig.y.Sol, USA)



Talks:


4.1 'Trusting Autonomous Systems'
by Michael Fisher (University of Manchester, UK)


4.2 'Trustworthiness in Urban Air Mobility'
by Tom Freund (Dig.y.Sol, USA)


4.3 'Verification of Maritime Autonomous Systems by Means of a Modelling and Simulation Framework'
by Arnau Carrera Viñas (NATO STO Centre for Maritime Research and Experimentation (CMRE), Italy)


4.4 'Abstractions for Formal Verification of AI Controlled Autonomous Systems'
by Pavithra Prabhakar (Kansas State University, USA)


4.5 'Towards a Tool for Managing Validation Arguments in Systems Engineering'
by Daniel Shapiro (University of California at Santa Cruz, USA)


4.6 'Safe Planning in Unstructured Semantic Environments'
by Yiannis Kantaros (Washington University in St. Louis, MO, USA)


Panel discussion moderated by session co-chairs:


How to increase trust in autonomous systems such as social robots?

16:30-17:00 EDT

Wrap up, summary of topics, follow up

Dejanira Araiza-Illan

Johnson & Johnson Ltd 

Singapore 


Michael Fisher

Univ. Manchester

Manchester, UK


Javier Ibanez-Guzman

Renault S.A 

Guyancourt, France 


Zvikomborero Murahwi 

Gratia ICT Projects Adv, Johannesburg, South Africa


Joanna Isabelle Olszewska

Univ. West of Scotland

Glasgow, UK


Alessandro Pinto

Raytheon Technologies Research Center, USA


Signe Redfield 

US Naval Research Lab Washington DC, USA