Difference between revisions of "Portal:Glossary"

From Verification Body of Knowledge
Jump to navigation Jump to search
(Created page with ";Human-robot interactions ;Acceptability ;Trust ;Safety in HRI ;Failure detection and recovery ;Redundant robots ;Robot safety ;Robotic Systems ;Assurance ;Functional assurance ;Assurance cases ;Specifications ;Requirements -workspace and conditions/context of operations ;Functional requirements ;Safety requirements ;Security requirements ;Requirements development processes Qualitative assessment Quantitative assessment Dependability Reliability Security...")
 
m (Added a few terms related to different types of robot operation)
 
(29 intermediate revisions by 6 users not shown)
Line 1: Line 1:
;Human-robot interactions
Offer different classifications and groupings - in main page
 
* Verification techniques (tree?)
* Metrics (grouping)
* Application area (e.g. ML systems, for HRI, for medical robots)
* Related concepts (e.g. all the specification and requirements terms together)
 
Contents of each linked page: 
* definitions with links to sources
* short discussion of how topic relates to verification of autonomous systems
 
'''''[[Template]]''''' for definition pages - please copy the markup from this page into your new definition page
 
 
;Acceptability
;Acceptability
;Trust
: exposure to loss (financial or otherwise) from a risk that an organization is willing to tolerate
;Safety in HRI
;Acceptance testing
;Accountability
;Activity
: set of tasks that describe the technical effort to accomplish a process and help generate expected outcome [https://www.nasa.gov/seh/index.html NASA Systems Engineering Handbook]
;[[Adaptive]]
;Assessment
;Assurance
;Assurance cases
;[[Autonomous System]]
: "system which has the ability to perform intended tasks based on current state, knowledge and sensing, without human intervention"  [https://shop.bsigroup.com/products/robots-and-robotic-devices-guide-to-the-ethical-design-and-application-of-robots-and-robotic-systems/standard BSI 8611:2016]
;Autonomous systems design
;Autonomy
;Automated
: a robot that does not make any decisions. It performs a precise sequence of actions without regard to external input. Even if a robot has sensors, that enable it to gather information about its environment, if they aren't used to modify its behavior, the robot is automated and acts as an automaton.
;Automation
: allocation of system functions to machines versus humans [https://www.nasa.gov/seh/index.html NASA Systems Engineering Handbook]
;Anomaly
: unexpected performance of intended function [https://www.nasa.gov/seh/index.html NASA Systems Engineering Handbook]
 
 
;Behavior verification for autonomous systems and ML systems
;Behavioral verification
 
 
;Combining V&V Techniques
;Contextual reasoning
;Correctness verification
;Credibility
;Cross-domain / multi-domain  validation
;Cyber-security
;Cyborg
: a system consisting of a combination of a core biological organism, usually human, and machine or mechanical components.
 
 
;Dependability
 
 
;Executable specifications
;Experiment design
;Explainability
 
;Failure
: inability of a system, subsystem or component to perform its required function within specified limits [https://www.nasa.gov/seh/index.html NASA Systems Engineering Handbook]
 
;Failure detection and recovery  
;Failure detection and recovery  
;Fault
: physical or logical cause, which explains a failure [https://www.nasa.gov/seh/index.html NASA Systems Engineering Handbook]
;Formal methods for autonomous  systems 
;Formal modelling
;Formal software verification
;Functional assurance
;Functional requirements
;Functional verification
;HRI (Human-robot interactions)
;HRI verification
;Hybrid Logical/Dynamical Planning and Verification
;Generalizability
;Integrity of autonomous systems
;Liability
;Licensing and  certification
;Model-based evaluation
;Objective evidence
: ISO 9000:2015 - "data, supporting the existence or verity of something" [https://bsol.bsigroup.com/Bibliographic/BibliographicInfoData/000000000030412048 ISO 9000:2015]
;Operational design domain
;Process
: "set of interrelated or interacting activities that use inputs to deliver an intended result" [https://bsol.bsigroup.com/Bibliographic/BibliographicInfoData/000000000030412048 ISO 9000:2015]
;Qualitative  assessment
;Quality requirement
: "requirement related to quality" [https://bsol.bsigroup.com/Bibliographic/BibliographicInfoData/000000000030412048 ISO 9000:2015]
;Quality plan
: "specification of the procedures and associated resources to be applied when and by whom to a specific object" [https://bsol.bsigroup.com/Bibliographic/BibliographicInfoData/000000000030412048 ISO 9000:2015]
;Quantitative  assessment
;Performance evaluation and benchmarking
;Privacy
;[[Reactive]]
;Redundant robots  
;Redundant robots  
;Robot safety  
;Reliability
;[[Remote Control]]
;Replicability
;Reproducibility
;Requirements
: Workspace and conditions/context of operations
 
;Requirement
: "need or expectation that is stated, generally implied or obligatory" [https://bsol.bsigroup.com/Bibliographic/BibliographicInfoData/000000000030412048 ISO 9000:2015]
 
;Requirements development processes
;Resilience
;[[Responsibility]]
: Either (a) responsible development and deployment of robotics, see [https://shop.bsigroup.com/products/robots-and-robotic-devices-guide-to-the-ethical-design-and-application-of-robots-and-robotic-systems/standard BSI 8611:2016] or
: (b) responsible decisions by the autonomous system itself, see [https://doi.org/10.1016/j.artint.2022.103677 ethics for autonomous systems]
;Robot
: See Robotic Systems
;Robot safety
;Robotic Systems  
;Robotic Systems  
;Assurance
: Robot systems engineering
;Functional assurance
;Robustness
;Assurance cases
;Run-time verification  and monitoring
;Specifications
 
;Requirements -workspace and conditions/context of operations
 
;Functional requirements
;Safety
;Safety and security interaction
;Safety in HRI
: safety, system function
: verification of any function in the system including ML & non-deterministic functions
;Safety-centric HRI protocols
;Safety requirements  
;Safety requirements  
;Security
;Security requirements
;Security requirements
;Requirements development processes
;[[Semi-autonomous]]
Qualitative  assessment
;Simulation-based testing  
Quantitative  assessment
;Sim-based evaluation
Dependability
;Social/legal/ethical requirements  
Reliability
;Software testing and debugging
Security
;Software tools for benchmarking and   
Cyber-security
;Software tools for testing  
Robustness
: software testing
Safety
;Specification
Trustworthiness
: "document stating requirements" [https://bsol.bsigroup.com/Bibliographic/BibliographicInfoData/000000000030412048 ISO 9000:2015]
Formal methods for autonomous systems 
;Synthetic environment
Resilience
: to develop high  level test scenarios, drive test case development
Behavior verification for autonomous systems and ML systems
 
Correctness verification - software testing  
 
Functional verification – safety, system function
;Teleoperated
Behavioral verification – verification of any function in the system including ML & non-deterministic functions
: a robot whose operator is controlling its every move, but where the command stream is not required to be continuous and the operator is using sensors on-board the robot for situational awareness. The operator may send higher level commands (e.g. "go to waypoint") and the robot will complete the last command it receives before it stops.
Cross-domain / multi-domain  validation
;Test cases for verification purposes
Safety and security interaction
;Testing
Social/legal/ethical requirements  
;Transparency  
Privacy
;Trust
Responsibility
;[[Trustworthiness]]
HRI verification
: Generally, [https://dictionary.cambridge.org/dictionary/english/trustworthy "able to be trusted"].
Safety-centric HRI protocols
: In autonomous systems, moving beyond standard cyber-physical systems where trustworthiess is synonymous with reliability and extending this with [https://doi.org/10.1007/978-3-030-69128-8_2 beneficiality], i.e. evidence that the system is working for our benefit.
Accountability
 
Liability
;Use-case scenario generation
Software tools for benchmarking and   
;Use Case testing  
Software tools for testing  
 
Explainability
 
Reproducibility
;Validation:
Hybrid Logical/Dynamical Planning and Verification
: further adds as "A system is able to accomplish its intended use, goals and objectives (i.e., meet stakeholder requirements) in the intended operational environment. The right system was built.” ISO/IEC/IEEE 15288:2015
Performance evaluation and benchmarking
;Verification:
Robot systems engineering
: defines verification as "Confirmation, through the provision of objective evidence, that specified requirements have been fulfilled" [https://bsol.bsigroup.com/Bibliographic/BibliographicInfoData/000000000030412048 ISO 9000:2015]
Software testing and debugging
: further adds as "Verification is a set of activities that compare a system or system element against the required characteristics. This includes, but is not limited to specified requirements, design description and the system itself. The system was built right." ISO/IEC/IEEE 15288:2015
Experiment design
;Verification and Validation Process
Acceptance testing
: Process of first, identifying the desired functionality and properties of the system, second, verifying that the system provides that functionality and possesses those properties, and third, validating that the system as delivered meets user needs.  
Model-based evaluation
 
Sim-based evaluation
 
Performance  evaluation
;Waldo
Replicability
: a mechanism that precisely repeats the actions of its operator (from the Robert Heinlein short story [http://en.wikipedia.org/wiki/Waldo_%28short_story%29 Waldo] from the book Waldo & Magic, Inc.)
Generalizability
 
Assessment
==Notes==
Transparency  
* Note that we need to discuss how we will handle definitions that are not part of standards and may not have agreed upon definitions
Formal software verification
Licensing and  certification
Run-time verification  and monitoring
Validation
Integrity of autonomous systems
Verification
Formal modelling
Credibility
Simulation-based testing
Operational design domain
Use Case testing  
Testing
Use-case scenario generation
Synthetic environment - to develop high  level test scenarios, drive test case  
development
Test cases for verification purposes
Executable specifications
Combining V&V Techniques
Contextual reasoning
Autonomous systems design

Latest revision as of 16:06, 31 October 2023

Offer different classifications and groupings - in main page

  • Verification techniques (tree?)
  • Metrics (grouping)
  • Application area (e.g. ML systems, for HRI, for medical robots)
  • Related concepts (e.g. all the specification and requirements terms together)

Contents of each linked page:

  • definitions with links to sources
  • short discussion of how topic relates to verification of autonomous systems

Template for definition pages - please copy the markup from this page into your new definition page


Acceptability
exposure to loss (financial or otherwise) from a risk that an organization is willing to tolerate
Acceptance testing
Accountability
Activity
set of tasks that describe the technical effort to accomplish a process and help generate expected outcome NASA Systems Engineering Handbook
Adaptive
Assessment
Assurance
Assurance cases
Autonomous System
"system which has the ability to perform intended tasks based on current state, knowledge and sensing, without human intervention" BSI 8611:2016
Autonomous systems design
Autonomy
Automated
a robot that does not make any decisions. It performs a precise sequence of actions without regard to external input. Even if a robot has sensors, that enable it to gather information about its environment, if they aren't used to modify its behavior, the robot is automated and acts as an automaton.
Automation
allocation of system functions to machines versus humans NASA Systems Engineering Handbook
Anomaly
unexpected performance of intended function NASA Systems Engineering Handbook


Behavior verification for autonomous systems and ML systems
Behavioral verification


Combining V&V Techniques
Contextual reasoning
Correctness verification
Credibility
Cross-domain / multi-domain validation
Cyber-security
Cyborg
a system consisting of a combination of a core biological organism, usually human, and machine or mechanical components.


Dependability


Executable specifications
Experiment design
Explainability
Failure
inability of a system, subsystem or component to perform its required function within specified limits NASA Systems Engineering Handbook
Failure detection and recovery
Fault
physical or logical cause, which explains a failure NASA Systems Engineering Handbook
Formal methods for autonomous systems
Formal modelling
Formal software verification
Functional assurance
Functional requirements
Functional verification


HRI (Human-robot interactions)
HRI verification
Hybrid Logical/Dynamical Planning and Verification


Generalizability


Integrity of autonomous systems


Liability
Licensing and certification


Model-based evaluation



Objective evidence
ISO 9000:2015 - "data, supporting the existence or verity of something" ISO 9000:2015


Operational design domain
Process
"set of interrelated or interacting activities that use inputs to deliver an intended result" ISO 9000:2015
Qualitative assessment
Quality requirement
"requirement related to quality" ISO 9000:2015
Quality plan
"specification of the procedures and associated resources to be applied when and by whom to a specific object" ISO 9000:2015
Quantitative assessment


Performance evaluation and benchmarking
Privacy
Reactive
Redundant robots
Reliability
Remote Control
Replicability
Reproducibility
Requirements
Workspace and conditions/context of operations
Requirement
"need or expectation that is stated, generally implied or obligatory" ISO 9000:2015
Requirements development processes
Resilience
Responsibility
Either (a) responsible development and deployment of robotics, see BSI 8611:2016 or
(b) responsible decisions by the autonomous system itself, see ethics for autonomous systems
Robot
See Robotic Systems
Robot safety
Robotic Systems
Robot systems engineering
Robustness
Run-time verification and monitoring


Safety
Safety and security interaction
Safety in HRI
safety, system function
verification of any function in the system including ML & non-deterministic functions
Safety-centric HRI protocols
Safety requirements
Security
Security requirements
Semi-autonomous
Simulation-based testing
Sim-based evaluation
Social/legal/ethical requirements
Software testing and debugging
Software tools for benchmarking and
Software tools for testing
software testing
Specification
"document stating requirements" ISO 9000:2015
Synthetic environment
to develop high level test scenarios, drive test case development


Teleoperated
a robot whose operator is controlling its every move, but where the command stream is not required to be continuous and the operator is using sensors on-board the robot for situational awareness. The operator may send higher level commands (e.g. "go to waypoint") and the robot will complete the last command it receives before it stops.
Test cases for verification purposes
Testing
Transparency
Trust
Trustworthiness
Generally, "able to be trusted".
In autonomous systems, moving beyond standard cyber-physical systems where trustworthiess is synonymous with reliability and extending this with beneficiality, i.e. evidence that the system is working for our benefit.
Use-case scenario generation
Use Case testing


Validation
further adds as "A system is able to accomplish its intended use, goals and objectives (i.e., meet stakeholder requirements) in the intended operational environment. The right system was built.” ISO/IEC/IEEE 15288:2015
Verification
defines verification as "Confirmation, through the provision of objective evidence, that specified requirements have been fulfilled" ISO 9000:2015
further adds as "Verification is a set of activities that compare a system or system element against the required characteristics. This includes, but is not limited to specified requirements, design description and the system itself. The system was built right." ISO/IEC/IEEE 15288:2015
Verification and Validation Process
Process of first, identifying the desired functionality and properties of the system, second, verifying that the system provides that functionality and possesses those properties, and third, validating that the system as delivered meets user needs.


Waldo
a mechanism that precisely repeats the actions of its operator (from the Robert Heinlein short story Waldo from the book Waldo & Magic, Inc.)

Notes

  • Note that we need to discuss how we will handle definitions that are not part of standards and may not have agreed upon definitions