Melanie McGrath
Melanie is a psychological scientist in CSIRO’s Collaborative Intelligence and Responsible Innovation Future Science Platforms. Her work addresses the integration of human and AI capabilities at the individual, group, and organisational levels. Melanie’s research is focused on understanding the role trust plays in working safely and effectively with AI, and how this understanding can be harnessed to develop and deploy responsible AI solutions.
Innovation title:
Trust-sensitive design for responsible AI
Abstract:
Without an understanding of the dynamics of human trust, the most promising AI innovations can – and often do – fail. Trust is critical to human willingness to rely on AI technology, however, safe and productive use of AI requires trust calibration; that is, the adaptation of human levels of trust to the capabilities of an AI system. Without effective trust calibration humans may place too much trust in a system that isn’t reliable, or insufficient trust in a system that is actually beneficial. To achieve this calibration we need to be able to identify the factors that shift trust in a given AI technology.
The goal of my research is to find the levers that influence trust in an AI technology, identify the optimal level of trust in that application to achieve performance that is effective and safe, and determine how those levers might be used to reach that optimal state of trust. To progress that goal I have developed a framework to model human trust in AI technologies that accounts for context-specific inputs of trust, the relation of trust to processes of human-AI interaction, and the outcomes that trust is expected to facilitate. The framework has been used to quantify the relative contribution of a set of design features to trust in an AI application supporting scientists engaged in genome annotation. The application of the framework to this agricultural use case demonstrates the utility of this approach for informing the design and development of trustworthy and trusted AI innovations.