How Humans Perceive Fundamental Building Blocks of Artificial Intelligence Explanations

Helpful, Misleading or Confusing:
How Humans Perceive Fundamental Building Blocks of Artificial Intelligence Explanations

Edward Small, Yueqing Xuan, Danula Hettiachchi, Kacper Sokol

Motivation: the Gap

Explainer advanced and complex
Explainees educational attainment, profession, literacy

Explainer advanced and complex
Explainees educational attainment, profession, literacy
Gap between design intention user perception

Measure the Gap:
3-C Evaluation Framework

Assess how users perceive AI explanations

Comprehension (Positive & Negative)
Confidence
Contentment

They are comprehension, confidence, and contentment.
Comprehension captures the gap between the information offered by an explanation and the information understood by a user. For example, we can have a fancy waterfall graph as explanation but a layperson struggles to understand the entire demonstration. We further categorise users' comprehension as positive and negative. With positive comprehension, users knows what information is offered and what information lacks of. On the other hand, with negative comprehension, users couldn't interpret the information correctly, and may even make up their own interpretations, which are most likely wrong.
The second measurement is confidence. This captures how confident users are with their own understanding of an explanation.
The third measurement is contentment. This measures the users' perception of how reliable and informative an explanation is.
Ideally, a good explanation is comprehensible to diverse audiences, so users are confident in their comprehension and give raise to a high contentment to the explainer. In the worst case, users make up their false interpretation but still feel very confident in their misunderstanding. This is very dangerous as it leads to users' misplaced trust to the model. And also users are vulnerable to harmful explainers that appear faithful but factually incorrect.

Quantify the Gap: User Study

Diverse participants

Numeracy, digital literacy, English proficiency, etc

Diverse participants

Numeracy, digital literacy, English proficiency, etc

Different types of explanations

Textual, visual, mathematical, interactive, etc

Explanation:
Your loan request was rejected because your income is $40,000

Comprehension:
Income was the strongest factor in your loan application

True
False

Can't say
Don't know

Comprehension:
Income was the strongest factor in your loan application

True
False

Can't say
Don't know

Confidence:
How confident are you in your previous answer?

Comprehension:
Income was the strongest factor in your loan application

True
False

Can't say
Don't know

Confidence:
How confident are you in your previous answer?

Contentment:
The explanation was easy to interpret

Close the Gap Together

3-C Framework: assess individual difference in perceptions
Equally comprehensible and trustworthy explanations
AI for everyone

Thanks

yueqing.xuan@student.rmit.edu.au

https://yueqingxuan.com

Paper link

Slides link