XAI
,Lecture 1
User Centered Design cycle
Discussion point
argument/statement that is well explained and grounded in the paper and potentially other
sources
Can include:
• criticisms and ideas on theoretical points and relation to other theories
• criticisms and ideas on methods used to test the theories
• criticisms on the conclusions drawn by the authors
• discussions of boundary conditions
• criticisms and ideas on new or different applications
,Important components
Argument: The argument is explicit, clearly explained, persuasive and well connected to the
literature
Structure: The structure of the DP is clear: it has a clear intro, argument and conclusion
Style, tone, grammar: The style and tone are appropriate to formal scientific writing and fit the
domain. Active rather than passive style of writing. Proper grammar usage.
Argument
Point made: Often writers forget to make the argument explicit
Argument strength: The argument is clearly explained, persuasive and well connected to the
literature
• you need to explain what you argue based on the paper
• go beyond the obvious, find one that is really persuasive
• good arguments also take the opposite perspective: “one might think that .. but ..”
• don’t try to argue more than one thing
Structure
It has a clear intro, position, argument and conclusion
Intro: it has a clear intro, position, argument and conclusion
• help the reader to put your argument into context
• for long papers: which part are you really arguing about
Claim/argumentation
Conclusion: Don’t forget to end with a clear conclusion that finished the argument
Style
Style, tone and grammar:
• Scientific does not mean complex writing ..
• “Prevent long sentences, that because of all sorts of qualifications, or additional thoughts,
given a specific topic that might be too vaguely defined, or have too many clauses”
• English language uses active voice
• Use word or other tools for spell / grammar checking
, Explainable AI
Means-end analysis compare current state with goal state, choose action that brings you
closer to the goal
Why need explainability
• model validation: avoid biases unfairness or overfitting, detect issues in the training data,
adhere to ethical/legal requirements
• Model debugging and improvement: improving the model fit, adverbial learning (fooling a
model with ‘hacked’ inputs), reliability & robustness (sensitivity to small input changes)
• Knowledge discovery: explanations provide feedback to the data scientist or user that can
result in new insights by revealing hidden underlying correlations/patterns
• Trust and technology acceptance: explanations might convince users to adopt the
technology more and have more control
Important properties for ML
• Accuracy: does the explanation predict unseen data? Is it as accurate as the model itself.
• Fidelity: does the explanation approximate the prediction of the model? Especially
important for black-box models (local fidelity)
• Consistency: same explanations for different models?
• Stability: similar explanations for similar instances?
• Comprehensibility: do humans get it?
What is good explanation (for humans)?
Confalonieri et al. (2020) & Molnar (2020) based on Miller:
• Contrastive: why was this prediction made instead of another? (counterfactual) What
should I change to get my loan approved rather than rejected? Only interested in those
factors that matter/change the situation: no complete explanation
• Selective: focus on a few important causes (not all features that contributed to the model)
• Social: Should fit the mental model of the explained / target audience, consider the social
context, and fit their prior belief (confirmation bias)
• Abnormalness: humans like rare causes (related to counterfactuals)
• Truthfulness: less important for humans then selectiveness
,Lecture 1
User Centered Design cycle
Discussion point
argument/statement that is well explained and grounded in the paper and potentially other
sources
Can include:
• criticisms and ideas on theoretical points and relation to other theories
• criticisms and ideas on methods used to test the theories
• criticisms on the conclusions drawn by the authors
• discussions of boundary conditions
• criticisms and ideas on new or different applications
,Important components
Argument: The argument is explicit, clearly explained, persuasive and well connected to the
literature
Structure: The structure of the DP is clear: it has a clear intro, argument and conclusion
Style, tone, grammar: The style and tone are appropriate to formal scientific writing and fit the
domain. Active rather than passive style of writing. Proper grammar usage.
Argument
Point made: Often writers forget to make the argument explicit
Argument strength: The argument is clearly explained, persuasive and well connected to the
literature
• you need to explain what you argue based on the paper
• go beyond the obvious, find one that is really persuasive
• good arguments also take the opposite perspective: “one might think that .. but ..”
• don’t try to argue more than one thing
Structure
It has a clear intro, position, argument and conclusion
Intro: it has a clear intro, position, argument and conclusion
• help the reader to put your argument into context
• for long papers: which part are you really arguing about
Claim/argumentation
Conclusion: Don’t forget to end with a clear conclusion that finished the argument
Style
Style, tone and grammar:
• Scientific does not mean complex writing ..
• “Prevent long sentences, that because of all sorts of qualifications, or additional thoughts,
given a specific topic that might be too vaguely defined, or have too many clauses”
• English language uses active voice
• Use word or other tools for spell / grammar checking
, Explainable AI
Means-end analysis compare current state with goal state, choose action that brings you
closer to the goal
Why need explainability
• model validation: avoid biases unfairness or overfitting, detect issues in the training data,
adhere to ethical/legal requirements
• Model debugging and improvement: improving the model fit, adverbial learning (fooling a
model with ‘hacked’ inputs), reliability & robustness (sensitivity to small input changes)
• Knowledge discovery: explanations provide feedback to the data scientist or user that can
result in new insights by revealing hidden underlying correlations/patterns
• Trust and technology acceptance: explanations might convince users to adopt the
technology more and have more control
Important properties for ML
• Accuracy: does the explanation predict unseen data? Is it as accurate as the model itself.
• Fidelity: does the explanation approximate the prediction of the model? Especially
important for black-box models (local fidelity)
• Consistency: same explanations for different models?
• Stability: similar explanations for similar instances?
• Comprehensibility: do humans get it?
What is good explanation (for humans)?
Confalonieri et al. (2020) & Molnar (2020) based on Miller:
• Contrastive: why was this prediction made instead of another? (counterfactual) What
should I change to get my loan approved rather than rejected? Only interested in those
factors that matter/change the situation: no complete explanation
• Selective: focus on a few important causes (not all features that contributed to the model)
• Social: Should fit the mental model of the explained / target audience, consider the social
context, and fit their prior belief (confirmation bias)
• Abnormalness: humans like rare causes (related to counterfactuals)
• Truthfulness: less important for humans then selectiveness
