Hello, my name is Ashley Barnett and I’m a philosopher at the Hunt Laboratory for Intelligence Research at The University of Melbourne. For over 10 years I have developed and delivered critical thinking training for intelligence agencies around the world. As part of a research project for the Intelligence Advanced Research Projects Activity (IARPA), I formulated a method of reasoning evaluation called the Reasoning Stress Test. In collaboration with the Australian Intelligence Community, the Reasoning Stress Test has been developed into a training program for intelligence managers that is run multiple times a year. For an earlier IARPA project, I worked on designing courses and training methodologies to improve the critical thinking skills of analysts using techniques such as argument mapping. My current research interests are analytic rigour and the nature of reasoning used in analysis, especially the abductive reasoning used to determine what is the best explanation for the available evidence.
One the resources page you can find the online and supplementary material for my seminars and courses.
You can contact me on barnett.a@unimelb.edu.au and follow me on Twitter @AshleyEBarnett.
Training courses
Advanced Analytic Rigour
Analytic rigour is essential for well-reasoned, trustworthy and high-impact analytic products. This course teaches a new and efficient method for evaluating analytic rigour called the Reasoning Stress Test that was developed with funding from the US Intelligence Community. In collaboration with the Australian National Intelligence Community, The Hunt Lab has developed training on the Reasoning Stress Test method that is delivered multiple times a year for intelligence managers from a range of organisations. Participants learn how to identify the different types of reasoning used in intelligence analysis, and the most important and common reasoning flaws. The Reasoning Stress Test methodology was formulated by Hunt Lab researcher Ashley Barnett and is based on extensive archival research on the types of arguments used in intelligence analysis. Participants will practice applying the Reasoning Stress Test method to declassified analysis, gaining an understanding of how the method can be applied to real products. Reasoning stress test evaluations are used to construct clear, precise, and actionable feedback. Additionally, the course provides participants with concepts and skills that will help them promote analytic rigour within their organisations.
Argument Mapping with Rationale: Building logical arguments using argument mapping software
An argument map is a way of visually representing the structure of the reasoning for an assessment or conclusion. This course teaches participants how to use specialist argument mapping software, called Rationale, developed by Tim van Gelder at the University of Melbourne. The software helps overcome cognitive limitations which can inhibit our analysis and evaluation of reasoning, and so can help people develop a deeper understanding of the reasoning that they read and write. By learning how to map arguments, participants will gain a better understanding of how arguments work and will be better equipped to formulate and clearly communicate their reasoning. Argument maps are useful for drafting, presenting, and reviewing arguments. And they can also be used to keep track of a debate, to help make sure all participants are ‘on the same page’. The course uses a training methodology which has been established through extensive research, as one of the best ways to teach reasoning skills including a course developed by the instructor, Ashley Barnett, and other experts for Intelligence Advanced Research Projects Activity (IARPA).
Reasoning Stratagems: constructing well-justified, high impact arguments
Learn how to construct concise, precise, and cogent arguments. When presenting an argument, it often seems that there is a tension between keeping it brief and adequately justifying the assessment. However, many arguments are needlessly verbose, confusingly structured, or poorly justified due to the author being unaware of how to explain how the reasoning works. Participants in this course learn how to distinguish between the different types of reasoning used in analysis and what they need to do to successfully establish a conclusion using these inferences. The course teaches not only the logical relationship between certain types of evidence and conclusions, but also how to clearly explain these relationships so the audience or reader can easily follow the reasoning. Participants will master these skills through practice with declassified historical products and more recent unclassified assessments. Some of the types of reasoning covered in this course are, 1) inference to the best explanation (i.e., abductive reasoning), 2) weighing causal drivers, 3) using indicators and signatures, 4) strategic calculations, 5) inferring generalisations from samples, 6) extrapolating trends, 7) appeals to analogy, 8) argument by elimination, 9) enumeration of causal pathways, and 10) corroborating sources.
Deception: The art and science of how to direct people’s attention and thinking
Adversarial deception is a main challenge for intelligence analysis, significantly increasing the degree of uncertainty analysts must grapple with. This course explains some of the core principles of deception and how to detect it. It covers three main topics. 1) The illusion of attention, covering psychological findings and demonstrations illustrating how we persistently and erroneously think that we notice more than we do. 2) The many guises of misdirection, where participants will learn the different ways in which our thinking can be manipulated into accepting a false explanation or theory. 3) Detecting deception, which covers the critical thinking skills that can help us check for and recognise deception. Participants will also learn several magic tricks and other demonstrations of deception that require a practical mastery of the principles covered in the course. By the end of the course, they’ll appreciate former deputy director of the CIA John McLaughlin’s claim that “Magic and Intelligence are really kindred arts”.