Thursday, 16 February 2017

What's the role of teacher intuition in evidence-based/informed practice?

At Thursday's Chartered College of Teaching Inaugural Conference the issue of the relationship between teacher intuition/instinct and evidence-based practice was briefly discussed.  In this post, I will consider  the role of professional intuition in evidence-based decision-making, and under what circumstances that intuition can be trusted.  To help us get to grips with this issue I will be turning to the work of (Kahneman and Klein, 2009) and (Kahneman and Klein, 2010) who represent two quite different schools of thought on decision-making  – naturalistic decision-making and heuristics and biases –  but nevertheless have reached a number of areas of agreement.  As such the rest of this section will:
  • define what is meant by skilled intuition;
  • describe the circumstances which may contribute to the quality of an intuitive judgment; 
  • determine whether the circumstances which support the development of skilled intuition exist in a school environment;
  • consider the implications of for school leaders of fractionation – having genuine expertise in some but not all aspects of a role.
  • consider the implications for evidence-based school leaders
What is skilled intuition?

Both Kahneman and Klein agree on Simon’s (1992) definition of skilled intuition as 

“The situation has provided a cue: This cue has given the expert access to information stored in memory, and the information provides the answer. Intuition is nothing more and nothing less than recognition” (p. 155). 

Kahneman and Klein note that this definition is extremely helpful in that it helps clarify what is meant by intuition, as it makes clear that intuition is not some mystical process but is just a process of pattern recognition

Under what circumstances is skilled intuition developed?

The determination of whether an intuitive judgment can be trusted is a function of both the environment is which the judgment is made – does the environment provide valid cues as to the issue or issues faced – and do decision-makers have the opportunity to learn the various cues.  Kahneman and Klein argue that medicine and firefighting take place in fairly ‘high validity’ environments, in that there are relatively stable relationships between the observed cues and the results of a range of decisions.  However, there are other environments such as financial and political forecasting which take place in what Kahneman and Klein describe as zero-validity environments. 

Do schools provide an appropriate environment to develop skill intuition?

Schools are likely to be a both a highly valid and uncertain environment.  By this I mean that a teacher may develop the skill to identify the most likely strategies to work with particular students, but that does not mean that it will work every time.  Head of departments and headteachers may have the skills and knowledge of what to do in trying to ‘turnaround’ an under-performing department or school, but that does not mean that this will lead to success on all occasions.   In these situations, the uncertainty is a function of the wide range of developments that can happen in any one class or school at any one time.

What are the problems with fractionation?

Kahneman and Klein state one of the challenges for decision-makers is that the fractionation of skill may lead to over-confidence.  School leaders more often than not have a number of different tasks, not all of which will they be as equally skilled. A school leader may have risen through the ranks of the school by being an expert teacher.  However, this may lead to them in their new role having to make judgment in areas in which they have no real knowledge and skills.  However, just because a school-leader has developed skills, for example in intuitively judging the academic success of a particular year group, that does not mean that those skills will transfer across to other areas such as staffing or financial decisions.  However, there is a real risk that the individual’ successes in other aspects of their role will lead to overconfidence – by both themselves and others – in their decision-making abilities, leading subsequently to poor decision-making.

What are the implications of this discussion for developing your skills as an evidence-based school leader?
  • On a day to day basis in the classroom - intuition and pattern recognition is going to be an important part of the teacher's decision-making process
  • However, when making less time-pressured and more deliberative decisions, professional intuition should not be trusted on its’ own.  Intuition should be seen as the starting point, and evidence which disconfirms your ‘gut’ feeling should be actively sought.
  • Make sure that a systematic search strategy for multiple sources of evidence is developed early in the decision-making process
  • Beware of over-confidence in your decision-making abilities, just because you have been able to make ‘good’ intuitive judgments in one part of your role, that does not mean you will be able to make such judgments in an area in which you are less skilled.
  • Ensure conditions are created where the evidence informing the decision can be genuinely and openly challenged, and where constructive dissent is rewarded rather than discouraged or punished.
  • Avoid jumping to conclusions and actively seek to defer judgment until all the evidence has been made available and appraised.
References

KAHNEMAN, D. & KLEIN, G. 2009. Conditions for intuitive expertise: a failure to disagree. American Psychologist, 64, 515.

KAHNEMAN, D. & KLEIN, G. 2010. Strategic decisions: When can you trust your gut. McKinsey Quarterly, 13, 1-10.

Saturday, 11 February 2017

The school research lead and the need for research literacy - the case of leadership and correlation to organisational climate

A major challenge for the school research lead is whether they are able to help colleagues develop the capacity and capability to interpret research findings.  In particular, can school research help colleagues to correctly interpret the statistical findings which are often found in research reports.  So this post, I will look at the specific challenges of using the correlation coefficient(r ) when seeking to interpreting Daniel Goleman’s  work on leadership and emotional intelligence (Goleman, 2000).  To help us do this I will:
  •  Summarise Goleman’s work on leadership that gets results.
  • Briefly explain what is meant by the term correlation coefficient and the implications for interpreting Goleman's work
  • Consider some broader points about the challenge of using correlation in educational leadership research and the implications for practice

Daniel Goleman and leadership that gets results

Goleman argues that leaders often mistakenly assume that their leadership is a function of their personality, rather than is something can be chosen to be meet the needs of a specific circumstance.  Goleman claims that successful leaders have strengths in six emotional intelligence competences* i.e.: self- awareness, self-regulation, motivation, empathy, and social skill.   Moreover, these components of emotional intelligence can be combined in different ways and which reflect six basic styles of leadership i.e. coercive, authoriative, affiliative, democratic, pace-setting and coaching, which are summarised in Table 1 Moreover, the very best leaders are not wedded to one particular approach and can change approaches depending upon the demands of the situation.

Table 1 Six approaches to leadership 

Leadership style
Brief explanation
Coercive
‘Do what I say’
Authoriative
‘Come with me’
Affiliative
‘People first’
Democratic
‘Let’s decide together’
Pace-setting
‘Let’s make this happen, and now.’
Coaching
‘People development

*note - I'm not going to get drawn into discussion about the evidence 'for and against' emotional intelligence - other than to suggest you have a look the work of Professor Rob Briner

The correlation coefficient and leadership styles

Based on research interviews with nearly 4000 executives out of a database of 20,000 executives worldwide,  Goleman goes onto to demonstrate the impact that different leadership styles  have on the organisational climate (working atmosphere).    Using the correlation coefficient (r ), Goleman seeks to to quantify both the strength and direction of a relationship between leadership style and organisational climate .     Now, the correlation coefficient (r) can range from +1 – where there is perfectly positive correlation to -1, where there is a perfectly negative correlation.  On the other hand, a value of 0 would suggest that there is no relationship between the two variables i.e. leadership style and organisational climate.

Goleman’s research indicates that  the relationship between leadership style and organisational climate reflects a range of intermediate values between -1 and +1.  Table 2 shows the relationship between different styles of leadership and aspects of organisational climate.  So looking at the table we can see that the coercive (-0.26) and pace-setting (- 0.25) styles have a negative overall correlation with organisational climate.  Whereas, the -authoritative (+0.54), affiliative (0.46), democratic (+0.43) and coaching (+0.42) – styles have a positive correlation correlation with organisational climate.  However, Goleman notes that all styles can have their uses, and no one style should be relied upon nor excluded, when seeking to tackle the ranges of circumstances which are faced by leaders.

Table 2 Leadership style and organisational climate (Goleman, 2000 p19)










Interpreting the correlation coefficient

(Cumming and Calin-Jageman, 2017) argue that when interpreting an r value – let’s say 0.54 – and which hints at some relationship between two variables X (leadership style)and Y (organisational climate), a number of different things could be going on.
  • ·        There may be a causal link between leadership style  and organisational climate changes in leadership style may lead to changes in organisational climate, or changes in organisational climate are leading to changes in organisational style.
  • ·         Something else may be going one, with other variables impacting on either leadership style  or organisational culture or both.
  • ·        There are no causal links, and what we are seeing is what Cumming and Calin Jageman describe as ‘seeing a face in the clouds’. 

As such Cumming and Calin Jageman note that all correlations do is to give use some form of guidance as to what to investigate further.  The r values may suggest what and where to look, however, it needs to be made clear that correlation does not imply causation.  In other words, just because X appears to be linked with changes in Y, that does not mean that changes in X are causing changes in Y

So far we have made no reference to how should interpret the size of different correlation coefficient or r values – be they -1, 0, 0.5. 0.7 or +1.  For example in Table 3, (Hinkle et al., 2003) provided the following guidance for interpreting correlation.

Table 3 – Rule of Thumb for Interpreting the Size of a Correlation Coefficient

Size of Correlation
Interpretation
0.90 to 1.00 (- 0.90 to –-1.0)
Very high positive (negative) correlation
0.70 to 0.90 (- 0.70 to - 0.90)
High positive (negative) correlation
0.50 to 0.70 ( - 0.50 to -0.70)
Moderate positive (negative) correlation
0.30 to 0.50 ( - 0.30 to -0.50)
Low positive (negative) correlation
0.00 to 0 0.30 (    0.00 to- 0.30)
Little if any correlation

If we were to use this table as a guide, this would suggest that in general, with the exception of the authoritative style, there is little/low levels of correlation between leadership styles and organisational climate.  Nevetheless, (Cumming and Calin-Jageman, 2017) argues that r values need to be interpreted in context, as correlation is used in such a wide-ranging number of ways and settings, that reference values for r provide little help in interpreting the data.  Indeed, Cummings and Calin Jageman argue that an r value of 0.3 may suggest there is a relationship between two variables, even though when graphed the data looks like a shot-gun blast.

What about the correlation determinant r squared.

However, if we are looking for the strength of association between the two variables, we may wish to use the coefficient of determination r squared – which gives you the proportion of shared variance.  So if we have two variables (A – the independent variable and B the dependent variable) and we have a correlation coefficient + 0.5 then our coefficient of determination will be 0.25. 

So what does a coefficient of determination of 0.25 mean, well Professor Steve Higgins (@profstig) of the University of Durham, has provided me some useful guidance.  Well at best
  • A might cause B 25% of the time OR
  • B might cause A 25% of the time OR
  • C causes them both (and is not present all of the time) OR
  • it is a pure coincidence OR
  • it is some of the above in some unknown combination
In other words,  there is some relationship explaining about a quarter of the overlap, but we don’t know anything about the cause

We can now calculate Goleman’s correlation coefficient into the coefficient determinant and interpret what it means for the relationship between leadership style and organisational climate

Table 4 Leadership style, correlation coefficient and correlation coefficient determinant

Leadership style
Correlation Coefficient  r
Coefficient determinant r  squared
Coercive
- 0.26
+ 0.07
Authoriative
+ 0.54
+ 0.29
Affiliative
+0.46
+ 0.21
Democratic
+0.43
+ 0.18
Pace-setting
-0.25
+ 0.06
Coaching
+ 0.42
+0.18

So what does this suggest, well for both the coercive and pace-setting styles relatively small amounts of variances in leadership style and organisational climate appear to be shared (-0.07 and 0.06 respectively) As for the relationship between an authoritative style and organisational climate the data suggests that at best only 29% of the total variance between the two variables is shared.

Other issues to take into account when interpreting the correlation coefficient

Again, I would like to thank Professor Steve Higgins for bringing these issues to my attention.  First, the correlation coefficient should be calculated from a random sample of the population.  In the case of Goleman’s work, this would appear to be the case with 4,000 executives randomly selected from a database of 20,000 executives worldwide.  That said, those 20,000 executives were not in themselves a random sample of executives.  Second, Goleman does not report the error associated with these correlations, as such, we do not know how precise they are, as no confidence limits have been reported.

Implications

It would seem to me that there are a number of implications in using ‘correlation’ when making recommendations about leadership and management.
  • Correlation does not mean there is a causal link between two variables – in other just because you have a positive correlation that does not mean the changes in leadership style have caused the change in organisational climate.  Indeed, it may be the change in organisational climate may cause the change in leadership style.
  • It’s important to have some rules of thumb for interpreting the size of the correlation coefficient.  In this example the largest correlation coefficient (authoritative 0.54) is around the borderline between positive and low correlation.  In other words, just because you have is for your study the largest correlation coefficient does not mean that the size of correlation coefficient is large.
  • If you are looking to make judgments about the strength of the relationship between two variables, then you need to calculate the coefficient of determination.  Again, in this example, we have two values (coercive 0.06 and pace-setting 0.07) which are close to zero, which suggests there very little of the variance in leadership style and organisational climate seem to be shared.
  • Before making recommendations about leadership and management which uses correlations, it might be useful to look at a basic statistical textbook.


References

CUMMING, G. & CALIN-JAGEMAN, R. 2017. Introduction to the New Statistics: Estimation, Open Science, and Beyond, Abingdon, Routledge.
GOLEMAN, D. 2000. Leadership that gets results. Harvard business review, 78, 4-17.

HINKLE, D. E., WIERSMA, W. & JURS, S. G. 2003. Applied statistics for the behavioral sciences.