NHS Education for Scotland, Glasgow, UK
NHS Ayrshire and Arran, Ayr, UK
NHS Ayrshire and Arran, Ayr, UK
NHS Ayrshire and Arran, Ayr, UK
NHS Education for Scotland, Glasgow, UK
Institute of Health and Wellbeing, University of Glasgow, Glasgow, UK
Submitted date: June 06, 2017; Accepted date: June 12, 2017; Published date: June 19, 2017
Background: In primary care, Significant Event Analysis (SEA) is an established method for learning and improving from patient safety incidents. The enhanced SEA method was developed to facilitate a deeper, systems approach to improve the effectiveness of analyses. Completion and submission of enhanced SEA reports to the national education and training authority for peer review was a component of an optional enhanced service for general practices in Ayrshire and Arran health board. The aim of this study was to theme incident types, improvement actions described and conducts inter-group comparisons of the quality of event analyses.
Methods: Two team members from participating general practices attended two half-day training sessions and completed and submitted an enhanced SEA report for peer review using a validated tool. Content of submitted reports was thematically analysed independently by two researchers. Peer review rating scores were used as a proxy indicator of quality and were compared with similar data from other GP groups. Quantitative data were analysed using descriptive and inferential statistics.
Results: Fifty-one of 55 practices participated (93%), submitting 52 SEA reports. 113 improvement actions across five different care systems were described (mean 2.2). Protocol creation or alteration was described most frequently (35%) whereas individual action was described in 5%. Of the 12 SEAs describing communication problems with other healthcare sectors, seven were discussed with professionals outside the practice (58%). Two SEAs described direct involvement of the patient (3.8%). The majority of the enhanced SEA reports described events whose consequences were negligible or minor (86.6%). Grading of reports were similar to prospective trainers and specialty trainees.
Conclusion: After training, GP teams were largely able to use the enhanced SEA method to analyse events and implement system-level actions. Consideration of how to improve the analysis of events across healthcare interfaces, involve patients to maximise learning and increase incident reporting is needed.
Significant event analysis; Human factors; Patient safety; Primary health care; Incident investigation
SEA: Significant Event Analysis; NES: NHS Education for Scotland; NHSAA: NHS Ayrshire and Arran
Recent evidence has suggested that between 2-3% of general practice (GP) consultations may include a patient safety incident – an event that caused harm, or had the potential to cause harm . Learning from these incidents by using significant event analysis (SEA) is a well-established educational, patient safety and quality improvement tool in many primary health care services worldwide [2,3]. However, there is evidence that SEA is often not performed effectively by care teams and is a missed opportunity for learning and improvement [2,4]. A key barrier to SEA effectiveness is that the ‘cause’ of the event is often attributed to ‘human error’ with a lack of a ‘systems approach’ underpinning the analytical process in line with latest safety science thinking and consensus [5-7]. This limited approach can have multiple negative impacts including, for example, emotional impacts on the care practitioners involved (the socalled ‘second victim syndrome’ , leading to the failure to implement robust, sustainable change to reduce the risk of occurrence of the event .
To address these deficiencies a process for undertaking SEA more effectively - based largely on Human Factors principles - has been developed and tested (termed ‘enhanced SEA’) . This approach aims to help care practitioners and teams reflect on the personal impacts of the event and adopt a ‘systems approach’ to the analysis to provide better insights into why things have gone wrong and so direct more meaningful improvements.
NHS Ayrshire and Arran (NHSAA) is one of 14 regional health boards in the Scottish Health Service. NHSAA encouraged the participation in localised safety and improvement work of general practice (GP) teams through an optional, financially incentivised, patient safety enhanced service. The specifications for the 2015-2016 patient safety enhanced service included completing and submitting an enhanced SEA to NHS Education for Scotland (NES) for peer review using a well-established system . NES is the national body with responsibility for the education and training of the healthcare workforce and is active in safety and improvement development and research. Participating GP teams were directed to submit enhanced SEA reports which described the analysis of a patient safety incident.
The aim of this study was to thematically summarise the types of significant events selected for analysis and submitted by local care teams and the agreed improvement actions. The quality of submissions, as judged by the trained NES peer reviewers, was also compared with those of other professional groups.
Participants, setting and incentivisation
All 55 general practice teams in NHSAA region were invited by the health board to participate in the ‘enhanced service’ on a voluntary basis. Participants were incentivised as part of local enhanced contractual arrangements and received additional ‘back-fill’ funding for attending related training and subsequently undertaking and submitting enhanced SEA reports.
Enhanced SEA training sessions
Participating practices were required to send two representatives, one of whom was a GP, to attend two halfday patient safety training sessions. These were designed and facilitated by two local GPs who were also the organisational clinical leads for patient safety in NHSAA. Both are experienced medical educators and researchers and were also trained in the enhanced SEA method. During each training session, 1 h was dedicated to learning focused on the enhanced SEA method. This included presentation on the theory of adopting a Human Factors approach to SEA, describing the enhanced SEA methodological process and participating in simulated analyses of example significant events (including their own events) as part of small group work which were assisted by a GP facilitator (Table 1).
|Training Topic||Education modality|
|Enhanced SEA background
•Current limitations of SEA process
•What is Human Factors and how can this approach be applied to SEA
•The limitations of blaming ‘human error’
•System complexity and performance variability
•PAcE analysis (people, activity, environment)
•Developing and implementing sustainable change
|Practical application of enhanced SEA process
•Applying the enhanced SEA process to example SEAs
|Small group work|
|Discussion of own SEAs •Discussed SEAs in small groups with staff from neighbouring practices||Small group work|
|Submission of SEA for peer review||Peer review and feedback|
Table 1: Delivered enhanced SEA training topics and educational modality.
Identification and analysis of Significant Events and submission of reports
GP teams identified ‘significant events’ for analyses during the study. Each team was required to analyse a single event as a practice team and apply the principles and approaches previously taught. Each participating practice were required to submit one enhanced SEA report to NES for educational peer review  within two months of the second training interventions. Participants were made aware that fair, constructive and developmental feedback would be provided by independent, external and trained GP peer reviewers. Enhanced SEA reports were completed using standardised pro-forma provided to teams and submitted via secure electronic mail to NES .
Peer review system and underlying theoretical principles
NES operate a well-established system for the peer review of SEAs based on sound educational principles that has previously been described and evaluated  (Figure 1). The theoretical approach underpinning the system is based on an adaptation of cognitive continuum theory. This helps to understand and frame the thinking used in performing different tasks and to attempt to improve the level of reflection on these tasks. How each task is accomplished is described using one of six ‘modes of practice’ that range from highly structured scientific experiment (mode 1) to intuitive judgements (mode 6) with peer review positioned between modes 4 (system-aided judgement) and 5 (peer-aided judgement). The peer review approach aims to improve the conclusions and decisions from the analysis by minimising the utilisation of mode 6 judgements (self-assessment) [10,11].
Submission of an SEA for peer review is a requirement of work placed based assessment for GP specialty trainees in the west of Scotland and for GPs who wish to train to become educational supervisors. The system is open to all qualified GPs on payment of a fee, however, Ayrshire and Arran GPs were able to submit SEA reports without cost.
The peer review team consists of qualified GPs (n=25) and a lead peer review co-ordinator who are trained and regularly calibrated in the SEA review and feedback processes by NES medical educators. Submitted SEA reports are annonymised prior to review although details of whether the report was submitted by a GP specialty trainee or a qualified GP are included. Reports are reviewed by two members of the peer review group who complete a feedback instrument  that includes grading the project (on a scale from 1=very poor to 7=outstanding) and giving formative feedback in several domains (Table 2). The grades given by peer reviewers in each domain listed in Table 2 are routinely inputted and stored in an electronic database by NES. A letter is generated by the lead co-ordinator based on the reviewers’ comments to give formative feedback to the author on the quality of event analysis in each of seven domains (the description of the event, its actual or potential impact, the exploration of contributing factors to why the event occurred, learning and reflection on the event, involvement of appropriate individuals, the action plan and the overall conclusion) and how this could be improved, where judged necessary, or provide reassurance and validation of good practice. GP teams receiving the feedback letters are under no obligation to act on this guidance, although the assumption is that feedback from two independent GP colleagues on an important safety issue will prompt action if considered to be relevant and feasible.
Table 2: SEA feedback instrument domains.
Each enhanced SEA was analysed independently by an NHSAA patient safety clinical lead (DM) and a NES safety and improvement project manager (PW). The involved practice system and the actions implemented were recorded using a previously published data collection framework . This was adapted iteratively to refine coding as the study progressed. Due to the complexity of some events many different care systems were involved. A pragmatic approach was taken to assign each report to the dominant system involved. For example, a prescription was issued for a live vaccine for a patient who was immune-compromised. Although a prescription was issued this event related more to the vaccination protocol rather than the prescribing system.
The involvement of patients in the event analysis, either directly or by explicitly obtaining and presenting their perspective, was recorded. The National Reporting and Learning Service (NRLS) risk matrix was used to grade the actual consequence of each event as described in the report using a five-point scale (where 1 - minor, 2 - one for negligible, 2 - minor, 3 - moderate, 4 - major and 5 - catastrophic) .
The grade given in each domain to NHSAA reports were compared with two other cohorts, GP specialty trainees and GP prospective trainers submitting SEA reports for peer review over the same period (31st March 2015 to 31st March 2016).
Data were analysed using simple described statistics such as frequency counts, percentages, means and 95% confidence intervals.
Study participation rate
A total of 51 NHSAA GP teams voluntarily participated in the enhanced service (51/55, 93.0%), attending the training sessions and submitting 52 enhanced SEA reports for peer review.
Main type of incident and examples
Significant events described in submitted event analysis reports were classified and themed across five main sub-systems (e.g. prescribing system and test result handling system). Table 3 shows the number of projects and examples of projects submitted in each system (Table 3).
|Theme||N (%)||Example SEAs|
|Prescribing system||14 (26.9)||•Prescription of unopposed oestrogen HRT to patient with intact uterus
•COCP prescribed inappropriately
•Incorrect dosage of antibiotic prescribed
•Treatment duration longer than intended
|Results handling systems||7 (13.5)||•Repeat blood tests were required but not arranged
•Abnormal INR not acted upon
•Result not acted upon causing delayed diagnosis
|Communication with care services
external to the practice
|12 (23.1)||•Lack of information within discharge letter
•Change in prescription between GP practice and pharmacy not communicated
•Lack of information provided to GP by nursing home staff
•Missed request for prescription issue for UTI – poor communication between secondary and primary care
|Clinical decisions and actions||11 (21.1)||•Vaginal discharge – no speculum examination carried out
•Incomplete assessment of patient with possible SIRS/Sepsis
•Good palliative care
•Cardiac arrest in waiting room
|Other practice systems||8 (15.4)||•Missed house visits
•Patient developed drug related renal failure – no blood test monitoring
•Wrong bloods taken
•Out of date cervical smear pots
Table 3: SEA incident system domains with frequency and examples of incident type.
Actions implemented or planned
A total of 113 implemented actions for system improvement were described in the 52 projects (mean 2.2 per project). This did not include discussion of the event within practices as this was required prior to submission of the reports. Eight themes of actions implemented were agreed. These are shown in Table 4 along with the frequency of this action (Table 4).
|Described action themes||Total number of times action described (%)|
|Discussion with secondary care||11 (8%)|
|Discussion with other areas in health and care (e.g. nursing homes, social work)||6 (5%)|
|Alteration to practice protocols (e.g. how messages sent and recorded, develop a recall system, checklist, electronic template development, incorporate new software, develop induction pack)||40 (35%)|
|Purchase of new equipment||3 (3%)|
|Audit or other QI project (e.g. trigger tool)||11 (10%)|
|Education session in practice||21 (19%)|
|Individual action only (e.g. increased vigilance)||6 (5%)|
|Shared with health board/interface group||15 (13%)|
Table 4: Overall frequency of actions in each domain.
The frequency of described actions in each of the main system domains are described in Table 5. The most frequently reported action was alteration of practice protocols. Overall, 35% of reports described changing protocols and 19% described educational interventions. Interventions that relied on individual action only, for example increased vigilance, was reported in only five percent of reports. Projects were infrequently discussed with other areas of health and care. Only seven of 12 SEAs that specifically related to interactions between general practice and other area of health and care were discussed with professionals outside the practice (58%).
|Domain||Example actions||Number of implemented actions (% of SEAs in the category with this action)|
|Discussion with secondary care colleagues||4 (29%)|
|Discussion with colleagues in other areas in health and care (e.g. nursing homes, social work)||2 (14%)|
|Alteration practice protocols (e.g. how messages sent and recorded, develop a recall system, checklist, electronic template development, incorporate new software, develop induction pack)||9 (64%)|
|Purchase of new equipment||0 (0%)|
|Audit or other QI project (e.g. trigger tool)||5 (36%)|
|Education session in practice||6 (43%)|
|Individual action only (e.g. increased vigilance)||3 (21%)|
|Shared with health board/interface group||3 (21%)|
|Test Results handling systems
|Discussion with secondary care||2 (29%)|
|Discussion with other areas in health and care (e.g. nursing homes, social work)||0 (0%)|
|Alteration practice protocols (e.g. how messages sent and recorded, develop a recall system, checklist, electronic template development, incorporate new software, develop induction pack)||7 (100%)|
|Purchase of new equipment||0|
|Audit or other QI project (e.g. trigger tool)||0|
|Education session in practice||2 (29%)|
|Individual action only (e.g. increased vigilance)||0|
|Shared with health board/interface group||1 (14%)|
|Communication with care services external to the practice (Total=12)||Discussion with secondary care||3 (25%)|
|Discussion with other areas in health and care (e.g. nursing homes, social work)||4 (33%)|
|Alteration practice protocols (e.g. how messages sent and recorded, develop a recall system, checklist, electronic template development, incorporate new software, develop induction pack)||8 (67%)|
|Purchase of new equipment||0|
|Audit or other QI project (e.g. trigger tool)||1 (8%)|
|Education session in practice||5 (42%)|
|Individual action only (e.g. increased vigilance)||2 (17%)|
|Shared with health board/interface group||5 (42%)|
|Clinical decisions and actions
|Discussion with secondary care||1 (9%)|
|Discussion with other areas in health and care (e.g. nursing homes, social work)||0|
|Purchase of new equipment||3 (27%)|
|Audit or other QI project (e.g. trigger tool)||1 (9%)|
|Education session in practice||5 (45%)|
|Individual action only (e.g. increased vigilance)||1 (9%)|
|Shared with health board/interface group||4 (36%)|
|Other practice systems
|Discussion with secondary care||1 (13%)|
|Discussion with other areas in health and care (e.g. nursing homes, social work)||0|
|Purchase of new equipment||0|
|Audit or other QI project (e.g. trigger tool)||4 (50%)|
|Education session in practice||3 (38%)|
|Individual action only (e.g. increased vigilance)||0|
|Shared with health board/interface group||2 (25%)|
Table 5: Reported improvement actions implemented in each incident system domain.
Two of the submitted reports (3.8%) described direct involvement of the patient in describing their perspective on what happened, the impact they experienced or in the design of the implemented change. Examples are the design of new systems to communicate results to patients and the design of a new system for INR monitoring.
Grading of consequences
The grading of the consequences of the events is shown in Table 6. The majority of the enhanced SEA reports described events whose consequences were negligible or minor (86.6%). Five events had consequences that were moderate or severe (9.6%).
|NRLS grading of consequences||Number of reports in this grading (%)||Example incident|
|Not graded – positive events||2 (3.8%)||Provision of good palliative care|
|Grade 1 – Negligible consequences||24 (46.2%)||Communication of test results to patient – informal complaint|
|Grade 2 – Minor consequences||21 (40.4%)||Prescribed too high a dose of warfarin – required extra monitoring and vitamin K|
|Grade 3 – Moderate consequences||4 (7.7%)||Inappropriate unopposed oestrogen HRT|
|Grade 4 – Major consequences||1 (1.9%)||Patient developed drug related renal failure – no blood test monitoring.|
|Grade 5 – Catastrophic consequences||0 (0%)|
Table 6: Grading of risk consequences of SEA reports (n=52).
Quality of SEA report
The mean grading given in each domain of the SEAs by NES peer reviewers is shown in Figure 2. Comparison is made between NHSAA SEA reports, those completed by prospective GP educational supervisors (n=38) and those completed by GP specialty trainees (n=98). All three groups have similar mean scores in all domains. The actions described in Ayrshire and Arran SEAs were graded lower than those of prospective trainers but higher than those of GP specialty trainees. The differences did not reach statistical significance.
This study examined enhanced SEA reports submitted by frontline GP teams in a single health board area as part of an incentivised regional initiative to improve learning and action from the analysis of patient safety incidents. The main findings suggest that participants were largely able to undertake ‘systemscentred’ event analyses as judged by external peer review.
Types of incidents and their consequences
A relatively large proportion of incidents described communication problems with external care services and a higher proportion of events with lower consequences were reported than have previously been described . As reports were submitted to the local health authority, low consequence events may have been purposively selected so that GP teams were less likely to be seen to be at fault. Furthermore, the submission of events where communication with external care sectors was problematic may have been prioritised in an attempt to emphasise the potential for harm at care interfaces and to stimulate improvement efforts at a Health Board level. The evidence base reports that patient safety incidents related to communication issues is one of the most common contributory factors in why things go wrong . The nature of communication problems between primary care and other care sectors often revolves around issues to do with transfer of accurate information in a timely manner and many of these challenges and their impacts were described the submitted reports .
Although incident reporting is well established in secondary care, reporting rates in primary care in the UK and internationally are very low, to the extent that routine engagement with formal reporting systems is virtually non-existent for many if not most teams – a circumstance mirrored in NHSAA . Submission of reports as part of this study may have been considered a form of incident reported by teams; however, less than 15% of event submissions in this study stated that the event had been formally and independently reported to the health board. Insufficient time, lack of feedback, fear of blame, perceived reputational damage and patient confidence have been found to be barriers for reporting of incidents by GPs . Adoption at all levels of health care of key principles for understanding and responding to patient safety incidents (a fundamental learning needs for care practitioners, educators and policy makers) may help overcome some of these barriers [18,19].
Rather than the propensity to frequently blame individual personal and professional failings (self and others), the adoption of a ‘system approach’ to inform more constructive and meaningful organisational learning following adverse events has been recommended in influential policy documents in the UK such as the ‘Organisation with a Memory’ report and more recently, the ‘Francis report’ [20,21]. In a recent study evaluating actions after a different incident investigation technique, Root Cause Analysis, the most common reported action was training . It is encouraging that SEAs in this study often described actions aimed at improving a wide range of care systems and procedures rather than focussing on being more vigilant or undertaking personal education or refresher training which are viewed as limited or weak improvements as part of the hierarchy of risk management interventions [23,24].
Many of the SEAs that centred around clinical decision making issues reported educational changes but all of these events also included changes to practice systems, which is gratifying given that ‘diagnostic error’ and related cognitive challenges are known to be influenced by wider system factors . It may be that rather than practitioners blaming themselves or colleagues and proposing to ‘try harder’ or learn more about a specific area, enhanced SEA has encouraged a ‘system approach’ and potentially more robust change ideas as a consequence, particularly compared with the previous research in this area [5,22].
There was limited evidence that SEA reports described discussions with other areas of health and care. Even when the event involved interactions with other professions just over half were discussed with these professionals. This may represent difficulties in contacting those involved, time available to contact them or lack of belief that such interactions will be worthwhile . Practices may have submitted SEAs describing interface issues with other parts of health and care in the hope that the act of submission will lead to change. Further research is needed to explore this in depth.
The design or redesign of practice protocols and procedures were described as the most frequently occurring actions for improvement in all system domains. This demonstrates to some extent the complex and imperfect nature of these care systems; an issue that goes to the heart of most patient safety incidents in general medical practice and wider healthcare . It also highlights a number of challenges around the design and implementation of practice procedures as barriers in safety management. The implementation of protocols is often seen as a ‘robust change’ however in the complex sociotechnical systems present in health care it can be difficult to specify work adequately to cover all conditions faced by staff. Work-asdone by frontline staff in order to achieve success often has to deviate from work-as-imagine in protocols. Obtaining multiple perspectives of staff on proposed changes and designing new protocols in a participatory manner may be more likely to result in work-as-done being closer to work-as-imagined .
Patient Involvement and early resolution of complaints
Involvement of patients in SEA is encouraged but has been reported as being difficult to achieve [28,29]. In this study only a small minority of practices reported the involvement of patients and families. Scottish and UK legislation will soon come into force regarding the organisational expectations around discussion of safety incidents with patients and families as part of a Duty of Candour – the professional expectation is already embedded in good clinical practice by regulators [30,31]. Further guidance and training on how this can be achieved more readily is needed if this is to be adopted more widely as part of a person-centred approach to quality care.
GPs have reported that completing SEA may be useful when a complaint has been received as this may demonstrate to patients that it has been taken seriously . Indeed, it may be that the use of SEA may aid the early resolution of complaints . There are potential barriers to reflecting openly when analysing significant events as reports could be requested by a court of law . Focusing on systems based learning rather than blaming individual actions may be a helpful approach.
Quality of reports
Measures of quality of SEA reports between Ayrshire and Arran GPs, prospective trainers and specialty trainees were similar. This demonstrates that after training frontline GP teams can perform SEA analyses of equal quality to previously identified early adopters and career motivated individuals. The quality can be improved further and the development of education interventions to enhance improvements is required.
Strengths and limitations
This study showed high engagement in a single health board and used a well-established and evidence based educational system but there were many limitations to this study. The numbers were small and so there is limited generalisability. Moreover, the type of incident analysed and submitted may have been influenced by the fact that this was part of an enhanced service. The intervention required the provision of dedicated training time that may not be possible for most GP teams across the UK. Ayrshire and Arran employed two GPs who had expertise and in enhanced SEA who were able to deliver the training. This approach may not be feasible for all health boards.
Implications for policy, practice and future research
To improve the quality of SEA reports in primary care and thus improve the learning and the effectiveness of change, priority has to be given to developing the knowledge and skills of frontline GP teams to perform SEA and the protection of time to complete SEA. For some health boards, capacity will need to be developed to train frontline teams.
Improving relations across care interfaces may be required before joint analysis of events can be undertaken but consideration needs to be given to how best to facilitate the involvement of other care sectors in analyses . This may include deciding on the best method of communication (telephone, electronic mail or through existing reporting systems), guidance on how communication should be conducted (for example by trying to understand why decisions were made in other areas rather than apportioning blame) and ensuring relevant feedback is given after analysis . If incident reporting in primary care is to be prioritised at a national level, GPs will need time and training on how to report events and which events to report. Further research into the barriers to patient involvement in SEA is also required if this area of SEA is to be developed.
This study has demonstrated that most GP practices, after minimal training, can apply the enhanced SEA method and largely adopt systems based interventions to improve care by analysing and respond to past harm or near misses. Practices described integrating learning to improve care systems which is a key component of Vincent’s Framework for measuring and monitoring safety . However, to be more comprehensive, consideration of how to improve the analysis of events across healthcare interfaces, involve patients to maximise learning, increase incident reporting and employ prospective hazard analysis methods, are also needed. Although this is a comparatively small study, it provides some important evidence that this ‘new’ approach is leading to more effective and meaningful event analyses by primary care teams - thereby potentially overcoming the key criticisms of the ‘old’ personfocused method which encouraged a limited, superficial and non-systematic investigation of a healthcare problem or issue.
Practices received funding from Ayrshire and Arran health board to participate in the enhanced service. No other funding was received for this evaluation.