Peer Review, quality of healthcare, referral, service development, variation, clinical practice

Introduction

General practitioner (GP) referrals have not histori-cally been routinely discussed with colleagues, and the quality and appropriateness of referrals has been dif-ficult to measure. Some GPs may be under-referring, introducing important quality and patient safety issues. Demand for hospital services is rising everywhere in the UK[1] and referrals need to be as appropriate and effective as possible.[2]

We know that individual GPs’ referral rates vary considerably, reflecting a range of appropriateness of referral, but it is not known how best to influence this key aspect of general medical practice.[2–5] One GP may consistently refer more or fewer patients than another and there is little understanding about why this is the case. In the UK no relationship has been found between referral rates and age of GP, years of experience or Membership of the Royal College of General Prac-titioners.[5] However, there has been a persistent rise in referrals across the UK.[1] Market reforms in the UK health system, practice-based commissioning and reduc-tion in waiting times have all been implicated in the rise.[6,7] However, even in settings such as Wales where there have been no market reforms there has been a similar increase in referral rates.[8] There is increasing attention on how to stem the rise in referral rates, and there is an assumption that this is based around ensuring appropriateness and quality of referrals.

Previously there was no system within general practice to look in detail at the process, appropriate-ness or quantity of referrals, compared with the many systems which look at prescribing, another very costly activity. There is also very little recent research into referral patterns within and between practices. There was some early experience gained from promoting prac-titioners to refer internally for further advice from other clinicians before deciding on hospital referral.[9,10] More recently several primary care organisations in the UK (principally primary care trusts) have instituted ‘Referral Management Centres’.[11] Experienced clinicians out-side the practice – also usually GPs – appraise the appropriateness of referrals according to criteria and may approve the referral, re-direct it to more appro-priate providers or refer back to the original GP to consider alternative ways forward for the patient concerned. Referral Management Centres have been viewed at best as a temporary measure as they do not fundamentally change the drivers of demand.[8] How-ever, in the UK, Peer Review and discussion of referral (as well as prescribing and emergency admission) rates is part of the Productivity Framework of the General Practice Quality and Outcomes Framework (incentivised performance criteria) from 2011/2012, indicating policy level support for external Peer Review of referrals.

Recent overviews have highlighted both the com-plex nature of the referral process[12] and how referral management centres have on the whole failed to achieve success in influencing referral behaviour.[13] Clinical triage also has inherent risks for patient safety.[13] These reports gave a clear steer that ‘a referral management strategy built around Peer Review and audit, supported by consultant feedback, with clear referral criteria and evidence-based guidelines is most likely to be both cost- and clinically-effective’. They also note there is a dearth of evidence about the impact of different approaches to demand management.

We previously reported the findings of a pilot scheme using these methods, which showed that quality of referrals as judged by doctors’ peers improved, referral rates to orthopaedics showed a reduction of up to 50% and understanding of other community based services and referrals through best local pathways for some common conditions were enhanced.[14] Here we report the experience gained and lessons learned from including a wider range of practices, first in Torfaen, South East Wales (then an autonomous unit for commissioning services) and then more widely in Gwent. The project, using practice-based GP Peer Review of referrals with formative feedback, alongside clinical engagement between GPs and consultants, continued for a further two years; Wave 1 in 2008 to 2009 and Wave 2 in 2009 to 2010. We examined whether the project affected the quality of referrals, variation in referral rates, overall referral rates and whether the effects appeared sustainable or were dependent on certain factors or local circumstances.

Method

Participants

In Wave 1 (2008 to 2009) ten out of the 13 practices in Torfaen took part (all were invited). These practices had list sizes ranging from 4600 to 11 500 (median 6450). Torfaen has high morbidity and deprivation indices, particularly in the northern half of the borough in post-industrialised ‘valleys’ towns (Townsend Dep-rivation scores range from –5.7 to 8.0, where 0 = all-Wales average (positive score more deprived);  www.  wales.nhs.uk/sites3/documents/368/Townsend%20-%  20Torfaen.xls). The following year 22 practices from across Gwent participated in a similar scheme, Wave 2 (2009 to 2010). The range of participants’ sessional commitments is characterised in Table 1. Twenty-one out of 53 GPs completing Wave 1 were female (median age of females 44, median for males 48.5). All referrers were included, but GP registrars and locums and doctors on maternity leave, sabbatical or who retired during the project were excluded from the final individual GP data analysis as their change in referral rates could not be calculated. There are no data about case-mix variations between doctors providing higher or lower numbers of sessions per week.

Intervention

The intervention was complex, engaging participating GPs and consultants in relevant specialties in different components. There was no specific pressure to reduce referrals, although feedback was given to the practices on their referral rates. The intervention included five elements.

1  The principal tool to influence future referral behaviour was a review of recent referrals. The practice was asked to meet, approximately weekly, to review recent referrals and discuss:

•    Consensus 1: agreement by the GPs on stan-dards of information in the letter (for complete-ness of clinical information, treatment to date, background social or employment factors and patient pReferences)

•    Consensus 2: on whether there had been appro-priate work-up beforehand

•    Consensus 3: whether guidelines had been used to inform the referral decision.

Discussion also took place on whether the peer group would have recommended alternative man-agement pathways or whether the referral was deemed ‘appropriate’. Appropriateness was judged internally by the practice, not according to external criteria, and although data were available on the proportion of referrals thought appropriate (see Results: Quality and Effectiveness) this consensus discussion should be viewed more as an integral part of the process of project participation, rather than as an outcome in itself. Meetings were intern-ally led, not facilitated.

2 The total number of referrals in the selected spe-cialties and appraisal of these referrals was recorded, including demographic and clinical features on specific datasheets.

3 Practices were divided into ‘clusters’ addressing designated clinical topics (see below) and they met at approximately six-weekly intervals to share ex-periences and make suggestions for improving the referral pathway across the district.

4 At the cluster meetings, practices were also pro-vided with actual referral rates across the practices, providing a ‘bench-marking’ influence.

5   Local consultants and representatives of other services (e.g. physiotherapy, optometry) in the relevant specialties also attended the meetings to engage GPs in discussion and to seek a consensus on the nature of appropriate referrals in the do-main, required work-up before referral and man-agement of certain patients in primary care without referral.

The only prior referral filters in the chosen specialties were in dermatology, where guidance existed on con-ditions that would normally be managed in primary care (e.g. warts, molluscum contagiosum, dermato-phyte infections, benign naevi and sebaceous cysts).

Wave 1 practices were divided into three cluster groups. All practices discussed orthopaedics. In ad-dition general surgery, neurology, ophthalmology, ENT, dermatology and nephrology were discussed within one cluster each (making three clinical topics per cluster). In Wave 2, 22 practices discussed these plus urology, cardiology, rheumatology and gynaecology. Combin-ations of specialties were identified so that total numbers of referrals per practice size would be equivalent for the review workload. Private referrals were included in order to keep practice referral rates comparable. As referral numbers were practice identified (rather than identified from Trust (hospital) figures), referral num-bers for the non-selected specialties were not available.

Specifically, in orthopaedics the emphasis was on the appropriate use of mainstream and specialist physiotherapy and podiatry and encouraging patient self-help; the dermatology discussions focused on some chronic skin diseases and minor surgery lesions staying within primary care; and general surgery and neurology discussions focused on appropriate work-up before referral. In ophthalmology, to which both GPs and optometrists can refer, the discussion in-cluded optometrists and ophthalmology consultants, and explored how optometrists could help GPs to manage patients within primary care and vice versa. Data were collected on alternative pathways, such as community based services either already in existence or that could be developed, to which referral could be made, thus saving a referral to hospital.

The practices were paid 80 pence per patient of list size to cover the administration costs and doctors’ time, under a local enhanced service (LES) directive,[14] totalling between £3700 and £9230 for the range of practice sizes. The total cost of payments for the ten practices in Wave 1 was £59 200.

Data processing

Data were collected on a specially designed (Excel) spreadsheet which had been further refined from the previous year to allow quick downloading to an Access database. This was analysed by a database manager and the Health Board’s Medical Advisor (EE). Rates of referral for each doctor were calculated and adjusted for sessions worked (as a proportion of ten sessions – i.e. referrals 10 divided by sessions worked). De-scriptive data about the numbers of referrals were derived, but without statistical analysis in view of the small sample size (of practices).

Results

In Wave 1 there were 2959 referrals over the year, with 53 of 58 GPs still in practice at the end of this year (five left for maternity leave, sabbatical etc). The following year (2009 to 2010) in Wave 2 there were 4135 referrals by 124 doctors over six months, across more specialities. The overall range of referral numbers was wide, between 22 and 122 for females and 12 and 126 for male GPs over six months. The rate of referral per doctor was similar in different specialities, i.e. if doctors referred at a high rate in one speciality they were likely to refer at an equivalent rate in other specialities (with excep-tions for gender specific specialities such as gynae-cology and urology).

Doctors referred at similar rates regardless of num-ber of sessions worked – the average was 26.6 referrals per three months for those working between one and three sessions per week, 28 for those for those working between four and six sessions per week and 25.5 for those working between seven and ten sessions. In keeping with other work on this subject, no corre-lation was found with age or gender of doctor, whether part- or full-time, size of practice or locality. Partici-pation in the project’s cluster meetings was in general excellent, a particularly valued component for both GPs and consultants, although those working between seven and ten sessions per week found it easier to attend than those working fewer sessions.

Figure 1 shows each GP’s referral rate (all special-ties; dark columns; weighted as if that GP had worked ten sessions). The highest number of referrals was 43 in the first quarter. These data are plotted with each GP’s change in number of referrals from first quarter to last quarter (light columns; also weighted as if working ten sessions). Where no light column is shown, the GP had the same number of referrals in the first and last quarter. Thus, in general, Figure 1 indicates that the highest referrers show a decrease while the lowest referrers may show an increase in referrals (and a significant negative correlation comparing the first month’s data to the change from first to last month, r=–0.719; P=0.019).

Figure 1: Individual doctors’ referral rate changes

These changes were also seen at practice level (see Table 2). The referral rate (per 1000 patients per quarter) range decreased from between 2.6 and 7.7 to between 3 and 6.5. The median referral rate fell from 5.5 to 4.3 per 1000 patients per quarter for the specialties in question. Overall the net result was a fall in number of referrals to secondary care as higher referrers reduced their referrals more than lower referrers increased theirs. In Wave 1 the decrease ranged from 5.2% (general surgery) to 16% (neurology; see Figure 2), in the context of a concurrent overall rise in referrals from practices in all four other localities not taking part in the scheme (e.g. orthopaedics; see Figure 3).

Figure 2: Change in referral rates

Figure 3: Orthopaedic referrals to local trust from June 2007 to March 2009 showing increasing trend lines for all localities, and a reduction in Torfaen

ENT (26% rise; see Figure 2) was a special case. Practices with unusual changes in referral rates in-cluded Practice E, which increased from approximately median to highest referrer although there was no obvious cause for this. Practice F had low referral rates in Quarter 1 and showed only a small rise during Wave 1; this practice was one of those that had participated in the pilot phase, thus indicating some sustainability effects from continued participation.

Sustainability of change

Data regarding sustainability of change were available for orthopaedics from two data sources, the data collected by practices and trust (hospital) data. First the three practices that participated in the pilot study continued during Wave 1, and one into Wave 2 (thus participating for two quarters longer than the others), representing up to 27 months participation. Their decreases in orthopaedic referral rates achieved in the pilot phase were sustained during Wave 1 (second quarter of 2008 to first quarter of 2009; see Figure 4).

Figure 4: Orthopaedic referrals: sustainability with pilot practices over two years

Second, seven practices left the scheme in March 2009, while three practices continued in Wave 2. The practices that continued in Wave 2 showed little increase in referral rates, while those practices leaving the scheme showed an over 40% increase in referrals after leaving (see Table 3). In parallel practices not participating in Wave 2 (73 practices across Gwent) the overall increase was 7% during those six months.

Third, in one speciality – ENT – no consultant was available to come to the cluster meetings. The scheme was stopped after Quarter 3, but data were still collected on referral numbers. The totals of referrals were 69 in Quarter 1, 58 in Quarter 2, 59 in Quarter 3 and 90 in Quarter 4, an increase of 30% from baseline after the GPs stopped discussing the referrals.

Quality and effectiveness

Less variation in itself might be associated with increased quality, and the GPs’ own assessment of their referrals was that they improved from 89% to 95% (Consensus 1) and 86% to 94% (Consensus 2), similar to findings in the pilot year.[14] At the same time the number of referrals over the year decreased, which may enhance quality by decreasing the overall workload in a cost-limited environment.

Discussion

Summary of main findings

General practices participated in this intervention to discuss referral pathways, using internal Peer Review of referrals complemented by feedback of benchmarked local data, and found cooperation with local consult-ants in the specialities studied extremely helpful. The intervention enabled clinicians to achieve a consensus over many referral pathways and improved quality (as judged by the GPs themselves). In addition both the actual referral rates and the variation in referral rates decreased considerably. The reduction appeared sus-tainable whilst the intervention continued, and referral rates rose in keeping with local trends once the inter-vention finished. Few characteristics of area, practice or doctors were identified to explain variation in referral rates.

Strengths and limitations of the study

This was a small ‘case-study’, capturing data from a service development project. It was not a trial with adequate control data. It is possible that the narrowing of variation of referral rates observed over the project period includes an element of regression towards the mean, and elements of seasonal variation (e.g. ENT) which we were unable to control for. However, the effects of the intervention were monitored both by data collection within the participating practices, and by comparison with routinely collected data from the localities involved. We did not rigorously assess ap-propriateness of referrals, primarily as the study was based on internal review, not external assessment, and also in order to minimise data collection requirements to ensure a high participation rate in the study. As-sessing completeness of information and the option of alternative management pathways was part of the process of the practices’ review of their own referrals, indicating an implicit focus on appropriateness, but this was not explicitly assessed. Referral numbers from participating and non-participating practices include some systematic differences – participating practices included private and out-of-area referrals so that the total activity could be assessed, whereas data from non-participating practices came from trust sources, thus not including private or out-of-area referrals, but potentially including some consultant–consultant referrals.

Comparison with existing literature

The intervention sought to engage the participants in a process of learning from themselves and others outside the practice. It used self-assessment and feedback of referral rates with benchmarking. The effects that were identified are consistent with other interventions for different aspects of general practice such as prescribing,[15,16] investigations,[17] organisational develop-ment[18,19] and earlier studies on referral.[9,10] These were based on similar underpinning principles, focusing on GPs reflecting and learning, in a formative educational process.

Implications for practice

Quality and quantity of referrals are linked in their effect; too many referrals can overload secondary care and contribute to longer waits and poorer outcomes. Too few referrals can compromise patient safety. This intervention to improve the quality of referrals was implemented and appeared effective in a large number of participating practices. However, the effect ceased as soon as the active intervention finished, indicating that active participation is key – reliance on a change of culture or referral behaviour to maintain reduced referral rates after a limited intervention cannot be supported from our experience. Furthermore, active participation of consultants appeared important. Some had previous experience of implementing the transfer of services to primary care, but others had had little previous interaction with GPs. The discussions did not always result in complete agreement, but were always useful. The general practitioner participation was resourced to provide protected time for review and discussion in the practice. Each of these features of the project – continued active participation by GPs, with resources and dialogue with consultants – ap-peared essential and require attention in planning further implementation of referral management strat-egies. This is consistent with the recommendations in the recent King’s Fund reviews.[12,13]

Anecdotally, the scheme resulted in improvement in communication and knowledge, and as such is beneficial to the healthcare system as a whole. It might be particularly applicable to practices and areas that find they have unusually large variations within and between practices. It may also be cost-effective, with the reduction in referrals tending to offset the costs of the scheme, especially when referrals are otherwise rising. A more detailed cost-effectiveness evaluation would be required.

Against this, primary care organisations may not be able to maintain Funding for referral management, whether externally or internally driven,[8,11] in the current economic climate. This study has shown that referral analysis can be effective. Alternative ways of achieving such effects should be considered and eval-uated, such as making referral review a more important part of the curriculum in GP education and appraisal.

Specific education on referral decisions has been rare in general practice to date. A requirement to do dedicated audits could be valuable, especially if in-volving peers in the review process. Other potential levers for improving referral practice could include peer-review mechanisms or the specification of tar-gets. In the case of the UK, this is now being imple-mented through incorporation of referral review (along with reviews of prescribing and emergency admis-sions) into the Quality and Outcomes Framework for GPs from 2011/2012,[20] a mechanism which has been shown to accelerate improvement trends across varied sectors of chronic disease management.[21] More could also be done to promote local community based services, to counteract the influence of special interest groups and to provide more objective local advice (as is done by pharmacist advisors for prescribing of medicines in general practice).

Implications for future research and development

There is a need for greater understanding of the influences on referral rates and this would need more data than is at present available from NHS systems. Data collection is always expensive and diffcult, and it contributes hugely to the costs of a project when collected by the practices themselves. The King’s Fund cautions against elaborate data systems for referral management, however, as referral is usually individual clinician behaviour, the intervention was based around review of individual referrals, and the collection of individual GP data was essential. Our data included reasons for referral, patient demand referrals, expected outcome and some limited clinical information (e.g. sub-specialty), which were very useful. If such infor-mation could be collected in a searchable form at the time of referral this would be very beneficial. Clinical systems will need to improve for accurate commission-ing, so this may provide such an opportunity. More comprehensive data may also identify potential strat-egies for further intervention, themselves requiring evaluation. These may complement a multifaceted inter-vention such as described here, or include quite different approaches, such as managing patient expectations and demand and promoting self-management in some situations. As above, the cost-effectiveness of a scheme such as described here requires further evalu-ation.

Conclusions

In the context of limited resources, it is imperative to ensure that all GPs put quality to the forefront of their referral decisions. This intervention, comprising internal review of referrals, feedback of referral rates and bench-mark data, and particularly with the involvement of local consultants, was acceptable to participating GPs and appeared to show an effect while the intervention was active, although this effect was not sustained after active project resources (time and Funding) ceased. It is based upon consensus not competition and can inform the development of community based services and extension of primary care services. As such this could help in the diffcult task ahead for GP com-missioning consortia.

Acknowledgementss

The authors are very grateful to members of the participating practices, consultants contributing to cluster meetings and staff at the Torfaen Local Health Board involved in data collection and analysis and to Dr Patrick Miller for statistical help.

References

1. Edwards N. What is causing GP referrals to rise? Health Service Journal 2008, Nov. 27;19.
2. Roland MO, Bartholomew J, Morrell DC, McDermott A and Paul E. Understanding hospital referral rates: a user’s guide. BMJ 1990;301:98–102.
3. Hugo P, Kendrick T, Reid F and Lacey H. GP referral to an eating disorder service: why the wide variation? British Journal of General Practice 2000;50:380–3.
4. Hippisley-Cox J, Hardy C, Pringle M et al. Are patients who present late with cancer registered with low refer-ring practices? British Journal of General Practice 1997; 47:731–2.
5. O’Donnell CA. Variation in GP referral rates: what can we learn from the literature? Family Practice 2000;17: 462–71.
6. Health Policy and Economic Research Unit. Factors Capable of Influencing an Increase in GP Referral Rates to Secondary Care. London: British Medical Association, 2009.
7. Mannion R and Street A. Managing activity and expen-diture in the new NHS market. Public Money and Management 2009;29:27–34.
8. National Leadership and Innovation Agency for Healthcare. Referral Management Pilots in Wales: follow up review. Cardiff: Welsh Assembly Government, 2007.
9. Maggs-Rapport F, Kinnersley P and Owen P. In-house referral: changing general practitioners’ roles in the referral of patients to secondary care. Social Science and Medicine 1998;46:131–6.
10. Kinnersley P, Rapport FM, Owen P and Stott N. In-house referral: a primary care alternative to immediate secondary care referral? Family Practice 1999;16:558–61.
11. Davies M and Elwyn G. Referral management centres: promising innovations or Trojan horses? BMJ 2006;332: 844–6.
12. Foot C, Naylor C and Imison C. The Quality of GP Diagnosis and Referral. London: The King’s Fund, 2010.
13. Imison C and Naylor C. Referral Management: lessons for success. London: The King’s Fund, 2010.
14. Evans E. The Torfaen referral evaluation project. Quality in Primary Care 2009;17:423–9.
15. Mowle S, Edwards A, Hood K and Kinnersley P. Benchmarking using simulated clinical scenarios: a feasibility project. British Journal of General Practice 2001; 51:658–60.
16. Simon SR, Smith DH, Feldstein AC et al. Computerized prescribing alerts and group academic detailing to reduce the use of potentially inappropriate medications in older people. Journal of the American Geriatric Society 2006;54:963–8.
17. Grupe P, Grinsted P, Møldrup M and Høilund-Carlsen P. Quality improvement of referrals from practising physicians. An example of a practicable strategy. (Article in Danish.) Ugeskr Laeger 2006;168:1434–8.
18. Rhydderch M, Elwyn G, Marshall M and Grol R. Organisational change theory and the use of indicators in general practice. Quality and Safety in Health Care 2004;13:213–17.
19. Eriksson T, Siersma VD, Løgstrup L et al. Documenting organisational development in general practice using a group-based assessment method: the Maturity Matrix. Quality and Safety in Health Care 2010;19:1–7.
20. General Practitioner Committee. Your Contract, Your Future. London: British Medical Association, 2002.
21. Campbell S, Reeves D, Kontopantelis E et al. Quality of primary care in England with the introduction of pay for performance. New England Journal of Medicine 2007; 357:181–90.

Funding

Not externally funded.

Ethical Approval

Ethical Approval was not sought for this service im-provement project.

Peer Review

Not commissioned; externally Peer Reviewed.

Conflicts of Interest

None.