Rheumatology Advance Access originally published online on March 1, 2006
Rheumatology 2006 45(9):1110-1115; doi:10.1093/rheumatology/kel042
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Experience of six years of a regional peer review scheme in rheumatology
Department of Rheumatology, University Hospitals of Coventry and Warwickshire NHS Trust, Walsgrave Hospital, Coventry, 1Staffordshire Rheumatology Centre, Haywood Hospital, Stoke-on-Trent, 2Highfield Unit, Worcestershire Acute Hospitals NHS Trust, Worcester and 3Department of Rheumatology, Russells Hall Hospital, Dudley, UK.
Correspondence to: H. Piper, Department of Rheumatology, University Hospitals of Coventry and Warwickshire NHS Trust, Walsgrave Hospital, Clifford Bridge Road, Coventry CV2 2DX, UK. E-mail: hollypiper{at}hotmail.com
 |
Abstract |
|---|
 Top Abstract IntroductionMethodsResultsDiscussionAcknowledgementsReferences |
|---|
Objective. Following discussions on peer review by the British Society for Rheumatology, the West Midlands Rheumatology Service and Training Committee established a peer review scheme for the West Midlands in 1998. We report our initial 6 yr of experience, during which all 14 units have been visited.
Methods. A rotating programme of visits was organized. Following this first cycle of peer review, questionnaires were sent to all consultants and senior allied health professionals in each visited unit and to all members of each visiting team to evaluate the process.
Results. There was clear consensus amongst staff from both visited units and visiting teams that a peer review visit is worthwhile and constructive. It is a good opportunity for education and exchange of ideas between staff and to promote the multidisciplinary team in rheumatology. Most recommendations from the reports were considered necessary. The most frequent recommendations were for an increase in consultants and therapy staff. Appointing further consultants has been successful. Opinion was only divided on whether the reports were viewed seriously by Trusts, whether peer review should be regional or national, and how to accurately assess the quality, as well as the quantity, of care provided. Staff would support further cycles of peer review visits.
Conclusions. This has been a successful initiative and a positive learning experience for all staff involved. Specifically, it helped to obtain more staff and secure facilities. We recommend developing this scheme and promoting it to other regions.
KEY WORDS: Peer review, Rheumatology.
|
Introduction |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionAcknowledgementsReferences |
|---|
The 1997 and 1998 Department of Health documents, respectively ‘The new NHS; Modern, dependable’ [1] and ‘A first class service’ [2], described how NHS organizations had to set, deliver and monitor quality standards. The Commission for Health Improvement was set up as the inspection body for the NHS and, under the heading of Clinical Governance, external inspection of hospital units by peer review was to be developed. Literally, peer review means evaluation by a colleague. It is formally defined as a ‘continuous, systematic, and critical reflection by a number of care providers, on their own and colleagues’ performance, using structured procedures, with the aim of achieving continuous improvement of the quality of care’ [3]. The structure of peer review involves preparation for the visit, selecting suitable topics, agreeing criteria, observing practice, implementing change, and follow-up [3].
In 1989 peer review in respiratory medicine was piloted with an inter-regional initiative linking the East Anglia and Yorkshire regions [4]. The British Thoracic Society, in 1992, then arranged a series of national, voluntary, interdepartmental peer review visits focusing on organizational aspects and training provided [5]. It was found to be both helpful and rewarding; it allowed free exchange of ideas between units during the visits, the reports identified many recommendations for change that had not been anticipated by the visited unit, and the feedback was also valuable in supporting requests for additional resources or staff. In conclusion the British Thoracic Society commended this process of peer review to other specialities. Subsequently, the Association of Paediatric Anaesthetists of Great Britain and Ireland developed a process for interdepartmental peer review [6], as has the British Cardiac Society and the British Diabetic Association [7]. Peer review schemes have also been reported in primary care [3] and amongst general dental practitioners [8].
In 1996 the British Society for Rheumatology (BSR) discussed the development of peer review visits in rheumatology. A regional peer review programme was already in place in Oxfordshire and their experience was used in developing the BSR Peer Review Scheme Assessment Proforma [9]. In 1998 the West Midlands Rheumatology Service and Training Committee (WMRSTC) decided to set up a scheme for peer review in the West Midlands region. The BSR were informed of this regional initiative. The ethos of the peer review scheme was to view the visits as a positive, non-threatening and educational experience. The visited unit should benefit from the opportunity for previsit intradepartmental introspection and identification of any problems in the quality and breadth of their rheumatology service provision. The visit should also be able to determine the strengths and weaknesses of how clinical governance is practised within the unit and be an educational opportunity for the multidisciplinary team. Recognition of the educational merit of the visit by the Royal College of Physicians resulted in the awarding of continuing medical education points for both the visitors and the visited. The visitors should benefit from the opportunity to compare and contrast how a different unit manages service provision, workloads, staffing levels, funding, education and research. This should allow reflection on the workings of their own unit. Both the visited and visitors were encouraged to have a free exchange of ideas during the visit.
We report experience of 6 yr of a rheumatology peer review scheme in the West Midlands, UK, during which all 14 units in the West Midlands have been visited.
|
Methods |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionAcknowledgementsReferences |
|---|
The visit
All units in the West Midlands Region were invited to participate in the peer review scheme. A designated member of the WMRSTC (J.D.) coordinated the organization of the programme of visits. The order of the visits was randomized, as was the selection of the visitors. A unit was given at least 6 months notice prior to their visit and were informed who the visitors would be. The unit was sent the previsit proforma to complete and also informed of the criteria by which their unit would be inspected. The completed previsit proforma was then sent to the visitors 2 weeks before the visit.
The previsit proforma contained basic demographic details of the unit to be visited. Questions were then grouped into: out-patients workload, out-patient staffing, out-patient facilities, general points on out-patients, in-patient workload, in-patient staffing, in-patient facilities, general points on in-patients, daycase workload and facilities, imaging facilities, pathology services, training and teaching with regard to consultants/junior medical staff/nursing/physiotherapy staff/general practice, research, audit, meetings, secretarial, medical records, budget-holding and local perceptions for change. This proforma was based on the BSR Peer Review Assessment Proforma [9]. Additional questions were included based on recent guidance on the role, facilities and essential standards of a rheumatology unit [10–12].
The visiting team consisted of two consultant rheumatologists (one nominated as chairman, who had had previous experience of peer review visits) from two different units. Each consultant recruited an allied health professional (clinical nurse specialist, occupational therapist, physiotherapist, senior ward nurse, chiropodist) from their unit to join the visiting team to make a total of four visitors. For some visits a specialist registrar attended and in some cases a business manager also attended. Each unit took it in turns to provide staff for the visiting teams.
One day was allocated for each visit. The timetable for the day was flexible depending on the size and location of the visited unit. Time was given to discussing issues raised by the previsit proforma, for meeting all staff, and touring in-patient and out-patient facilities. At the end of the day the visitors met together to discuss their findings, which would shape the subsequent formal report. At the end of the visit formal verbal feedback on the findings of the visiting team was provided for the staff of the unit and representatives of the Trust management team. This took the form of highlighting the areas of good practice identified during the visit followed by areas where the visiting team felt that practice could be enhanced.
The chairman of the visiting team was responsible for writing the report, with input from the other members of the visiting team. The final report was then agreed by all team members. It was suggested that this take no longer than a month following the visit. A template for writing the final report was provided, the headings being dictated by those of the previsit proforma. This described areas of excellence as observed by the visiting team. Findings and recommendations were then given for each of the following areas: out-patient workload and staffing, out-patient facilities, general points on out-patients, in-patient facilities, general points on in-patients, imaging facilities, pathology facilities, training and teaching, audit and research, secretarial support, medical records, points arising during the tour of the rheumatology department, and other issues arising during assessment visit.
The organization of the programme of visits, criteria used on the proforma and format of the final report were discussed, modified and agreed by all members of the WMRSTC.
The final confidential report was sent to all consultant rheumatologists whose units were visited and to the chief executive of the host Trust. A copy was also given to the chairman of the WMRSTC.
Evaluation of the peer review experience
Following the first cycle of peer review visits (all 14 rheumatology units having been visited) it was thought appropriate to evaluate the process. Questionnaires were sent to all consultants and senior allied health professionals (AHP) (nurse, physiotherapist, occupational therapist and clinical nurse specialist) in each visited unit. All consultants and AHPs in the visiting teams were sent a different questionnaire to analyse the peer review process from this perspective. Each questionnaire was composed of statements concerning the peer review scheme with which the reader was asked to express agreement or otherwise, on a four-point Likert scale. These statements included positive and negative comments to reduce bias. The opportunity was given for free text comments about the peer review process. Individuals were asked to indicate who they felt should have access to the final report.
All consultants in each visited unit were also provided with a list of the recommendations that they had originally been sent and asked to indicate against each recommendation whether it (i) had been acted on and change implemented, (ii) was going to be acted on, (iii) was necessary but not acted upon, or (iv) was not considered necessary.
|
Results |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionAcknowledgementsReferences |
|---|
All 14 of the units in the West Midlands chose to participate in the peer review scheme. The visits took place between 2000 and 2004. At the time of this analysis, reports for 12 of the 14 units were completed.
Ninety-three health professionals from 12 different rheumatology units were sent questionnaires. Fourteen of these individuals were confirmed not to be still in post and so could not reply. Fifty-nine of the 79 people available to reply did so (74.6%). Of these 59 people, several both worked in a visited unit and were a visitor to another unit. These people completed two questionnaires appropriately. Thus (excluding those health professionals not still in post) 28 out of 32 (87.5%) consultants and 24 out of 35 (68.5%) allied health professionals whose units were visited replied. Twenty four out of the 32 (75%) visitors replied.
Preparation of the previsit proforma took between 4 and 12 h and was normally done in electronic format. This was always done by a consultant. Four out of 11 consultants found obtaining the information easy. Seven out of 11 consultants found it troublesome obtaining information due to both information not being available in an appropriate format and inadequate help from administrators in accessing information. The visitors spent between 1 and 6 h (median 4 h) assimilating this information provided prior to the visit. All but two people involved felt 1 day was the appropriate amount of time for a peer review visit. Two people felt up to 2 days should be allocated in units with outreach services.
Table 1 shows the questionnaire responses from consultants and AHPs in the visited units. There was clear consensus on many issues. Participants agreed that previsit intradepartmental introspection was valuable, the balance of health professionals in the visiting team was appropriate, people were comfortable in being reviewed by people they knew well, the report was an accurate assessment, the recommendations were agreed with and peer review was both worthwhile and a constructive component of continuing professional development. People consistently disagreed that the visit caused undue anxiety, that knowing the reviewers affected the objectivity of the report, that the report failed to address the important issues, and that the peer review cycle should not be repeated. Issues where opinion was divided included AHPs not having access to the report and not being involved in implementing pertinent recommendations. In addition, there was no clear consensus on whether the report was viewed seriously by Trusts and Primary Care Trusts (PCTs) or whether peer review is best organized as a regional rather than national process.
|
There was also consensus of opinion amongst the visiting teams. They agreed that the balance of health professionals in their teams was appropriate, that they were comfortable in reviewing peers they knew well, they felt able to objectively review a department and accurately assess the quantity of work achieved, it was a constructive experience for them personally, and again peer review was worthwhile and a constructive component of continuing professional development. Staff on the visiting teams consistently disagreed that knowing peers in a visited unit affected the objectivity of their report, that they would not be inspired to alter their own practice having learnt from the unit they reviewed, and that the peer review cycle should not be repeated. The one issue where the visiting team did not reach a clear consensus was whether, on a peer review visit, they were able to accurately assess the quality of care provided; five visitors did not feel they were able to assess the quality whereas 19 visitors did.
Writing the report from the peer review visit took between 4 h and 2 days. The cumulative salient recommendations from the reports can be considered in terms of staffing, facilities, organization and education and training (Table 2). All 28 consultants assessed each recommendation suggested to their unit. This gave a total of 576 responses. Two hundred and eleven responses stated the recommendation had been acted on, 101 responses stated the recommendation will be acted on and 230 responses stated the recommendation was thought to be necessary but had not yet been acted on. Thirty-four responses stated that the recommendation was not thought to be necessary. There was not always consensus of opinion between consultants in the same unit as to whether recommendations had been acted on. Table 2 also evaluates the salient recommendations. The most frequent recommendations were appointing further consultants, increasing staffing in therapy services, improving access to orthotics and podiatry, developing an on-site DEXA (dual energy X-ray absorbtiometry) service as well as developing regular multidisciplinary rheumatology meetings, improving disabled parking and implementing electronic patient recording. Of these, appointing further consultants and developing regular multidisciplinary meetings have frequently been implemented but increasing staffing in therapy services and improving access to orthotics and podiatry have not yet been implemented despite being considered necessary.
|
Forty-eight people replied to the question on who should have access to the final report. All agreed the report should be available to clinicians of the visited team and the medical director of the visited unit. Most people thought the chief executive of the visited team and the chairman of the WMRSTC (46 people and 43 people respectively) should have access to the report. Fewer felt it should be available to all members of the WMRSTC (11 people) or free access to the public (eight people). Other suggestions included making the report available to the PCT or having a lay worded summary available on each hospital website.
Suggestions for when the peer review cycle should be repeated ranged from annually to every 7 yr, with a median of 5 yr.
Free text responses reflecting on this process of peer review revealed a number of perceived positive outcomes. These included helping obtain more consultants and secure daycase facilities, setting strategies for the next 5 yr and focusing the managerial team on the requirements of the department. Staff found it a positive learning environment where strengths could be transferred between units, and found it very helpful in promoting the multidisciplinary team. The negative outcomes of this scheme included realizing that change does not automatically follow, especially when management felt unable to deliver on many points due to financial pressures. Comments on the practical process of this peer review included that it involved a lot of work and was time-consuming, but this was time well spent. Suggestions for the future included a faster turn-round of reports, grading recommendations as vital, recommended or advised, and giving a time frame to complete the vital and recommended suggestions, giving the Trust Executive the opportunity to respond to the report and getting managerial commitment to the process. Reviewers should rotate their sites for future reviews and ensure all staff in the region are invited to participate.
|
Discussion |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionAcknowledgementsReferences |
|---|
The WMRSTC embarked on this peer review process with certain aims in mind, as outlined in the introduction; in several areas this evaluation has shown a clear consensus of opinion about its impact.
The intended ethos of a peer review visit was to be a non-threatening, positive and educational experience. The visits were not found to cause undue anxiety; health professionals were comfortable being reviewed and overall found it a worthwhile exercise. The previsit intradepartmental introspection was found to be valuable and the previsit discussion between the lead consultants from the visited unit and the visiting team was found to be helpful in highlighting any current problems with service provision so that the day could be structured accordingly. The visits themselves were recognized to be of educational value and our free text feedback confirmed that the visits had been a positive learning experience with free exchange of ideas between staff. The visits promoted the role of the multidisciplinary team within rheumatology. Peer review was intended to benefit the visitors as well as the visited unit; visitors consistently agreed that visiting another unit was a constructive experience for them personally independently of the advice they could offer the visited unit and, given the opportunity to then reflect on their own current practice, would be inspired to alter practice within their own unit as a result.
The reports highlighted and addressed shortfalls in services and where necessary helped obtain more staff and secure facilities. This has been successful with the appointment of further consultants but, so far, the recommended increase in staffing in physiotherapy and occupational therapy has not been implemented and this must now be considered a priority. The peer review identified many other recommendations, some of which the visited unit were not previously aware of, and already many of these recommendations have been acted on and change implemented; for example, in appraising and improving the organization and educational activities of a unit or the development of a programme of regular multidisciplinary meetings. The reviews also identified resource issues that were relevant to risk management as well as assessing, and giving recommendations on, each unit's internal audit programme, protocols (for example, for the safe monitoring of disease-modifying drugs) and educational meetings to encourage evidence-based practice.
The evaluation itself involved all members of the multidisciplinary team and there was a good response rate to the questionnaires, particularly from consultants. The methods of analysis used included both objective and subjective assessments. As well as assessing opinion on the process and impact of a peer review visit, the analysis also sought to quantify how much change had been implemented as a result of peer review recommendations. However, there was some intradepartmental disagreement about whether certain recommendations had been acted on already and so this may not be reliable. Whilst all attempts were made to ensure the evaluation was as objective as possible, all people questioned had been involved in the peer review scheme and had an interest in improving rheumatology departments and promoting rheumatology within a Trust. This may therefore be a source of bias. In addition, the chief executives were not questioned to assess how helpful they had found a peer review visit to their Trust or to seek their opinion on why they had been unable to support the implementation of some of the recommendations.
There were only a few practical problems with this peer review scheme: namely, some AHPs not having seen the report or not being involved in implementing recommendations pertinent to their therapy service. This will need to be addressed by individual units in future reviews.
The health professionals in rheumatology involved in peer review support repeating the peer review cycle. The central aim of peer review is to improve quality of care and, although analysis of facilities, staffing levels and working practices are valuable in this regard, further work is required to incorporate measures of quality into future reviews. Nationally accepted standards of care, such as those recently published by the Arthritis and Musculoskeletal Alliance (ARMA), will be valuable in this regard [13, 14], although it must be recognized that collection of evidence regarding these standards poses major challenges to individual units in terms of resources and time.
Although our study did not show a clear consensus of opinion, the majority of health professionals supported peer review as a regional rather than a national process. Both the visited unit and the visitors felt comfortable with peers they knew well and did not think it affected the objectivity of the report. A nationally based peer review scheme such as that operated by the British Thoracic Society may benefit from relative anonymity of the visitors, but has significantly increased time and resource implications. Further consideration could, however, be given to having an external professional member from another region on the reviewing team. Additionally, our study revealed that health professionals would welcome managers and specialist registrars on the reviewing teams, but there was division of opinion on the question of involving a lay person in the visiting team on future visits. Involvement of lay members and patients will, however, be important issues in the future development of the peer review process.
Maximizing the influence of the final report, possibly by targeting both the chief executive of the Trust and local PCT leads with copies of the report, could be considered. Requesting feedback after a set interval on action that was being taken may also be valuable. Access to the report by other parties, including patient and lay groups, will need further discussion. It is important to ensure, prior to setting up a peer review scheme, that there is a mechanism in place to manage any serious evidence of malpractice that may be revealed and to discuss and agree who should have access to the final report.
Since this evaluation, initial discussions at the BSR have taken place comparing regional peer review programmes with the aim of sharing experiences and developing a unified approach to peer review. The assessment proforma is being updated into a standardized format with inclusion of tools to further assess quality of care, which can potentially be used in all regions. The new proforma will also benefit from input from members of the British Health Professionals in Rheumatology (BHPR) to improve the multidisciplinary focus.
In conclusion, the West Midlands rheumatology regional peer review scheme has been a successful, worthwhile initiative with a powerful potential to facilitate clinical governance. We commend this scheme to all other regions and would welcome further discussion at a national level.
|
Acknowledgements |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionAcknowledgementsReferences |
|---|
We would like to thank the other members of the West Midlands Rheumatology Services and Training Committee: M. Allen, S. Bowman, C. Buckley, S. Rigby, K. Grindulis, D. Mulherin, G. Kitas, N. Erb, D. Situnayake, R. Williams. We would like to thank all participating consultants and allied health professionals involved in this peer review scheme from the following units in the West Midlands: City Hospital NHS Trust, Dudley Group of Hospitals NHS Trust, Hereford Hospitals NHS Trust, Mid-Staffordshire General Hospital NHS Trust, Robert Jones and Agnes Hunt Hospital NHS Trust, Sandwell and West Birmingham NHS Trust, University Hospitals Birmingham NHS Trust, University Hospitals of Coventry and Warwickshire NHS Trust, University Hospital of North Staffordshire NHS Trust, Walsall Hospitals NHS Trust, Warwick Hospital NHS Trust, and Worcestershire Royal Hospital NHS Trust.
No conflicts of interest have been declared by the authors.
|
References |
|---|
|
TopAbstractIntroductionMethodsResultsDiscussionAcknowledgementsReferences |
|---|
- . Department of Health. (1997) The new NHS; modern, dependable (Stationary Office, London).
- . Department of Health. (1998) A first class service: quality in the new NHS (Stationary Office, London).
- Grol R. (1994) Quality improvement by peer review in primary care: a practical guide. Quality in Health Care 3:147–52.
[Free Full Text] - O’Brien JM, Price AJ, Wallace IR. (1992) Peer review: collaboration between two regions. Norfolk Norwich Inst Med Educ J 9:41–3.
- Page RL and Harrison BD. (1995) Setting up interdepartmental peer review. The British Thoracic Society's scheme. J R Coll Physicians Lond 29:319–24.[Web of Science][Medline]
- Crean PM, Stokes MA, Williamson C, Hatch DJ. (2003) Quality in paediatric anaesthesia: a pilot study of interdepartmental peer review. Anaesthesia 58:543–8.[CrossRef][Web of Science][Medline]
- Van Weert C. (2000) Developments in professional quality assurance towards quality improvement: some examples of peer review in the Netherlands and the United Kingdom. Int J Qual Health Care 12:239–42.
[Abstract/Free Full Text] - Eaton KA, Fleming WG, Rich JL. (1998) A report of an evaluation of the pilot peer review scheme for general dental practitioners working in the general dental services in England. BDJ 184:178–82.[Medline]
- . British Society for Rheumatology. (1994) BSR Peer Review Scheme Assessment Proforma (British Society for Rheumatology, London).
- . British League Against Rheumatism. (1997) Arthritis: standards of care. Towards meeting people's needs (British League Against Rheumatism, London).
- . British Society for Rheumatology. (1992) Musculoskeletal disorders: providing for the patient's needs. The role of rheumatology departments (British Society for Rheumatology, London).
- . British Society for Rheumatology. (1994) Facilities for rheumatology units. Newsletter No. 3 (Clinical Affairs Committee and British Health Professionals in Rheumatology, London).
- . Arthritis and Musculoskeletal Alliance. (2004) Standards of care (Arthritis and Musculoskeletal Alliance, London).
- Kennedy T, McCabe C, Struthers G, et al. (2005) BSR guidelines on standards of care for persons with rheumatoid arthritis. Rheumatology 44:553–6.
[Free Full Text]
CiteULike 
Connotea 
Del.icio.us What's this?
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||