Skip Navigation


Rheumatology Advance Access originally published online on March 5, 2008
Rheumatology 2008 47(4):514-518; doi:10.1093/rheumatology/ken004
This Article
Right arrow Abstract Freely available
Right arrow FREE Full Text (PDF) Freely available
Right arrow All Versions of this Article:
47/4/514    most recent
ken004v1
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrowRequest Permissions
Right arrow Disclaimer
Google Scholar
Right arrow Articles by Taylor, W. J.
Right arrow Articles by Sew Hoy, M.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Taylor, W. J.
Right arrow Articles by Sew Hoy, M.
Related Collections
Right arrow Rheumatoid Arthritis
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?

© The Author 2008. Published by Oxford University Press on behalf of the British Society for Rheumatology. All rights reserved. For Permissions, please email: journals.permissions@oxfordjournals.org

Disease Activity Score 28-ESR bears a similar relationship to treatment decisions across different rheumatologists, but misclassification is too frequent to replace physician judgement

W. J. Taylor1,2, A. A. Harrison1,2, J. Highton3,4, P. Chapman5, L. Stamp5,6, J. Dockerty4, F. McQueen7, P. B. B. Jones7,8, D. Ching9, D. Porter10, C. Rajapakse2, S. R. Rudge2, G. Taylor11, S. Kumar12, T. Macedo12 and M. Sew Hoy13

1University of Otago, Wellington, 2Wellington Regional Rheumatology Unit, Hutt Valley District Health Board, Lower Hutt, 3University of Otago, 4Otago District Health Board, Dunedin, 5Canterbury District Health Board, 6University of Otago, Christchurch, 7University of Auckland, Auckland, 8QE Health, Rotorua, 9South Canterbury District Health Board, Timaru, 10Nelson-Malborough District Health Board, Nelson, 11Wanganui District Health Board, Wanganui, 12Counties-Manukau District Health Board, Auckland and 13193 Memorial Avenue, Christchurch, New Zealand.

Correspondence to: W. J. Taylor, Department of Medicine, University of Otago Wellington, PO Box 7343, Wellington 6242, New Zealand. E-mail: will.taylor{at}otago.ac.nz


   Abstract
Top
 Abstract
Introduction
 Methods
 Results
 Discussion
 Acknowledgements
 References
 
Objectives. To determine whether physician factors are associated with disease activity status in RA, independently of 28-joint disease activity score (DAS28)-ESR and to re-evaluate DAS28-ESR misclassification rates for identifying active disease in usual practice.

Methods. A prospective observational study of outpatients with RA seen by 17 rheumatologists across New Zealand. Active disease was defined by an increase in therapy together with a reason of ‘active disease’; very low disease activity was defined by a decrease in therapy together with a reason of ‘patient well’. The independent physician effect was assessed using logistic regression. Sensitivity and specificity of current DAS28-ESR thresholds were calculated.

Results. In 511 patients, 178 had active disease, 220 had low disease activity, 37 had very low disease activity and 76 had uncertain disease activity status. There was no independent effect of physician upon active disease status (P = 0.16) with DAS28-ESR [(OR) 3.7] explaining around 50% of the variability in active disease status. There was a trend towards an independent effect of physician upon very low disease activity status (P = 0.06) and greater variability in the distribution of DAS28-ESR for patients in very low disease activity. DAS28-ESR thresholds showed a significant risk of misclassification for active disease.

Conclusions. DAS28-ESR discriminates satisfactorily between groups of patients with active and non-active disease, with no evidence of additional physician-specific factors to explain disease activity status. However, DAS28-ESR is not as good for discriminating remission from non-remission status. There are appreciable probabilities of misclassification error, which make DAS28-ESR inappropriate as a sole guide for treatment decisions.

KEY WORDS: Rheumatoid arthritis, Disease activity score, Remission criteria, Reliability, Validity, Health status, Treatment decisions, Psychometrics, Physician-factor, Methodology


   Introduction
Top
 Abstract
 Introduction
Methods
 Results
 Discussion
 Acknowledgements
 References
 
The disease activity score (DAS) is a useful measure of inflammatory disease activity in RA and clearly distinguishes between placebo and effective treatment in clinical trials. It has also been used to identify patients whose therapy should be changed because of unacceptably high levels of DAS, leading to the promotion of clinical practice that is strongly guided by DAS (or other composite score) monitoring [1–3]. Such an approach, in combination with monthly rheumatology consultation, has led to excellent outcomes in a randomized controlled trial [4]. This concept has also helped national rheumatology societies to define entry criteria for new biologic agents in terms of DAS. For example, a DAS28 of at least 5.1 is required for biologic therapy and achievement of ≤3.2 is required to continue therapy, according to the British Society for Rheumatology [5].

The DAS was developed from the mathematical association of key disease indicators with a ‘gold standard’ of disease activity being treatment decisions by the patients’ rheumatologist. Discriminant function analysis was used to calculate the appropriate weighting of disease indicators in order to most accurately discriminate patients into high or low disease activity states. Subsequently, the distribution of DAS scores were used to define two thresholds: a lower threshold that defined low disease activity as the 75th percentile of scores amongst those with low disease activity (defined by rheumatologist treatment decision) and a higher threshold that defined high disease activity as the 25th percentile of scores amongst those with high disease activity [6]. Thus, three categories of disease activity are formed: low, moderate and high. DAS28-ESR, one of the most popular forms of the DAS, uses the following disease indicators: tender joint count (28 joints), swollen joint count (28 joints), ESR and patient-reported general health status on a 100 mm visual analogue scale (VAS) [7].

Despite its success in clinical trials, some issues remain. First, it is unclear whether variability in physician treatment behaviour could affect the concordance of DAS with the presumptive ‘gold standard’ of active disease, which is intensification of therapy. It is possible that a different relationship exists between the disease indicators of the DAS formulation and treatment decisions across different physicians. This would affect the appropriate thresholds of DAS when considering disease activity states. Second, the accuracy of the DAS thresholds for use in the individual clinical encounter needs to be better defined [8]. Significant variation between DAS or other composite scores and clinically determined judgements might mean that their use for clinical or regulatory purposes is not soundly based [9]. We therefore set out to investigate the extent of mismatch between physician-based treatment decisions and DAS-driven judgements. This study of unselected patients with RA seen in general rheumatology clinics by New Zealand rheumatologists, participating in the New Zealand Rheumatology Research Network, aimed to address this issue. Clinical encounters in this context were not informed by DAS measurement and treatment decisions were made on the basis of usual physician judgement.


   Methods
Top
 Abstract
 Introduction
 Methods
Results
 Discussion
 Acknowledgements
 References
 
Rheumatologists who were part of the New Zealand Rheumatology Research Network participated in this observational study. As part of a usual clinical encounter, a brief case record form was completed by the doctor and patient, which included the following data: tender (TJC) and swollen (SJC) 28-joint counts, acute-phase marker (ESR or CRP according to local practice), patient general health on a 100 mm VAS, modified HAQ Disability Index (mHAQ) and details of treatment changes.

The treatment decisions were classified as:

  1. No change: no changes made.
  2. Treatment increased: DMARD start, DMARD dose increase, change to a different DMARD, intra-articular/intra-muscular/intravenous glucocorticoid injection (IA, IM, IV), prednisone start or prednisone dose increase.
  3. Treatment decreased: DMARD dose decrease, DMARD stop, prednisone dose decrease or prednisone stop.
  4. Ambiguous decision: More than one treatment change in different directions.

Biologic treatment was not easily available in New Zealand at the time this study was conducted and no patients in this study were being treated with biologic agents.

The reason for treatment change was also recorded: patient well, active disease, drug side-effect or patient preference. ‘Active disease’ was thus defined as ‘treatment increased’ and ‘active disease’. ‘Non-active disease’ was defined as no change/treatment decreased and ‘patient well’.

To explore the concept of ‘remission’ or very low disease activity, disease activity status was defined in a separate analysis as ‘active disease’ (same as above), ‘low disease activity’ (no change in treatment and ‘patient well’) and ‘very low disease activity’ (treatment decreased and ‘patient well’). The distribution of DAS28-ESR scores by physician and by these groups, defined by treatment decision was plotted with reference to the proposed DAS28-ESR thresholds of 2.6 (remission), 3.2 (moderate disease activity) and 5.1 (high disease activity).

DAS28 was calculated from the clinical records submitted and did not form part of the clinical consultation. DAS28 scores were not available to the treating clinician. ESR was measured in 431 people and CRP was measured in 67 people. DAS28 can be calculated using either acute-phase marker [10] and the scores correlate highly, but are not interchangeable. A conversion formula to convert DAS28-CRP into DAS28-ESR was applied [11]. If TJC, SJC, patient global VAS or acute-phase marker was missing, a DAS28-ESR could not be calculated.

The ability of DAS28-ESR to discriminate between active disease and non-active disease was assessed using the area under the curve (AUC) of a receiver operating characteristic (ROC) curve that plots true positive rate (sensitivity) against false positive rate (1 – specificity) for each value of DAS28-ESR in distinguishing active disease from non-active disease. An AUC of 1.00 indicates that the measure perfectly discriminates between the two disease states. In addition, the threshold of DAS28-ESR that best distinguished active disease from non-active disease in this sample was calculated at the maximum value of Youden's index, which is (sensitivity + specificity)/2.

A similar analysis was performed to determine the ability of DAS28-ESR to discriminate between remission (very low disease activity) and non-remission groups.

Since DAS28-ESR is validated against a standard of disease activity that depends upon physician treatment behaviour, it is possible that different levels of DAS correspond to the same level of disease activity, depending on the physician. The consistency of the relationship was tested using logistic regression analysis, modelling the disease activity state (active/non-active) by physician, DAS28-ESR and the interaction term. Statistical significance of the physician variable or the interaction term would indicate that some physicians judge disease activity differently for patients with the same DAS28-ESR level compared with other physicians. SPSS 15.0.0 was used for all analyses.

The study protocol was approved by the New Zealand Health and Disability Multicentre Ethics Committee.


   Results
Top
 Abstract
 Introduction
 Methods
 Results
Discussion
 Acknowledgements
 References
 
Seventeen rheumatologists contributed data on 511 patients (range 16–50 patients each) with RA. The mean (S.D.) age of the patients was 59 (13); 73% were female; 45% had RA for >10 yrs and 31% for < 5 yrs; 75% had radiographic erosions; 86% were RF-positive; the mean (S.D.) mHAQ was 0.54 (0.53).

Non-active disease status was assigned to 257 patients (220 with no change in treatment and 37 with a decrease in treatment, who also had a treatment decision reason of ‘patient well’). Active disease status was assigned to 178 patients (increase in treatment for those who also had a treatment decision reason of ‘active disease’). The remaining 76 patients had unclear disease activity status. The reasons for this are shown in Table 1.


View this table:
[in this window]
[in a new window]

  		TABLE 1. Patients with unclear disease activity status

 
DAS28-ESR scores were available from 500 patients with a mean (S.D.) of 3.99 (1.53) and ranged from 0.08 to 7.85. For 249 patients with non-active disease and DAS28-ESR available, the mean (S.D.) score was 3.12 (1.13). For 178 patients with active disease and available DAS28-ESR, the mean (S.D.) score was 4.99 (1.34). For 73 patients whose disease activity status was unclear and available DAS28-ESR, the mean (S.D.) score was 4.50 (1.47).

The AUC was calculated from the DAS28-ESR scores of the 249 patients with non-active disease and the 178 patients with active disease, and was 0.86 (95%CI 0.82, 0.89). The maximum Youden's index was 0.794, which occurred at a DAS28-ESR value of 4.1 (sensitivity 76.8%, specificity 81.9%). The sensitivity and specificity for this threshold are compared with previously proposed thresholds in Table 2.


View this table:
[in this window]
[in a new window]

  		TABLE 2. Performance of different DAS28-ESR thresholds for determining active disease

 
Using the same approach as van Gestel et al. [6] to determine DAS thresholds, in this cohort of patients, the 75th percentile for patients with non-active disease was 3.9 and the 25th percentile for patients with active disease was 4.1.

A logistic regression model of disease activity status (active/non-active) was first assessed without the interaction term, using the variables of physician and DAS28-ESR. This model was statistically significant (model chi-square = 211, P < 0.001, R2 = 0.53) and showed that the physician variable was not significant (overall P = 0.16). This means that there was no physician effect on disease activity status, after adjusting for DAS28-ESR. Since the main effect of the physician variable was non-significant, we did not go on to test a model with the interaction term. DAS28-ESR was significantly associated with disease activity status with OR 3.75 (95% CI 2.89, 4.85).

The distribution of DAS28-ESR scores by physician and treatment defined disease activity status as shown in Fig. 1. This shows much more variability in how DAS28-ESR scores corresponded to the very low disease activity group amongst different physicians and that there was only modest concordance with the DAS28-ESR definition of remission (<2.6) and disease activity defined by a decrease in treatment. Figure 2 shows the overall distribution of DAS28-ESR scores, divided into the treatment-defined disease activity groups. It is readily apparent that there is substantial overlap in the distribution of scores across the treatment-defined disease activity groups.


Figure 1
View larger version (30K):
[in this window]
[in a new window]
[Download PowerPoint slide]
  		FIG. 1. Box-plot distribution of DAS28-ESR scores by individual physician- and treatment-defined disease activity state (very low disease activity—treatment decreased; low disease activity—treatment unchanged; active disease—treatment increased). The horizontal lines show currently used DAS28-ESR thresholds.

 

Figure 2
View larger version (35K):
[in this window]
[in a new window]
[Download PowerPoint slide]
  		FIG. 2. Overall histogram distribution of DAS28-ESR by treatment-defined disease activity state. The vertical lines show currently used DAS28-ESR thresholds.

 
Furthermore, a logistic regression analysis that modelled very low disease activity (by treatment decision) using DAS28-ESR and physician as explanatory variables, showed a trend towards statistical significance of an independent physician effect (overall P = 0.06), suggesting that physician-determined factors other than DAS28-ESR could be important in determining very low disease activity.

An ROC analysis of DAS28-ESR dividing the sample into ‘remission’ and ‘non-remission’ (defined by treatment decrease) found an AUC of 0.80 (95% CI 0.74, 0.86). The maximal value of Youden's index was 0.73, which occurred at a DAS28-ESR value of 2.66 (sensitivity 79%, specificity 68%).


   Discussion
Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
Acknowledgements
 References
 
This study of patients who were observed in ordinary clinical practice shows that physicians are similar in how their treatment decisions relate to DAS28-ESR scores. The study also confirms that DAS28-ESR discriminates between patients with active and non-active disease. These findings add weight to the argument that the score may be helpful in clinical practice and not just in research settings.

However, the thresholds for determining active disease identified in this patient group are different to thresholds previously identified for ‘high disease activity’ and ‘low disease activity’ (3.2 and 5.1). These two activity thresholds divide patients into three groups: high disease activity, low disease activity and a middle group (moderate disease activity). In this data set, 41% of the patients would be classified in the middle group and 43% of those patients were considered as having active disease that required an increase in treatment. The percentage of RA patients classified as moderate disease activity was remarkably similar to that seen in a French cohort study [12]. This group of patients therefore forms an important part of the clinical workload, but it is not clear how DAS28-ESR scores classified under the original thresholds would assist in the treatment decision-making. In contrast, the thresholds determined in the present study identified a much smaller proportion of patients with ‘moderate disease activity’ and tended to classify more patients with high disease activity than the currently used thresholds.

Also of interest is that the optimal threshold of DAS28-ESR (4.1) observed in this study is very similar to the threshold of 4.0 observed to distinguish between those who achieved and those who did not achieve an ACR-20 response in a study of patients treated with infliximab [13]. This convergence tends to support the use of this threshold for making treatment decisions.

However, even using a threshold of 4.1, the accuracy of the DAS28-ESR leads to significant number of patients being misclassified. Given the probability of physician-defined active disease observed in this study (178/435 = 0.409), the number of patients misclassified as having active disease when they did not (using the DAS28-ESR threshold of 4.1) is 47 and the number of patients misclassified as having non-active disease when they did is 41. The use of different thresholds and in contexts in which the a priori likelihood of high disease activity is different will lead to different misclassification rates. For example, it might be appropriate to use higher DAS28-ESR scores to determine eligibility for biologic therapy; nonetheless, it should be recognized that this may come at a cost of misclassifying even more patients with active disease.

At the other end of the disease activity spectrum, remission as defined by DAS28-ESR as ≤2.6, was observed in 107 patients in this cohort. Of these, 9% still had active disease as judged by the treating rheumatologist (Table 3). This finding tends to support the view that the 28-joint count of the DAS28 can miss some people with active disease in the feet or ankles [14]. Furthermore, there was substantial overlap in the distribution of DAS28-ESR scores for groups defined by decreased treatment compared with treatment unchanged, even though the best cut-off from this sample was similar to the previously identified value of 2.6. This implies that DAS28-ESR does not discriminate well at the lower end of the disease activity continuum. Since it is likely that some patients in whom treatment was unchanged will also be in remission, the ‘treatment decreased’ group will plausibly identify patients in remission with poor sensitivity but greater specificity. The fact that many such patients had DAS28-ESR scores above 2.6, supports the concept that DAS28-ESR does not perform so well at the lower end of the spectrum. It is of interest that a study that compared DAS28-ESR with DAS28-CRP found larger fluctuations in the relationship between the two scores at lower end of the scale suggesting that the DAS28 system itself may have some inherent uncertainty for lower values of DAS28 [11]. One other possible explanation for this could be the biological behaviour of ESR, with a floor effect for ESR caused by hypergammaglobulinaemia.


View this table:
[in this window]
[in a new window]

  		TABLE 3. Classification of patients by previously proposed DAS28 thresholds according to physician judgement of disease activity

 
We conclude that the DAS28-ESR scores relate to similar concepts of disease activity across different rheumatologists. It is a useful measure to discriminate between active and non-active disease at a group level, which makes it very suitable for clinical trials or observational cohort studies, but misclassification error makes it less suitable as a guide to individual patient care. Further work to refine remission criteria using DAS28 is necessary.

Formula


   Acknowledgements
Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 Acknowledgements
References
 
Funding: This study was funded by the Wellington Regional Rheumatology Trust, Queen Elizabeth Hospital Community Trust and the University of Otago.

Disclosure statement: The authors have declared no conflicts of interest.


   References
Top
 Abstract
 Introduction
 Methods
 Results
 Discussion
 Acknowledgements
 References
 

  1. Fransen J, Moens HB, Speyer I, van Riel PLCM. Effectiveness of systematic monitoring of rheumatoid arthritis disease activity in daily practice: a multicentre, cluster randomised controlled trial. Ann Rheum Dis (2005) 64:1294–8.[Abstract/Free Full Text]
  2. Creemers MC, Fransen J, de Groot-Vos EH, Welsing P, Kievit W, van Riel PL. Implementation of the DAS28 as a routine assessment in daily clinical practice. Ann Rheum Dis (2005) 64(Suppl. 3):603.
  3. Smolen JS, Aletaha D. Target remission. The Rheumatologist (2007) 1:14–16.
  4. Grigor C, Capell H, Stirling A, et al. Effect of a treatment strategy of tight control for rheumatoid arthritis (the TICORA study): a single-blind randomised controlled trial. Lancet (2004) 364:263–9.[CrossRef][Web of Science][Medline]
  5. Ledingham J, Deighton C, on behalf of the British Society for Rheumatology Standards GaAWG. Update on the British Society for Rheumatology guidelines for prescribing TNF{alpha} blockers in adults with rheumatoid arthritis (update of previous guidelines of April 2001). Rheumatology (2005) 44:157–63.[Free Full Text]
  6. van Gestel AM, Prevoo ML, van't Hof MA, van Rijswijk MH, van de Putte LB, van Riel PL. Development and validation of the European league against rheumatism response criteria for rheumatoid arthritis. Comparison with the preliminary American College of Rheumatology and the World Health Organization/International League Against Rheumatism Criteria. Arthritis Rheum (1996) 39:34–40.[Medline]
  7. Prevoo ML, van't Hof MA, Kuper HH, van Leeuwen MA, van de Putte LB, van Riel PL. Modified disease activity scores that include twenty-eight-joint counts. Development and validation in a prospective longitudinal study of patients with rheumatoid arthritis. Arthritis Rheum (1995) 38:44–8.[Web of Science][Medline]
  8. Gardiner PV, Bell AL, Taggart AJ, et al. A potential pitfall in the use of the Disease Activity Score (DAS28) as the main response criterion in treatment guidelines for patients with rheumatoid arthritis. Ann Rheum Dis (2005) 64:506–7.[Free Full Text]
  9. Wolfe F, Michaud K, Pincus T, Furst D, Keystone E. The disease activity score is not suitable as the sole criterion for initiation and evaluation of anti-tumor necrosis factor therapy in the clinic - discordance between assessment measures and limitations in questionnaire use for regulatory purposes. Arthritis Rheum (2005) 52:3873–9.[CrossRef][Web of Science][Medline]
  10. Fransen J, Welsing PMJ, De Keijzer RMH, Van Riel PLCM. Development and validation of the DAS28 using CRP. Ann Rheum Dis (2003) 62(Suppl. 1):10.[Abstract/Free Full Text]
  11. Inoue E, Yamanaka H, Hara M, Tomatsu T, Kamatani N. Comparison of Disease Activity Score (DAS) 28-erythrocyte sedimentation rate and DAS28-C-reactive protein threshold values. Ann Rheum Dis (2007) 66:407–9.[Abstract/Free Full Text]
  12. Saraux A, Devauchelle, Pensec V, Engerran L, Flipo RM. Most rheumatologists are conservative in active rheumatoid arthritis despite methotrexate therapy: results of the PRISME survey. J Rheumatol (2006) 33:1258–65.[Web of Science][Medline]
  13. Vander Cruyssen B, Van Looy S, Wyns B, et al. DAS28 best reflects the physician's clinical judgment of response to infliximab therapy in rheumatoid arthritis patients: validation of the DAS28 score in patients under infliximab treatment. Arthritis Res Ther (2005) 7:R1063–71.[CrossRef][Web of Science][Medline]
  14. Landewe R, van der Heijde D, van der Linden S, Boers M. Twenty-eight-joint counts invalidate the DAS28 remission definition owing to the omission of the lower extremity joints: a comparison with the original DAS remission. Ann Rheum Dis (2006) 65:637–41.[Abstract/Free Full Text]
Submitted 31 October 2007; revised version accepted 3 January 2008.
Add to CiteULike CiteULike   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us    What's this?



This Article
Right arrow Abstract Freely available
Right arrow FREE Full Text (PDF) Freely available
Right arrow All Versions of this Article:
47/4/514    most recent
ken004v1
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrowRequest Permissions
Right arrow Disclaimer
Google Scholar
Right arrow Articles by Taylor, W. J.
Right arrow Articles by Sew Hoy, M.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by Taylor, W. J.
Right arrow Articles by Sew Hoy, M.
Related Collections
Right arrow Rheumatoid Arthritis
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?