Rheumatology

This Article Abstract FREE Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Similar articles in ISI Web of Science Similar articles in PubMed Alert me to new issues of the journal Add to My Personal Archive Download to citation manager Search for citing articles in: ISI Web of Science (4) Request Permissions Disclaimer Google Scholar Articles by Taylor, W. Articles by McPherson, K. Search for Related Content PubMed PubMed Citation Articles by Taylor, W. Articles by McPherson, K. Related Collections Diagnostics and Imaging Procedures Rheumatoid Arthritis Social Bookmarking          What's this?

Rheumatology 2001; 40: 757-762
© 2001 British Society for Rheumatology

Original Papers

Evidence for inadequate construct validity of the Disease Repercussions Profile in people with rheumatoid arthritis

W. Taylor, J. Myers, H. McNaughton and K. McPherson

Rehabilitation Teaching and Research Unit, Wellington School of Medicine, University of Otago, PO Box 7343, Wellington, New Zealand

	Abstract

Top Abstract Introduction Patients and methods Results Discussion References

 
Objective. To re-evaluate the construct validity of the Disease Repercussions Profile (DRP), a measure of handicap in arthritis populations.

Methods. We used the multitrait–multimethod approach to determine convergent and discriminant validity in a postal survey of randomly selected patients with rheumatoid arthritis who had attended the Wellington Regional Rheumatology Unit since 1988. Respondents (n=142) completed the following self-report instruments: Disease Repercussions Profile, EuroQol EQ-5D, Health Assessment Questionnaire, London Handicap Scale and WHOQOL-BREV.

Results and conclusions. The pattern of correlation supported the construct validity of the dichotomous response question (DRP domain affected or not) for the activity and relationships domain, provided ambiguous support for the appearance and emotion domain and no support for the social and financial domains. There was no support for construct validity of any of the domains on the importance rating part of the DRP. We suggest that the DRP be interpreted cautiously in aggregated group data.

KEY WORDS: Disease Repercussions Profile, Construct validity, Rheumatoid arthritis, Handicap.

	Introduction

Top Abstract Introduction Patients and methods Results Discussion References

 
The Disease Repercussions Profile (DRP) is explicitly designed to be a measure of the International Classification of Impairment, Disabilities and Handicaps (ICIDH-1) concept of handicap [1]. Handicap is defined by the ICIDH-1 [2] as being a reduction in the ability to fulfil the social role considered normal for the person's age, gender and culture within the defined dimensions of orientation, physical independence, mobility, occupation, social integration and economic self-sufficiency. Handicap has been seen by some authors as the aspect of health outcome most relevant to patients and therefore should be a primary focus of the measurement of outcome, at least in rehabilitation settings [3]. There is little empirical evidence for this notion, attractive though it seems.

The DRP was developed for use as a clinical tool in people with arthritis. It was seen as filling a useful gap in the measurement armamentarium of the rheumatology community, which has for many years focused upon the measurement of impairment and disability [4, 5]. The evidence for construct validity rests upon its demonstrated correlation with other instruments measuring similar constructs. However, the reported pattern of correlation is open to other interpretations. For instance, Carr et al. [1] found that many of the correlation coefficients between the domains of the DRP and impairment indices are higher than that between the DRP domains and domains of the London Handicap Scale (LHS), an instrument which also attempts to obtain a measurement of handicap. This is not consistent with both the DRP and LHS measuring handicap. Furthermore, some of the reported correlations between distinct domains of the DRP are greater than the correlations between similar domains measured by other instruments [6]. This does not support construct validity. There is also little reported data on discriminant validity—the extent to which dissimilar constructs are not correlated.

Construct validity is a necessary psychometric property of any measurement instrument [7]. It is part of what the OMERACT (Outcome Measures in Arthritis Clinical Trials) filter has termed ‘truth’ [8]. That is, the construct validity of a measure reflects the degree to which the instrument actually measures what it purports to measure. Using instruments that lack construct validity is a little like using a set of scales to measure height. Unfortunately, ‘handicap’ is rather more conceptual than length and weight.

The usual method of demonstrating that a health measurement instrument has adequate construct validity is to consider how the construct of interest should relate to other aspects of health or demography, bearing in mind some theoretical or empirical model that predicts the form of these relationships. Often, this is termed ‘convergent validity’ (when instruments that should be measuring the same thing demonstrate high correlation) or ‘discriminant validity’ (when instruments that should be measuring different things demonstrate poor correlation). A more sophisticated but conceptually similar method involves structural equation modelling [9]. In this approach it is assumed that the comparator instruments have adequate construct validity themselves. Because it is difficult to be entirely sure about this, it is necessary to use multiple measures of similar constructs so as to arrive at a more accurate measurement model. For instance, it is necessary to examine the correlation between the dimension of ‘financial security’ between multiple other indicators of economic well-being, such as being in employment and the LHS dimension of economic self-sufficiency.

In the psychological literature, a classic method of determining convergent and discriminant validity uses the multitrait–multimethod matrix [10]. This method is based upon the notion that correlations among different measures of the same trait (or construct) should be greater than correlations among different traits using the same measurement method.

As part of a study exploring the ICIDH-1 model in people with rheumatoid arthritis (RA), we tested the construct validity of the DRP using the multitrait–multimethod matrix approach by concurrent measurement of a number of health-status and quality-of-life instruments in a randomly selected cohort of people with RA.

	Patients and methods

Top Abstract Introduction Patients and methods Results Discussion References

 
The Wellington Regional Rheumatology Unit provides the secondary and tertiary level rheumatology care to a predominantly urban population of about 400 000 people. From a disease register maintained by the Unit since 1988, 324 out of 3000 patients with clinic-diagnosed RA were selected randomly, using computer-generated random numbers, and were contacted by telephone to confirm their location and obtain informed consent. Of these patients, 190 (59%) were posted questionnaire booklets and a postage-paid return envelope. Reasons for non-inclusion were inability to locate the patient (17%), the patient was deceased (16%), and the patient declined or was unable to participate (9%). A follow-up telephone call was made after 3 weeks to remind respondents to return their questionnaires and answer any questions. A second reminder phone call was made after 5–6 weeks. Of the 190 patients from whom verbal consent was obtained, 142 questionnaires were received (75% response rate). Fifteen patients withdrew after having given verbal consent and 33 patients failed to send back completed questionnaires despite reminders.

The following self-administered questionnaires were included in the study booklet: EuroQol EQ-5D, DRP, Health Assessment Questionnaire (HAQ) disability index, LHS, and WHOQOL-BREV. Respondents were able to contact a research associate if they had any problems answering the questions.

The DRP asks respondents to judge whether their arthritis is currently affecting any of the following areas of their life: day-to-day activities, bodily appearance, social activities, financial security, relationships and emotional feelings. If there is judged to be any effect, the respondent is asked to rate the importance of that effect on a Likert horizontal scale numbered 0–10, lower scores indicating less importance.

The EuroQol EQ-5D [11] is a measure of global health status. The first part categorizes respondents into one of 243 health states based on their responses to five questions. The second part asks respondents to gauge their overall health using a vertical visual analogue scale. It can be scored in a number of ways. We chose to use the visual analogue scale rating that reflected the respondents’ own views of their overall health status. It results in a single score between 0 and 100, lower scores indicating poorer health. The EQ-5D can also be scored by reference to normal population valuations of health state [12].

The HAQ disability index [13] is a very commonly used measure of physical disability in rheumatology. It asks respondents to rate the degree of difficulty with activities under eight subheadings (dressing and grooming, rising, eating, walking, hygiene, reach, grip, day-to-day activities) on a four-point scale (none, some, much, can't do), and these ratings are averaged to produce a single disability index between 0 and 3. Higher scores indicate more disability.

The LHS [14] is based upon the ICIDH-1 framework and asks respondents to rate (on a six-point descriptive scale) how their health interferes with each of the domains of mobility, self-care, work and leisure activities, social relationships, economic independence and awareness of one's environment. The profiles of scores range from 1 to 6, lower scores indicating less interference. The score can be weighted and combined into a single index to give an overall measure of handicap. The weights are derived from a general population survey. As we were interested in the patients’ perspective, we retained the raw score as a profile.

The WHOQOL-BREV [15] is a brief version of the WHOQOL-100 [16], an instrument developed by the World Health Organization to measure quality of life across cultures and countries with equivalent versions. Factor analysis showed the WHOQOL-BREV to aggregate into four domains: environment, psychological well-being, social well-being and physical well-being. The scores of each domain are calibrated to lie in the range 0–100, higher scores indicating better quality of life.

As the structure of the DRP is in two distinct parts, we evaluated each part separately. The first part dealt with whether the domain was judged by the respondent to be affected by their arthritis, resulting in a ‘yes’ or ‘no’ response. To determine the construct validity of this, we calculated point-biserial correlation coefficients for each of the domains against each of the other quality of life or health status scores. This pattern of correlations was compared with the within-DRP pattern of point-biserial correlation coefficients.

The second part asked those who judged that an effect was present to rate the importance of that effect. The Spearman correlation coefficient was calculated for the DRP importance ratings on each domain against each of the other instruments, and this pattern of association was compared to the within-DRP pattern of correlation coefficients. The size of the coefficients was depicted graphically in a way similar to a method used by Stockler et al. [17]. The quantitative significance of the correlation coefficients was based on the recommendations by Burnand et al. [18]: r<0.30, insignificant; r=0.30–0.45, moderate; r=0.45–0.60, substantial; r>0.60, high.

We also postulated that demographic features may be associated with handicap scores, although we acknowledge that the model that underlies these predicted relationships is not well established. We predicted that being married may have some association with social and relationship scores, that the number of joint operations may have some association with appearance scores and that being in employment may have some association with the financial security domain. Associations among dichotomous variables were tested with the ² statistic and associations of ordinal with nominal variables were tested by Kruskal–Wallis analysis of variance.

The predicted convergent relationships between domains of the DRP and other measures and demographics are summarized in Table 1. We used an approach similar to that of Campbell and Fisk [10], which required at least that there was a clinically and statistically significant correlation between different methods of measuring similar constructs and that a construct–method combination (e.g. social domain measured with DRP) should correlate better with an independent method of measuring the same construct (e.g. social domain measured by WHOQOL-BREV) than with other constructs measured using the same method (e.g. mobility domain measured by DRP).

View this table: [in this window] [in a new window]   TABLE 1. Predicted convergent relationships between DRP domains and other measures or demographic features

 
We report mainly coefficients that were calculated with a statistical significance of P<0.001. Coefficients with P<0.01 are also reported, but they should be interpreted cautiously given the large number of statistical tests. With a conventional level of significance of P=0.05, Bonferroni correction allows 50 tests to be made with minimal chance of a type I error at P<0.001. All analyses were performed using SPSS version 7 for Windows.

The study was given ethical approval by the Wellington Ethics Committee.

	Results

Top Abstract Introduction Patients and methods Results Discussion References

 
Questionnaires were returned by 142 respondents. The mean (SD) age was 60.7 (14.4) yr, 102 (72%) were female, 128 (90.1%) were of European ethnicity, 88 (62.0%) were married, 29 (20.4%) were working full-time, 21 were working (14.8%) part-time, 58 (40.8%) were retired and 10 (7%) were supported by accident compensation, sickness benefit or unemployment benefit. In general, these subjects had moderately severe disease as judged by a median disease duration of 12.6 yr (interquartile range 6.8, 25.1), use of a median of three (interquartile range 0.25–5.75) disease-modifying anti-rheumatic drugs and a median of 0 (range 0–15) operations for arthritis.

The first analysis looked at the dichotomous response of effect/no effect for each DRP domain. The second analysis looked at the importance rating given to each domain if it was judged to be affected. The numbers of respondents who judged their health to be affected (and therefore rated its importance) were 99 (activities), 33 (appearance), 39 (financial security), 24 (relationships), 65 (social activities) and 44 (emotions). The correlations with other instruments are shown in Table 2 and the correlation matrix between different domains of the DRP is shown in Table 3. A summary of the interpretation of these correlation matrices in relation to the expected pattern of correlation, and the expected relationship between within-DRP correlations and within-similar construct correlations, is shown in Table 4.

View this table: [in this window] [in a new window]   TABLE 2. Correlation matrix for DRP domains with other instrument scores

 

View this table: [in this window] [in a new window]   TABLE 3. Correlation between different domains of the DRP

 

View this table: [in this window] [in a new window]   TABLE 4. Summary of results in terms of the multitrait–multimethod method of assessing construct validity

 
There were no significant differences for any of the DRP domains on the dichotomous response section between married/not married and working/not working, and no significant correlation with number of joint surgeries and the DRP appearance domain. There was no significant association between DRP financial security importance rating and vocational status, but, in comparison, there was an association between the LHS economic independence rating and vocational status (Kruskal–Wallis, P=0.001).

	Discussion

Top Abstract Introduction Patients and methods Results Discussion References

 
These findings provide some support for the construct validity of the dichotomous response question of whether or not health affects each handicap domain for the domains of day-to-day activities and relationships, ambiguous support for the appearance and emotion domains and no support for the social and financial security domains. There is no support for construct validity of the Likert scale importance question for any of the domains.

A major problem in determining the construct validity of a health status instrument is the absence of a gold standard. Other instruments against which the new instrument may be compared may themselves lack construct validity in the absence of a criterion standard. For instance, it may be argued that the LHS may not be actually measuring handicap. In such circumstances, ‘triangulation in the construct space’ is necessary. We attempted to accomplish this by using multiple measures of similar (though not identical) constructs together with some features of demography. For example, we showed that vocational status is associated with the economic domain of the LHS, suggesting that these two indicators measure a similar construct, but neither indicator was strongly associated with the financial security domain of the DRP.

We should also note that the rigorous assessment of construct validity may be lacking in other higher-order measures of health in RA, particularly handicap and quality of life. The shortcomings of the DRP may be common to other instruments in this field, which possibly illustrates the lack of a well-developed and coherent theory of health and quality of life [19, 20].

In the context of the clinical use of this instrument, these shortcomings in construct validity may not be quite so serious as it first appears. This is because the DRP questionnaire also incorporates a free response section in which respondents can write down what they actually mean when they judge that any particular aspect of their life is affected by their health and the importance to which they attach to this effect. For the individual–clinician encounter, interpretation of the questionnaire may therefore be reasonably straightforward. This is one of the principle uses of the questionnaire: as a tool for clinical use [1].

However, the situation is quite different when groups of patients are being evaluated. For instance, the instrument has been used to suggest that patients with osteoarthritis and low back pain have greater levels of handicap than patients with RA [21]. In this context, the inadequate construct validity of the DRP renders aggregated data impossible to interpret. We suggest that caution be exercised when using this instrument unless patient-specific interpretation is available, particularly in evaluating the handicap of groups of patients.

	Acknowledgments

 
We thank Hutt Valley Health for permission to access the database of the Rheumatology Unit. We gratefully acknowledge the Otago Research Fund, whose grant supported this study, and the Arthritis Research Campaign/Arthritis Foundation of NZ, who supported W.T. while the paper was being written (grant T0520).

	Notes

 
Correspondence to: W. J. Taylor, Rehabilitation Teaching and Research Unit, Wellington School of Medicine, University of Otago, PO Box 7343, Wellington, New Zealand.

	References

Top Abstract Introduction Patients and methods Results Discussion References

 

1. Carr AJ. A patient-centred approach to evaluation and treatment in rheumatoid arthritis: the development of a clinical tool to measure patient-perceived handicap. Br J Rheumatol 1996;35:921–32.[Abstract/Free Full Text]
2. World Health Organization. Classification of impairments, disabilities and handicaps. Geneva: World Health Organization, 1980.
3. Whiteneck GG. Measuring what matters: key rehabilitation outcomes. Arch Phys Med Rehabil 1994;75:1073–6.[Web of Science][Medline]
4. Carr AJ, Thompson PW. Towards a measure of patient-perceived handicap in rheumatoid arthritis. Br J Rheumatol 1994;33:378–82.[Abstract/Free Full Text]
5. Bellamy N. Principles of outcome assessment. In: Klippel JH, Dieppe PA, eds. Rheumatology, edn 2. London: Mosby, 1998:3.14.1–3.14.10.
6. Harwood RH, Carr AJ, Thompson PW, Ebrahim S. Handicap in inflammatory arthritis. Br J Rheumatol 1996;35:891–7.[Abstract/Free Full Text]
7. McDowell I, Newell, C. Measuring health. A guide to rating scales and questionnaires, edn 2. New York: Oxford University Press, 1996.
8. Boers M, Brooks P, Strand V, Tugwell P. The OMERACT filter for outcome measures in rheumatology. J Rheumatol 1998;25:198–9.[Web of Science][Medline]
9. Schumaker RE, Lomax RG. A beginner's guide to structural equation modelling. Mahwah, NJ: Lawrence Erlbaum, 1996.
10. Campbell DT, Fiske DW. Convergent and discriminative validation by the multitrait–multimethod matrix. Psychol Bull 1959;56:81–105.[Web of Science][Medline]
11. The EuroQol Group. EuroQol—a new facility for the measurement of health-related quality of life. Health Policy 1990;16:199–208.[Web of Science][Medline]
12. Dolan P, Gudex C, Kind P, Williams A. A social tariff for EuroQol: Results from a UK general population survey. York: University of York, 1995. Discussion Paper 138.
13. Fries JF, Spitz P, Kraines RG, Holman HR. Measurement of patient outcomes in arthritis. Arthritis Rheum 1980;23:137–45.[Web of Science][Medline]
14. Harwood RH, Gompertz P, Ebrahim S. Handicap one year after a stroke: validity of a new scale. J Neurol Neurosurg Psychiatry 1994;57:825–9.[Abstract/Free Full Text]
15. The WHOQOL Group. Development of the World Health Organization WHOQOL–BREF quality of life assessment. Psychol Med 1998;28:551–8.[Web of Science][Medline]
16. The WHOQOL Group. The World Health Organization Quality of Life Assessment (WHOQOL): development and general psychometric properties. Soc Sci Med 1998;46:1569–85.
17. Stockler MR, Osoba D, Corey P, Goodwin PJ, Tannock IF. Convergent discriminative, and predictive validity of the Prostate Cancer Specific Quality of Life Instrument (PROSQOLI) assessment and comparison with analogous scales from the EORTC QLQ-C30 and a trial-specific module. J Clin Epidemiol 1999;52:653–66.[Web of Science][Medline]
18. Burnand B, Kernan WN, Feinstein AR. Indexes and boundaries for ‘quantitative significance’ in statistical decisions. J Clin Epidemiol 1990;43:1273–84.[Web of Science][Medline]
19. Fallowfield L. Quality of quality-of-life data. Lancet 1996;348:421–2.[Web of Science][Medline]
20. Leplege A, Hunt S. The problem of quality of life in medicine. J Am Med Assoc 1997;278:47–50.[Abstract/Free Full Text]
21. Carr AJ. Beyond disability: measuring the social and personal consequences of osteoarthritis. Osteoarthritis Cartilage 1999;7:230–8.[Web of Science][Medline]

Submitted 19 June 2000; Accepted 31 January 2001

CiteULike    Connotea    Del.icio.us    What's this?

This article has been cited by other articles:

				L. Goodacre, J. Smith, D. Meddis, and J. Goodacre Development and validation of a patient-centred Measure of Activity Limitation (MAL) in rheumatoid arthritis Rheumatology, April 1, 2007; 46(4): 703 - 708. [Abstract] [Full Text] [PDF]

				P P Katz, A Morris, and E H Yelin Prevalence and predictors of disability in valued life activities among individuals with rheumatoid arthritis Ann Rheum Dis, June 1, 2006; 65(6): 763 - 769. [Abstract] [Full Text] [PDF]

				L Sharpe, F Denton, and L Schrieber Validity of the Disease Repercussion Profile in patients with systemic lupus erythematosus Lupus, June 1, 2004; 13(6): 428 - 435. [Abstract] [PDF]

				L. Sharpe, T. Sensky, W. J. Taylor, and K. McPherson Validity of the Disease Repercussion Profile Rheumatology, August 1, 2002; 41(8): 945 - 946. [Full Text] [PDF]

This Article Abstract FREE Full Text (PDF) Alert me when this article is cited Alert me if a correction is posted Services Email this article to a friend Similar articles in this journal Similar articles in ISI Web of Science Similar articles in PubMed Alert me to new issues of the journal Add to My Personal Archive Download to citation manager Search for citing articles in: ISI Web of Science (4) Request Permissions Disclaimer Google Scholar Articles by Taylor, W. Articles by McPherson, K. Search for Related Content PubMed PubMed Citation Articles by Taylor, W. Articles by McPherson, K. Related Collections Diagnostics and Imaging Procedures Rheumatoid Arthritis Social Bookmarking          What's this?

Online ISSN 1462-0332 - Print ISSN 1462-0324

Copyright © 2009 British Society for Rheumatology

Oxford Journals Oxford University Press

• Site Map
• Privacy Policy
• Frequently Asked Questions

Other Oxford University Press sites: Oxford University Press Oxford Journals China Oxford Journals Japan American National Biography Booksellers' Information Service Children's Fiction and Poetry Children's Reference Corporate & Special Sales Dictionaries Dictionary of National Biography Digital Reference English Language Teaching Higher Education Textbooks Humanities International Education Unit Law Medicine Music Online Products Oxford English Dictionary Reference Rights and Permissions Science School Books Social Sciences Very Short Introductions World's Classics