| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Rheumatology 2001; 40: 914-919
© 2001 British Society for Rheumatology
Paediatric Rheumatology |
Discordance between proxy-reported and observed assessment of functional ability of children with juvenile idiopathic arthritis
Paediatric Rheumatology/Series Editor: P. Woo
Dipartimento di Scienze Pediatriche dell'Università
1 Servizio di Epidemiologia Clinica e Biometria and
2 Laboratorio di Informatica Medica, IRCCS Policlinico S. Matteo, Pavia, Italy
Abstract
Objective. To determine the level of agreement between parents and clinicians in rating dysfunction in children with juvenile idiopathic arthritis (JIA).
Methods. A parent of each patient completed the Italian version of the Childhood Health Assessment Questionnaire (CHAQ). Subsequently, an examiner assessed, in a specially equipped room, the child's performance of tasks as described by the CHAQ. Demographic and clinical variables were recorded for all patients.
Results. Seventy consecutive JIA patients and their parents were included. The mean proxy-reported and observed CHAQ score was 0.64±0.53 and 0.47±0.62 respectively, the difference ranging from -1.75 to 1.5. There were 30 cases (43%) of agreement (difference 
0.25 CHAQ units) between the parent's and clinician's ratings, whereas in 40 cases (57%) there was discordance (difference >0.25 CHAQ units). In 30 cases the parent rated the child's functional ability as worse than that observed by the clinician (i.e. the parent underestimated the child's function), whereas in 10 cases the parent rated the child's functional ability as better than that observed by the clinician (i.e. overestimated the child's function). Multivariate regression analysis showed that children's functional ability was overestimated by parents with increasing erythrocyte sedimentation rate and global articular severity score and underestimated with increasing level of pain. Among the functional areas of the CHAQ, the level of agreement was poorest in the areas of eating and hygiene and was best for activities.
Conclusions. Discordance between proxy-reported and observed functional ability was frequent in our patients with JIA. The children's functional ability was overestimated by parents as the severity of arthritis increased and underestimated as the level of pain increased.
KEY WORDS: Parent–physician agreement, Functional ability, Juvenile idiopathic arthritis.
Functional ability is regarded as an important outcome measure in the care of children with juvenile idiopathic arthritis (JIA). Assessment of functioning is helpful in obtaining a ‘point in time’ estimate of the patient's status, in monitoring the progression of disease and the effectiveness of therapeutic interventions, in identifying the specific needs and concerns of patients and families, and in attempting to predict the outcome and prognosis early in the course of the disease [1]. Furthermore, the measurement of functional ability has been included among the core set of measures of the preliminary definition of improvement for juvenile arthritis [2]. In recent years, numerous comprehensive indices of health status have been designed specifically for use in children with JIA [3]. These indices are generally based on questionnaires aimed at evaluating the child's ability in performing daily activities. Although the level of function can be self-reported by the child (generally when aged 
9 yr), these questionnaires are usually completed by the parents (or guardians), most often by the mother. Indeed, good agreement between the parent's and the child's assessment of physical function has been observed [4, 5]. The reliability of these questionnaires is based on the assumption that the perceived level is in accordance with the objective level of functioning. However, studies in adult patients with rheumatoid arthritis have shown that patients and clinicians may disagree in the assessment of the patient's functioning [6, 7]. To our knowledge, no study has compared reported with observed functional ability in children with JIA.
The aim of the present study was to investigate whether there is a discordance between proxy-reported and observed functional ability in children with JIA, as measured by the Childhood Health Assessment Questionnaire (CHAQ). Furthermore, the contribution of demographic, clinical and psychosocial factors to the possible overestimation or underestimation of functional ability was examined.
Patients and methods
Patient selection
All consecutive patients aged less than 18 yr meeting the criteria for the diagnosis of JIA developed by the Classification Taskforce of the Paediatric Standing Committee of the International League of Associations for Rheumatology (Durban, 1997) [8] who were seen at the Department of Paediatrics of the University of Pavia between March 1998 and March 1999 were enrolled in the study. To be included, patients had to have active arthritis (defined as swelling or, if no swelling was present, limitation of movement with either pain upon movement or tenderness) in at least one joint or, if no active arthritis was present, a limited range of motion of at least 10% of the full movement in at least one joint. Patients were excluded if they had musculoskeletal abnormalities (other than JIA) or other severe diseases that affected functional health status. Informed consent was obtained by a parent of all patients included in the study.
Clinical assessment
The following information was recorded for each patient: sex, age, disease duration, JIA onset subtype, articular course, disease-modifying anti-rheumatic drug (DMARD) treatment in the previous 3 months, and ongoing steroid treatment. The physician's and parent's/ patient's global assessments of the disease status and the parent's/patient's assessment of pain were measured on an anchored horizontal 15-cm visual analogue scale, as reported previously [9]. The number of active joints and the global articular severity score were assessed in a total of 64 joints (the joints that are included in the normal clinical evaluation) and calculated as reported previously [9]. A radiographic joint lesion was defined as the presence of joint space narrowing or bone erosion in at least one joint. The erythrocyte sedimentation rate (ESR) was measured by the Westergren method and C-reactive protein (CRP) concentration by nephelometry.
Assessment of functional ability
A parent of each patient (generally the mother) was asked to complete the Italian version of the CHAQ [10]. The CHAQ was also administered to many patients (aged 9 yr or more), who filled in the questionnaire separately from their parents. The CHAQ measures the child's ability in performing functions included in eight areas (Dressing and Grooming, Arising, Eating, Walking, Hygiene, Reach, Grip, and Activities). Two to five items are evaluated in each area, giving a total of 30 items. Respondents are directed to note only those difficulties caused by arthritis. Each question is scored from 0 to 3 (0=no difficulty, 1=some difficulty, 2=much difficulty, 3=unable to perform the task). The question with the highest score determines the score for that functional area. If aids or devices are used or help is needed to complete tasks in a certain area, a minimum score of 2 is recorded for the corresponding functional area. The scores for each of the eight functional areas are averaged to calculate the CHAQ disability index (DI), which ranges from 0 (best) to 3 (worst).
After the CHAQ had been completed, the parent was asked to provide information on his/her education level (scored as follows: primary school=1; secondary school=2; high school=3; degree=4) and to complete the Italian translation of Goldberg's General Health Questionnaire (GHQ) [11]. The GHQ was developed as a screening instrument for mental disorder in the general population, but has been used widely as a measure of psychological distress. The version we used is composed of 30 items, each including four possible responses. Responses are scored 1–4, resulting in a total score ranging from 30 (best) to 120 (worst).
Prior to the objective assessments of functional ability, two paediatric rheumatologists (AR or SV) were asked to describe in detail their opinion about the meaning of ‘with some difficulty’ and ‘with much difficulty’ for the items of the CHAQ. After discussion, consensus was reached about a standardized method of determining functional ability and a scoring system for each item included in the CHAQ. All items were selected except ‘Run errands and shop’ and ‘Do household chores’, because these activities were not considered reproducible in an indoor setting. Agreement between clinicians was reached easily through the simulation of all selected functions and relative degrees of limitation by using the same instrument or tools as those employed by patients to perform tasks. The results of the consensus meeting for each item were written down in a special form and used in the observer assessment of functional ability. The measurements were made in a specially equipped room. Each patient was asked to perform all the selected activities of the CHAQ and his or her ability in performing tasks was judged by the same observer (AR or SV). The same items and utensils were used by all patients. The examiner and the parent or patient were not allowed to discuss the questionnaire. The objective measurement of functional ability and the clinical assessment were done by different rheumatologists, who were unaware of their respective findings and of the CHAQ responses given by the parent/patient. All the assessments were done in the morning between 9:00 and 12:00 h.
Statistics
The level of agreement between parents' and clinicians' assessments of functional ability was represented graphically according to the method proposed by Bland and Altman [12], by plotting the difference between the proxy-reported and observed CHAQ DI against the mean of the two assessments. The 
2 test was used to compare the frequencies in each group; comparison of quantitative variables was made with the Mann–Whitney U-test or the Kruskal–Wallis test; the correlation between variables was investigated by the Spearman rank order correlation procedure. All tests were two-tailed. Discordance between proxy-reported and observed assessments of functional ability was defined as a difference of more than 0.25 CHAQ units. The relationship of clinical variables with the amount of discordance between proxy-reported and observed CHAQ DI was analysed with a multivariate regression model. The level of agreement between the parent's and clinician's ratings in each of the eight functional areas of the CHAQ was estimated with the weighted kappa coefficient [13, 14] and was defined, according to the criteria of Landis and Koch [15], according to the following values of kappa: 0.20, poor; 0.21–0.40, fair; 0.41–0.60, moderate; 0.61–0.80, good; 0.81–1.00, almost perfect.
Results
Seventy consecutive patients with JIA, whose demographic and clinical data are reported in Table 1
, and their parents participated in this study. The mean proxy-reported CHAQ DI was 0.64±0.53 and the mean observed CHAQ DI was 0.47±0.62. The difference between proxy-reported and observed CHAQ DI ranged from -1.75 to 1.5 (mean 0.17±0.58). There were 30 cases of agreement (difference 0.25 CHAQ units) between the parent's and clinician's ratings, representing 43% of the total cases, whereas in 40 cases (57%) there was discordance (difference >0.25 CHAQ units) between the two assessments. In 30 cases (43%), the parent rated the child's functional ability as more impaired than that observed by the clinician (i.e. the parent underestimated the child's functional ability). In 10 cases (14%), the parent rated the child's functional ability as less impaired than that observed by the clinician (i.e. the parent overestimated the child's functional ability) (Fig. 1). The CHAQ DI of the 28 children who self-reported their functional ability was strongly correlated with that of their parents (r=0.82; P<0.0001).
|
|
The comparison of the clinical variables between groups of JIA patients differing with respect to agreement or disagreement (underestimation vs overestimation) is shown in Table 2
. The score of the parent's assessment of pain was significantly lower in the group of patients for whom there was agreement between the proxy-reported and the observed level of function (P=0.024), whereas the use of steroids was significantly more common in patients whose functional ability was overestimated by the parents (P=0.0004) and the presence of radiographic joint lesions was significantly more frequent in patients whose functional ability was underestimated by the parents (P=0.002). Moreover, although the differences did not reach significance, patients whose functional ability was overestimated by parents were characterized by a longer disease duration, more severe joint involvement and a higher ESR and CRP. Patients for whom there was agreement had less disability than those with disagreement because the CHAQ was rated as 0 (e.g. normal functional ability) in 50 and 47% of them by the parents and clinicians respectively, compared with 0 and 25% respectively in patients with disagreement. Moreover, these patients had a higher frequency of X-rays without signs of joint damage than those for whom there was disagreement. The amount and direction of discordance were not influenced by the parents' level of psychological distress, as measured by the GHQ and level of education.
|
The discordance between proxy-reported and observed CHAQ DI was further analysed by means of multivariate regression analysis (Table 3
). According to the regression model, the children's functional ability was overestimated by parents as ESR and global articular severity score increased, and underestimated with increasing number of active joints and level of pain as assessed by parents.
|
The weighted kappa statistic was used as a measure of the extent of agreement between the patients' and clinicians' ratings of each of its eight functional areas (Table 4
). When interpreted according to the guidelines proposed by Landis and Koch (see above) [15], this indicated that the level of agreement was poorest in the Eating and Hygiene areas and best in the Activities area.
|
Discussion
Substantial disagreement between the parent's and the clinician's rating of health status can lead to difficulty in assessing the effectiveness of therapy or in evaluating the need for additional interventions. To give an example, it could result in a situation in which the clinician concludes that a particular treatment was beneficial and the parent says it was not, or vice versa. Furthermore, poor clinician–parent concordance may lead to parent dissatisfaction and decreased compliance.
We estimated the degree of concordance between parent's and clinician's assessments of the child's physical health status in a cohort of children with JIA. Our patient group was representative of the main onset subtypes of the disease and was characterized by a wide spectrum of extent and severity of joint involvement. We chose the CHAQ as a measure of function because it is widely used in the assessment of disability and discomfort, has been adapted to several languages and cultures, and has shown good reliability, validity, discriminative properties and responsiveness [16]. Furthermore, the CHAQ includes at least one question for each functional area that is relevant to children of all ages, thereby removing bias due to developmental differences.
We found that parents and clinicians often disagree in their assessments of the child's functioning. By means of multivariate regression analysis, this disagreement was found to differ in direction and magnitude depending on the severity of the disease and the level of pain. The parents judged the children's functional ability as better than that observed by the clinicians (i.e. the parents overestimated their children's functional ability) with greater severity of the child's arthritis, as expressed by the articular severity score, and with greater degree of systemic inflammation, as reflected by a higher ESR. In addition, by univariate analysis children whose functional ability was overestimated by parents were found to be more commonly on steroid therapy, though this variable did not prove significant in the best-fitting model of multivariate regression. Conversely, parents and clinicians tended to agree in their assessment of physical functioning at low levels of disease severity and disability. This suggests that in children with more aggressive disease the physician may intuitively adjust the amount of impairment that they see to the objective signs of disease activity presented by the patient, whereas the parents may be more inclined to adapt to the loss of their child's functional ability. According to the multivariate regression model, the parents rated the children's functional status as more impaired than that observed by the clinicians (i.e. the parents underestimated their children's functional ability) with increasing extent of the child's arthritis, as expressed by the number of active joints, and increasing level of their own perception of their children's pain. Thus, the presence of joint pain may have a relevant influence on proxy-reported functional ability, leading the parents to emphasize their child's disability. Of note, previous studies have shown that in children with arthritis the level of pain might not be accurately perceived by parents [4] and that parents' and doctors' ratings of pain correlated significantly with a measure of joint inflammation but the children's rating did not [17]. In contrast, we found a very high correlation between parents' and children's assessments of pain (r=0.84, P<0.0001), though this measure was obtained in only 28 patients. Psychological distress or education level did not demonstrate a significant influence on proxy-reported functional ability in our patient cohort.
We must acknowledge that by describing the parents as under- or overestimating their child's ability, we cannot imply that the clinician's assessment is the correct one. Indeed, parents and physician were measuring something different: the CHAQ done by parents was supposed to reflect the child's average performance over the preceding week, whereas the clinicians were assessing the child at one point in time only. In addition, some of the difference could be due to the fact that the child was in pain or had bad disease. To give an example, children with pain could feel obliged to make a special effort and put up with pain for a special hospital examination and the results could therefore not reflect their normal functional level. Disagreement might also be related to the fact that some activities may have been less suitable for observation due to difficulties in reproducing the home environment and that the instruments or tools used to perform some tasks were different from those used in everyday life. Indeed, the poorest level of agreement was observed in the Eating and Hygiene areas of the CHAQ, which include some daily activities that may be less easily reproducible in a single morning evaluation in the hospital setting (e.g. ‘Cut his/her own meat’, ‘Wash and dry entire body’). Moreover, children with longer duration of disease are likely to have found cutlery, bowls and other pottery that suit them and therefore perform better at home than in hospital, where they are given unfamiliar equipment. Disagreement was not related to the complexity of the rating scale, because no parent or physician indicated any difficulty in completing the rating.
We conclude that discordance between proxy-reported and objective assessment of functional ability, as assessed by the CHAQ, was frequent in our patients with JIA. The severity of arthritis and the concomitant pain were found to be the main contributors to disagreement.
Acknowledgments
This work was supported in part by the Istituto di Ricovero e Cura a Carattere Scientifico Policlinico S. Matteo, Pavia, Italy.
Notes
Correspondence to: A. Martini, Clinica Pediatrica, IRCCS, S. Matteo, P. le Golgi, 2, 27100 Pavia, Italy. 
References
- Murray KJ, Passo MH. Functional measures in children with rheumatic diseases. Pediatr Clin N Am1995; 42:1127–53.[Medline]
- Giannini EH, Ruperto N, Ravelli A, Lovell DJ, Felson DT, Martini A. Preliminary definition of improvement in juvenile arthritis. Arthritis Rheum1997;40:1202–9.[ISI][Medline]
- Duffy CM, Watanabe Duffy KN. Health assessment in the rheumatic diseases of childhood. Curr Opin Rheumatol1997;9:440–7.[Medline]
- Doherty E, Yanni G, Conroy RM, Bresnihan B. A comparison of child and parent ratings of disability and pain in juvenile chronic arthritis. J Rheumatol1993;20:1563–6.[Medline]
- Duffy CM, Arsenaults L, Watanabe Duffy KN. Level of agreement between parents and children in rating dysfunction in juvenile rheumatoid arthritis and juvenile spondyloarthritides. J Rheumatol1993;20:2134–9.[Medline]
- Kwoh CK, O'Connor GT, Regan-Smith MG et al. Concordance between clinician and patient assessment of physical and mental health status. J Rheumatol1992;19:1031–7.[ISI][Medline]
-
Van Den Ende CHM, Hazes JMW, Le Cessie S, Breedveld FC, Dijkmans BAC. Discordance between objective and subjective assessment of functional ability of patients with rheumatoid arthritis. Br J Rheumatol1995;34:951–5.
[Abstract/Free Full Text] - Petty RE, Southwood TR, Baum J et al. Revision of the proposed classification criteria for juvenile idiopathic arthritis: Durban, 1997. J Rheumatol1998; 25:1991–4.[Medline]
-
Ravelli A, Viola S, Ruperto N, Corsi B, Ballardini G, Martini A. Correlation between conventional disease activity measures in juvenile chronic arthritis. Ann Rheum Dis1997;56:197–200.
[Abstract/Free Full Text] - Fantini F, Corvaglia G, Bergomi P et al. Validation of the Italian version of the Stanford Childhood Health Assessment Questionnaire for measuring functional status in children with chronic arthritis. Clin Exp Rheumatol1995;13:785–91.[ISI][Medline]
- Goldberg DP, Williams P. The user's guide to the general health questionnaire. Slough: NEFR Nelson,1988.
- Bland JM, Altman DG. Statistical methods for assessing agreement between two methods of clinical measurement. Lancet1986;1:307–10.[ISI][Medline]
- Cohen J. A coefficient of agreement for nominal scales. Educ Psychol Meas1960;20:37–46.[ISI]
- Fleiss JL, Cohen J. The equivalence of weighted kappa and the intraclass correlation coefficient as measures of reliability. Educ Psychol Meas1973;33:613–9.[ISI]
- Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics1977; 33:363–74.[ISI][Medline]
- Burgos-Vargas R. Assessment of quality of life in children with rheumatic diseases. J Rheumatol1999;26:1432–5.[ISI][Medline]
-
Ilowite NT, Walco GA, Pochaczevsky R. Assessment of pain in patients with juvenile rheumatoid arthritis: relation between pain intensity and degree of joint inflammation. Ann Rheum Dis1992;51:343–6.
[Abstract/Free Full Text]
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  F. Sztajnbok, D. L. Coronel-Martinez, A. Diaz-Maldonado, C. Novarini, A. Pistorio, S. Viola, N. Ruperto, A. Buoncompagni, A. Martini, and A. Ravelli Discordance between physician's and parent's global assessments in juvenile idiopathic arthritis Rheumatology, January 1, 2007; 46(1): 141 - 145. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. G. Sawyer, J. N. Whitham, D. M. Roberton, J. E. Taplin, J. W. Varni, and P. A. Baghurst The relationship between health-related quality of life, pain and coping strategies in juvenile idiopathic arthritis Rheumatology, March 1, 2004; 43(3): 325 - 330. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||