| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Rheumatology 2001; 40: 1274-1278
© 2001 British Society for Rheumatology
Paediatric Rheumatology |
Measuring the function of children with juvenile arthritis
Paediatric Rheumatology/Series Editor: P. Woo
Rheumatology & Rehabilitation Research Unit, School of Medicine, The University of Leeds and
1 United Leeds Teaching Hospitals, General Infirmary, Leeds, UK
Abstract
Objective. Juvenile idiopathic arthritis (JIA) can affect a child's performance across a range of activities necessary to normal childhood development. Although there are now several available measures of disability in JIA, none have been validated for use with children in the UK. Consequently, a study was undertaken to compare and validate four such measures, together with a locally developed function test.
Methods. Fifty-three children between the ages of 5 and 16 yr were recruited into the study. The mean age was 10.4 yr and mean duration of disease 4 yr. Seventy per cent were female.
Results. Internal consistency was adequate in three of five measures. Four of five measures showed the expected associations between disease activity and function (P<0.05). The level of reliability was poor for tests that involved direct assessment by therapists. Most showed poor levels of unidimensionality.
Conclusion. Until new measures become available, the CHAQ appears to be the current ‘best buy’ for measuring function in children with arthritis.
KEY WORDS: JIA, Outcome, Function, Reliability, Validity, Rasch.
A complex and changing pattern of signs and symptoms is present in juvenile idiopathic arthritis (JIA). These can compromise a child's performance in a range of activities, disadvantaging the child in many areas of life. In adult inflammatory arthritis, it is agreed that disease activity is measured principally as the level of acute-phase reactants in conjunction with measures of synovitis, principally joint counts. In children, there is reluctance to do repeated blood tests unless absolutely necessary and greater emphasis is placed upon function.
In children, changes in function may affect development and may also presage disadvantage, which could last for 70 yr. It is thus extremely important to maximize function as quickly and as effectively as possible.
In this study we followed the nomenclature of the International Classification of Impairments, Disabilities and Handicaps (ICIDH) [1], in which symptoms or signs are mainly impairments, while the performance of tasks, sometimes called function, relates to disabilities. These domains have been referred to traditionally as health status [2–4]. The disadvantage that accrues to the child is currently designated handicap as it attempts to summarize the consequences of disease, impairment and disability in the context of extrinsic factors, such as the environment [5]. Handicap also belongs within the concept of health status, and was recognized as such long before if was set out in the ICIDH [6]. More recently it has become usual to describe these dimensions of health status as ‘health-related quality of life’ (HRQoL) [7].
A number of outcome measures are available to assess health status, particularly function or disability, among children with JIA. For example, there is an adapted version of the Stanford Health Assessment Questionnaire (HAQ) [8], the Childhood Stanford Health Assessment Questionnaire (CHAQ) [9]. A family of measures was created around the Juvenile Arthritis Functional Assessment Report (JAFAR) [10]. These included an observer-assessed version (JAFAS), a parent-completed version (JAFAR-P) and a version that can be completed by the child (JAFAR-C) [11]. More recently, Ruperto et al. [12] have reviewed some of the measures in an Italian setting. Although there is evidence to support the reliability, validity and responsiveness of these measures, none has been validated for use with children in the UK. Consequently, a study was undertaken to compare these measures with each other and a previously piloted, locally developed measure, the Turner Observed Functional Test (TOFT), to ascertain their reliability, validity, responsiveness and acceptability in UK children. At the time the study began (1995), little use was made of such standardized measures in routine clinical practice.
Methods
Subjects
A pragmatic study was undertaken of 53 children between the ages of 5 and 16 yr attending, with their parents, the Regional Juvenile Arthritis Centre held at the General Infirmary at Leeds. Data were collected from July 1995 to September 1997. Interobserver reliability of two experienced occupational therapists was tested with 21 children. In addition, 24 children took part in a study of responsiveness. No alteration was made to routine clinical practice, in that, for example, children were assessed for disease activity by C-reactive protein only when the clinical situation demanded this.
Measures
The Physician's Global Assessment of Disease Activity was used as the main measure of disease activity. Background demographic data, joint count, mode of onset and disease classification were recorded independently of the functional assessments undertaken by the paediatric occupational therapists. Four North American measures were chosen as being feasible within a routine clinical setting. Of these, only the JAFAS is a timed observed function test. The JAFAR-C asks questions of the child (i.e. the therapist asks the questions as opposed to the child self-completing); the JAFAR-P and CHAQ are parent questionnaires (although the latter can be given to children over the age of 14 yr, but was completed by the parents in this study). The locally developed observed test (TOFT) was also included in the study. Table 1
summarizes the characteristics of these measures.
|
For both the JAFAS and the TOFT, standardized methods of undertaking individual tasks were followed. These were prescribed in the documentation of the JAFAS, and agreed in detail for the TOFT. Assessment of the children gave rise to 561 sets of data.
Statistical analysis
Traditional psychometrics
Each measure was evaluated for internal consistency by the use of Cronbach's 
. An value of 0.85 was considered acceptable [13]. Inter-rater reliability was assessed using the 
statistic [14]. Construct validity was evaluated by associations with the Physicians’ Global Assessment of Disease Activity, active joint count and mode of onset. Responsiveness was evaluated as the effect size [15].
Modern psychometrics
In addition to the traditional psychometrics, data from the measures were applied to the one-parameter item response theory (IRT) model to examine the unidimensionality of the measures and the way in which they work at the individual item level. Many health status measures have not been tested formally for unidimensionality, relying entirely on high internal consistency as a measure of this property. Unfortunately, a high Cronbach's value does not confirm unidimensionality [16].
IRT is a general statistical theory about question (item) and scale performance and how that performance relates to the perceptions that are measured by the items in the scale [17]. Thus, an understanding of the way in which the children respond to items and how the items measure the construct is gained by the application of IRT. Based upon the work of the Danish mathematician Georg Rasch [18], the one-parameter IRT model is a unidimensional model that assumes that the children and items can be ordered uniquely in terms of the child's function and the difficulty of the items. The probability that a child will affirm an item or category within an item (or perform the task) is seen as a logistic function of the difference between their ability (
) and the difficulty of the item (b), and only a function of that difference:
|
| (001) |
) is the probability that children with ability level will affirm item i, and b is the item difficulty. Ethics approval was given for the study by the Leeds Healthcare/United Leeds Teaching Hospitals NHS Trust. Consent was obtained from the parents and children, who received written information about the study. The children who took part in the inter-rater reliability study were seen twice on the same day. Most children completed the assessments during a routine visit to the Regional Juvenile Arthritis Clinic at Leeds General Infirmary. Where children completed the instruments (as opposed to parents), they were all of a valid age for the particular test.
Results
The study enrolled 53 children with a diagnosis of JIA. The mean age at entry into the study was 10.4 yr (S.D. 3.1, range 5–16) and the mean age at onset was 6.5 yr (S.D. 3.8); the mean duration of disease was thus 4 yr (S.D. 3.4). Over half were classified as having oligoarthritis. Approximately one in seven (14%) was classified as having systemic and polyarticular (negative for rheumatoid factor) respectively. Extended oligoarthritis accounted for only one in fourteen (7%). The mean number of active joints was 1.8 (S.D. 2.6), quiescent joints 2.7 (S.D. 7.6) and total joint count 4.5 (SD 8.0). Almost two-thirds (64%) had no or mild disease activity on the Physician's Global Assessment of Disease Activity scale. A baseline mean CHAQ of 0.71 indicated that most children also had only mild disability, although only one in eight children (13%) had a CHAQ of zero, a total joint count of zero and a physician's global assessment of zero. In contrast, 13 children had more severe disability (HAQ>1.5).
Traditional test theory
Reliability
Internal consistency was adequate in three of the tests (Table 2), but the JAFAS and JAFAR-C failed to meet the acceptable level of 0.85. The inter-rater reliability of the two observed measures (JAFAS and TOFT) was highly variable by task. For example, getting out of the bath or bed, brushing the hair and picking up from the floor displayed poor levels of agreement. The value ranged from 0.17 to 1.00 for the 20 items of the TOFT (six items had perfect agreement) and from 0.07 to 1.00 on the JAFAS, where one item showed perfect agreement.
|
Validity
The association between function and measures such as the Physician's Global Assessment of Disease Activity and the active joint count would only be expected to be moderate. This is consistent with the relationship between disease, impairment and disability. Table 3 shows that all scales showed a moderate association. Only the CHAQ (P=0.003) showed a significant relationship with mode of onset (Kruskal–Wallis test).
|
Responsiveness
It was not possible to calculate the non-parametric effect size as the median change score on four of the measures was zero. The CHAQ showed the greatest responsiveness (Table 4), as determined by the parametric effect size. Its responsiveness was more than twice that of the JAFAS, JAFAR-P and JAFAR-C. Too few children were evaluated by the TOFT to obtain a meaningful analysis of responsiveness.
|
Modern test theory
The fit of the data to the Rasch model is a test of unidimensionality. Two fit statistics, INFIT and OUTFIT, are used to determine which items fit the model. For this number of cases, the required ranges for INFIT and OUTFIT are 0.7–1.3 and 0.2–1.8 respectively [19]. Items that have a value above the required range are said to show unexpected responses, which may derive from their failure to belong to the same construct. Items with a value below the required range indicate redundancy in that their information adds to that already present in the other items. Table 5 shows that the CHAQ showed the greatest unidimensionality, having only one problematic item (Hygiene) with a consequent misfit rate of 13%. Similarly, the JAFAR-C had a misfit rate of 17%. All other scales showed varying degrees of misfit to the model. Items that involve grip appear to be most problematic. So ‘cutting up food with a knife and fork’ (therapeutic putty in our case) and ‘button shirt’, did not display a consistent probabilistic relationship with other items in the scale. For example, children with a high level of disability would be expected to report problems with cutting up food, as it is a common problem, but some do not. Thus, at a given level of disability the task does not behave in a consistent way with regard to the construct being measured. With an acceptable level of reliability for the item, this would suggest that the activity may belong to a different construct.
|
Discussion
When the present study began, there were few functional scales in routine use with children with arthritis in the UK. Four such scales, all developed in North America, together with a locally developed observed measure, were examined for their traditional psychometric properties, and for unidimensionality using Rasch analysis. All scales showed some weaknesses, to varying extents.
Both the classical and modern psychometric analysis of the two observed tests (JAFAS and TOFT) showed disappointing results. This is not easy to explain. The lack of reliability, despite standardized protocols, compromises the utility of the scales. However, the inter-rater reliability study was undertaken within the same day and it is possible that the children became tired, or indeed that morning stiffness had worn off, although the children were not tested very early in the morning. Another explanation might be the unfamiliarity of the situation in which the child was tested. All the JAFAS items and one of the TOFT items were timed tests, and it is possible that some tasks that were completed just within the allotted time during the first assessment failed to be done on time approximately one and a half hours later, when the retest was undertaken.
All tests take account of developmental issues. However, as the tasks are not graded for developmental age they tend to have a considerable time-span allocated, ranging from 2.4 to 27.2 s for the JAFAS. It would be possible for a child to show considerable deterioration in a particular task, say doubling the time taken, but still be within the allotted time to ‘pass’ on a particular task. This has implications for detecting change in longitudinal studies.
With a mean of 0.71 on the CHAQ, most children had only mild to moderate disability. Only the JAFAR-P displayed a ceiling effect, and for only two children. In contrast, all measures displayed a floor effect to a greater or lesser extent. This ranged from 20.4% for the TOFT to 37.7% for the JAFAR-P. Clearly, there is some scope for identifying tasks which can discriminate even milder levels of disability.
It is known that, for JIA, parent's ratings of disease activity are only moderately correlated with actual disease activity [20]. For function, parent-completed measures only achieve a modest correlation with responses by the child. For example, mothers have been shown to consistently rate their child's perceived competence more negatively than the self-rating of the child [21]. The correlation between the different measures in the present study supports these findings. For example, the JAFAR-C comprised a set of items asked of the child, whereas the JAFAR-P consisted of the same set of questions asked of the parents. The correlation between the two was 0.50. In contrast, even though some different items were involved, the correlation between the CHAQ, completed by the parents, and the JAFAR-P, also completed by the parents, was 0.69. Likewise, the two observed tests achieved a correlation of 0.78. However, the correlation between the observed tests and the administered or self-completed questionnaires never exceeded 0.57. Thus, the context for obtaining the information and the time-span being considered may play a crucial part in the measurement of function. Both observed function and questionnaires may be of value, although they express different facets of function. The question then becomes which type of test should be used in what situation.
Both parent-completed measures (JAFAR-P and CHAQ) performed well, although the JAFAR-P has a high misfit rate to the underlying construct. Items such as ‘lift an empty glass to the mouth’ and ‘button shirt’, appear to have no association with the underlying construct. The results from the present study suggest that activities related to manual dexterity and grip may be measuring a domain different from the general activity and mobility items that commonly make up these functional tests.
It is possible that the limitations of the functional measures in the present study may be overcome by new measures that have emerged since the commencement of this study. For example, attempts have been made to develop questionnaires that are highly responsive to change over time by offering the child a choice of items that are currently causing problems. The Juvenile Arthritis Quality of Life Questionnaire [JAQQ], designed for self-report for children over 9 yr of age, contains 74 items in four domains (of HRQoL) within which a child can select the five items that cause most difficulty [22]. However, because children choose subsets of items, some doubts have been expressed about the ability of this measure to offer comparison across children [23]. Such doubts are misplaced. They can be overcome by the application of IRT and the concept of item banking [24], assuming that the items within each domain do belong to a single construct. Whether the measure meets both traditional and modern psychometric criteria remains to be seen. As important, perhaps, is the anxiety that the JAQQ may prove too cumbersome and too time-consuming for routine use.
Most children in this study, as world-wide, had minor rather than major disabilities. The CHAQ works well for children with severe disability as none were found at the ceiling of the scale. However, approximately one in five children were at the floor of the CHAQ. As more minor disability is the context in which we expect to be working in the future, this is a limitation of the scale. An instrument is needed to measure subtle changes in minor disability.
The present study raised a dilemma. The information required for clinical practice is that obtained from an observed functional assessment, whereas that needed for demonstrating the efficacy of an intervention was best provided by those tests which depended on the reporting of functional difficulties and which met both traditional and modern psychometric standards. Pending further work on the newer measures and solutions to the problems highlighted with the older measures, the CHAQ appears to be the current ‘best buy’ for measuring function in children with arthritis.
Acknowledgments
The authors wish to thank J. Stowe for her help in developing the original idea for the study. We also thank the staff, children and parents at the paediatric outpatients department at the General Infirmary at Leeds, and the Arthritis Research Campaign for its financial support.
Notes
Correspondence to: A. Tennant, Rheumatology & Rehabilitation Research Unit, University of Leeds, 36 Clarendon Road, Leeds LS2 9NZ, UK. 
References
- World Health Organization. The International Classification of Impairments, Disabilities and Handicaps. Geneva: WHO, 1980.
- Meenan RF, Gertman PM, Mason JH. Measuring health status in arthritis: The Arthritis Impact Measurement Scales. Arthritis Rheum1980;23:146–52.[Web of Science][Medline]
- Bergner M. Health status as a measure of health promotion and disease prevention: unresolved issues and the agenda for the 1990's. In: Rockville MD, ed. Proceedings of the 1989 Public Health Conference on Records and Statistics. Washington: National Centre for Health Statistics, 1989.
- Fitzpatrick R. The measurement of health status and quality of life in rheumatological disorders. Bailliere’s Clin Rheumatol1993;7:297–317.[Medline]
- Badley EM. The genesis of handicap: definition, models of disablement, and role of external factors. Disabil Rehabil1995;17:53–62.[Web of Science][Medline]
- Lerner M. Conceptualisation of health and well-being. Health Serv Res1973;8:6–12.[Medline]
- Schumaker S, Berzon R, eds. The international assessment of health related quality of life. Oxford: Rapid Communications, 1995.
- Fries JF, Spitz PW, Young DY. The dimension of health outcomes: The Health Assessment Questionnaire. J Rheumatol1982;9:789–93.[Web of Science][Medline]
- Singh G, Athreya B, Fries J, Goldsmith D. Measurement of health status in children. Arthritis Rheum1994;37:1761–9.[Web of Science][Medline]
- Lovell DJ, Howe S, Shear ES et al. Development of a disability measurement tool for juvenile rheumatoid arthritis. Arthritis Rheum1989;32:1390–5.[Medline]
- Howe S, Levinson J, Shear ES et al. Development of a disability measurement tool for juvenile rheumatoid arthritis. Arthritis Rheum1991;34:873–80.[Web of Science][Medline]
- Ruperto N, Ravelli A, Migliavacca D et al. Responsiveness of clinical measures in children with oligoarticular juvenile chronic arthritis. J Rheumatol1999;26:1827–30.[Web of Science][Medline]
- Streiner DL, Norman GR. Health Measurement Scales. 2nd edn. Oxford: Oxford University Press, 1995.
- Cohen JA. Coefficient of agreement for nominal scales. Educ Psychol Meas1960;20:37–46.[Web of Science]
- Kaziz L, Anderson JJ, Meenan RF. Effect sizes for interpreting changes in health status. Med Care1989;27:S178–9.
- Cortina JM. What is coefficient alpha? An examination of theory and applications. J Appl Psychol1993;78:98–104.[Web of Science]
- Hambleton RK, Jones RW, Russell W. Comparison of classical test theory and item response theory and their applications to test development. Educ Meas Issues Pract1993;12:38–47.
- Rasch G. Probabilistic models for some intelligence and attainment tests. Chicago: University of Chicago Press, 1980.
- Smith RM, Schumacher RE, Bush MJ. Using item mean squares to evaluate fit in the Rasch model. J Outcome Meas1998;2:66–78.[Medline]
- Rapoff MA, Lindsley CB, Purviance MR. The validity and reliability of parental ratings of disease activity in juvenile rheumatoid arthritis. Arthritis Care Res1991;4:136–9.[Medline]
-
Ennett ST, DeVellis BM, Earp JA, Kredich D, Warren RW, Wilhelm CL. Disease experience and psychosocial adjustment in children with juvenile rheumatoid arthritis: Children's versus mother's reports. J Pediatr Psychol1991;16:557–68.
[Abstract/Free Full Text] - Duffy CM, Aresenault L, Watanabe KN, Paquin JD, Stawczynski H. The Juvenile Arthritis Quality of Life Questionnaire: development of a new response index for juvenile rheumatoid arthritis. J Rheumatol1997;24:738–46.[Web of Science][Medline]
- Tucker LB. Measuring health status and outcome in adolescents with rheumatic disease. In: Isenberg DA, Miller, JM III, eds. Adolescent rheumatology. London: Martin Dunitz, 1999.
- Dobby J, Duckworth D. Objective assessment by means of item banking. Schools Council Examination Bulletin 40. Evans/Methuen Educational, 1979.
CiteULike 
Connotea 
Del.icio.us What's this?
This article has been cited by other articles:
|
|
 |
|
  T. M. Palermo, A. C. Long, A. S. Lewandowski, D. Drotar, A. L. Quittner, and L. S. Walker Evidence-based Assessment of Health-related Quality of Life and Functional Impairment in Pediatric Psychology J. Pediatr. Psychol., October 1, 2008; 33(9): 983 - 996. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 S Magni-Manzoni, A Pistorio, E Labo, S Viola, P Garcia-Munitis, S Panigada, C Visconti, A Buoncompagni, A Martini, and A Ravelli A longitudinal analysis of physical functional disability over the course of juvenile idiopathic arthritis Ann Rheum Dis, August 1, 2008; 67(8): 1159 - 1164. [Abstract] [Full Text] [PDF]
|
|
|
|
 |
|
 T. Ding, A. Hall, K. Jacobs, and J. David Psychological functioning of children and adolescents with juvenile idiopathic arthritis is related to physical disability but not to disease status Rheumatology, May 1, 2008; 47(5): 660 - 664. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. Batthish, R. Schneider, A. V. Ramanan, C. Achonu, N. L. Young, and B. M. Feldman What does 'active disease' mean? patient and parent perceptions of disease activity in the systemic arthritis form of juvenile idiopathic arthritis (SO-JIA) Rheumatology, June 1, 2005; 44(6): 796 - 799. [Abstract] [Full Text] [PDF]
|
|
|
|
|
|
M. G. Sawyer, J. N. Whitham, D. M. Roberton, J. E. Taplin, J. W. Varni, and P. A. Baghurst The relationship between health-related quality of life, pain and coping strategies in juvenile idiopathic arthritis Rheumatology, March 1, 2004; 43(3): 325 - 330. [Abstract] [Full Text] [PDF]
|
|
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||