Skip Navigation


Rheumatology Advance Access originally published online on March 10, 2006
Rheumatology 2006 45(9):1154-1157; doi:10.1093/rheumatology/kel073
This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow All Versions of this Article:
45/9/1154    most recent
kel073v1
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrow Search for citing articles in:
ISI Web of Science (1)
Right arrowRequest Permissions
Right arrow Disclaimer
Google Scholar
Right arrow Articles by van Winkelen, P.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by van Winkelen, P.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?

© The Author 2006. Published by Oxford University Press on behalf of the British Society for Rheumatology. All rights reserved. For Permissions, please email: journals.permissions@oxfordjournals.org

How evidence-based are advertisements in journals regarding the subspecialty of rheumatology?

P. van Winkelen, J. S. van Denderen, C. Y. Vossen, T. W. J. Huizinga1, F. W. Dekker for the SEDUCE study group

Department of Clinical Epidemiology and 1Department of Rheumatology, Leiden University Medical Center, Leiden, The Netherlands.

Correspondence to: F. W. Dekker, Department of Clinical Epidemiology, Leiden University Medical Center, PO Box 9600, 2300 RC Leiden, The Netherlands. E-mail: F.W.Dekker{at}lumc.nl


   Abstract
Top
 Abstract
Introduction
 Materials and methods
 Results
 Discussion
 Appendix
 References
 
Objectives. In rheumatology, five different billion-dollar drugs have emerged in recent years, making this subspecialty the focus of extensive advertising campaigns. Considering this development and the fact that the scientific content of advertisements has been questioned, we initiated a study to determine how evidence-based advertisements are in four leading journals of rheumatology.

Methods. Advertisements were extracted from the journals Arthritis and Rheumatism, Rheumatology, Seminars in Arthritis and Rheumatism and the Journal of Rheumatology, and were evaluated using a standardized score form.

Results. In total, 353 advertisements were obtained, of which 84 were unique. Of the 300 references provided by these unique advertisements, 53 (18%) were considered ‘supporting’. In addition, 87 (29%) of the 300 references referred to randomized controlled trials (RCTs), of which 49% supported the claim. The vast majority of RCTs (97%) were sponsored by the advertising company. In the 84 unique advertisements 288 claims were made, of which 190 provided one or more references. Of these 190 claims, 33 (17%) were judged ‘well supported’. In total, only four (5%) of the 84 different advertisements were judged ‘well supported’ and 11 (13%) of the 84 were judged ‘misleading’ because they contained one or more misleading claims, i.e. claims with references to articles contradicting the claim.

Conclusions. This study indicates that few advertisements in rheumatology journals are entirely evidence-based.

KEY WORDS: Advertisements, Evidence-based, Rheumatology.


   Introduction
Top
 Abstract
 Introduction
Materials and methods
 Results
 Discussion
 Appendix
 References
 
Medical journals are the major source of information for doctors, through both peer-reviewed articles and advertisements. The scientific content of advertisements, trying to meet commercial and scientific standards at the same time, has been questioned [1–9]. The World Health Organization (WHO) formulated ethical and scientific criteria for pharmaceutical drug promotion in 1968. According to these criteria, advertisements should provide ‘scientifically based, balanced factual information without ambiguities’ [10]. Another attempt to establish norms for promotional quality was made by the International Federation of Pharmaceutical Manufacturers and Associations (IFPMA). The code became operational in 1981 and the accompanying complaints procedure in 1982. Information is judged according to whether it is up to date, meets ethical standards, is not misleading, reveals its commercial nature, and provides qualification of keywords such as ‘safe’. The promotional material should be ‘based on an up-to-date evaluation of evidence’. A number of pharmaceutical companies have adopted the IFPMA code as their guideline for advertising [11].

Several studies have tried to substantiate the value of pharmaceutical advertisements as a source of information for doctors, thereby focusing on a wide range of issues. Some researchers focused on the presence of important information such as drug interactions and contraindications in advertisements [2]. References supporting the promotional claims can make the claim more impressive, so other researchers focused on the references accompanying the claims. Cooper described the kinds of documents cited in support of claims and assessed the availability of these documents [7]. Lankinen, Villanueva and Loke examined references in more detail. The study of Lankinen et al. indicated that 21% of references were either totally irrelevant to the claim, or at least did not provide scientific support to it [1]. In the study of Villanueva et al., the promotional statement was not supported by the reference in 41% of claims [3]. Loke et al. found that, of 855 quantitatively substantiated claims, 55% referred to a randomized controlled trial or a systematic review as their source of information [4]. Three studies [5, 6, 8] assessed the methodological quality of articles that were referred to, but only Lexchin and Holbrook used an official score to assess the methodological quality of articles cited [8]. Further, of all studies only two focused on a single medical discipline [5, 9].

None of the researchers focused on the discipline of rheumatology. This subspecialty, however, has been the subject of important changes during the last 6 yr. According to the 2005 president's speech of the American College of Rheumatology, five different billion-dollar drugs have emerged in rheumatology in recent years, making this subspecialty the focus of the ‘most aggressive advertising campaign this side of Cialis’ [12]. Considering this development and the results of the studies mentioned above, it is interesting to evaluate the quality of information given in these advertising campaigns. Therefore, the aim of the present study was to measure the quality of evidence behind advertisements in four leading journals of rheumatology. This was done by focusing on claims and evaluating the accompanying references. References to trials were evaluated with regard to their methodological properties and potential to support the claim. For the methodological properties of randomized controlled trials, the modified score of Chalmers was used.


   Materials and methods
Top
 Abstract
 Introduction
 Materials and methods
Results
 Discussion
 Appendix
 References
 
Advertisements used in this study were extracted from four journals: Arthritis and Rheumatism, January to October 2004; Rheumatology, 2003; Seminars in Arthritis and Rheumatism, 2002 and 2003; and Journal of Rheumatology, 2004. A team of four researchers searched the above journals and included all drug-promoting advertisements with one or more claims. Identical advertisements were copied only once but the frequency of appearance was noted. The advertisements found were distributed among the research group and evaluated using a standardized score form. Meetings to discuss definitions and qualifications were organized to diminish inter-observer variability.

Each claim was evaluated. A claim was defined as all statements concerning the safety or efficacy of a drug, and other statements if they included a reference. Prescribing information in advertisements (always included in separate textboxes) was not checked for claims because this was regarded as non-promotional information. The quality of evidence-based advertising was evaluated by determining the number of claims with references, their availability on PubMed (http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed), and by comparing the claim with the factual information in the references. Points of comparison were those essential for the validity of the reference serving as evidence for the claim. Thus, we asked whether the population, the controls and the intervention in the advertisement corresponded to the article referred to. Furthermore, all references were divided into the following categories: systematic review; randomized controlled trial (RCT); research other than an RCT or systematic review; prescribing information; data on file; and ‘other’ (e.g. web sites, proceedings, articles not traceable on PubMed). The methodological properties of the RCTs referred to were also assessed. For this purpose, a score described by Chalmers et al. [13] was used, which is a more detailed variant of the more widely used Jadad score [14]. Both scoring systems rate the methodology of the study by focusing on the reports of randomization, blinding and the handling of withdrawals. The Chalmers score rates these three items on a scale of 0–3. The sum of the three subscores forms the total score, which can be grouped into the following categories: <3, negative; 3–5, neutral; and 6+, positive.

Conclusions were drawn using the following definitions. When references were of neutral or positive methodological quality, and in accordance with the claim, they were considered ‘supporting’. When the evidence for the claim was poor (differences in populations, controls or interventions), they were called ‘non-supporting’. Because only articles available on PubMed were considered accessible and scientifically interpretable for practitioners, references were also called ‘non-supporting’ when the reference category was ‘prescribing information’, ‘data on file’ or ‘other’. At the claim level, the outcome could be ‘well supported’, ‘poorly supported’ or ‘misleading’. Claims referring to traceable systematic reviews or trials, which supported the claim, were labelled ‘well supported’. Claims without a traceable reference or with an insufficiently supporting reference were labelled ‘poorly supported’. Claims that referred to an article that contradicted the claim were labelled ‘misleading’. Advertisements were rated ‘well supported’, ‘poorly supported’, ‘misleading’ or as containing both well and poorly supported claims. Advertisements with nothing but well supported claims were labelled ‘well supported’. Advertisements with one or more misleading claims were considered ‘misleading’. Score forms were given to two researchers, who entered the data in SPSS (Chicago, Illinois, USA).


   Results
Top
 Abstract
 Introduction
 Materials and methods
 Results
Discussion
 Appendix
 References
 
From the sample of four journals, 353 advertisements were obtained, representing 31 different products. After exclusion of identical advertisements, 84 advertisements remained. These 84 advertisements provided a total of 300 references. These references were evaluated; 53 (18%) were considered ‘supporting’, indicating that the article referred to was a traceable, methodologically correct RCT (n = 43), a systematic review (n = 2) or other research (n = 8) with sufficient methodological quality (neutral or positive Chalmers’ score), which dealt with population, intervention and control characteristics comparable to the claim. Of the 247 ‘non-supporting’ references, 181 referred to ‘data on file’, ‘prescribing information’ and ‘other sources (e.g. web sites, proceedings, research not traceable on PubMed), 44 referred to RCTs and 22 to other research. Of the 300 references, 87 referred to RCTs. However, 84 (97%) of the 87 RCTs were sponsored by the advertising company. In none of the 87 cases did the RCT referred to get a negative Chalmers score: 57 RCTs got a positive score and the remainder a neutral score. Forty-nine per cent of the references to RCTs were supporting the claim, indicating that the Chalmers score was neutral or positive and no discordance was found when comparing the advertisement and the article referred to with respect to the populations, controls and interventions.

The 353 advertisements obtained represented 31 different products from the whole spectrum of drugs used in rheumatology. A total of 116 advertisements (33%) were published on three different TNF-{alpha} blockers. Four coxibs formed the second largest group, with 59 advertisements (17%), followed by cholinergic agonists (10%), bisphosphonates, viscosupplements and NSAIDs (7% each) and a group ‘other’, representing 19% of all advertisements. Misleading advertisements were found for all the different drug types except coxibs and biphosphanates.

In the 84 unique advertisements, 288 claims were found. Ninety-eight claims did not refer to any source, so they were categorized as ‘poorly supported’. One hundred and ninety claims had one or more references. Of these, 16 were judged ‘misleading’, 33 ‘well supported’ and 141 ‘poorly supported’. Ninety-nine of 141 poorly supported claims were so judged because they provided only one reference, referring to data on file, prescribing information or other data (the three poorly supporting categories of information) (Fig. 1)


Figure 1
View larger version (16K):
[in this window]
[in a new window]
[Download PowerPoint slide]
  		FIG. 1. Assessment of claims.

 
Five per cent of the 84 different advertisements were labelled ‘well supported’, meaning that their claims were all well supported by their references. Thirteen per cent of the 84 different advertisements were labelled ‘misleading’ because they contained one or more misleading claims, i.e. claims with references to articles contradicting the claim (Table 1).


View this table:
[in this window]
[in a new window]

  		TABLE 1. Assessment of advertisements with and without referenced claims

 

   Discussion
Top
 Abstract
 Introduction
 Materials and methods
 Results
 Discussion
Appendix
 References
 
This is the first study that has assessed the quality of information provided by advertisements in journals covering the subspecialty of rheumatology. Remarkably, only four (5%) out of 84 advertisements were fully supported by high-quality evidence. Many of the references accompanying claims did not refer to a proper paper traceable on PubMed. Three categories—data on file, prescribing information, and other (e.g. web sites and readers)—accounted for 60% of the references; the category most referred to was data on file (30%). There is a possibility that data on file will be published later and will become available for validation of the claim. Therefore, the term ‘non-supporting’ should not be interpreted as a definitive judgement for this category of references. Moreover, the relatively high proportion of references to data on file may be caused by the tendency of pharmaceutical companies to advertise for new indications for which the data have been presented to the regulatory authorities but have not yet been published in the public literature. However, as a way to distribute information, the availability of this information can be questioned. Cooper and Schriger investigated the availability of cited documents: only 20% of the requests for data on file produced a document, showing that data on file are not easy available [7]. Thus, it may be more logical to refer to the formal label of the drug instead of data on file, because this information is in the public domain.

Companies do not present a balanced list of references representing pro and contra articles in their advertisement. This could lead to a biased opinion when practitioners do not realize this enough. In the present study we did not assess the potential influence on how evidence-based an advertisement is.

We only assessed the various forms of research (RCT, systematic review and other research methods) available on PubMed as possible evidence to support claims, because these articles give the opportunity to draw conclusions based on the results given. However, in 44 (51%) of 87 references referring to an RCT, the RCT did not support the claim properly. The problem was never the methodological quality, as all the RCTs got a neutral or positive Chalmers score. The major problem was that the advertisements tended to extrapolate trial results to other patient groups, dosages or treatments. In addition, the advertising companies sponsored 97% of the RCTs referred to. This is relevant as the results from RCTs differ according to whether they are or are not company-sponsored trials [15].

This study included four rheumatology journals. We have not formally tested whether the advertisements in these journals are a proper reflection of all journals regarding rheumatology, but there are no data to indicate that advertisements differ in the different journals. Three of the four journals examined were from the American side of the Atlantic. Because both the American and the European pharmaceutical industry have adopted the IFMPA Code of Pharmaceutical Marketing Practices as their advertising guideline [11], we do not expect large differences between American and European advertising practices as a result of differences in regulatory requirements. The volumes of the journals we used formed a relative small sample, but an up-to-date one. The assessment itself was subjective in its core, but researchers used standardized score forms and discussed difficulties in interpretation. A small pilot study was performed to train researchers in using the score form, and to discover shortcomings of the form.

In the present study, a rather small percentage (5%) of the advertisements proved to be evidence-based. In a comparable study, Gutknecht classified 18% of distinctive advertisements (n = 187) as evidence-based. In his study, ‘evidence-based’ meant: ‘citing numerical data from controlled clinical trials, and presumably seeking to influence readers through provision of that evidence’ [2]. We used a wider definition (not restricted to the citing of numerical data) and still found a smaller percentage, probably because we made a comparison between the claim and the information in the article and found many incompatibilities. Only when a referred-to trial supported a claim was the claim regarded evidence-based. In conclusion, this study indicates that there is room for improvement in the quality of information in advertisements in rheumatology.

Formula

No conflict of interest has been declared by the authors.


   Appendix
Top
 Abstract
 Introduction
 Materials and methods
 Results
 Discussion
 Appendix
References
 
The SEDUCE study groups consists of 14 medical students of the Leiden University Medical Center: P. van Winkelen, J. S. van Denderen, R. A. van Adrichem, M. M. de Boer, M. S. Bruijn, D. J. Bruijn, P. D. Dijk, B. P. C. Hoppe, C. J. Kortmann, S. van der Marel, A. G. van de Merwe, J. van Rhenen, S. Saadatmand and V. Vennemann; and of C. Y. Vossen, T. W. J. Huizinga and F. W. Dekker.


   References
Top
 Abstract
 Introduction
 Materials and methods
 Results
 Discussion
 Appendix
 References
 

  1. Lankinen KS, Levola T, Marttinen K, Puumalainen I, Helin-Salmivaara A. (2004) Industry guidelines, laws and regulations ignored: quality of drug advertising in medical journals. Pharmacoepidemiol Drug Saf 13:789–95.[CrossRef][ISI][Medline]
  2. Gutknecht DR. (2001) Evidence-based advertising? A survey of four major journals. J Am Board Fam Pract 14:197–200.
  3. Villanueva P, Peiro S, Librero J, Pereiro I. (2003) Accuracy of pharmaceutical advertisements in medical journals. Lancet 36:27–32.
  4. Loke TW, Koh FC, Ward JE. (2002) Pharmaceutical advertisement claims in Australian medical publications. Med J Aust 177:291–3.[ISI][Medline]
  5. Bhattacharyya T, Tornetta P 3rd, Healy WL, Einhorn TA. (2003) The validity of claims made in orthopaedic print advertisements. J Bone Joint Surg Am 85A:1224–8.[Abstract/Free Full Text]
  6. Wilkes MS, Doblin BH, Shapiro MF. (1992) Pharmaceutical advertisements in leading medical journals: experts’ assessments. Ann Intern Med 116:912–9.[ISI][Medline]
  7. Cooper RJ and Schriger DL. (2005) The availability of references and the sponsorship of original research cited in pharmaceutical advertisements. CMAJ 172:487–91.[Abstract/Free Full Text]
  8. Lexchin J and Holbrook A. (1994) Methodologic quality and relevance of references in pharmaceutical advertisements in a Canadian medical journal. CMAJ 151:47–54.[Abstract]
  9. Sutherland AG, Craig N, Maffulli N, Brooksbank A, Moir JS. (2000) Accuracy of references in the orthopaedic literature. J Bone Joint Surg Br 82:9–10.
  10. WHO. (1988) Resolution WHA41.17 adopted by the Forty-first World Health Assembly13 May 1988Geneve: World Health Organization Available: at http://www.who.int/.
  11. IFPMA. (1994) IFPMA code of pharmaceutical marketing practices. (International Federation of Pharmaceutical Manufacturers and Associations, Geneva) Available at: http://www.ifpma.org.
  12. Wofsy D. (2005) Living in a different world. Arthritis Rheum 52:395–401.[CrossRef][ISI][Medline]
  13. Chalmers TC, Smith H Jr, Blackburn B, et al. (1981) A method for assessing the quality of a randomized control trial. Control Clin Trials 2:31–49.[CrossRef][ISI][Medline]
  14. Jadad AR, Moore RA, Carroll D, et al. (1996) Assessing the quality of reports of randomized clinical trial: is blinding necessary? Control Clin Tials 17:1–12.
  15. Bekelman JE, Li Y, Gross CP. (2003) Scope and impact of financial conflicts of interest in biomedical research: a systematic review. JAMA 289:454–65.[Abstract/Free Full Text]
Submitted 22 November 2005; revised version accepted 7 February 2006.
Add to CiteULike CiteULike   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us    What's this?



This Article
Right arrow Abstract Freely available
Right arrow Full Text (PDF)
Right arrow All Versions of this Article:
45/9/1154    most recent
kel073v1
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Similar articles in ISI Web of Science
Right arrow Similar articles in PubMed
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrow Search for citing articles in:
ISI Web of Science (1)
Right arrowRequest Permissions
Right arrow Disclaimer
Google Scholar
Right arrow Articles by van Winkelen, P.
Right arrow Search for Related Content
PubMed
Right arrow PubMed Citation
Right arrow Articles by van Winkelen, P.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?