Validity of Health Information
Validation of information presented in a publication entails the evaluation of (1) the goals of the organization involved in the publication, (2) conflicts of interest, (3) major stakeholders, (4) the knowledge, expertise, and qualifications of the editorial board, and (5) the editorial process. The members of the Editorial Board of a publication interest are the gatekeepers of scientific information.
Australian Medical Handbook (AMH)
The Australian Medicines Handbook (AMH) was developed as a resource for healthcare practitioners interested in the quality use of pharmaceuticals in Australia (AMH, 2012). AMH is independent of both Government and the pharmaceutical industry; therefore, the AMH has no conflict of interest and selects information based solely on its relevance and validity (Thistlethwaite, 2010). Good data quality also depends on the efficacy of the internal organization (Foster, 2012). AMH is owned by the Royal Australian College of General Practitioners (RACGP), the Pharmaceutical Society of Australia (PSA) and the Australasian Society of Clinical and Experimental Pharmacologists and Toxicologists (ASCEPT) (AMH, 2012). These organizations have vast experience in the healthcare field and are amongst the top healthcare organizations in Australia and as such have a vested interest in the validity of health information presented to its members.
Knowledge and experience influence the validity of data (Forster, 2012). The AMH editorial staff and Editorial Advisory Board members are chosen for their high level of expertise, contribution, and achievement in the medical and pharmacological fields. For example, AMH chairman Anthony J. Nunan is former president of PSA (PSA, 2012); AMH Editorial Advisor John Dowden is Chairman, Expert Group for TGL and Editor-in-Chief of Australian Prescriber, Australia’s national journal of drugs and therapeutics (TGL, 2011); and AMH content advisor Leanne Stafford is an eminent clinical pharmacist and researcher in the field of cardiology and rheumatoid arthritis (AMH, 2012). Thus, the most qualified experts in the field vet and validate the information to be included in the AMH.
AMH editors then prepare a draft document, which undergoes a series of reviews by other AMH editors until some form of consensus is reached regarding the new content. Studies have shown that open peer-review by multiple reviewers is superior to anonymous peer-review by an independent reviewer (Vinther, 2012). The draft document is then sent out to experts in the field for external comments and feedback; external review has also been shown to increase the validity of information (Thistlethwaite, 2012).
AMH editors evaluate comments from external reviewers and edit the draft document to include the new information, and the draft document undergoes another series of internal reviews. In short, each new source of information undergoes a much more rigorous and in-depth evaluation of its validity than it undergoes at its initial publication, strongly suggesting that the validity of the information presented in the AMH is of the highest caliber. Figure 1 below summarizes the AMH editorial and evaluation process.
Figure 1. The AMH editorial process
Herbs and Natural Supplements, 3rd Edition: An Evidence-based Guide
Herbs and Natural Supplements, 3rd Edition: An evidence-based guide is a reference guide that provides scientific evidence on the 130 most widely used herbs, nutrients and food supplements used in alternative medicine across Australia and New Zealand (Braun & Cohen, 2010). The data provided for each substance is similar to the data provided in a standard drug formulary; i.e. chemical components, pharmacological actions, clinical indications, dosage, toxicity, adverse reactions, drug interactions, contraindications, and precautions (IAHPC, 2012). However, although each substance is well referenced, not all substances are backed by evidence from clinical trials and this impacts their validity as therapeutics (Vantongelen, Rotmensz, & van der Schueren, 1989). This by no means reflects any lack of diligence in the part of the authors, but rather underscores the paucity of clinical data available on natural medications, for it is not until recently that scientists have begun to turn their attention towards phyto- medication and pharmacology. But there are other factors to consider when evaluating the validity of the health information in Herbs and Natural Supplements, and one of them is the credentials of the authors (Forster, 2012). Braun is a pharmacist, naturopath and herbalist, and Senior Lecturer at the Australian College of Natural Medicine; and Cohen, is a professor and Head of the Department of Complementary Medicine at RMIT University in Melbourne, Australia. Thus, both authors are highly qualified to judge the material that should be included in their guide. Moreover, Herbs and Natural Supplements has undergone considerable post-publication peer-review, though nothing approximating the review process involved in the selection of the information included in the AMH. Post-publication review has been shown to be more effective than pre-publication review, especially in this case where the whole editorial process lay in the hands of but two individuals (Forster et al., 2012, Herron, 2012). For example, the guide has been reviewed by the International Association for Hospice & Palliative Care (IAHPC), whose member palliative care centres often include the use of the natural medications in their healthcare management plans, and thus can provide in-house verification of the information presented in Herbs and provide valuable feedback (IAHPC, 2012).
Braun and Cohen have tried to validate the information in Herbs within the limitations imposed by the system and Herbs remains a valuable resource for healthcare professionals seeking information on natural medicines.
Therapeutic Guidelines
Therapeutic Guidelines Limited (TGL) is a comprehensive compendium of common clinical disorders (TGL, 2011). Topics are chosen by the board of directors on advice by the healthcare community, and updates reflect changes in clinical practice. The Board of Directors includes members at the top of their professions, and thus are highly qualified to choose the topics to be included in the TGL. Each set of guidelines is developed by a multidisciplinary panel of experts chosen for their scholarship, relevant knowledge of the literature, and position in their respective fields. No conflicts of interests are allowed.
Members of the panel consult with other colleagues in their field for feedback and verification of the data, to enhance validity (Vinther et al., 2012). Any recommendation made by a panel member must be accompanied by a document outlining the rationale for the recommendation, which includes a literature review and external recommendations. Thus, the final manuscript cannot be attributed to a single author, but is rather the result of extensive internal review and revision by a multi-disciplinary team of experts, enhancing the validity of the data (Vinther et al., 2012). TGL aims to include evidence-based guidelines. However, not all disease management protocols used in clinical practice are supported by evidence from clinical trials (Vantongelen, Rotmensz, & van der Schueren, 1989); thus, recommendations must take other factors into consideration, including the known pathophysiological patterns of the disease, the clinical expertise of the multi-disciplinary members of the panel, comparative adverse effects of the various therapeutic protocols, relevance and feasibility in clinical practice, and collateral factors such as patient characteristics and comorbidities (Forster, 2012). Studies have shown that a comprehensive approach to evaluation of data increases their validity (Vinther et al., 2012), especially in light of the limitations of the peer-reviewed process (Berquist, 2012).
Proven therapeutics are favored over novel drugs (TGL, 2011). Therefore, due to the holistic approach in the choosing of a particular treatment protocol, and the lack of clinical studies in many cases, it is not appropriate to rate the recommendations on the basis of scientific evidence alone. Nevertheless, all the data in TGL are well referenced and the guidelines represent the current gold standards of treatment. Once the manuscript has passed internal evaluation, it is then released for external evaluation and endorsement by major relevant specialty healthcare organizations. Multiple evaluations increase the validity of information (Sweitzer & Cullen, 1994). The final evaluation takes place after the publication of the TGL. TGL takes a proactive approach and encourages feedback from its large network of users. Post-publication review is one of the most valuable tools in evaluating the validity of data (Herron, 2012). Thus, TGL follows an extensive editorial process to ensure the validity of the healthcare information it includes in its guidelines; as such, the validity of the information in TGL is rated high. Figure 2 provides a summary of the TGL editorial and evaluation process.
Figure 2. The TGL editorial and evaluation process
References
Australian Medicines Handbook (AMH). (2012). Adelaide: Australian Medicines
Handbook Pty Ltd.
Retrieved from https://shop.amh.net.au/
Berquist, T. H. (2012). Peer review: is the process broken? AJR Am J
Roentgenol.199(2):243
Braun, L. & Cohen, M. (2010). Herbs and Natural Supplements, 3rd Edition: An
evidence-based guide, London: Churchill Livingstone.
Retrieved from http://www.elsevierhealth.com.au/Health-Professions/
specialty/book/9780729539104/Herbs-and-Natural-Supplements/
