How do we know what evidence to use and to trust? The NHS tells us we should make evidence-based commissioning decisions but there is a lot of information available and we need to be assured of the quality of evidence before we can implement it.
We have provided below some guidance on reading and understanding research findings. This may be helpful if you are presented with a research paper that you would like to appraise so you can understand whether the findings are of value to you.
If you have published findings that you would like us to add to our repository, please contact Gemma.
There is a large quantity of evidence available online and in print. The sheer quantity can be overwhelming and it is important to filter down the evidence to a manageable amount and to the most valuable and reliable evidence.
There are many sources of evidence and lots of these are collected into some of the large medical research databases.
NHS staff are able to register for an NHS OpenAthens account, which gives access to hundreds of health research articles via journals and healthcare databases such as MEDLINE, EMBASE, CINAHL and the British Nursing Index. To register for an OpenAthens account visit NHS Evidence.
Many health research articles are available for free in open access journals, which do not usually require a login. You can search for journals through The National Institute for Health and Care Excellence.
National Institute for Health Research (NIHR) funds much of the research which takes place in the UK, and publishes accounts of its research which can be accessed for free from the NIHR Journals Library.
In order to make sound evidence-based decisions, we understand that you will usually want to assess the body of evidence around a particular issue, rather than basing a decision on one paper or a small number of papers. In these cases we recommend that you access a systematic review which, as the name suggests, is a systematic consideration of all research papers published on a particular topic. A systematic review will exclude studies with poor methods, too-small sample sizes or other methodological failings, and will focus on the well-conducted studies. It will provide a summary of the findings and where possible will synthesize the findings of comparable trials into a meta-analysis – which effectively ‘adds together’ comparable studies’ findings to give an overall result. Systematic reviews can be found in various journals and databases, but an ideal place to check first is the Cochrane Database of Systematic Reviews which is an open access source of reviews which come from the Cochrane Collaboration, an international network of collaborators and recognised as the gold standard for high quality, trusted information. Also you can access Cochrane Podcasts.
You can then assess the systematic review using the CASP checklist to see if it is valid, reviewing the results to determine whether they will apply locally. Download the checklist through the Critical Appraisal Skills Programme (CASP) website.
Finding suitable evidence can be difficult. The databases are huge and sometimes article titles do not fully describe their content. A good search strategy can help you to identify relevant documents.
Avon Primary Care Research Collaborative, together with their local Academic Health Science Network and the NIHR, have put together a toolkit which you may find useful in finding, appraising and applying evidence.
You can find more information about this on the 'reading and understanding research findings' and 'recommended reading' sections of our repository.
Airedale, Wharfedale and Craven CCG, Bradford City CCG & Bradford Districts CCG
Resources on finding evidence can be accessed via the following websites: Bradford Teaching Hospitals NHS Foundation Trust, Bradford District Care NHS Foundation Trust and Airedale NHS Foundation Trust. These Trusts work collaboratively and provide a comprehensive service for all NHS staff and students across the Bradford and Airedale District. BTHFT will carry out literature searches on behalf of eligible NHS staff. This service is offered to staff at Bradford Teaching Hospitals NHS Foundation Trust as well as to GPs, dentists, optometrists, primary care and commissioning staff in Bradford and Airedale. To make a request please visit the BTHFT library services page.
Calderdale CCG, Greater Huddersfield CCG & North Kirklees CCG
Library services are available through Calderdale and Huddersfield NHS Foundation Trust. Please contact their Library Knowledge Services team to discuss.
Leeds North CCG, Leeds South and East CCG & Leeds West CCG
The NHS Leeds library service has a page dedicated to resources that can be accessed by commissioners. There is a wealth of information on the NHS Leeds library service website that you might find helpful. There is a section of the website that links to some of the latest evidence available to help you keep up to date with developments.
The Mid Yorkshire Hospitals NHS Trust library can be contacted to advise on services available to CCG staff.
A good research article should always provide enough information so that the reader could, if they chose, recreate the study. This means the method, sampling strategy and all other elements of the study should be clearly described. This information also allows you to identify whether the study was well-run, whether an appropriate sampling strategy was used, what the statistical power of the study was, whether the study had any bias or conflicts of interest.
There is considered to be a hierarchy of evidence in clinical trials which is important as you can recognise whether the study type is of a high standard.
Health Knowledge describes the hierarchy of evidence, with randomised controlled trials being the ‘gold standard’ of a research study as a randomly selected group is studied with groups being randomised to different treatments. Control groups are used to evidence that any change is not due to other external factors. Any biases or confounding factors should be highlighted and controlled for where possible.
Above the randomised controlled trial in the hierarchy is a meta-analysis or systematic review which involves reviewing and, where possible, compiling, the results of a number of randomised controlled trials and sometimes other types of study. This highlights trends across multiple studies and can also highlight any publication bias, where some negative studies may not have been published.
As mentioned in the above article, this does not mean that other types of study are never useful as they can be. The hierarchy of evidence also does not incorporate qualitative research, as this methodology does not usually attempt to be generalizable or randomised, so cannot be approached in the same way. However qualitative research has a place in research - see this Nursing Times article for a discussion of the value of qualitative research.
Bear in mind that, although the label ‘randomised controlled trial’ indicates a high quality methodology, the detail of the study should be examined to make sure it was well-conducted and attempted to reduce bias.
Reading and understanding research findings, sometimes called ‘critical appraisal’, is a useful skill and can help you to recognise good research. Critical appraisal is useful for all types of research paper, including qualitative research.
Understanding health research talks you through doing a critical appraisal which can help you to identify whether a piece of research has been properly and rigorously carried out – in other words; is it worth reading, and trying to apply the results to your context? The tool walks you through the paper with a series of questions to help you to review the paper’s quality, and is very straightforward to use with hints, explanations and things to think about at each stage. You can also stop at any point and get a summary.
For training please see the following links:
The Critical Appraisal Skills Programme (CASP) also provides useful resources (tools and checklists) for reading and appraising research papers.
You may also like to read Trisha Greenhalgh’s book How to read a paper which is available in many libraries or bookshops.
Useful terms in critical appraisal are explained in the glossary available here.
As we are all short of time, it can be useful to use another source which has already done the critical appraisal for us. This is why we recommend the Cochrane Library of systematic reviews and also NICE guidance. Watch a video on systematic reviews here.There is also information on NICE about how they put together the guidelines. There is a useful site the National Elf Service which produces summaries of evidence in a number of subject areas. However, it is useful to have the ability to appraise a research paper as you may be presented with a piece of research and asked for a response.
Evidence based decision making
An e-learning course from Health Education England (HEE) and NICE offers practical advice on using evidence in daily decision-making. Follow this link to find out more and register.
A glossary of some of the key terms used in research is available from the National Institute of Health Research.
Library and information services
Sources of evidence
Critical Appraisal Skills – training courses are available across the region and dates will be posted in our events section.
BBC Radio 4’s More or Less looks critically at statistics reported in the news.
This online community (WeCATS – Critical Appraisal Twitter Session) regularly critically appraises research papers via a Twitter chat.
by David Bowers, Allan House, David Owens, Bridgette Bewick
by David Bowers, Allan House, David Owens
How to Read a Paper: The Basics of Evidence-Based Medicine - 28 Mar 2014
by Trisha Greenhalgh