How do we know what evidence to use and to trust? The NHS tells us we should make evidence-based commissioning decisions but there is a lot of information available and we need to be assured of the quality of evidence before we can implement it.
We have provided below some guidance on reading and understanding research findings. This may be helpful if you are presented with a research paper that you would like to appraise so you can understand whether the findings are of value to you.
If you have published findings that you would like us to add to our repository, please contact us.
There is a large quantity of evidence available online and in print. The sheer quantity can be overwhelming and it is important to filter down the evidence to a manageable amount and to the most valuable and reliable evidence.
There are many sources of evidence and lots of these are collected into some of the large medical research databases.
NHS staff are able to register for an NHS OpenAthens account, which gives access to hundreds of health research articles via journals and healthcare databases. To register for an OpenAthens account visit the National Institute for Health and Care Excellence (NICE) website.
Many health research articles are available for free in open access journals, which do not usually require a login. You can search for journals through the NHS Knowledge and Library Hub.
National Institute for Health and Research (NIHR) funds much of the research which takes place in the UK, and publishes accounts of its research which can be accessed for free from the NIHR Journals Library.
In order to make sound evidence-based decisions, we understand that you will usually want to assess the body of evidence around a particular issue, rather than basing a decision on one paper or a small number of papers. In these cases we recommend that you access a systematic review which, as the name suggests, is a systematic consideration of all research papers published on a particular topic. A systematic review will exclude studies with poor methods, too-small sample sizes or other methodological failings, and will focus on the well-conducted studies. It will provide a summary of the findings and where possible will synthesize the findings of comparable trials into a meta-analysis – which effectively ‘adds together’ comparable studies’ findings to give an overall result. Systematic reviews can be found in various journals and databases, but an ideal place to check first is the Cochrane Database of Systematic Reviews which is an open access source of reviews which come from the Cochrane Collaboration, an international network of collaborators and recognised as the gold standard for high quality, trusted information. Also you can access Cochrane Podcasts.
You can then assess the systematic review using the CASP checklist to see if it is valid, reviewing the results to determine whether they will apply locally. Download the checklist through the Critical Appraisal Skills Programme (CASP) website.
Finding suitable evidence can be difficult. The databases are huge and sometimes article titles do not fully describe their content. A good search strategy can help you to identify relevant documents.
Avon Primary Care Research Collaborative, together with their local Academic Health Science Network and the NIHR, have put together a toolkit which you may find useful in finding, appraising and applying evidence.
You can find more information about this on the 'reading and understanding research findings' and 'recommended reading' sections of our repository.
Bradford District and Craven
Resources on finding evidence can be accessed via the following websites: Bradford Teaching Hospitals NHS Foundation Trust, Bradford District Care NHS Foundation Trust and Airedale NHS Foundation Trust. These Trusts work collaboratively and provide a comprehensive service for all NHS staff and students across the Bradford and Airedale District. BTHFT will carry out literature searches on behalf of eligible NHS staff. This service is offered to staff at Bradford Teaching Hospitals NHS Foundation Trust as well as to GPs, dentists, optometrists, primary care and ICB staff in Bradford and Airedale. To make a request please visit the BTHFT library services page.
Calderdale and Kirklees
Library services are available through Calderdale and Huddersfield NHS Foundation Trust. Please contact their Library Knowledge Services team to discuss.
Leeds
The NHS Leeds library service - there is a wealth of information on the NHS Leeds library service website that you might find helpful.
Wakefield
The Mid Yorkshire Hospitals NHS Trust library can be contacted to advise on services available to staff.
In the below video West Yorkshire Integrated Care Board (ICB) lead Rob Webster explores why ICBs need to use knowledge and evidence:
A good research article should always provide enough information so that the reader could, if they chose, recreate the study. This means the method, sampling strategy and all other elements of the study should be clearly described. This information also allows you to identify whether the study was well-run, whether an appropriate sampling strategy was used, what the statistical power of the study was, whether the study had any bias or conflicts of interest.
There is considered to be a hierarchy of evidence in clinical trials which is important as you can recognise whether the study type is of a high standard.
Health Knowledge describes the hierarchy of evidence, with randomised controlled trials being the ‘gold standard’ of a research study as a randomly selected group is studied with groups being randomised to different treatments. Control groups are used to evidence that any change is not due to other external factors. Any biases or confounding factors should be highlighted and controlled for where possible.
Above the randomised controlled trial in the hierarchy is a meta-analysis or systematic review which involves reviewing and, where possible, compiling, the results of a number of randomised controlled trials and sometimes other types of study. This highlights trends across multiple studies and can also highlight any publication bias, where some negative studies may not have been published.
As mentioned in the above article, this does not mean that other types of study are never useful as they can be. The hierarchy of evidence also does not incorporate qualitative research, as this methodology does not usually attempt to be generalizable or randomised, so cannot be approached in the same way. However qualitative research has a place in research - see this Nursing Times article for a discussion of the value of qualitative research.
Bear in mind that, although the label ‘randomised controlled trial’ indicates a high quality methodology, the detail of the study should be examined to make sure it was well-conducted and attempted to reduce bias.
Reading and understanding research findings, sometimes called ‘critical appraisal’, is a useful skill and can help you to recognise good research. Critical appraisal is useful for all types of research paper, including qualitative research.
Understanding health research talks you through doing a critical appraisal which can help you to identify whether a piece of research has been properly and rigorously carried out – in other words; is it worth reading, and trying to apply the results to your context? The tool walks you through the paper with a series of questions to help you to review the paper’s quality, and is very straightforward to use with hints, explanations and things to think about at each stage. You can also stop at any point and get a summary.
For training please see the following links:
The Critical Appraisal Skills Programme (CASP) also provides useful resources (tools and checklists) for reading and appraising research papers.
You may also like to read Trisha Greenhalgh’s book How to read a paper which is available in many libraries.
As we are all short of time, it can be useful to use another source which has already done the critical appraisal for us. This is why we recommend the Cochrane Library of systematic reviews and also NICE guidance. Watch a video on systematic reviews here.There is also information on NICE about how they put together the guidelines. There is a useful site the National Elf Service which produces summaries of evidence in a number of subject areas. However, it is useful to have the ability to appraise a research paper as you may be presented with a piece of research and asked for a response.
Glossary
A glossary of some of the key terms used in research is available from the National Institute of Health Research.
Evidence sources and library services
Cochrane Library of systematic reviews
National Elf Service – critically appraised expert commentaries
Bradford District Care Trust Library
Mid Yorkshire Hospitals NHS Trust Library and Information Service
Calderdale and Huddersfield NHS Foundation Trust Library Services
Critical appraisal
Leeds Libraries for Health Critical Appraisal Training
Critical Appraisal Skills Programme (CASP)
How to read a scientific paper
BBC Radio 4’s More or Less looks critically at statistics reported in the news.
This online community (WeCATS – Critical Appraisal Twitter Session) regularly critically appraises research papers via a Twitter chat.
Guides
Research methodologies
Statistical analysis and research methodology
Conducting research
Trial methodology – other training
Short courses in clinical research
Children and clinical research
Effectively promoting your research
Useful books
Understanding Clinical Papers Paperback – 8 Nov 2013 by David Bowers, Allan House, David Owens, Bridgette Bewick
Getting Started in Health Research - 10 Jun 2011 by David Bowers, Allan House, David Owens
How to Read a Paper: The Basics of Evidence-Based Medicine - 28 Mar 2014 by Trisha Greenhalgh
- WAAW 2022: Antimicrobial stewardship and Lowering AntiMicrobial Prescribing (LAMP) - 18th November 2022. Written by Tasneem Khan.
- WAAW Day 7: Lowering AntiMicrobial Prescribing (LAMP) - 24th November 2021. Written by Tasneem Khan.
- WAAW Day 6: Taking AMR seriously in Leeds - 23rd November 2021. Visit Seriously Resistant.
- WAAW Day 5: Antibiotics and the Environment: What's the Link? - 22nd November 2021. Written by Costas Vasiliou.
- WAAW Day 4: Prescribing of antibiotics in Calderdale and West Yorkshire throughout the pandemic and issues surrounding remote prescribing of empirical antibiotics - 21st November 2021. Written by Nicola Booth.
- WAAW Day 3: A Patient's Story - 20th November 2021. Written by John.
- WAAW Day 2: The Origins and Resistance of Antimicrobials: A Community Pharmacist’s Advice to AMR - 19th November 2021. Written Robina Rowley-Conwy and Farial Majid.
- WAAW Day 1: World Antimicrobial Awareness Week & Lowering AntiMicrobial Prescribing (LAMP) - 18th November 2021. Written by Paul Carder.
- COVID-19: One Year On - 31st March 2021. Written by Stella Johnson.
- The Health Inequalities Academy - 10th February 2021. Written by Rose Dewey.
- LAMP in lockdown - the digital transformation - 30th June 2020. Written by Imran Mohammed.
- Increasing our primary care research activity – a practice manager’s perspective - 24th June 2020. Written by Nick Gwatkin.
- Research in COVID-19 - 15th April 2020. Written by Rose Dewey.
- Primary Care Networks: an emerging opportunity for research in general practice? - 29th January 2020. Written by Damian Reynolds.
- Lowering Anti-Microbial Prescribing (LAMP) - 19th November 2019. Written by Paul Carder.
- The 4th Annual International Symposium on Advancing the Science and Impact of Audit & Feedback - 30th July 2019. Written by Paul Carder.
- Thoughts following 2019’s R&D Forum - 31st May 2019. Written by Rose Dewey.
- A PhD Researcher’s Experience of WY R&D - 25th February 2019. Written by Kathryn Chater.
- A week in the (work) life! - 21st August 2018. Written by Rose Dewey.
- Research Delivery Officer Introduction - 30th July 2018. Written by Damian Reynolds.
- Research Co-ordinator Introduction - 20th July 2018. Written by Imran Mohammed.
- Reflecting on this year’s R&D Forum - 4th June 2018. Written by Rebecca Harper.
- Research in General Practice - 22nd March 2018 - Written by Emma Neil.
- A look back at the history of medical research - 12th February 2018 - Written by Emma Neil.
- Research Governance Lead Introduction - 30th January 2018 - Written by Rebecca Harper.
- Research Capability Funding Case Study - 31st August 2017. Written by Dr Matt Mulvey.
