6 Questions to ask yourself when reading research articles in public health

Students and professionals in global and public health spend a large amount of time gathering online evidence-based information to support their work (e.g. writing research papers, implementing health programs, developing health policies, etc.).

As it becomes easier to publish a paper in a journal, and with the amount of information out there, how do you know which research papers to trust? To help you out, we’ve compiled a checklist with six questions to ask yourself when reading the next paper.

And please let us know what key tips we are missing by sharing in the comments!

1. Did anyone pay for this research?

Specifying who paid for the research projects as well as the researcher’s fees helps to identify if there is any potential for bias and any conflict in interests. You can find this information at the bottom of a research article under “funding”.

For instance, several studies have shown that research on drugs and medical devices have a greater likelihood to show positive results when they have been funded or sponsored by the manufacturer (Wareham et al., 2017).

Keep in mind though that the fact that the project has been funded by a pharmaceutical company does not necessarily mean that the researcher is being paid by this company.

Furthermore, most clinical trials follow very strict guidelines to avoid issues such as sponsorship bias. One critical point is that the original clinical protocol is registered in a site such as clinicaltrials.gov, so there is no chance to change the protocol afterwards.

2. Who is it written by?

Assessing who the author(s) are of the research paper and their credentials (e.g. degrees) helps to understand what authority and qualification they have to write on the topic. Check out what institutional affiliations they have which help to build their credibility and have a look at the author(s)’ previous articles and expertise on the topic.

However, not having previously published articles does not always indicate a lack of expertise. Many great researchers from low- and middle-income countries (LMICs) are not well-represented in global health research and face many barriers in academia and for publishing (including discrimination, inadequate funding and support for publishing from their institutions, language barriers and underrepresentation of women and minority groups). So yes, check people’s experience, but also check the article’s quality.

It’s a good idea to check that the author is analyzing the topic from an objective point of view and includes more than one perspective on the issue. Most articles these days include a section in the discussion on study gaps and alternative interpretations.

3. When and where was it published?

Depending on the research topic, evidence on a certain subject can change rapidly over the years. It is important to check the date or year that the research was published, (in many cases, the more recent the better) to ensure the evidence has not become obsolete or worse, been refuted.

Nowadays, with so many paid journals, it’s become easier to have a research paper published which makes it more challenging to differentiate between high and low quality research. “Journal ranking” assesses the academic journals’ impact and quality. It also assesses how difficult it is to get published in that field and thus has associate prestige. Keep in mind, however, that ranking is not everything as quality research can be found in many journals which are not necessarily highly ranked.

4. Who else agrees with the paper?

Check who are experts in that field of research and see what they have to say about the paper you are looking at. You can, for example, check how many times that research has been mentioned and cited in other papers, by other researchers. Sometimes, for key articles, journals will also run commentaries by other experts when publishing the article. Check for those as well.

5. What’s the sample size and who is represented?

The sample of participants needs to be a good representation of the population studied. You can start by looking at the sample size and the sample diversity.

The sample size is based on how many people the condition affects, for example, if the research is based on a medical condition that is extremely rare, a large sample size would not be as feasible. Whereas if it is based on a more common health condition, such as obesity, a larger sample size should be expected.

Variation is also an important characteristic of the quality of a sample. For example, if a study looks at obesity in French adults and the population sample includes mostly white individuals from Paris aged between 25-35 years old, it does not do a good job of representing the diversity of the population of French adults.

You should also look at how the sample has been selected. For field studies and surveys, for example, random selection takes a random portion of the entire population and each individual has an equal chance of being selected. This method of sampling can enable a representation of the total population.

However, for other types of studies, such as clinical trials, true random sampling is not possible as people need to be invited to participate, and thus some sort of selection and self selection happens. In that case, all patients who fill out the inclusion criteria are invited to participate, in a sample of selected hospitals, and then can be randomly allocated to intervention and control groups. Check to see if efforts have been made to have a diversity of patients in the sample. In many studies, women and/or minorities tend not to be as well represented.

6. What methods did they use?

In deciding what research is of quality you can use the hierarchy of evidence or also called the evidence pyramid.

- On top of the pyramid you will find systematic reviews and meta-analyses, which take into account all high quality research available on a topic, in a methodologically sound way (if well conducted).
- Then, we have “randomized control trials” which also provide high levels of evidence as in these studies participants are allocated into a control or intervention group randomly. Most RCTs also use “double blinding” where both researcher and participant are unaware if they are in the control or intervention group, reducing biases.
- After this, from higher to lower, there are cohort studies, case-control studies, case series/reports, these have their uses and are sometimes the only option to carry out certain studies.
  - Cohort studies: In this type of study, a group of people is observed over a period of time (e.g. several months or years) to assess, for example, the frequency of a certain condition. Typically, you will find one group that is exposed to a risk factor and one group which isn’t. For example, the study tries to determine the effect of heavy alcohol consumption on liver disease, where one group drinks alcohol frequently and the other not.
  - Case-control studies: Here, you will find one group of people with a certain health condition and one without, both groups have similar characteristics (e.g. age, sex, ethnicity). This is often a retrospective type of study as researchers then try to find out which risk factors they have been exposed to in the past which explain their current health state.
  - Case series/reports: A case report is a report of the diagnosis, treatment, or response to treatment of one patient whereas a case series is a multitude of such reports (analyzing the same treatment).

- At the bottom of the pyramid you will find texts such as expert opinions and editorials, which have the lowest level but can still be used to understand different perspectives on research topics.

All these different types of methodologies and studies have to follow certain guidelines to improve their quality. Here are some examples:

PRISMA the Preferred Reporting Items for Systematic reviews and Meta-Analyses has a checklist and flowchart for systematic reviews and meta-analyses. This enables transparency as it indicates why a review was done, what the author did (e.g. how they found evidence), and what evidence they found.
CONSORT checklist and flowchart for RCTs; Consolidated Standards of Reporting Trials. This method offers transparency and reports how a trial was designed, analyzed, and interpreted.
STROBE checklist for cohort and case-control studies; STrengthening the Reporting of Observational studies in Epidemiology. As indicated in the name, the STROBE guidelines aim to enhance reporting of data by providing recommendations to authors on how to report what was planned, done, and found in a study and what these findings mean.
CARE checklist for CAse REports; these guidelines help authors to increase transparency in their research, decrease bias, and report clearly what treatment works for which patient.

We hope this gives you a better idea of what to look at when reading research papers in global and public health! Do share in the comments what other guidelines you follow when looking at articles!