Paper 15: Livestock ownership, household food security and childhood anaemia in rural Ghana

References

Christian AK, Wilson ML, Aryeetey RNO, Jones AD (2019) Livestock ownership, household food security and childhood anaemia in rural Ghana. PLoS ONE 14(7): e0219310. https://doi.org/10.1371/journal.pone.0219310

Disclosure

This reproducibility project was conducted to the best of our ability, with careful attention to statistical methods and assumptions. The research team comprises four senior biostatisticians (three of whom are accredited), with 20 to 30 years of experience in statistical modelling and analysis of healthcare data. While statistical assumptions play a crucial role in analysis, their evaluation is inherently subjective, and contextual knowledge can influence judgements about the importance of assumption violations. Differences in interpretation may arise among statisticians and researchers, leading to reasonable disagreements about methodological choices.

Our approach aimed to reproduce published analyses as faithfully as possible, using the details provided in the original papers. We acknowledge that other statisticians may have differing success in reproducing results due to variations in data handling and implicit methodological choices not fully described in publications. However, we maintain that research articles should contain sufficient detail for any qualified statistician to reproduce the analyses independently.

Methods used in our reproducibility analyses

There were two parts to our study. First, 100 articles published in PLOS ONE were randomly selected from the health domain and sent for post-publication peer review by statisticians. Of these, 95 included linear regression analyses and were therefore assessed for reporting quality. The statisticians evaluated what was reported, including regression coefficients, 95% confidence intervals, and p-values, as well as whether model assumptions were described and how those assumptions were evaluated. This report provides a brief summary of the initial statistical review.

The second part of the study involved reproducing linear regression analyses for papers with available data to assess both computational and inferential reproducibility. All papers were initially assessed for data availability and the statistical software used. From those with accessible data, the first 20 papers (from the original random sample) were evaluated for computational reproducibility. Within each paper, individual linear regression models were identified and assigned a unique number. A maximum of three models per paper were selected for assessment. When more than three models were reported, priority was given to the final model or the primary models of interest as identified by the authors; any remaining models were selected at random.

To assess computational reproducibility, differences between the original and reproduced results were evaluated using absolute discrepancies and rounding error thresholds, tailored to the number of decimal places reported in each paper. Results for each reported statistic, e.g., regression coefficient, were categorised as Reproduced, Incorrect Rounding, or Not Reproduced, depending on how closely they matched the original values. Each paper was then classified as Reproduced, Mostly Reproduced, Partially Reproduced, or Not Reproduced. The mostly reproduced category included cases with minor rounding or typographical errors, whereas partially reproduced indicated substantial errors were observed, but some results were reproduced.

For models deemed at least partially computationally reproducible, inferential reproducibility was further assessed by examining whether statistical assumptions were met and by conducting sensitivity analyses, including bootstrapping where appropriate. We examined changes in standardized regression coefficients, which reflect the change in the outcome (in standard deviation units) for a one standard deviation increase in the predictor. Meaningful differences were defined as a relative change of 10% or more, or absolute differences of 0.1 (moderate) and 0.2 (substantial). When non-linear relationships were identified, inferential reproducibility was assessed by comparing model fit measures, including R², Akaike Information Criterion (AIC), and Bayesian Information Criterion (BIC). When the Gaussian distribution was not appropriate for the dependent variable, alternative distributions were considered, and model fit was evaluated using AIC and BIC.

Results from the reproduction of the Christian et al. (2019) paper are presented below. An overall summary of results is presented first, followed by model-specific results organised within tab panels. Within each panel, the Original results tab displays the linear regression outputs extracted from the published paper. The Reproduced results tab presents estimates derived from the authors’ shared data, along with a comprehensive assessment of linear regression assumptions. The Differences tab compares the original and reproduced models to assess computational reproducibility. Finally, the Sensitivity analysis tab evaluates inferential reproducibility by examining whether identified assumption violations meaningfully affected the results.

Summary from statistical review

The paper examines the relationship between household livestock ownership and anaemia in children in rural Ghana. The dependent variable in the linear regression was animal-source food (ASF) consumption, measured as a count of the number of different ASF types consumed by each child the week before the interview. Simple descriptive statistics for ASF consumption indicated that the mean and standard deviation were both 1.2, with no discussion of the distribution or normality of residuals, which raised the question of whether a Poisson regression would have been more appropriate.

Data availability and software used

Data were available in the Supporting Information and on Figshare in Stata format, with a partial data dictionary. The authors have included a full dataset with 1399 variables. Stata was used for analyses of linear regression models.

Regression sample

Two regressions were identified, both multivariable with ASF Diversity as the outcome. A mediation analysis was also identified, but was not considered because it had a different purpose: to provide direct and indirect effects.

Computational reproducibility results

The linear regression results for the paper could not be reproduced due to ambiguity in the ASF outcome variable: there are two possible options, ASF and ASF2. The simple mean and standard deviation for ASF by livestock ownership were reproduced. Identification of demographic variables for adjustment seemed reasonable. However, the results for the number of animals owned do not align with the published findings. Attempts to dichotomise the three livestock categories (poultry, sheep & goats, pigs) did not produce results that resemble the original analysis. Therefore, this paper was not found to be computationally reproducible.

Inferential reproducibility results

As this paper was not computationally reproducible, inferential reproducibility was not considered, since the original analyses could not be reproduced and therefore, statistical assumptions could not be meaningfully compared or interpreted.

Recommended Changes

• Ensure variables were used in analyses and can be matched to the data dictionary.
• As the dataset contained many additional variables, the authors should provide a minimal dataset including only the variables used in the paper.
• Evaluate the assumptions of the linear regression models by examining residuals, identifying influential outliers, and assessing multicollinearity among predictors. If any assumptions are violated, address them using appropriate methods.

Model 1

Original results

Model results for ASF Diversity

Term	B	SE	Lower	Upper	t	p-value
Intercept
poultry	0.02		0.01	0.03		<0.05
Sheep_goats
pigs
SE = Standard error; Lower = lower confidence interval; Upper = upper confidence interval.

Fit statistics for ASF Diversity

R	R2	R2Adj	AIC	RMSE	F	DF1	DF2	p-value
R2 Adj = Adjusted R2; AIC = Akaike Information Criterion; RMSE = The Root Mean Squared Error; DF1 = Degrees of freedom for the model; DF2 = Degrees of freedom for the residuals.

ANOVA table for ASF Diversity

Term	SS	DF	MS	F	p-value
poultry
Sheep_goats
pigs
Residuals
SS = Sum of Squares; DF = Degrees of freedom; MS = Mean Square.

Reproduced results

Model results for ASF Diversity

Term	B	SE	Lower	Upper	t	p-value
Intercept	1.177	0.075	1.030	1.325	15.697	<0.001
poultry	0.005	0.024	−0.042	0.051	0.190	0.8496
Sheep_goats	0.017	0.014	−0.011	0.044	1.170	0.2430
pigs	0.014	0.033	−0.052	0.079	0.420	0.6750
SE = Standard error; Lower = lower confidence interval; Upper = upper confidence interval.

Fit statistics for ASF Diversity

R	R2	R2Adj	AIC	RMSE	F	DF1	DF2	p-value
0.075	0.006	−0.004	958.412	1.176	0.555	3	296	0.6452
R2 Adj = Adjusted R2; AIC = Akaike Information Criterion; RMSE = The Root Mean Squared Error; DF1 = Degrees of freedom for the model; DF2 = Degrees of freedom for the residuals.

ANOVA table for ASF Diversity

Term	SS	DF	MS	F	p-value
poultry	0.050	1	0.050	0.036	0.8496
Sheep_goats	1.916	1	1.916	1.368	0.2430
pigs	0.247	1	0.247	0.176	0.6750
Residuals	414.585	296	1.401
SS = Sum of Squares; DF = Degrees of freedom; MS = Mean Square; Calculated using type III SS.

Visualisation of regression model

The blue line shows the best line of fit with shading representing 95% confidence intervals, while holding all other covariates constant. The dots show partial residuals, which reflect the observed data adjusted for all other predictors except the one being plotted.

Differences

Forest plot showing Original and Reproduced coefficients and 95% confidence intervals for ASF Diversity

Change in regression coefficients

term	O_B	R_B	Change.B	reproduce.B
Intercept		1.1771
poultry	0.02	0.0045	−0.0155	Not Reproduced
Sheep_goats		0.0165
pigs		0.0140
O_B = original B; R_B = reproduced B; Change.B = change in R_B - O_B; Reproduce.B = B reproduced.

Change in lower 95% confidence intervals for coefficients

term	O_lower	R_lower	Change.lci	Reproduce.lower
Intercept		1.0295
poultry	0.01	−0.0422	−0.0522	Not Reproduced
Sheep_goats		−0.0113
pigs		−0.0515
O_lower = original lower confidence interval; R_lower = reproduced lower confidence interval; change.lci = change in R_lower - O_lower; Reproduce.lower = lower confidence interval reproduced.

Change in upper 95% confidence intervals for coefficients

term	O_upper	R_upper	Change.uci	Reproduce.upper
Intercept		1.3247
poultry	0.03	0.0512	0.0212	Not Reproduced
Sheep_goats		0.0443
pigs		0.0794
O_upper = original upper confidence interval; R_upper = reproduced upper confidence interval; change.uci = change in R_upper - O_upper; Reproduce.upper = upper confidence interval reproduced.

Change in p-values

Term	O_p	R_p	Change.p	Reproduce.p	SigChangeDirection
Intercept		0.0490
poultry	<0.05	0.8496	0.8006	Not Reproduced	Sig to non-sig, B changes direction
Sheep_goats		0.2430
pigs		0.6750
O_p = original p-value; R_p = reproduced p-value; Changep = change in p-value R_p - O_p; Reproduce.p = p-values reproduced. SigChangeDirection = statistical significance and B change between original and reproduced models. Note, p-values that were <0.05 were set to 0.049 for the purposes of comparison.

Results for p-values

• The p-value for this model was not reproduced.

Conclusion computational reproducibility

This model was not computationally reproducible.

Sensitivity analysis

As this model was not computationally reproducible, inferential reproducibility was not considered, since the original analyses could not be reproduced and therefore, statistical assumptions could not be meaningfully compared or interpreted.

Model 2

Original results

Model results for ASF Diversity

Term	B	SE	Lower	Upper	t	p-value
Intercept
poultry	0.02	0.01	0.01	0.03		<0.05
Sheep_goats	0.02	0.02	−0.01	0.06		0.23
pigs	−0.02	0.01	−0.05	0.01		0.12
Owns_any_livestock:
Yes – No	−0.36	0.17	−0.70	−0.02		0.04
Household_wealth:
Medium – Low	0.56	0.16	0.25	0.88		<0.05
High – Low	0.63	0.17	0.30	0.96		<0.05
household_size	0.02	0.03	−0.03	0.08		0.36
Sex_of_household_head:
Female – Male	0.31	0.22	−0.13	0.74		0.17
Caregiver_Marital_Status:
Married – Single	0.57	0.18	0.22	0.93		<0.05
Education:
Preschool/Primary – No	0.38	0.16	0.06	0.70		0.02
Secondary – No	0.41	0.17	0.08	0.75		0.02
Child_sex:
Male – Female	−0.12	0.14	−0.40	0.15		0.39
childage	−0.01	0.01	−0.02	0.01		0.30
SE = Standard error; Lower = lower confidence interval; Upper = upper confidence interval.

Fit statistics for ASF Diversity

R	R2	R2Adj	AIC	RMSE	F	DF1	DF2	p-value
R2 Adj = Adjusted R2; AIC = Akaike Information Criterion; RMSE = The Root Mean Squared Error; DF1 = Degrees of freedom for the model; DF2 = Degrees of freedom for the residuals.

ANOVA table for ASF Diversity

Term	SS	DF	MS	F	p-value
poultry
Sheep_goats
pigs
Owns_any_livestock
Household_wealth
household_size
Sex_of_household_head
Caregiver_Marital_Status
Education
Child_sex
childage
Residuals
SS = Sum of Squares; DF = Degrees of freedom; MS = Mean Square.

Reproduced results

Model results ASF Diversity

Term	B	SE	Lower	Upper	t	p-value
Intercept	0.300	0.383	−0.454	1.054	0.784	0.4339
poultry	−0.004	0.023	−0.050	0.042	−0.173	0.8625
Sheep_goats	0.028	0.016	−0.003	0.059	1.764	0.0788
pigs	0.002	0.033	−0.063	0.067	0.069	0.9448
Owns_any_livestock:
Yes – No	−0.164	0.157	−0.473	0.146	−1.041	0.2986
Household_wealth:
Medium – Low	0.520	0.164	0.197	0.844	3.165	0.0017
High – Low	0.608	0.163	0.289	0.928	3.743	<0.001
household_size	0.011	0.025	−0.037	0.060	0.460	0.6462
Sex_of_household_head:
Female – Male	0.313	0.199	−0.079	0.705	1.574	0.1167
Caregiver_Marital_Status:
Married – Single	0.487	0.209	0.076	0.899	2.334	0.0203
Education:
Preschool/Primary – No	0.365	0.162	0.046	0.684	2.251	0.0251
Secondary – No	0.435	0.172	0.097	0.773	2.532	0.0119
Child_sex:
Male – Female	−0.106	0.135	−0.371	0.160	−0.784	0.4336
childage	−0.005	0.006	−0.017	0.006	−0.869	0.3855
SE = Standard error; Lower = lower confidence interval; Upper = upper confidence interval.

Model fit for ASF Diversity

R	R2	R2Adj	AIC	RMSE	F	DF1	DF2	p-value
0.326	0.106	0.066	946.382	1.114	2.616	13	286	0.0019
R2 Adj = Adjusted R2; AIC = Akaike Information Criterion; RMSE = The Root Mean Squared Error; DF1 = Degrees of freedom for the model; DF2 = Degrees of freedom for the residuals.

ANOVA table for ASF Diversity

Term	SS	DF	MS	F	p-value
poultry	0.039	1	0.039	0.030	0.8625
Sheep_goats	4.054	1	4.054	3.112	0.0788
pigs	0.006	1	0.006	0.005	0.9448
Owns_any_livestock	1.413	1	1.413	1.084	0.2986
Household_wealth	21.156	2	10.578	8.120	<0.001
household_size	0.275	1	0.275	0.211	0.6462
Sex_of_household_head	3.226	1	3.226	2.476	0.1167
Caregiver_Marital_Status	7.095	1	7.095	5.446	0.0203
Education	9.868	2	4.934	3.787	0.0238
Child_sex	0.801	1	0.801	0.615	0.4336
childage	0.984	1	0.984	0.755	0.3855
Residuals	372.603	286	1.303
SS = Sum of Squares; DF = Degrees of freedom; MS = Mean Square; Calculated using type III SS.

Visualisation of regression model

The blue line shows the best line of fit with shading representing 95% confidence intervals, while holding all other covariates constant. The dots show partial residuals, which reflect the observed data adjusted for all other predictors except the one being plotted.

Differences

Forest plot showing Original and Reproduced coefficients and 95% confidence intervals for ASF Diversity

Change in regression coefficients

term	O_B	R_B	Change.B	reproduce.B
Intercept		0.3000
poultry	0.02	−0.0040	−0.0240	Not Reproduced
Sheep_goats	0.02	0.0279	0.0079	Incorrect Rounding
pigs	−0.02	0.0023	0.0223	Not Reproduced
Owns_any_livestock:
Yes – No	−0.36	−0.1636	0.1964	Not Reproduced
Household_wealth:
Medium – Low	0.56	0.5202	−0.0398	Not Reproduced
High – Low	0.63	0.6084	−0.0216	Not Reproduced
household_size	0.02	0.0113	−0.0087	Incorrect Rounding
Sex_of_household_head:
Female – Male	0.31	0.3133	0.0033	Reproduced
Caregiver_Marital_Status:
Married – Single	0.57	0.4874	−0.0826	Not Reproduced
Education:
Preschool/Primary – No	0.38	0.3649	−0.0151	Not Reproduced
Secondary – No	0.41	0.4349	0.0249	Not Reproduced
Child_sex:
Male – Female	−0.12	−0.1057	0.0143	Not Reproduced
childage	−0.01	−0.0051	0.0049	Reproduced
O_B = original B; R_B = reproduced B; Change.B = change in R_B - O_B; Reproduce.B = B reproduced.

Change in lower 95% confidence intervals for coefficients

term	O_lower	R_lower	Change.lci	Reproduce.lower
Intercept		−0.4535
poultry	0.01	−0.0499	−0.0599	Not Reproduced
Sheep_goats	−0.01	−0.0032	0.0068	Incorrect Rounding
pigs	−0.05	−0.0626	−0.0126	Not Reproduced
Owns_any_livestock:
Yes – No	−0.70	−0.4728	0.2272	Not Reproduced
Household_wealth:
Medium – Low	0.25	0.1967	−0.0533	Not Reproduced
High – Low	0.30	0.2885	−0.0115	Not Reproduced
household_size	−0.03	−0.0371	−0.0071	Incorrect Rounding
Sex_of_household_head:
Female – Male	−0.13	−0.0786	0.0514	Not Reproduced
Caregiver_Marital_Status:
Married – Single	0.22	0.0763	−0.1437	Not Reproduced
Education:
Preschool/Primary – No	0.06	0.0458	−0.0142	Not Reproduced
Secondary – No	0.08	0.0968	0.0168	Not Reproduced
Child_sex:
Male – Female	−0.40	−0.3711	0.0289	Not Reproduced
childage	−0.02	−0.0166	0.0034	Reproduced
O_lower = original lower confidence interval; R_lower = reproduced lower confidence interval; change.lci = change in R_lower - O_lower; Reproduce.lower = lower confidence interval reproduced.

Change in upper 95% confidence intervals for coefficients

term	O_upper	R_upper	Change.uci	Reproduce.upper
Intercept		1.0535
poultry	0.03	0.0418	0.0118	Not Reproduced
Sheep_goats	0.06	0.0589	−0.0011	Reproduced
pigs	0.01	0.0672	0.0572	Not Reproduced
Owns_any_livestock:
Yes – No	−0.02	0.1456	0.1656	Not Reproduced
Household_wealth:
Medium – Low	0.88	0.8437	−0.0363	Not Reproduced
High – Low	0.96	0.9283	−0.0317	Not Reproduced
household_size	0.08	0.0597	−0.0203	Not Reproduced
Sex_of_household_head:
Female – Male	0.74	0.7052	−0.0348	Not Reproduced
Caregiver_Marital_Status:
Married – Single	0.93	0.8986	−0.0314	Not Reproduced
Education:
Preschool/Primary – No	0.70	0.6839	−0.0161	Not Reproduced
Secondary – No	0.75	0.7730	0.0230	Not Reproduced
Child_sex:
Male – Female	0.15	0.1596	0.0096	Incorrect Rounding
childage	0.01	0.0064	−0.0036	Reproduced
O_upper = original upper confidence interval; R_upper = reproduced upper confidence interval; change.uci = change in R_upper - O_upper; Reproduce.upper = upper confidence interval reproduced.

Change in Standard error

term	O_SE	R_SE	Change.SE	Reproduce.SE
Intercept		0.3828
poultry	0.01	0.0233	0.0133	Not Reproduced
Sheep_goats	0.02	0.0158	−0.0042	Reproduced
pigs	0.01	0.0330	0.0230	Not Reproduced
Owns_any_livestock:
Yes – No	0.17	0.1571	−0.0129	Not Reproduced
Household_wealth:
Medium – Low	0.16	0.1644	0.0044	Reproduced
High – Low	0.17	0.1625	−0.0075	Incorrect Rounding
household_size	0.03	0.0246	−0.0054	Incorrect Rounding
Sex_of_household_head:
Female – Male	0.22	0.1991	−0.0209	Not Reproduced
Caregiver_Marital_Status:
Married – Single	0.18	0.2089	0.0289	Not Reproduced
Education:
Preschool/Primary – No	0.16	0.1621	0.0021	Reproduced
Secondary – No	0.17	0.1718	0.0018	Reproduced
Child_sex:
Male – Female	0.14	0.1348	−0.0052	Incorrect Rounding
childage	0.01	0.0059	−0.0041	Reproduced
O_SE = original standard error; R_SE = reproduced standard error; Change.SE = change in R_SE - O_SE; Reproduce.lower = standard error reproduced.

Change in p-values

Term	O_p	R_p	Change.p	Reproduce.p	SigChangeDirection
Intercept		0.4339
poultry	<0.05	0.8625	0.8135	Not Reproduced	Sig to non-sig, B changes direction
Sheep_goats	0.23	0.0788	−0.1512	Not Reproduced	Remains non-sig, B same direction
pigs	0.12	0.9448	0.8248	Not Reproduced	Remains non-sig, B changes direction
Owns_any_livestock:
Yes – No	0.04	0.2986	0.2586	Not Reproduced	Sig to non-sig, B changes direction
Household_wealth:
Medium – Low	<0.05	0.0490	0.0000	Reproduced	Remains sig, B same direction
High – Low	<0.05	0.0490	0.0000	Reproduced	Remains sig, B same direction
household_size	0.36	0.6462	0.2862	Not Reproduced	Remains non-sig, B same direction
Sex_of_household_head:
Female – Male	0.17	0.1167	−0.0533	Not Reproduced	Remains non-sig, B same direction
Caregiver_Marital_Status:
Married – Single	<0.05	0.0490	0.0000	Reproduced	Remains sig, B same direction
Education:
Preschool/Primary – No	0.02	0.0490	0.0290	Not Reproduced	Remains sig, B same direction
Secondary – No	0.02	0.0490	0.0290	Not Reproduced	Remains sig, B same direction
Child_sex:
Male – Female	0.39	0.4336	0.0436	Not Reproduced	Remains non-sig, B same direction
childage	0.30	0.3855	0.0855	Not Reproduced	Remains non-sig, B same direction
O_p = original p-value; R_p = reproduced p-value; Changep = change in p-value R_p - O_p; Reproduce.p = p-values reproduced. SigChangeDirection = statistical significance and B change between original and reproduced models. Note, p-values that were <0.05 were set to 0.049 for the purposes of comparison.

Bland Altman Plot showing differences between Original and Reproduced p-values for ASF Diversity

Results for p-values

• The p-values for this model were not reproduced.

Conclusion computational reproducibility

This model was not computationally reproducible.

Sensitivity analysis

As this model was not computationally reproducible, inferential reproducibility was not considered, since the original analyses could not be reproduced and therefore, statistical assumptions could not be meaningfully compared or interpreted.