One of the most frequent questions in quantitative research is how to assess and rank the relevance of variables in multiple regression models. This uncertainty often arises when researchers prioritize data mining over established economic theory. Recently, a social media contact of mine posed this exact dilemma:
“Does anyone know of a way to demonstrate the importance of a variable in a regression model, apart from using the standard regression coefficient? Has anyone used Johnson’s epsilon or alternatives to solve this issue?”
In this post, I want to share the advice I offered him. I emphasized that his goal was to justify a variable’s inclusion beyond its weighted effect on the dependent variable. In robust scientific research, we must prioritize sound structural modeling over “p-hacking” or “regression fishing.”
His core concern pertained to model specification. To tackle this, researchers must establish the relative relevance of a regressor beyond the coefficients’ absolute values—a process requiring a mix of intuition and careful data exploration. I advised him to evaluate the appropriateness of the variables independently before analyzing them jointly. Scrutinizing the variables in isolation is always the crucial first step.
Specifically, I recommended checking four dimensions for each variable:
- Data quality: Assess data availability and the degree of measurement error.
- Theoretical alignment: Ensure every variable is consistent with your underlying theory.
- Core assumptions: Verify that standard econometric assumptions hold.
- Robustness: Test rival models that attempt to explain the same phenomenon.
But what if all variables pass these baseline checks, the model aligns with theory, and everything looks sound? If the researcher still needs to narrow down the model, I suggest moving to the second phase: post-estimation diagnostics.
This diagnostic phase involves analyzing the variables after running the regressions to detect specification or measurement errors. Because specification errors are common, I suggested nested hypothesis testing to determine if the model omitted relevant variables (misspecification) or included irrelevant ones (overfitting). In his case, the error was the latter.
The bottom line is that regardless of the specific test you choose, the critical task will always be to track and analyze the nuances within the error terms of your competing models.
At Econometricus we deliver quality mixed-methods research that ranks high on both academic and social media impact. We build cogent data-rooted rationales that machines (AI) can not match. Sources wise, we have more than fifteen years of experience leveraging most public sources datasets.
Feel free to get in touch to discuss your research needs.
Categories: Statistics and Time Series.
