The use of bifactor models has increased substantially in the past decade. However, bifactor models are prone to a nonidentification problem in the context of prediction that is not well recognized in the general research community. Moreover, the practical consequences of adopting different conceptualizations of hierarchical constructs when examining their predictive validity has received little attention. Therefore, Study 1 examined the statistical performance of bifactor models and investigated the effectiveness of an augmentation strategy to remedy the nonidentification problem. Monte Carlo simulations showed that the augmentation strategy is effective. The second simulation study demonstrated that researchers may arrive at different conclusions regarding the predictive validity of hierarchical constructs depending on their choice of models. In general, augmented bifactor models, which are restricted variants of the more general bifactor-(S·I-1) model, reasonably recovered the overall predictive validity ( R 2 ) of hierarchical constructs and led to correct substantive conclusions regarding the incremental validity of facets regardless of the true data-generation model given a sufficiently large sample ( n ≥ 600). The authors discussed implications of those findings and made practical recommendations for further users of bifactor models.

References

Aichholzer, J., Danner, D., Rammstedt, B. ( 2018 ). Facets of personality and “ideological asymmetries.” Journal of Research in Personality, 77, 90 – 100 .

Google Scholar Crossref

Anderson, J. C., Gerbing, D. W. ( 1984 ). The effect of sampling error on convergence, improper solutions, and goodness-of-fit indices for maximum likelihood confirmatory factor analysis . Psychometrika, 49(2), 155 – 173 .

Google Scholar Crossref | ISI

Ashton, M. C., Lee, K. ( 2007 ). Empirical, theoretical, and practical advantages of the HEXACO model of personality structure . Personality and Social Psychology Review, 11(2), 150 – 166 .

Google Scholar SAGE Journals | ISI

Asparouhov, T., Muthén, B. ( 2009 ). Exploratory structural equation modeling . Structural Equation Modeling: A Multidisciplinary Journal, 16(3), 397 – 438 .

Google Scholar Crossref | ISI

Asparouhov, T., Muthén, B. ( 2019 ). Nesting and equivalence testing for structural equation models . Structural Equation Modeling: A Multidisciplinary Journal, 26(2), 302 – 309 .

Google Scholar Crossref

Bentler, P. M., Satorra, A. ( 2010 ). Testing model nesting and equivalence . Psychological Methods, 15(2), 111 – 123 .

Google Scholar Crossref | Medline

Bonifay, W., Cai, L. ( 2017 ). On the complexity of item response theory models . Multivariate Behavioral Research, 52(4), 465 – 484 .

Google Scholar Crossref | Medline

Boomsma, A. ( 1985 ). Nonconvergence, improper solutions, and starting values in LISREL maximum likelihood estimation . Psychometrika, 50(2), 229 – 242 .

Google Scholar Crossref

Bosco, F. A., Aguinis, H., Singh, K., Field, J. G., Pierce, C. A. ( 2015 ). Correlational effect size benchmarks . Journal of Applied Psychology, 100(2), 431 – 449 .

Google Scholar Crossref | Medline | ISI

Box, G. E. ( 1976 ). Science and statistics . Journal of the American Statistical Association, 71(356), 791 – 799 .

Google Scholar Crossref | ISI

Brunner, M., Keller, U., Dierendonck, C., Reichert, M., Ugen, S., Fischbach, A., Martin, R. ( 2010 ). The structure of academic self-concepts revisited: The nested Marsh/Shavelson model . Journal of Educational Psychology, 102(4), 964 – 981 .

Google Scholar Crossref | ISI

Brunner, M., Nagy, G., Wilhelm, O. ( 2012 ). A tutorial on hierarchically structured constructs . Journal of Personality, 80(4), 796 – 846 .

Google Scholar Crossref | Medline | ISI

Cai, L. ( 2010 ). A two-tier full-information item factor analysis model with applications . Psychometrika, 75(4), 581 – 612 .

Google Scholar Crossref | ISI

Cai, L., Wirth, R. J. ( 2013 ). flexMIRT®: A numerical engine for flexible multilevel multidimensional item analysis and test scoring (Version 2.00) [Computer software] . Vector Psychometric Group .

Google Scholar

Cai, L., Yang, J. S., Hansen, M. ( 2011 ). Generalized full-information item bifactor analysis . Psychological Methods, 16(3), 221 – 248 .

Google Scholar Crossref | Medline

Caspi, A., Houts, R. M., Belsky, D. W., Goldman-Mellor, S. J., Harrington, H., Israel, S., Meier, M. H., Ramrakha, S., Shalev, I., Poulton, R., Moffitt, T. E. ( 2014 ). The p factor: one general psychopathology factor in the structure of psychiatric disorders? Clinical Psychological Science, 2(2), 119 – 137 .

Google Scholar SAGE Journals

Castro-Schilo, L., Grimm, K. J., Widaman, K. F. ( 2016 ). Augmenting the correlated trait–correlated method model for multitrait–multimethod data . Structural Equation Modeling: A Multidisciplinary Journal, 23(6), 798 – 818 .

Google Scholar Crossref

Chalmers, R. P . ( 2012 ). mirt: A multidimensional item response theory package for the R environment . Journal of Statistical Software, 48(6), 1 – 29 .

Google Scholar Crossref | ISI

Chen, F., Bollen, K. A., Paxton, P., Curran, P. J., Kirby, J. B. ( 2001 ). Improper solutions in structural equation models: Causes, consequences, and strategies . Sociological Methods & Research, 29(4), 468 – 508 .

Google Scholar SAGE Journals | ISI

Chen, F. F., Hayes, A., Carver, C. S., Laurenceau, J. P., Zhang, Z. ( 2012 ). Modeling general and specific variance in multifaceted constructs: A comparison of the bifactor model to other approaches . Journal of Personality, 80(1), 219 – 251 .

Google Scholar Crossref | Medline | ISI

Chen, F. F., Jing, Y., Hayes, A., Lee, J. M. ( 2013 ). Two concepts or two approaches? A bifactor analysis of psychological and subjective well-being . Journal of Happiness Studies, 14(3), 1033 – 1068 .

Google Scholar Crossref | ISI

Chen, F. F., West, S. G., Sousa, K. H. ( 2006 ). A comparison of bifactor and second-order models of quality of life . Multivariate Behavioral Research, 41(2), 189 – 225 .

Google Scholar Crossref | Medline | ISI

Christensen, H., Mackinnon, A. J., Korten, A., Jorm, A. F. ( 2001 ). The “common cause hypothesis” of cognitive aging: Evidence for not only a common factor but also specific associations of age with vision and grip strength in a cross-sectional analysis . Psychology and Aging, 16(4), 588 – 599 .

Google Scholar Crossref | Medline

Costa, P. T., McCrae, R. R. ( 1995 ). Domains and facets: Hierarchical personality assessment using the Revised NEO Personality Inventory . Journal of Personality Assessment, 64(1), 21 – 50 .

Google Scholar Crossref | Medline | ISI

Debusscher, J., Hofmans, J., De Fruyt, F. ( 2017 ). The multiple face(t)s of state conscientiousness: Predicting task performance and organizational citizenship behavior . Journal of Research in Personality, 69, 78 – 85 .

Google Scholar Crossref

Eid, M., Geiser, C., Koch, T., Heene, M. ( 2017 ). Anomalous results in G-factor models: Explanations and alternatives . Psychological Methods, 22(3), 541 – 562 .

Google Scholar Crossref | Medline

Eid, M., Krumm, S., Koch, T., Schulze, J. ( 2018 ). Bifactor models for predicting criteria by general and specific factors: Problems of nonidentifiability and alternative solutions . Journal of Intelligence, 6(3), 1 – 23 .

Google Scholar Crossref

Fraley, R. C., Hudson, N. W., Heffernan, M. E., Segal, N. ( 2015 ). Are adult attachment styles categorical or dimensional? A taxometric analysis of general and relationship-specific attachment orientations . Journal of Personality and Social Psychology, 109(2), 354 – 368 .

Google Scholar Crossref | Medline | ISI

Gibbons, R. D., Bock, R. D., Hedeker, D., Weiss, D. J., Segawa, E., Bhaumik, D. K., Kupfer, D. J., Frank, E., Grochocinski, V. J., Stover, A. ( 2007 ). Full-information item bifactor analysis of graded response data . Applied Psychological Measurement, 31(1), 4 – 19 .

Google Scholar SAGE Journals | ISI

Gibbons, R. D., Hedeker, D. R. ( 1992 ). Full-information item bi-factor analysis . Psychometrika, 57(3), 423 – 436 .

Google Scholar Crossref | ISI

Gignac, G. E. ( 2009 ). Partial confirmatory factor analysis: Described and illustrated on the NEO–PI–R . Journal of Personality Assessment, 91(1), 40 – 47 .

Google Scholar Crossref | Medline

Gignac, G. E. ( 2016 ). The higher-order model imposes a proportionality constraint: That is why the bifactor model tends to fit better . Intelligence, 55, 57 – 68 .

Google Scholar Crossref

Grewal, R., Cote, J. A., Baumgartner, H. ( 2004 ). Multicollinearity and measurement error in structural equation models: Implications for theory testing . Marketing Science, 23(4), 519 – 529 .

Google Scholar Crossref | ISI

Gustafsson, J. E., Balke, G. ( 1993 ). General and specific abilities as predictors of school achievement . Multivariate Behavioral Research, 28(4), 407 – 434 .

Google Scholar Crossref | Medline | ISI

Hancock, G. R., Mueller, R. O. ( 2001 ). Rethinking construct reliability within latent variable systems . In Cudeck, R., du Toit, S., Sorbom, D. (Eds.), Structural equation modeling: Present and future (pp. 195 – 216 ). Scientific Software International .

Google Scholar

Heggestad, E. D., Scheaf, D. J., Banks, G. C., Monroe Hausfeld, M., Tonidandel, S., Williams, E. B. ( 2019 ). Scale adaptation in organizational science research: A review and best-practice recommendations . Journal of Management, 45(6), 2596 – 2627 .

Google Scholar SAGE Journals | ISI

Hinkin, T. R. ( 1995 ). A review of scale development practices in the study of organizations . Journal of Management, 21(5), 967 – 988 .

Google Scholar SAGE Journals | ISI

Holzinger, K. J., Swineford, F. ( 1937 ). The bi-factor method . Psychometrika, 2(1), 41 – 54 .

Google Scholar Crossref

Idaszak, J. R., Bottom, W. P., Drasgow, F. ( 1988 ). A test of the measurement equivalence of the revised Job Diagnostic Survey: Past problems and current solutions . Journal of Applied Psychology, 73(4), 647 – 656 .

Google Scholar Crossref

Jennrich, R. I., Bentler, P. M. ( 2011 ). Exploratory bi-factor analysis . Psychometrika, 76(4), 537 – 549 .

Google Scholar Crossref | Medline | ISI

Jennrich, R. I., Bentler, P. M. ( 2012 ). Exploratory bi-factor analysis: The oblique case . Psychometrika, 77(3), 442 – 454 .

Google Scholar Crossref | Medline | ISI

Jensen, A. R . ( 1998 ). The g factor and the design of education In Sternberg, R. J., Williams, W. M. (Eds.), Intelligence, instruction, and assessment: Theory into practice (pp. 111 – 131 ). Lawrence Erlbaum Associates Publishers .

Google Scholar

Jeon, M., Rijmen, F., Rabe-Hesketh, S. ( 2014 ). Flexible item response theory modeling with FLIRT . Applied Psychological Measurement, 38(5), 404 – 405 .

Google Scholar SAGE Journals | ISI

Judge, T. A., Rodell, J. B., Klinger, R. L., Simon, L. S., Crawford, E. R. ( 2013 ). Hierarchical representations of the five-factor model of personality in predicting job performance: Integrating three organizing frameworks with two theoretical perspectives . Journal of Applied Psychology, 98(6), 875 – 925 .

Google Scholar Crossref | Medline

Kell, H., Lang, J. ( 2017 ). Specific abilities in the workplace: More important than g? Journal of Intelligence, 5(2), 13 – 31 .

Google Scholar Crossref

Kell, H., Lang, J. ( 2018 ). The great debate: General ability and specific abilities in the prediction of important outcomes? Journal of Intelligence, 6(3), 1 – 23 .

Google Scholar Crossref

Kretzschmar, A., Gignac, G. E. ( 2019 ). At what sample size do latent variable correlations stabilize? Journal of Research in Personality, 80, 17 – 22 .

Google Scholar Crossref

Lance, C. E., Beck, S. S., Fan, Y., Carter, N. T. ( 2016 ). A taxonomy of path-related goodness-of-fit indices and recommended criterion values . Psychological Methods, 21(3), 388 – 404 .

Google Scholar Crossref | Medline

Mansolf, M., Reise, S. P. ( 2016 ). Exploratory bifactor analysis: The Schmid-Leiman orthogonalization and Jennrich-Bentler analytic rotations . Multivariate Behavioral Research, 51(5), 698 – 717 .

Google Scholar Crossref | Medline

Markon, K. E. ( 2019 ). Bifactor and hierarchical models: Specification, inference, and interpretation . Annual Review of Clinical Psychology, 15, 51 – 69 .

Google Scholar Crossref | Medline

Marsh, H. W. ( 1987 ). The hierarchical structure of self-concept and the application of hierarchical confirmatory factor analysis . Journal of Educational Measurement, 24(1), 17 – 39 .

Google Scholar Crossref | ISI

Maslach, C., Jackson, S. E., Leiter, M. P. ( 1996 ). Maslach Burnout Inventory manual ( 3rd ed .). Consulting Psychologists Press .

Google Scholar

Mason, C. H., Perreault, W. D., ( 1991 ). Collinearity, power, and interpretation of multiple regression analysis . Journal of Marketing Research, 28(3), 268 – 280 .

Google Scholar SAGE Journals | ISI

McAbee, S. T., Oswald, F. L., Connelly, B. S. ( 2014 ). Bifactor models of personality and college student performance: A broad versus narrow view . European Journal of Personality, 28(6), 604 – 619 .

Google Scholar

McDonald, R. P., Ho, M. H. R. ( 2002 ). Principles and practice in reporting structural equation analyses . Psychological Methods, 7(1), 64 – 82 .

Google Scholar Crossref | Medline | ISI

McGrew, K. S. ( 2009 ). CHC theory and the human cognitive abilities project: Standing on the shoulders of the giants of psychometric intelligence research . Intelligence, 37, 1 – 10 .

Google Scholar Crossref | ISI

Merkle, E. C., You, D., Preacher, K. J. ( 2016 ). Testing nonnested structural equation models . Psychological Methods, 21(2), 151 – 163 .

Google Scholar Crossref | Medline

Mike, A., King, H., Oltmanns, T. F., Jackson, J. J. ( 2018 ). Obsessive, compulsive, and conscientious? The relationship between OCPD and personality traits . Journal of Personality, 86(6), 952 – 972 .

Google Scholar Crossref | Medline

Moshagen, M., Hilbig, B. E., Zettler, I. ( 2018 ). The dark core of personality . Psychological Review, 125(5), 656 – 688 .

Google Scholar Crossref | Medline

Murray, A. L., Johnson, W. ( 2013 ). The limitations of model fit in comparing the bi-factor versus higher-order models of human cognitive ability structure . Intelligence, 41(5), 407 – 422 .

Google Scholar Crossref | ISI

Muthén, L. K., Muthén, B. O. ( 2002 ). How to use a Monte Carlo study to decide on sample size and determine power . Structural Equation Modeling: A Multidisplinary Journal, 9(4), 599 – 620 .

Google Scholar Crossref | ISI

Ng, V., Cao, M., Marsh, H. W., Tay, L., Seligman, M. E. ( 2017 ). The factor structure of the Values in Action Inventory of Strengths (VIA-IS): An item-level exploratory structural equation modeling (ESEM) bifactor analysis . Psychological Assessment, 29(8), 1053 – 1058 .

Google Scholar Crossref | Medline

Nye, C. D., Chernyshenko, O. S., Stark, S., Drasgow, F., Phillips, H. L., Phillips, J. B., Campbell, J. S. ( 2020 ). More than g: Evidence for the incremental validity of performance-based assessments for predicting training performance . Applied Psychology: An International Review, 69(2), 302 – 324 .

Google Scholar Crossref

Olea, M. M., Ree, M. J. ( 1994 ). Predicting pilot and navigator criteria: Not much more than g . Journal of Applied Psychology, 79(6), 845 – 851 .

Google Scholar Crossref

Raykov, T., Marcoulides, G. A., Menold, N., Harrison, M. ( 2019 ). Revisiting the bi-factor model: Can mixture modeling help assess its applicability? Structural Equation Modeling: A Multidisciplinary Journal, 26(1), 110 – 118 .

Google Scholar Crossref

Ree, M. J., Carretta, T. R., Teachout, M. S. ( 2015 ). Pervasiveness of dominant general factors in organizational measurement . Industrial and Organizational Psychology, 8(3), 409 – 427 .

Google Scholar Crossref

Ree, M. J., Earles, J. A. ( 1991 ). Predicting training success: Not much more than g . Personnel Psychology, 44(2), 321 – 332 .

Google Scholar Crossref

Ree, M. J., Earles, J. A., Teachout, M. S. ( 1994 ). Predicting job performance: Not much more than g . Journal of Applied Psychology, 79(4), 518 – 524 .

Google Scholar Crossref

Reise, S. P. ( 2012 ). The rediscovery of bifactor measurement models . Multivariate Behavioral Research, 47(5), 667 – 696 .

Google Scholar Crossref | Medline | ISI

Reise, S. P., Kim, D. S., Mansolf, M., Widaman, K. F. ( 2016 ). Is the bifactor model a better model or is it just better at modeling implausible responses? Application of iteratively reweighted least squares to the Rosenberg Self-Esteem Scale . Multivariate Behavioral Research, 51(6), 818 – 838 .

Google Scholar Medline

Reise, S. P., Moore, T. M., Haviland, M. G. ( 2010 ). Bifactor models and rotations: Exploring the extent to which multidimensional data yield univocal scale scores . Journal of Personality Assessment, 92(6), 544 – 559 .

Google Scholar Crossref | Medline | ISI

Reise, S. P., Morizot, J., Hays, R. D. ( 2007 ). The role of the bifactor model in resolving dimensionality issues in health outcomes measures . Quality of Life Research, 16(1), 19 – 31 .

Google Scholar Crossref | Medline | ISI

Reise, S. P., Scheines, R., Widaman, K. F., Haviland, M. G. ( 2013 ). Multidimensionality and structural coefficient bias in structural equation modeling: A bifactor perspective . Educational and Psychological Measurement, 73(1), 5 – 26 .

Google Scholar SAGE Journals | ISI

Revelle, W . ( 2018 ). psych: Procedures for personality and psychological research. https://CRAN.R-project.org/package=psychVersion=1.8.12.

Google Scholar

Richard, F. D., Bond, C. F., Stokes-Zoota, J. J. ( 2003 ). One hundred years of social psychology quantitatively described . Review of General Psychology, 7(4), 331 – 363 .

Google Scholar SAGE Journals | ISI

Rindskopf, D. ( 1984 ). Structural equation models: Empirical identification, Heywood cases, and related problems . Sociological Methods & Research, 13(1), 109 – 119 .

Google Scholar SAGE Journals | ISI

Roberts, B. W., Chernyshenko, O. S., Stark, S., Goldberg, L. R. ( 2005 ). The structure of conscientiousness: An empirical investigation based on seven major personality questionnaires . Personnel Psychology, 58(1), 103 – 139 .

Google Scholar Crossref

Rodriguez, A., Reise, S. P., Haviland, M. G. ( 2016 ). Evaluating bifactor models: Calculating and interpreting statistical indices . Psychological Methods, 21(2), 137 – 150 .

Google Scholar Crossref | Medline

Rosseel, Y. ( 2012 ). Lavaan: An R package for structural equation modeling and more. Version 0.5–12 (BETA) . Journal of Statistical Software, 48(2), 1 – 36 .

Google Scholar Crossref | ISI

Salthouse, T. A. ( 1998 ). Relation of successive percentiles of reaction time distributions to cognitive variables and adult age . Intelligence, 26(2), 153 – 166 .

Google Scholar Crossref

Samuel, D. B., Widiger, T. A. ( 2008 ). A meta-analytic review of the relationships between the five-factor model and DSM-IV-TR personality disorders: A facet level analysis . Clinical Psychology Review, 28(8), 1326 – 1342 .

Google Scholar Crossref | Medline | ISI

Schaufeli, W. B., Bakker, A. ( 2003 ). Utrecht work engagement scale (UWES) preliminary manual [Version 1, November 2003]. Utrecht University : Occupational Health Psychology Unit .

Google Scholar

Schmid, J., Leiman, J. M. ( 1957 ). The development of hierarchical factor solutions . Psychometrika, 22(1), 53 – 61 .

Google Scholar Crossref | ISI

Schmidt, F. L., Hunter, J. ( 2004 ). General mental ability in the world of work: Occupational attainment and job performance . Journal of Personality and Social Psychology, 86(1), 162 – 173 .

Google Scholar Crossref | Medline | ISI

Schmiedek, F., Li, S. C. ( 2004 ). Toward an alternative representation for disentangling age-associated differences in general and specific cognitive abilities . Psychology and Aging, 19(1), 40 – 56 .

Google Scholar Crossref | Medline

Sigal, M. J., Chalmers, R. P. ( 2016 ). Play it again: Teaching statistics with Monte Carlo simulation . Journal of Statistics Education, 24(3), 136 – 156 .

Google Scholar Crossref

Soto, C. J., John, O. P. ( 2017 ). The next Big Five Inventory (BFI-2): Developing and assessing a hierarchical model with 15 facets to enhance bandwidth, fidelity, and predictive power . Journal of Personality and Social Psychology, 113(1), 117 – 143 .

Google Scholar Crossref | Medline

Su, R., Zhang, Q., Liu, Y., Tay, L. ( 2019 ). Modeling congruence in organizational research with latent moderated structural structural equations . Journal of Applied Psychology, 104(11), 1404 – 1433 .

Google Scholar Crossref | Medline

Thoemmes, F., Rosseel, Y., Textor, J. ( 2018 ). Local fit evaluation of structural equation models using graphical criteria . Psychological Methods, 23(1), 27 – 41 .

Google Scholar Crossref | Medline

Wee, S., Newman, D. A., Joseph, D. L. ( 2014 ). More than g: Selection quality and adverse impact implications of considering second-stratum cognitive abilities . Journal of Applied Psychology, 99(4), 547 – 563 .

Google Scholar Crossref | Medline

Wee, S. ( 2018 ). Aligning predictor-criterion bandwidths: Specific abilities as predictors of specific performance . Journal of Intelligence, 6, 1 – 14 .

Google Scholar Crossref

Williams, L. J., O’Boyle, E. ( 2011 ). The myth of global fit indices and alternatives for assessing latent variable relations . Organizational Research Methods, 14(2), 350 – 369 .

Google Scholar SAGE Journals | ISI

Wolf, E. J., Harrington, K. M., Clark, S. L., Miller, M. W. ( 2013 ). Sample size requirements for structural equation models: An evaluation of power, bias, and solution propriety . Educational and Psychological Measurement, 73(6), 913 – 934 .

Google Scholar SAGE Journals | ISI