Thoughts and Theory
If you’ve made it this far, you’ve probably read my Exploratory Factor Analysis (EFA) text. If you haven’t read it, check it out before jumping straight into Confirmatory Factor Analysis (CFA), as the two analyzes have some important similarities! In this text, we will present the fundamentals of the CFA, how it works, and we will compare the CFA with the EFA.
How do I represent my construct?
Like EFA, CFA uses the common factor model, that is, it sees the covariance between observed variables as a reflection of the influence of one or more factors and also a variance that is not explained. This would be different from network analysis, which allows the covariance between items to have a cause between them. In other words, the psychometric model of factor analysis generally believes that the covariance of items is only because there is a latent factor that explains it. This is a very important assumption to keep in mind, as perhaps your Construct does not fit the common factor model, but rather a network model. I will explain with an example given by Borsboom and Cramer (2013).
Below, we see a factor model of an instrument that measures major depression. In it, the items measure aspects such as: feeling depressed, weight gain, sleeping problems, motor problems, fatigue, concentration problems, etc. We see from the image that the variation in scores on the items has a common cause, depression (i.e., the higher the person’s level of depression, the more they report having these symptoms).
However, we can think that some items have relationships among themselves that are not just due to depression. An example of this is the cause of the concentration problem and its relationship to other symptoms. People who have trouble sleeping become fatigued and, therefore, have concentration problems (sleep problems → fatigue → concentration problems). In other words, it is possible to infer a causal relationship between one observable variable and another, which "breaks" with the common factor model. A possible representation of this model is in the image below, where the items have causal relationships with each other.
So how do I know if my construct follows the common factor model or is more like a network model? Well, based on theory! I know that researchers for a long time only cared about statistics to guide everything, but it is important for us to rethink our constructs theoretically and then test the theory empirically.
Okay, you decided that the most suitable model for your construct is the factor model, now what?
What is it and when do we apply a CFA?
The CFA is a multivariate statistic that serves to estimate the structure of an instrument, verifying how well the measured variables represent the number of constructs. That is, it verifies whether an instrument’s structure can be, but is not necessarily, true. For this, we need to state which structure we want to test. Generally, the CFA is used when there is a previous study that tells us the dimensionality of that instrument. For instance, we would have a North American study that uses an EFA to verify the instrument’s dimensionality and you use a CFA to verify how well this structure happens with Brazilian data. However, this is not the only way you can use the CFA! You can, for example, have the EFA in the same study (to explore the dimensionality), but still test different theoretical models using the CFA.
Thus, both EFA and CFA are applied when you want to estimate the dimensionality of an instrument (note that I said estimate, not explore/discover dimensionality). For example, we can apply the CFA in self-report instruments, where items represent behaviors, thoughts, or feelings. Another example, we can apply it in a set of other measures, such as psychophysical measures of anxiety. Thus, the CFA applies to instruments that measure some attributes such as well-being, anxiety, prejudice, etc.
Differences between Saturated/Unrestricted Model and Restricted Model
The EFA model can be called the saturated/unrestricted model. This is because all latent dimensions explain the variation in all items, as exemplified in the image below.
As for the CFA, we can call it the Restricted Model, that is, we impose some restrictions on the model, for example, not having cross-loadings of a factor with items from another factor. The restricted model is exemplified in the image below.
Of course, there are some practical differences between one model and another. The first is that, generally, the output of the factor loadings from a CFA is different from the EFA. While in EFA we have cross loads on all factors, in the CFA some loadings are set at 0, as in the image below.
I made a table that shows the differences between the unrestricted model (EFA) and the restricted model (CFA).
We see in the table above that, for the confirmatory Factor Analysis, we need to have a defined hypothesis, that is, there must be a theory behind that will directly guide our analyses, we cannot just keep exploring without a proper justification. This is a little different from EFA, which has a theory behind the structure, but you test whether this structure will be corroborated in the data (through parallel analysis and the like). Of course, in EFA we can extract the factors based on theory, which, in a way, would resemble CFA in terms of the hypothesis guiding the analyzes directly.
It is also important to emphasize again that in the CFA we can test different models, being able to make modifications and allow residual correlations. See the image below.
We can even test more complex models, such as a hierarchical model or a bifactor model.
In short, because CFA makes restrictions on the model, we have the possibility to test a multitude of things! One use of CFA is through multi-group CFA, which I wrote in a previous text.
Model Identification
We have to deal with a "problem" called model identification when we talk about a restricted model (Bollen, 1989). In other words, we need our data to have enough "information" to be able to do the necessary statistics.
In the image above, we are estimating unidimensionality using the CFA. We see that we estimate 4 factor loadings (lambdas; one per item) and 4 residuals (epsilons; one per item), that is, we have 8 information to be estimated. The pieces of information we have are the scores of V1, V2, V3, and V4 (that is, how much the person scored for each variable observed) and the correlation between them.
Thus, we have 4 scores + 6 correlations = 10 pieces of information. In other words, with 4 items we can estimate the 8 pieces of factor loadings and residues since we have 10 pieces of information in our hands. Following this logic, it is easy to see that, in order to be able to identify the model, the minimum number of items is 3 items per latent factor. See, in a unifactorial model with 3 items, we will estimate 3 factor loadings + 3 residues = 6 necessary information. We have information for 3 items + 3 correlations = 6 information in our sleeve. So we will have 0 degrees of freedom (DF).
- If DF < 0, the unidentified model (nothing will be estimated);
- If DF = 0, the model is under-identified (only factor loadings will be arbitrarily estimated; no fit indexes will be generated);
- If DF> 1, the overidentified model (everything can be estimated).
A model should only be interpreted if DF> 1, as this is the only way to solve the covariance equation of items and Latent Variables, allowing the output of fit indices.
Fit Indices
Fit indices represent how plausible the model you are estimating is. A well-fitting model reduces the discrepancy between the Sigma matrix (population covariance matrix, i.e., the pattern you put in the model) and the S matrix (sample covariance matrix, i.e., your empirical matrix).
There are several fit indices, and each has its cutoff point (Brown, 2015; Mair, 2018). I made another table showing the descriptions of the fit indices and their recommendations.
Advanced Topics in Fit Indices
A model can have great fit indices, but it cannot be generalized to the population. In other words, we have to be careful with fit indices! More complex models generally have better fit indices. We can adjust our model to explain the data from a linear equation, a quadratic equation, or even a 6th-degree equation. Sometimes, the 6th-degree equation explains the data perfectly! However, this model is less generalizable given the complexity of the equation. Thus, we have a cost-benefit relationship between model complexity and fit indices, where we have to be careful not to have an overfitting (or exaggerated fit).
A paper by Bonifay and Cai (2017) verified how well each of the models below fits in general. That is, whether a model always presents good fit indices or not. For this, a semi-unrestricted model was tested (with only 2 loads being restricted; model in black); a bifactor model (green model); a hierarchical model (in blue and yellow); and a unidimensional model (red). For this, the analysis of the fit indices in 1000 simulated datasets was performed.
What they found is summarized in the image below. We see that within all the possible fit indices in the sample space, the semi-unrestricted model and the bifactor model almost always presented good fit indices. This implies that we cannot interpret fit indices the same way as other models in these cases. For example, if you are going to compare a bifactor model with a unidimensional model, you will most likely find better fit indices in the bifactor model, but this is not necessarily the best model to explain the data. Of course, one would have to compare fit indices of nested models, but the example serves as an illustration.
With that in mind, there is a shiny app that simulates the most appropriate cut-off points for your data (McNeish and Wolf, 2020). I won’t go into details to explain how it works because the site itself has an explanation. With this, we have the chance to check the most adequate fit indices for unidimensional and multidimensional models and to test equivalence. The site is represented below.
Thank you so much for reading the text so far! I hope you enjoyed and learned a lot! If you liked it, don’t forget to share your knowledge with colleagues and friends! Soon, we will make a tutorial on how to do a CFA in a dataset. Follow me on Medium!
This text was first published in Brazillian Portuguese on PsicoData, with me being the author as well.
Contact
LinkedIn ResearchGate Google Scholar For papers partnerships or statistical consulting, get in touch by email ([email protected])
References
K. A. Bollen, Confirmatory Factor Analysis. In K. A. Bollen (Ed.), Structural Equations with latent variables, 1989, Willey.
W. Bonifay and L. Cai, On the complexity of item response theory models, 2017, Multivariate Behavioral Research, 52(4), 465–484.
D. Borsboom and A. O. Cramer, Network analysis: an integrative approach to the structure of psychopathology, 2013, Annual review of clinical psychology, 9, 91–121.
T. A. Brown, Confirmatory factor analysis for applied research Second Edition, 2015, The Guilford Press.
P. Mair, Modern Psychometrics with R, 2018, Springer.
D. McNeish and M. G. Wolf. Dynamic Fit Index Cutoffs for Confirmatory Factor Analysis Models, 2020, July 7, https://doi.org/10.31234/osf.io/v8yru
