The multi trait multi method (MTMM) framework is an alternative to the standard CFA approach that allows for assessing interaction effects. It involves using a set of latent variables to model the variance shared between measures that are correlated with one another. These shared variance components are assumed to represent a combination of both trait and method variance. This enables researchers to evaluate the extent to which a correlation between measures is driven by the intended trait or by systematic measurement influences – sometimes referred to as “method effects”.
Campbell and Fiske introduced two new forms of validity – convergent validity MT-MM and discriminant validity – in 1959. The former has to do with the degree to which concepts that should be interrelated theoretically are actually interrelated in reality and the latter has to do with the degree to which a measure is meaningful.
A widely used test of construct validity is the multitrait-multimethod matrix. In a typical multitrait-multimethod matrix, multiple traits are measured with different methods and the correlations between measures of each trait are evaluated. If there is good construct validity, the correlations between measures that share a common trait should be high (this is referred to as convergent validity) and the correlations between the same method measuring different traits should be low (this is referred to as discriminant validity).
For example, if sociability, extroversion, and altruism are all traits and they are measured with self-report questionnaires, reports of friends, and behavioral observations, then these three measures might be correlated. The correlation between sociability and extroversion would be high and the correlation between sociability and altruism would be low. This is a good indication that the three measures are measuring the same thing and that the measure of sociability has some construct validity.
However, it is also possible that the correlation between sociability and extroversion or sociability and altruism is driven by a systematic variation in the measurements. This systematic variation is sometimes referred to as a method factor and it has been shown that it tends to inflate the coefficients on a standardized Item-Continuity Reliability (ICR) model when it is used to estimate construct validity.
To address this problem, a number of approaches have been developed that include trait and method factors in the model. These include a variety of CFA-MTMM mixture models that are based on latent difference score variables and a number of multigroup CFA-MTMM mixture models. Although these models can be quite useful, they suffer from several limitations. First, they require that the subgroups to be compared are known in advance. This is an important limitation as it limits the scope of the analysis. Moreover, these models can be difficult to interpret and do not allow the researcher to investigate differences in convergent and discriminant validity within the different subgroups. The present study addresses these problems by developing a MTMM mixture model that can be used to assess MTMM with data that does not have pre-specified subgroups.