Tests for preference heterogeneity in conjoint experiments
amce_diffs( data, formula, by, id = ~0, weights = NULL, feature_order = NULL, feature_labels = NULL, level_order = c("ascending", "descending"), alpha = 0.05, ... ) cj_anova(data, formula, id = NULL, weights = NULL, by = NULL, ...) mm_diffs( data, formula, by, id = ~0, weights = NULL, feature_order = NULL, feature_labels = NULL, level_order = c("ascending", "descending"), alpha = 0.05, h0 = 0, ... )
| data | A data frame containing variables specified in |
|---|---|
| formula | A formula specifying a model to be estimated. All variables should be factors; all levels across features should be unique. |
| by | A formula containing only RHS variables, specifying grouping factors over which to perform estimation. For |
| id | Ignored. |
| weights | An (optional) RHS formula specifying a variable holding survey weights. |
| feature_order | An (optional) character vector specifying the names of feature (RHS) variables in the order they should be encoded in the resulting data frame. |
| feature_labels | A named list of “fancy” feature labels to be used in output. By default, the function looks for a “label” attribute on each variable in |
| level_order | A character string specifying levels (within each feature) should be ordered increasing or decreasing in the final output. This is mostly only consequential for plotting via |
| alpha | A numeric value indicating the significance level at which to calculate confidence intervals for the MMs (by default 0.95, meaning 95-percent CIs are returned). |
| ... | |
| h0 | A numeric value specifying a null hypothesis value to use when generating z-statistics and p-values (only used for |
amce_diffs and mm_diffs return a data frame similar to the one returned by cj, including a BY column (with the value “Difference”) for easy merging with results returned by that function.
cj_anova returns an anova object.
cj_anova takes a model formula (“reduced” model) and generates a “full” model with two-way interactions between the variables specified in by and all RHS variables in formula, then computes an F-test comparing the two models, providing a test for whether preferences vary across levels of by. This is, in essence, a test of whether all such interaction coefficients are distinguishable from zero. (Because the test depends on overall model fit, not the coefficient variances, clustering is irrelevant.)
mm_diffs provides a data frame of differences in marginal means (literally differencing the results from mm across levels of by. This provides the clearest direct measure of preference differences from a conjoint design.
amce_diffs provides a data frame of differences in AMCEs (the coefficient on an interaction between each RHS factor and the variable in by). This provides an estimate of the difference in causal effects of each factor level relative to the baseline level (i.e., the difference in conditional AMCEs). This quantity is easily misinterpreted as the difference in preferences, which it is not. Rather it is a difference in the effect of the factor on preferences relative to the baseline/reference category of that feature. If preferences in the reference category differ across levels of by, the the difference in conditional AMCEs will have an unpredictable sign and significance, making differences in marginal means a more sensible quantity of interest. See amce_by_reference for a diagnostic.
Note: amce_diffs does not work with constrained designs. To obtain such differences, subset the design by constraints and calculate differences within each subset.
data("immigration") immigration$contest_no <- factor(immigration$contest_no) # Test for heterogeneity by profile order cj_anova(immigration, ChosenImmigrant ~ Gender + Education + LanguageSkills, by = ~ contest_no)#> Analysis of Deviance Table #> #> Model 1: ChosenImmigrant ~ Gender + Education + LanguageSkills #> Model 2: ChosenImmigrant ~ Gender + Education + LanguageSkills + contest_no + #> Gender:contest_no + Education:contest_no + LanguageSkills:contest_no #> Resid. Df Resid. Dev Df Deviance F Pr(>F) #> 1 13949 3353.0 #> 2 13905 3343.9 44 9.088 0.8589 0.7334# Test for heterogeneity by CountryOfOrigin feature cj_anova(immigration, ChosenImmigrant ~ Gender + Education, by = ~ CountryOfOrigin)#> Analysis of Deviance Table #> #> Model 1: ChosenImmigrant ~ Gender + Education #> Model 2: ChosenImmigrant ~ Gender + Education + CountryOfOrigin + Gender:CountryOfOrigin + #> Education:CountryOfOrigin #> Resid. Df Resid. Dev Df Deviance F Pr(>F) #> 1 13952 3407.8 #> 2 13880 3374.5 72 33.286 1.9016 6.756e-06 *** #> --- #> Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1# \donttest{ # Differences in MMs by Gender feature mm_diffs(immigration, ChosenImmigrant ~ LanguageSkills + Education, ~ Gender, id = ~ CaseID)#> BY statistic outcome feature #> 1 Male - Female mm_difference ChosenImmigrant Language Skills #> 2 Male - Female mm_difference ChosenImmigrant Language Skills #> 3 Male - Female mm_difference ChosenImmigrant Language Skills #> 4 Male - Female mm_difference ChosenImmigrant Language Skills #> 5 Male - Female mm_difference ChosenImmigrant Educational Attainment #> 6 Male - Female mm_difference ChosenImmigrant Educational Attainment #> 7 Male - Female mm_difference ChosenImmigrant Educational Attainment #> 8 Male - Female mm_difference ChosenImmigrant Educational Attainment #> 9 Male - Female mm_difference ChosenImmigrant Educational Attainment #> 10 Male - Female mm_difference ChosenImmigrant Educational Attainment #> 11 Male - Female mm_difference ChosenImmigrant Educational Attainment #> level estimate std.error z p #> 1 Fluent English -0.015436300 0.01564954 -0.9863741 0.323949592 #> 2 Broken English -0.047683617 0.01593767 -2.9918819 0.002772635 #> 3 Tried English but Unable -0.009770039 0.01591358 -0.6139436 0.539252607 #> 4 Used Interpreter -0.030670893 0.01594758 -1.9232320 0.054450926 #> 5 No Formal -0.008679690 0.02149994 -0.4037077 0.686427677 #> 6 4th Grade -0.007051326 0.02162325 -0.3260992 0.744349269 #> 7 8th Grade -0.029815233 0.02129659 -1.4000003 0.161513221 #> 8 High School -0.018077430 0.02147494 -0.8417919 0.399904458 #> 9 Two-Year College -0.049321323 0.02130677 -2.3148192 0.020622828 #> 10 College Degree -0.021520237 0.02136533 -1.0072502 0.313814576 #> 11 Graduate Degree -0.044846098 0.02138557 -2.0970264 0.035991236 #> lower upper Gender #> 1 -0.04610884 0.0152362347 Male #> 2 -0.07892087 -0.0164463646 Male #> 3 -0.04096008 0.0214199996 Male #> 4 -0.06192757 0.0005857869 Male #> 5 -0.05081879 0.0334594130 Male #> 6 -0.04943212 0.0353294660 Male #> 7 -0.07155578 0.0119253167 Male #> 8 -0.06016754 0.0240126790 Male #> 9 -0.09108183 -0.0075608190 Male #> 10 -0.06339552 0.0203550500 Male #> 11 -0.08676104 -0.0029311580 Male# Differences in AMCEs by Gender feature (i.e., feature interactions) amce_diffs(immigration, ChosenImmigrant ~ LanguageSkills + Education, ~ Gender, id = ~ CaseID)#> BY outcome statistic feature #> 1 Male - Female ChosenImmigrant amce_difference Educational Attainment #> 2 Male - Female ChosenImmigrant amce_difference Educational Attainment #> 3 Male - Female ChosenImmigrant amce_difference Educational Attainment #> 4 Male - Female ChosenImmigrant amce_difference Educational Attainment #> 5 Male - Female ChosenImmigrant amce_difference Educational Attainment #> 6 Male - Female ChosenImmigrant amce_difference Educational Attainment #> 7 Male - Female ChosenImmigrant amce_difference Language Skills #> 8 Male - Female ChosenImmigrant amce_difference Language Skills #> 9 Male - Female ChosenImmigrant amce_difference Language Skills #> level Gender estimate std.error z p #> 1 4th Grade Male -0.005966999 0.03172945 -0.1880587 0.8508306 #> 2 8th Grade Male -0.030625179 0.03090707 -0.9908793 0.3217445 #> 3 College Degree Male -0.014844848 0.03107888 -0.4776507 0.6328989 #> 4 Graduate Degree Male -0.042143177 0.03134662 -1.3444249 0.1788110 #> 5 High School Male -0.013956325 0.03121208 -0.4471450 0.6547704 #> 6 Two-Year College Male -0.047183288 0.03112293 -1.5160299 0.1295118 #> 7 Broken English Male -0.029903918 0.02329320 -1.2838049 0.1992102 #> 8 Tried English but Unable Male 0.004543744 0.02291366 0.1982985 0.8428115 #> 9 Used Interpreter Male -0.013293411 0.02371292 -0.5605978 0.5750718 #> lower upper #> 1 -0.06815558 0.05622158 #> 2 -0.09120193 0.02995157 #> 3 -0.07575833 0.04606864 #> 4 -0.10358142 0.01929507 #> 5 -0.07513088 0.04721823 #> 6 -0.10818311 0.01381653 #> 7 -0.07555774 0.01574991 #> 8 -0.04036620 0.04945369 #> 9 -0.05976988 0.03318306# } # preferences differ for Male and Female immigrants with 'Broken English' ability (m1 <- mm_diffs(immigration, ChosenImmigrant ~ LanguageSkills, ~ Gender, id = ~ CaseID))#> BY statistic outcome feature #> 1 Male - Female mm_difference ChosenImmigrant Language Skills #> 2 Male - Female mm_difference ChosenImmigrant Language Skills #> 3 Male - Female mm_difference ChosenImmigrant Language Skills #> 4 Male - Female mm_difference ChosenImmigrant Language Skills #> level estimate std.error z p #> 1 Fluent English -0.015436300 0.01564954 -0.9863741 0.323949592 #> 2 Broken English -0.047683617 0.01593767 -2.9918819 0.002772635 #> 3 Tried English but Unable -0.009770039 0.01591358 -0.6139436 0.539252607 #> 4 Used Interpreter -0.030670893 0.01594758 -1.9232320 0.054450926 #> lower upper Gender #> 1 -0.04610884 0.0152362347 Male #> 2 -0.07892087 -0.0164463646 Male #> 3 -0.04096008 0.0214199996 Male #> 4 -0.06192757 0.0005857869 Male# yet differences in conditional AMCEs depend on the reference category amce_diffs(immigration, ChosenImmigrant ~ LanguageSkills, ~ Gender, id = ~ CaseID)#> BY outcome statistic feature #> 1 Male - Female ChosenImmigrant amce_difference Language Skills #> 2 Male - Female ChosenImmigrant amce_difference Language Skills #> 3 Male - Female ChosenImmigrant amce_difference Language Skills #> level Gender estimate std.error z p #> 1 Broken English Male -0.032247317 0.02366186 -1.3628393 0.1729332 #> 2 Tried English but Unable Male 0.005666262 0.02313999 0.2448688 0.8065580 #> 3 Used Interpreter Male -0.015234592 0.02388354 -0.6378698 0.5235584 #> lower upper #> 1 -0.07862372 0.01412908 #> 2 -0.03968728 0.05101980 #> 3 -0.06204548 0.03157629immigration$LanguageSkills2 <- relevel(immigration$LanguageSkills, "Used Interpreter") amce_diffs(immigration, ChosenImmigrant ~ LanguageSkills2, ~ Gender, id = ~ CaseID)#> BY outcome statistic feature #> 1 Male - Female ChosenImmigrant amce_difference LanguageSkills2 #> 2 Male - Female ChosenImmigrant amce_difference LanguageSkills2 #> 3 Male - Female ChosenImmigrant amce_difference LanguageSkills2 #> level Gender estimate std.error z p #> 1 Broken English Male -0.01701272 0.02372866 -0.7169694 0.4733930 #> 2 Fluent English Male 0.01523459 0.02388354 0.6378698 0.5235584 #> 3 Tried English but Unable Male 0.02090085 0.02366212 0.8833043 0.3770719 #> lower upper #> 1 -0.06352005 0.02949460 #> 2 -0.03157629 0.06204548 #> 3 -0.02547605 0.06727776# while differences in MMs do not depend on the reference cateory (m2 <- mm_diffs(immigration, ChosenImmigrant ~ LanguageSkills2, ~ Gender, id = ~ CaseID))#> BY statistic outcome feature #> 1 Male - Female mm_difference ChosenImmigrant LanguageSkills2 #> 2 Male - Female mm_difference ChosenImmigrant LanguageSkills2 #> 3 Male - Female mm_difference ChosenImmigrant LanguageSkills2 #> 4 Male - Female mm_difference ChosenImmigrant LanguageSkills2 #> level estimate std.error z p #> 1 Used Interpreter -0.030670893 0.01594758 -1.9232320 0.054450926 #> 2 Fluent English -0.015436300 0.01564954 -0.9863741 0.323949592 #> 3 Broken English -0.047683617 0.01593767 -2.9918819 0.002772635 #> 4 Tried English but Unable -0.009770039 0.01591358 -0.6139436 0.539252607 #> lower upper Gender #> 1 -0.06192757 0.0005857869 Male #> 2 -0.04610884 0.0152362347 Male #> 3 -0.07892087 -0.0164463646 Male #> 4 -0.04096008 0.0214199996 Male