All datasets generated and used for this study are openly available on GitHub: https://github.com/HazelvdW/context-framed-listening.
This document analyses lexical semantic similarity between music-influenced mental content (MIMC) descriptions at the aggregated document level using TF-IDF cosine similarity. Analyses are organised around three research questions:
Note on method: TF-IDF is a bag-of-words method optimised for document-level analysis. Individual short MIMC texts (~10–50 words) produce extremely sparse vectors with minimal lexical overlap; similarity values at that scale would not reflect semantic relationships. This analysis therefore works exclusively with aggregated combMIMC documents (one per clip × context cell, N = 64). Individual- level analysis is handled by MiniLM.
The
SC_SX(same clip, same context) condition is unavailable at the TF-IDF level — each cell is one document and cannot be self-paired. RQ3b (combined alignment advantage) is therefore deferred to the MiniLM pipeline.This document-level lexical analysis complements the MiniLM semantic similarity pipeline; condition categories are identical across both methods to enable direct cross-method comparison.
Using virtual environment '/Users/hazelaileenvanderwalle/.virtualenvs/r-reticulate' ...+ /Users/hazelaileenvanderwalle/.virtualenvs/r-reticulate/bin/python -m pip install --upgrade --no-user pandas numpy scikit-learn scipy matplotlib seaborn wordcloudOutput directory: /Users/hazelaileenvanderwalle/Library/CloudStorage/OneDrive-Personal/work/study/PhD/_PROJECT/_framedListening_NLP/context-framed-listening/NLP_outputs/TFIDFPython 3.9.6 (default, Jan 9 2026, 11:03:41)
[Clang 17.0.0 (clang-1700.6.4.2)]---------------------------------------------------- numpy 2.0.2 OK
pandas 2.3.3 OK
scipy 1.13.1 OK
sklearn 1.6.1 OK
matplotlib 3.9.4 OK
seaborn 0.13.2 OK
wordcloud 1.9.6 OK
All package versions OK.def sig_marker(p):
"""Return significance marker string for a p-value."""
if p < 0.001: return "***"
if p < 0.01: return "**"
if p < 0.05: return "*"
return "n.s."
def welch_t(g1, g2):
"""Welch's t-test returning (t, p, df, mean_diff, ci_low, ci_high)."""
n1, n2 = len(g1), len(g2)
v1, v2 = g1.var(ddof=1), g2.var(ddof=1)
se = np.sqrt(v1/n1 + v2/n2)
t = (g1.mean() - g2.mean()) / se
df = (v1/n1 + v2/n2)**2 / ((v1/n1)**2/(n1-1) + (v2/n2)**2/(n2-1))
p = 2 * (1 - stats.t.cdf(abs(t), df))
mean_diff = float(g1.mean() - g2.mean())
ci_low = mean_diff - 1.96 * se
ci_high = mean_diff + 1.96 * se
return float(t), float(p), float(df), mean_diff, ci_low, ci_high
def cohens_d(g1, g2):
"""Compute Cohen's d effect size."""
pooled = np.sqrt((g1.std()**2 + g2.std()**2) / 2)
return float((g1.mean() - g2.mean()) / pooled) if pooled > 0 else np.nanLoad combMIMC_lvl2a/b.csv — Level 2 preprocessed text data of participants’ MIMC descriptions aggregated into combMIMC documents, generated using NLP-text-prep.qmd.
data_path = f"{base_path}/NLP_outputs/combMIMC_lvl2b.csv"
df = pd.read_csv(data_path)
# lvl2a = aggressive preprocessing with "thought" stop words removed
# lvl2b retains "thought" words
print(f"Shape: {df.shape}")Shape: (64, 7)print(f"Columns: {df.columns.tolist()}")Columns: ['context_word', 'genre_code', 'clip_name', 'ClipContext_pair', 'GenreContext_pair', 'textMIMC_lvl2b', 'text_original']print(f"\nUnique clips: {sorted(df['clip_name'].unique())}")
Unique clips: ['80s_LOW_02', '80s_LOW_06', '80s_MED_08', '80s_MED_13', 'Ele_LOW_09', 'Ele_LOW_14', 'Ele_MED_19', 'Ele_MED_20', 'Jaz_LOW_19', 'Jaz_LOW_21', 'Jaz_MED_02', 'Jaz_MED_07', 'Met_LOW_09', 'Met_LOW_14', 'Met_MED_19', 'Met_MED_20']print(f"Unique contexts: {sorted(df['context_word'].unique())}")Unique contexts: ['bar', 'concert', 'movie', 'videogame']print(f"Unique genres: {sorted(df['genre_code'].unique())}")Unique genres: ['80s', 'Ele', 'Jaz', 'Met']print(f"\nTotal combMIMC documents: {len(df)}")
Total combMIMC documents: 64df.head(4) context_word ... text_original
0 bar ... kind of sad melancholy not happy or upbeat emo...
1 bar ... the music felt really 80s and so it was an ima...
2 bar ... i think about an old 80s movie that takes plac...
3 bar ... i could only think about sitting on the couch ...
[4 rows x 7 columns]TF-IDF matrix shape: (64, 3315) -> 64 documents x 3315 termsMatrix density: 0.0637
Top 20 terms by mean TF-IDF:like 0.063314
imagine 0.062485
sound 0.051336
scene 0.046185
think 0.044116
make 0.042154
people 0.041130
remind 0.037347
dance 0.036263
character 0.034620
something 0.034355
get 0.033599
feel 0.033350
one 0.032823
thought 0.032441
old 0.032322
start 0.032077
hear 0.031700
go 0.031046
show 0.030752Cosine similarity matrix shape: (64, 64)Value range: [0.0661, 1.0000]Mean off-diagonal similarity: 0.1428The pairwise matrix is unpacked into a tidy long-format DataFrame. Each row is a unique document pair annotated with condition labels.
| Condition | Description |
|---|---|
SC_SX |
Same clip and same context — combined alignment |
SC_dx |
Same clip, different context — isolates clip influence |
dc_SX_SG |
Different clip, same context, same genre — joint context + genre alignment |
dc_dx_SG |
Both different, same genre |
dc_SX_dg |
Different clip, same context, different genre — isolates pure context influence |
dc_dx_dg |
Entirely different — dissimilarity baseline |
Total unique pairs: 2016
Condition distribution:condition
SC_dx 96
dc_SX_SG 96
dc_SX_dg 384
dc_dx_SG 288
dc_dx_dg 1152
Saved similarity dataLinear mixed models (LMMs) provide a complementary, model-based approach to the pairwise t-tests above. The outcome is pairwise cosine similarity; fixed effects are context_comb (the context labels from the pair) and genre_comb (the genres of items in the pair); item_i and item_j are included as a random intercept to account for the fact that each individual response appears in multiple pairs.
Three models are fitted in sequence. Model 1 tests main effects of context, genre, and clip — clip is included as both a fixed and random effect to separate clip-specific idiosyncrasies from the genre signal, since genre is fully nested within clip. Model 2 adds mean believability (from the supplementary study) as a covariate at the clip × context cell level (64 unique values, one per clip–context pairing) — this gives genuine within-context variation in believability and avoids the collinearity that would arise from collapsing to context-level means. Model 3 tests the context × genre interaction while retaining clip as a covariate. Model comparisons use likelihood ratio tests (ANOVA on nested models) to determine whether each additional term improves fit.
Following the LMMs, estimated marginal means (emmeans) unpack pairwise context differences while controlling for genre. Finally, a Random Forest classifier and a Linear Discriminant Analysis (LDA) are used to assess how well context condition can be recovered from similarity scores alone.
# ── Packages ──────────────────────────────────────────────────────────────────
required <- c("lme4", "lmerTest", "MuMIn", "emmeans", "ggdist",
"randomForest", "MASS", "dplyr", "ggplot2",
"car", "cowplot", "patchwork")
for (pkg in required) {
if (!requireNamespace(pkg, quietly = TRUE))
install.packages(pkg, repos = "https://cloud.r-project.org")
library(pkg, character.only = TRUE)
}Loading required package: Matrix
Attaching package: 'lmerTest'The following object is masked from 'package:lme4':
lmerThe following object is masked from 'package:stats':
stepWelcome to emmeans.
Caution: You lose important information if you filter this package's results.
See '? untidy'randomForest 4.7-1.2Type rfNews() to see new features/changes/bug fixes.
Attaching package: 'randomForest'The following object is masked from 'package:MuMIn':
importance
Attaching package: 'dplyr'The following object is masked from 'package:MASS':
selectThe following object is masked from 'package:randomForest':
combineThe following objects are masked from 'package:stats':
filter, lagThe following objects are masked from 'package:base':
intersect, setdiff, setequal, union
Attaching package: 'ggplot2'The following object is masked from 'package:randomForest':
marginRegistered S3 method overwritten by 'car':
method from
na.action.merMod lme4Loading required package: carData
Attaching package: 'car'The following object is masked from 'package:dplyr':
recode
Attaching package: 'patchwork'The following object is masked from 'package:cowplot':
align_plotsThe following object is masked from 'package:MASS':
area# Normalise emmeans CI column names — at large N emmeans uses asymp.LCL/UCL
# instead of lower.CL/upper.CL; this ensures ggplot code works consistently
fix_emm_ci <- function(df) {
names(df)[names(df) == "asymp.LCL"] <- "lower.CL"
names(df)[names(df) == "asymp.UCL"] <- "upper.CL"
df
}
# ── Paths ─────────────────────────────────────────────────────────────────────
base_path <- paste0(
"/Users/hazelaileenvanderwalle/Library/CloudStorage/",
"OneDrive-Personal/work/study/PhD/_PROJECT/",
"_framedListening_NLP/context-framed-listening"
)
output_dir <- file.path(base_path, "NLP_outputs", "TFIDF")
sim_df_path <- file.path(output_dir, "TFIDF_similarity_by_condition.csv")
sim_df <- read.csv(sim_df_path, stringsAsFactors = FALSE)
cat("Loaded similarity data:", nrow(sim_df), "pairs\n")Loaded similarity data: 2016 pairscat("Columns:", paste(names(sim_df), collapse = ", "), "\n")Columns: item_i, item_j, similarity, same_clip, same_context, same_genre, condition, clip_i, clip_j, context_i, context_j, genre_i, genre_j # Using sum-to-zero (effects) coding so no reference level is used —
# each coefficient represents deviation from the grand mean,
# anova() gives Type III tests for each level independently
options(contrasts = c("contr.sum", "contr.poly"))
# Define similarity pair combination variables
sim_df$genre_comb <- paste0(sim_df$genre_i, "_",sim_df$genre_j)
sim_df$context_comb <- paste0(sim_df$context_i,"_",sim_df$context_j)
sim_df$clip_comb <- paste0(sim_df$clip_i, "_",sim_df$clip_j)
cat("genre_comb levels:\n"); print(table(sim_df$genre_comb))genre_comb levels:
80s_80s 80s_Ele 80s_Jaz 80s_Met Ele_80s Ele_Ele Ele_Jaz Ele_Met Jaz_80s Jaz_Ele
120 160 160 160 96 120 160 160 96 96
Jaz_Jaz Jaz_Met Met_80s Met_Ele Met_Jaz Met_Met
120 160 96 96 96 120 cat("\ncontext_comb levels:\n"); print(table(sim_df$context_comb))
context_comb levels:
bar_bar bar_concert bar_movie bar_videogame
120 256 256 256
concert_concert concert_movie concert_videogame movie_movie
120 256 256 120
movie_videogame videogame_videogame
256 120 # clip_comb levels levels not pasted as it's quite long
# Factor predictors — comb variables as factors for the LMMs
sim_df$genre_comb <- factor(sim_df$genre_comb)
sim_df$context_comb <- factor(sim_df$context_comb)
sim_df$clip_comb <- factor(sim_df$clip_comb)
sim_df$item_i <- factor(sim_df$item_i)
sim_df$item_j <- factor(sim_df$item_j)
# Note: combMIMC has no PROLIFIC_PID — item_i/item_j (document indices) used as
# random intercepts instead. Each document appears in multiple pairs.
cat("\ncontext_comb levels:", nlevels(sim_df$context_comb), "\n")
context_comb levels: 10 cat("\ngenre_comb levels: ", nlevels(sim_df$genre_comb), "\n")
genre_comb levels: 16 cat("\nclip_comb levels: ", nlevels(sim_df$clip_comb), "\n")
clip_comb levels: 256 # ── Believability covariate ───────────────────────────────────────────────────
supp_path <- file.path(base_path, "dataSUPPstudy.csv")
supp <- read.csv(supp_path, stringsAsFactors = FALSE)
believe_cell <- supp |>
group_by(clip_name, context_word) |>
summarise(believability = mean(rating_likelihood.response, na.rm = TRUE),
.groups = "drop")
believe_cell$clip_merge <- tolower(sapply(
strsplit(sub("\\.mp3$", "", believe_cell$clip_name), "_"),
function(x) {
prefix <- switch(tolower(x[1]),
"electronic" = "ele", "jazz" = "jaz", "metal" = "met", tolower(x[1]))
paste(c(prefix, x[2], x[3]), collapse = "_")
}
))
believe_cell$cntx_merge <- trimws(tolower(believe_cell$context_word))
# Merge believability for item i
sim_df$clip_i_merge <- trimws(tolower(as.character(sim_df$clip_i)))
sim_df$cntx_i_merge <- trimws(tolower(as.character(sim_df$context_i)))
sim_df <- merge(sim_df, believe_cell[, c("clip_merge", "cntx_merge", "believability")],
by.x = c("clip_i_merge", "cntx_i_merge"),
by.y = c("clip_merge", "cntx_merge"),
all.x = TRUE)
names(sim_df)[names(sim_df) == "believability"] <- "believability_i"
# Merge believability for item j
sim_df$clip_j_merge <- trimws(tolower(as.character(sim_df$clip_j)))
sim_df$cntx_j_merge <- trimws(tolower(as.character(sim_df$context_j)))
sim_df <- merge(sim_df,
believe_cell[, c("clip_merge", "cntx_merge", "believability")],
by.x = c("clip_j_merge", "cntx_j_merge"),
by.y = c("clip_merge", "cntx_merge"),
all.x = TRUE)
names(sim_df)[names(sim_df) == "believability"] <- "believability_j"
# Pair-level believability = mean of both items' cell ratings
sim_df$believability_pair <- rowMeans(
sim_df[, c("believability_i", "believability_j")], na.rm = FALSE)
cat("\nRows with full pair believability:", sum(!is.na(sim_df$believability_pair)),
"of", nrow(sim_df), "\n")
Rows with full pair believability: 2016 of 2016 cat("Rows with NA (unmatched):", sum(is.na(sim_df$believability_pair)), "\n")Rows with NA (unmatched): 0 `summarise()` has regrouped the output.
ℹ Summaries were computed grouped by context_comb and genre_comb.
ℹ Output is grouped by context_comb.
ℹ Use `summarise(.groups = "drop_last")` to silence this message.
ℹ Use `summarise(.by = c(context_comb, genre_comb))` for per-operation grouping
(`?dplyr::dplyr_by`) instead.# A tibble: 136 × 4
# Groups: context_comb [10]
context_comb genre_comb M SD
<fct> <fct> <dbl> <dbl>
1 bar_bar 80s_80s 0.161 0.0327
2 bar_bar 80s_Ele 0.153 0.0261
3 bar_bar 80s_Jaz 0.176 0.0290
4 bar_bar 80s_Met 0.144 0.0188
5 bar_bar Ele_Ele 0.151 0.0191
6 bar_bar Ele_Jaz 0.153 0.0186
7 bar_bar Ele_Met 0.143 0.0240
8 bar_bar Jaz_Jaz 0.191 0.0447
9 bar_bar Jaz_Met 0.141 0.0161
10 bar_bar Met_Met 0.183 0.0359
# ℹ 126 more rows# A tibble: 10 × 3
context_comb M SD
<fct> <dbl> <dbl>
1 bar_bar 0.156 0.0285
2 bar_concert 0.150 0.0341
3 bar_movie 0.146 0.0350
4 bar_videogame 0.139 0.0388
5 concert_concert 0.150 0.0315
6 concert_movie 0.140 0.0349
7 concert_videogame 0.133 0.0357
8 movie_movie 0.146 0.0348
9 movie_videogame 0.140 0.0376
10 videogame_videogame 0.140 0.0313# A tibble: 16 × 3
genre_comb M SD
<fct> <dbl> <dbl>
1 80s_80s 0.160 0.0287
2 80s_Ele 0.137 0.0256
3 80s_Jaz 0.157 0.0315
4 80s_Met 0.124 0.0264
5 Ele_80s 0.139 0.0216
6 Ele_Ele 0.158 0.0446
7 Ele_Jaz 0.133 0.0235
8 Ele_Met 0.140 0.0353
9 Jaz_80s 0.147 0.0284
10 Jaz_Ele 0.127 0.0258
11 Jaz_Jaz 0.177 0.0553
12 Jaz_Met 0.122 0.0268
13 Met_80s 0.131 0.0186
14 Met_Ele 0.135 0.0301
15 Met_Jaz 0.127 0.0216
16 Met_Met 0.175 0.0347cat("———MODEL 1: similarity ~ context_comb + genre_comb
+ (1 | clip_comb) + (1 | item_i) + (1 | item_j)———\n")———MODEL 1: similarity ~ context_comb + genre_comb
+ (1 | clip_comb) + (1 | item_i) + (1 | item_j)———model1 <- lmer(similarity ~ context_comb + genre_comb
+ (1 | clip_comb) + (1 | item_i) + (1 | item_j),
data = sim_df, REML = FALSE)
print(anova(model1))Type III Analysis of Variance Table with Satterthwaite's method
Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
context_comb 0.018521 0.0020579 9 91.256 6.0361 1.309e-06 ***
genre_comb 0.055928 0.0037285 15 211.715 10.9363 < 2.2e-16 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1cat("\n———Omnibus fixed effects — no reference level———\n")
———Omnibus fixed effects — no reference level———print(car::Anova(model1, type = 3))Analysis of Deviance Table (Type III Wald chisquare tests)
Response: similarity
Chisq Df Pr(>Chisq)
(Intercept) 4056.666 1 < 2.2e-16 ***
context_comb 54.325 9 1.638e-08 ***
genre_comb 164.044 15 < 2.2e-16 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1cat("\n———Pseudo-R² (Nakagawa & Schielzeth), variance explained:———\n")
———Pseudo-R² (Nakagawa & Schielzeth), variance explained:———print(MuMIn::r.squaredGLMM(model1)) R2m R2c
[1,] 0.2760024 0.7405304cat("\n———EMMean, context combos with 95% CI———\n")
———EMMean, context combos with 95% CI———emm1_ctx <- emmeans(model1, ~ context_comb)Registered S3 methods overwritten by 'broom':
method from
nobs.fitdistr MuMIn
nobs.multinom MuMInprint(summary(emm1_ctx, infer = c(TRUE, FALSE), level = 0.95)) context_comb emmean SE df lower.CL upper.CL
bar_bar 0.161 0.00438 143 0.152 0.170
bar_concert 0.150 0.00391 119 0.142 0.158
bar_movie 0.146 0.00391 119 0.138 0.153
bar_videogame 0.139 0.00391 119 0.131 0.147
concert_concert 0.153 0.00419 156 0.145 0.162
concert_movie 0.140 0.00391 120 0.132 0.147
concert_videogame 0.133 0.00391 120 0.126 0.141
movie_movie 0.148 0.00419 157 0.140 0.157
movie_videogame 0.140 0.00391 120 0.132 0.147
videogame_videogame 0.147 0.00428 154 0.138 0.155
Results are averaged over the levels of: genre_comb
Degrees-of-freedom method: kenward-roger
Confidence level used: 0.95 cat("\n———EMMean, genre combos with 95% CI———\n")
———EMMean, genre combos with 95% CI———emm1_gen <- emmeans(model1, ~ genre_comb)
print(summary(emm1_gen, infer = c(TRUE, FALSE), level = 0.95)) genre_comb emmean SE df lower.CL upper.CL
80s_80s 0.163 0.00680 338 0.149 0.176
80s_Ele 0.137 0.00665 326 0.124 0.150
80s_Jaz 0.158 0.00664 325 0.145 0.171
80s_Met 0.125 0.00664 325 0.112 0.139
Ele_80s 0.142 0.00689 351 0.129 0.156
Ele_Ele 0.165 0.00675 343 0.152 0.178
Ele_Jaz 0.134 0.00665 326 0.121 0.147
Ele_Met 0.142 0.00664 325 0.129 0.155
Jaz_80s 0.147 0.00690 352 0.134 0.161
Jaz_Ele 0.130 0.00683 356 0.116 0.143
Jaz_Jaz 0.181 0.00675 342 0.167 0.194
Jaz_Met 0.122 0.00665 326 0.109 0.135
Met_80s 0.133 0.00693 353 0.119 0.147
Met_Ele 0.139 0.00686 357 0.126 0.153
Met_Jaz 0.133 0.00685 356 0.119 0.146
Met_Met 0.180 0.00677 342 0.167 0.193
Results are averaged over the levels of: context_comb
Degrees-of-freedom method: kenward-roger
Confidence level used: 0.95 cat("\n———random effects variance check———\n")
———random effects variance check———print(summary(model1)$varcor) Groups Name Std.Dev.
clip_comb (Intercept) 0.0208919
item_i (Intercept) 0.0072712
item_j (Intercept) 0.0110012
Residual 0.0184643# ── Context emmeans pointrange plot ───────────────────────────────────────────
emm1_ctx_df <- fix_emm_ci(as.data.frame(
summary(emm1_ctx, infer = c(TRUE, FALSE), level = 0.95)))
p_emm1_ctx <- ggplot(emm1_ctx_df,
aes(x = reorder(context_comb, emmean), y = emmean)) +
geom_pointrange(aes(ymin = lower.CL, ymax = upper.CL),
size = 0.7, linewidth = 1, colour = "#2c7bb6") +
geom_hline(yintercept = mean(emm1_ctx_df$emmean),
linetype = "dashed", colour = "grey50", linewidth = 0.5) +
coord_flip() +
labs(title = "Model 1: Estimated Marginal Means — Context Combination",
subtitle = "Points = emmean; bars = 95% CI; dashed = grand mean",
x = NULL, y = "Cosine similarity (emmean)") +
scale_y_continuous(limits = c(0.10, 0.22)) +
theme_minimal(base_size = 11) +
theme(plot.title = element_text(face = "bold"))
# ── Genre emmeans pointrange plot ─────────────────────────────────────────────
emm1_gen_df <- fix_emm_ci(as.data.frame(
summary(emm1_gen, infer = c(TRUE, FALSE), level = 0.95)))
p_emm1_gen <- ggplot(emm1_gen_df,
aes(x = reorder(genre_comb, emmean), y = emmean)) +
geom_pointrange(aes(ymin = lower.CL, ymax = upper.CL),
size = 0.7, linewidth = 1, colour = "#d7191c") +
geom_hline(yintercept = mean(emm1_gen_df$emmean),
linetype = "dashed", colour = "grey50", linewidth = 0.5) +
coord_flip() +
labs(title = "Model 1: Estimated Marginal Means — Genre Combination",
subtitle = "Points = emmean; bars = 95% CI; dashed = grand mean",
x = NULL, y = "Cosine similarity (emmean)") +
scale_y_continuous(limits = c(0.10, 0.22)) +
theme_minimal(base_size = 11) +
theme(plot.title = element_text(face = "bold"))
p_stacked <- cowplot::plot_grid(p_emm1_ctx, p_emm1_gen, ncol = 1)
print(p_stacked)
ggsave(file.path(output_dir, "TFIDF_emmeans_model1_stacked.png"),
p_stacked, width = 10, height = 14, dpi = 300)
# ── Pairwise context contrasts forest plot ────────────────────────────────────
cat("\n———Pairwise context contrasts (Tukey), with 95% CI———\n")
———Pairwise context contrasts (Tukey), with 95% CI———emm1_ctx_pairs <- pairs(emm1_ctx, adjust = "tukey")
print(summary(emm1_ctx_pairs, infer = c(TRUE, TRUE), level = 0.95)) contrast estimate SE df lower.CL
bar_bar - bar_concert 0.010900 0.00483 73.7 -0.004872
bar_bar - bar_movie 0.015419 0.00483 74.0 -0.000351
bar_bar - bar_videogame 0.021846 0.00483 74.0 0.006077
bar_bar - concert_concert 0.007451 0.00572 118.5 -0.010995
bar_bar - concert_movie 0.021287 0.00556 106.3 0.003331
bar_bar - concert_videogame 0.027642 0.00556 106.3 0.009687
bar_bar - movie_movie 0.012542 0.00572 119.2 -0.005910
bar_bar - movie_videogame 0.021390 0.00556 106.3 0.003434
bar_bar - videogame_videogame 0.014404 0.00579 119.5 -0.004266
bar_concert - bar_movie 0.004519 0.00442 61.3 -0.009992
bar_concert - bar_videogame 0.010946 0.00442 61.4 -0.003563
bar_concert - concert_concert -0.003449 0.00352 102.7 -0.014834
bar_concert - concert_movie 0.010387 0.00520 93.1 -0.006468
bar_concert - concert_videogame 0.016742 0.00520 93.2 -0.000113
bar_concert - movie_movie 0.001642 0.00541 109.7 -0.015822
bar_concert - movie_videogame 0.010489 0.00520 93.5 -0.006364
bar_concert - videogame_videogame 0.003503 0.00548 109.9 -0.014204
bar_movie - bar_videogame 0.006428 0.00442 62.1 -0.008076
bar_movie - concert_concert -0.007968 0.00541 108.8 -0.025442
bar_movie - concert_movie 0.005868 0.00319 67.6 -0.004562
bar_movie - concert_videogame 0.012223 0.00520 93.4 -0.004630
bar_movie - movie_movie -0.002876 0.00352 98.5 -0.014262
bar_movie - movie_videogame 0.005971 0.00520 93.5 -0.010882
bar_movie - videogame_videogame -0.001015 0.00548 110.4 -0.018721
bar_videogame - concert_concert -0.014396 0.00541 108.9 -0.031869
bar_videogame - concert_movie -0.000559 0.00520 93.4 -0.017413
bar_videogame - concert_videogame 0.005796 0.00319 67.6 -0.004634
bar_videogame - movie_movie -0.009304 0.00541 109.2 -0.026770
bar_videogame - movie_videogame -0.000457 0.00319 65.9 -0.010896
bar_videogame - videogame_videogame -0.007443 0.00363 85.8 -0.019241
concert_concert - concert_movie 0.013836 0.00467 77.1 -0.001363
concert_concert - concert_videogame 0.020192 0.00467 77.2 0.004993
concert_concert - movie_movie 0.005092 0.00559 124.6 -0.012911
concert_concert - movie_videogame 0.013939 0.00541 109.9 -0.003531
concert_concert - videogame_videogame 0.006953 0.00565 123.9 -0.011264
concert_movie - concert_videogame 0.006355 0.00442 62.1 -0.008148
concert_movie - movie_movie -0.008744 0.00352 98.4 -0.020130
concert_movie - movie_videogame 0.000103 0.00520 93.5 -0.016750
concert_movie - videogame_videogame -0.006883 0.00548 110.4 -0.024589
concert_videogame - movie_movie -0.015100 0.00541 109.2 -0.032566
concert_videogame - movie_videogame -0.006253 0.00319 65.8 -0.016693
concert_videogame - videogame_videogame -0.013239 0.00363 85.7 -0.025037
movie_movie - movie_videogame 0.008847 0.00466 78.1 -0.006338
movie_movie - videogame_videogame 0.001861 0.00565 123.5 -0.016333
movie_videogame - videogame_videogame -0.006986 0.00363 85.1 -0.018787
upper.CL t.ratio p.value
0.02667 2.255 0.4303
0.03119 3.189 0.0607
0.03762 4.519 0.0009
0.02590 1.303 0.9512
0.03924 3.831 0.0079
0.04560 4.975 0.0001
0.03100 2.192 0.4683
0.03935 3.850 0.0074
0.03307 2.488 0.2859
0.01903 1.022 0.9897
0.02546 2.476 0.3015
0.00794 -0.980 0.9928
0.02724 1.998 0.6028
0.03360 3.220 0.0531
0.01911 0.304 1.0000
0.02734 2.017 0.5892
0.02121 0.639 0.9997
0.02093 1.454 0.9045
0.00951 -1.473 0.8998
0.01630 1.841 0.7075
0.02908 2.351 0.3672
0.00851 -0.818 0.9981
0.02282 1.148 0.9780
0.01669 -0.185 1.0000
0.00308 -2.661 0.2030
0.01629 -0.108 1.0000
0.01623 1.818 0.7217
0.00816 -1.721 0.7814
0.00998 -0.143 1.0000
0.00436 -2.049 0.5676
0.02904 2.966 0.1056
0.03539 4.328 0.0017
0.02309 0.911 0.9959
0.03141 2.577 0.2413
0.02517 1.230 0.9661
0.02086 1.438 0.9102
0.00264 -2.486 0.2888
0.01696 0.020 1.0000
0.01082 -1.255 0.9611
0.00237 -2.793 0.1521
0.00419 -1.961 0.6283
-0.00144 -3.645 0.0157
0.02403 1.897 0.6707
0.02005 0.330 1.0000
0.00481 -1.924 0.6533
Results are averaged over the levels of: genre_comb
Degrees-of-freedom method: kenward-roger
Confidence level used: 0.95
Conf-level adjustment: tukey method for comparing a family of 10 estimates
P value adjustment: tukey method for comparing a family of 10 estimates emm1_pairs_df <- fix_emm_ci(as.data.frame(
summary(emm1_ctx_pairs, infer = c(TRUE, TRUE), level = 0.95)))
# Filtered: same-context vs same-context only
is_same_ctx <- function(label) {
parts <- strsplit(label, " - ")[[1]]
all(sapply(parts, function(p) {
tokens <- strsplit(p, "_")[[1]]
length(tokens) == 2 && tokens[1] == tokens[2]
}))
}
emm1_pairs_key <- emm1_pairs_df[
sapply(as.character(emm1_pairs_df$contrast), is_same_ctx), ]
clean_label <- function(label) {
parts <- strsplit(label, " - ")[[1]]
ctxs <- sapply(parts, function(p) strsplit(p, "_")[[1]][1])
paste0(tools::toTitleCase(ctxs[1]), " vs ", tools::toTitleCase(ctxs[2]))
}
emm1_pairs_key$label <- sapply(as.character(emm1_pairs_key$contrast), clean_label)
p_contrasts_key <- ggplot(emm1_pairs_key,
aes(x = reorder(label, estimate), y = estimate)) +
geom_pointrange(aes(ymin = lower.CL, ymax = upper.CL),
size = 0.8, linewidth = 1.1,
colour = ifelse(emm1_pairs_key$p.value < 0.05,
"#2c7bb6", "grey60")) +
geom_hline(yintercept = 0, linetype = "dashed",
colour = "grey30", linewidth = 0.6) +
coord_flip() +
labs(title = "Model 1: Same-Context Pairwise Contrasts",
subtitle = "Blue = significant (Tukey); bars = 95% CI",
x = NULL, y = "Estimated difference in cosine similarity") +
theme_minimal(base_size = 12) +
theme(plot.title = element_text(face = "bold"),
axis.text.y = element_text(size = 11))
print(p_contrasts_key)
ggsave(file.path(output_dir, "TFIDF_emmeans_model1_contrasts.png"),
p_contrasts_key, width = 8, height = 5, dpi = 300)cat("\n———MODEL 2: context_comb × genre_comb interaction———\n")
———MODEL 2: context_comb × genre_comb interaction———# At N=2016 pairs this is tractable without downsampling
model2 <- lmer(similarity ~ context_comb * genre_comb
+ (1 | clip_comb) + (1 | item_i) + (1 | item_j),
data = sim_df, REML = FALSE)fixed-effect model matrix is rank deficient so dropping 24 columns / coefficientsprint(anova(model2))Missing cells for: context_combbar_bar:genre_combEle_80s, context_combconcert_concert:genre_combEle_80s, context_combmovie_movie:genre_combEle_80s, context_combvideogame_videogame:genre_combEle_80s, context_combbar_bar:genre_combJaz_80s, context_combconcert_concert:genre_combJaz_80s, context_combmovie_movie:genre_combJaz_80s, context_combvideogame_videogame:genre_combJaz_80s, context_combbar_bar:genre_combJaz_Ele, context_combconcert_concert:genre_combJaz_Ele, context_combmovie_movie:genre_combJaz_Ele, context_combvideogame_videogame:genre_combJaz_Ele, context_combbar_bar:genre_combMet_80s, context_combconcert_concert:genre_combMet_80s, context_combmovie_movie:genre_combMet_80s, context_combvideogame_videogame:genre_combMet_80s, context_combbar_bar:genre_combMet_Ele, context_combconcert_concert:genre_combMet_Ele, context_combmovie_movie:genre_combMet_Ele, context_combvideogame_videogame:genre_combMet_Ele, context_combbar_bar:genre_combMet_Jaz, context_combconcert_concert:genre_combMet_Jaz, context_combmovie_movie:genre_combMet_Jaz, context_combvideogame_videogame:genre_combMet_Jaz.
Interpret type III hypotheses with care.Type III Analysis of Variance Table with Satterthwaite's method
Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
context_comb 0.017517 0.00194629 9 95.85 6.3806 4.739e-07 ***
genre_comb 0.047249 0.00314995 15 206.82 10.3266 < 2.2e-16 ***
context_comb:genre_comb 0.067274 0.00060607 111 487.14 1.9869 3.459e-07 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1cat("\n———Omnibus fixed effects — no reference level———\n")
———Omnibus fixed effects — no reference level———print(car::Anova(model2, type = 3))Analysis of Deviance Table (Type III Wald chisquare tests)
Response: similarity
Chisq Df Pr(>Chisq)
(Intercept) 627.796 1 < 2.2e-16 ***
context_comb 37.853 9 1.852e-05 ***
genre_comb 147.921 15 < 2.2e-16 ***
context_comb:genre_comb 220.545 111 3.021e-09 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1cat("\n———Pseudo-R²:———\n")
———Pseudo-R²:———print(MuMIn::r.squaredGLMM(model2)) R2m R2c
[1,] 0.3181222 0.7660277cat("\n———EMMean, context combos with 95% CI———\n")
———EMMean, context combos with 95% CI———emm2_ctx <- emmeans(model2, ~ context_comb)fixed-effect model matrix is rank deficient so dropping 24 columns / coefficientsNOTE: Results may be misleading due to involvement in interactionsprint(summary(emm2_ctx, infer = c(TRUE, FALSE), level = 0.95)) context_comb emmean SE df lower.CL upper.CL
bar_bar nonEst NA NA NA NA
bar_concert 0.150 0.00397 143 0.142 0.158
bar_movie 0.146 0.00397 144 0.138 0.153
bar_videogame 0.139 0.00397 144 0.131 0.147
concert_concert nonEst NA NA NA NA
concert_movie 0.140 0.00397 146 0.132 0.147
concert_videogame 0.133 0.00397 145 0.125 0.141
movie_movie nonEst NA NA NA NA
movie_videogame 0.140 0.00397 145 0.132 0.147
videogame_videogame nonEst NA NA NA NA
Results are averaged over the levels of: genre_comb
Degrees-of-freedom method: kenward-roger
Confidence level used: 0.95 cat("\n———EMMean, genre combos with 95% CI———\n")
———EMMean, genre combos with 95% CI———emm2_gen <- emmeans(model2, ~ genre_comb)fixed-effect model matrix is rank deficient so dropping 24 columns / coefficients
NOTE: Results may be misleading due to involvement in interactionsprint(summary(emm2_gen, infer = c(TRUE, FALSE), level = 0.95)) genre_comb emmean SE df lower.CL upper.CL
80s_80s 0.164 0.00695 370 0.151 0.178
80s_Ele 0.137 0.00678 337 0.124 0.150
80s_Jaz 0.157 0.00678 337 0.143 0.170
80s_Met 0.124 0.00678 337 0.111 0.137
Ele_80s nonEst NA NA NA NA
Ele_Ele 0.162 0.00694 372 0.149 0.176
Ele_Jaz 0.133 0.00678 337 0.120 0.146
Ele_Met 0.140 0.00678 338 0.127 0.154
Jaz_80s nonEst NA NA NA NA
Jaz_Ele nonEst NA NA NA NA
Jaz_Jaz 0.177 0.00694 372 0.164 0.191
Jaz_Met 0.122 0.00678 338 0.109 0.135
Met_80s nonEst NA NA NA NA
Met_Ele nonEst NA NA NA NA
Met_Jaz nonEst NA NA NA NA
Met_Met 0.181 0.00694 372 0.167 0.194
Results are averaged over the levels of: context_comb
Degrees-of-freedom method: kenward-roger
Confidence level used: 0.95 cat("\n———EMMean, context × genre cell means with 95% CI———\n")
———EMMean, context × genre cell means with 95% CI———emm2_int <- emmeans(model2, ~ context_comb | genre_comb)fixed-effect model matrix is rank deficient so dropping 24 columns / coefficientsprint(summary(emm2_int, infer = c(TRUE, FALSE), level = 0.95))genre_comb = 80s_80s:
context_comb emmean SE df lower.CL upper.CL
bar_bar 0.1703 0.01260 362 0.1456 0.195
bar_concert 0.1627 0.00999 351 0.1430 0.182
bar_movie 0.1573 0.00999 362 0.1377 0.177
bar_videogame 0.1518 0.00999 366 0.1322 0.171
concert_concert 0.1649 0.01190 698 0.1416 0.188
concert_movie 0.1624 0.00999 363 0.1428 0.182
concert_videogame 0.1498 0.00999 366 0.1302 0.169
movie_movie 0.1889 0.01190 714 0.1656 0.212
movie_videogame 0.1682 0.00999 364 0.1486 0.188
videogame_videogame 0.1669 0.01190 707 0.1435 0.190
genre_comb = 80s_Ele:
context_comb emmean SE df lower.CL upper.CL
bar_bar 0.1533 0.00999 310 0.1336 0.173
bar_concert 0.1411 0.00999 356 0.1215 0.161
bar_movie 0.1397 0.00999 365 0.1201 0.159
bar_videogame 0.1228 0.00999 368 0.1031 0.142
concert_concert 0.1411 0.00999 356 0.1215 0.161
concert_movie 0.1331 0.00999 365 0.1135 0.153
concert_videogame 0.1254 0.00999 368 0.1057 0.145
movie_movie 0.1442 0.00999 363 0.1246 0.164
movie_videogame 0.1243 0.00999 366 0.1047 0.144
videogame_videogame 0.1443 0.00999 359 0.1246 0.164
genre_comb = 80s_Jaz:
context_comb emmean SE df lower.CL upper.CL
bar_bar 0.1761 0.00999 334 0.1565 0.196
bar_concert 0.1571 0.00999 360 0.1374 0.177
bar_movie 0.1652 0.00999 366 0.1455 0.185
bar_videogame 0.1542 0.00999 369 0.1346 0.174
concert_concert 0.1527 0.00999 360 0.1331 0.172
concert_movie 0.1573 0.00999 366 0.1377 0.177
concert_videogame 0.1437 0.00999 369 0.1241 0.163
movie_movie 0.1625 0.00999 364 0.1429 0.182
movie_videogame 0.1425 0.00999 367 0.1228 0.162
videogame_videogame 0.1536 0.00999 360 0.1340 0.173
genre_comb = 80s_Met:
context_comb emmean SE df lower.CL upper.CL
bar_bar 0.1443 0.00999 346 0.1247 0.164
bar_concert 0.1333 0.00999 363 0.1137 0.153
bar_movie 0.1220 0.00999 368 0.1023 0.142
bar_videogame 0.1045 0.00999 370 0.0848 0.124
concert_concert 0.1420 0.00999 362 0.1224 0.162
concert_movie 0.1208 0.00999 367 0.1012 0.140
concert_videogame 0.1111 0.00999 370 0.0915 0.131
movie_movie 0.1256 0.00999 365 0.1059 0.145
movie_videogame 0.1048 0.00999 367 0.0852 0.124
videogame_videogame 0.1327 0.00999 360 0.1131 0.152
genre_comb = Ele_80s:
context_comb emmean SE df lower.CL upper.CL
bar_bar nonEst NA NA NA NA
bar_concert 0.1414 0.00999 352 0.1218 0.161
bar_movie 0.1346 0.00999 363 0.1150 0.154
bar_videogame 0.1456 0.00999 367 0.1260 0.165
concert_concert nonEst NA NA NA NA
concert_movie 0.1269 0.00999 362 0.1073 0.147
concert_videogame 0.1386 0.00999 366 0.1190 0.158
movie_movie nonEst NA NA NA NA
movie_videogame 0.1482 0.00999 364 0.1285 0.168
videogame_videogame nonEst NA NA NA NA
genre_comb = Ele_Ele:
context_comb emmean SE df lower.CL upper.CL
bar_bar 0.1667 0.01210 603 0.1429 0.190
bar_concert 0.1722 0.00999 357 0.1525 0.192
bar_movie 0.1586 0.00999 365 0.1389 0.178
bar_videogame 0.1567 0.00999 368 0.1371 0.176
concert_concert 0.1751 0.01190 707 0.1518 0.198
concert_movie 0.1538 0.00999 365 0.1342 0.173
concert_videogame 0.1572 0.00999 368 0.1376 0.177
movie_movie 0.1668 0.01190 717 0.1435 0.190
movie_videogame 0.1671 0.00999 365 0.1474 0.187
videogame_videogame 0.1506 0.01190 697 0.1272 0.174
genre_comb = Ele_Jaz:
context_comb emmean SE df lower.CL upper.CL
bar_bar 0.1526 0.00999 334 0.1329 0.172
bar_concert 0.1365 0.00999 360 0.1168 0.156
bar_movie 0.1389 0.00999 366 0.1193 0.159
bar_videogame 0.1299 0.00999 368 0.1103 0.150
concert_concert 0.1294 0.00999 359 0.1098 0.149
concert_movie 0.1272 0.00999 366 0.1075 0.147
concert_videogame 0.1257 0.00999 368 0.1060 0.145
movie_movie 0.1367 0.00999 363 0.1171 0.156
movie_videogame 0.1311 0.00999 365 0.1115 0.151
videogame_videogame 0.1204 0.00999 355 0.1007 0.140
genre_comb = Ele_Met:
context_comb emmean SE df lower.CL upper.CL
bar_bar 0.1432 0.00999 346 0.1235 0.163
bar_concert 0.1430 0.00999 363 0.1234 0.163
bar_movie 0.1339 0.00999 368 0.1142 0.154
bar_videogame 0.1161 0.00999 369 0.0965 0.136
concert_concert 0.1598 0.00999 362 0.1402 0.179
concert_movie 0.1348 0.00999 367 0.1152 0.154
concert_videogame 0.1283 0.00999 369 0.1087 0.148
movie_movie 0.1505 0.00999 364 0.1309 0.170
movie_videogame 0.1451 0.00999 366 0.1254 0.165
videogame_videogame 0.1502 0.00999 355 0.1305 0.170
genre_comb = Jaz_80s:
context_comb emmean SE df lower.CL upper.CL
bar_bar nonEst NA NA NA NA
bar_concert 0.1520 0.00999 352 0.1323 0.172
bar_movie 0.1452 0.00999 363 0.1256 0.165
bar_videogame 0.1570 0.00999 367 0.1373 0.177
concert_concert nonEst NA NA NA NA
concert_movie 0.1359 0.00999 362 0.1163 0.156
concert_videogame 0.1384 0.00999 366 0.1187 0.158
movie_movie nonEst NA NA NA NA
movie_videogame 0.1530 0.00999 362 0.1333 0.173
videogame_videogame nonEst NA NA NA NA
genre_comb = Jaz_Ele:
context_comb emmean SE df lower.CL upper.CL
bar_bar nonEst NA NA NA NA
bar_concert 0.1369 0.00999 357 0.1173 0.157
bar_movie 0.1397 0.00999 366 0.1200 0.159
bar_videogame 0.1255 0.00999 369 0.1059 0.145
concert_concert nonEst NA NA NA NA
concert_movie 0.1202 0.00999 365 0.1006 0.140
concert_videogame 0.1138 0.00999 368 0.0942 0.133
movie_movie nonEst NA NA NA NA
movie_videogame 0.1263 0.00999 364 0.1066 0.146
videogame_videogame nonEst NA NA NA NA
genre_comb = Jaz_Jaz:
context_comb emmean SE df lower.CL upper.CL
bar_bar 0.2020 0.01200 664 0.1785 0.226
bar_concert 0.1832 0.00999 360 0.1635 0.203
bar_movie 0.1962 0.00999 367 0.1766 0.216
bar_videogame 0.1828 0.00999 369 0.1631 0.202
concert_concert 0.1740 0.01190 712 0.1507 0.197
concert_movie 0.1778 0.00999 366 0.1581 0.197
concert_videogame 0.1642 0.00999 368 0.1445 0.184
movie_movie 0.1760 0.01190 717 0.1527 0.199
movie_videogame 0.1726 0.00999 364 0.1529 0.192
videogame_videogame 0.1454 0.01200 665 0.1218 0.169
genre_comb = Jaz_Met:
context_comb emmean SE df lower.CL upper.CL
bar_bar 0.1414 0.00999 346 0.1217 0.161
bar_concert 0.1338 0.00999 362 0.1142 0.153
bar_movie 0.1245 0.00999 367 0.1049 0.144
bar_videogame 0.1037 0.00999 369 0.0841 0.123
concert_concert 0.1473 0.00999 361 0.1277 0.167
concert_movie 0.1202 0.00999 366 0.1005 0.140
concert_videogame 0.0986 0.00999 368 0.0790 0.118
movie_movie 0.1222 0.00999 363 0.1025 0.142
movie_videogame 0.1097 0.00999 364 0.0901 0.129
videogame_videogame 0.1180 0.00999 346 0.0984 0.138
genre_comb = Met_80s:
context_comb emmean SE df lower.CL upper.CL
bar_bar nonEst NA NA NA NA
bar_concert 0.1389 0.00999 351 0.1192 0.159
bar_movie 0.1271 0.00999 362 0.1074 0.147
bar_videogame 0.1349 0.00999 366 0.1153 0.155
concert_concert nonEst NA NA NA NA
concert_movie 0.1279 0.00999 361 0.1082 0.148
concert_videogame 0.1253 0.00999 365 0.1056 0.145
movie_movie nonEst NA NA NA NA
movie_videogame 0.1293 0.00999 360 0.1097 0.149
videogame_videogame nonEst NA NA NA NA
genre_comb = Met_Ele:
context_comb emmean SE df lower.CL upper.CL
bar_bar nonEst NA NA NA NA
bar_concert 0.1386 0.00999 357 0.1190 0.158
bar_movie 0.1401 0.00999 365 0.1205 0.160
bar_videogame 0.1340 0.00999 368 0.1144 0.154
concert_concert nonEst NA NA NA NA
concert_movie 0.1350 0.00999 364 0.1154 0.155
concert_videogame 0.1295 0.00999 367 0.1098 0.149
movie_movie nonEst NA NA NA NA
movie_videogame 0.1311 0.00999 362 0.1115 0.151
videogame_videogame nonEst NA NA NA NA
genre_comb = Met_Jaz:
context_comb emmean SE df lower.CL upper.CL
bar_bar nonEst NA NA NA NA
bar_concert 0.1397 0.00999 360 0.1200 0.159
bar_movie 0.1318 0.00999 367 0.1122 0.151
bar_videogame 0.1281 0.00999 369 0.1085 0.148
concert_concert nonEst NA NA NA NA
concert_movie 0.1298 0.00999 365 0.1102 0.149
concert_videogame 0.1185 0.00999 368 0.0989 0.138
movie_movie nonEst NA NA NA NA
movie_videogame 0.1158 0.00999 363 0.0961 0.135
videogame_videogame nonEst NA NA NA NA
genre_comb = Met_Met:
context_comb emmean SE df lower.CL upper.CL
bar_bar 0.1923 0.01190 689 0.1689 0.216
bar_concert 0.1900 0.00999 362 0.1703 0.210
bar_movie 0.1733 0.00999 367 0.1537 0.193
bar_videogame 0.1776 0.00999 369 0.1580 0.197
concert_concert 0.2105 0.01190 715 0.1873 0.234
concert_movie 0.1710 0.00999 366 0.1513 0.191
concert_videogame 0.1644 0.00999 367 0.1447 0.184
movie_movie 0.1704 0.01190 714 0.1471 0.194
movie_videogame 0.1636 0.00999 363 0.1439 0.183
videogame_videogame 0.1935 0.01220 554 0.1697 0.217
Degrees-of-freedom method: kenward-roger
Confidence level used: 0.95 cat("\n———random effects variance check———\n")
———random effects variance check———print(summary(model2)$varcor) Groups Name Std.Dev.
clip_comb (Intercept) 0.0210355
item_i (Intercept) 0.0061429
item_j (Intercept) 0.0101842
Residual 0.0174652# ── Faceted interaction plot ───────────────────────────────────────────────────
emm2_int_df <- fix_emm_ci(as.data.frame(
summary(emm2_int, infer = c(TRUE, FALSE), level = 0.95)))
emm2_int_df$genre_i <- sub("_.*", "", emm2_int_df$genre_comb)
emm2_int_df$genre_j <- sub(".*_", "", emm2_int_df$genre_comb)
emm2_int_df$pair_type <- ifelse(
emm2_int_df$genre_i == emm2_int_df$genre_j, "Same genre", "Cross genre")
genre_colours <- c("80s" = "#fc8a1e", "Electronic" = "#7dcea0",
"Jazz" = "#8552b6", "Metal" = "#9e1100")
p_emm2_int <- ggplot(emm2_int_df,
aes(x = context_comb, y = emmean,
colour = genre_j, group = genre_comb,
alpha = pair_type, linewidth = pair_type)) +
geom_line() +
geom_pointrange(aes(ymin = lower.CL, ymax = upper.CL),
size = 0.45, linewidth = 0.5,
position = position_dodge(width = 0.25)) +
scale_colour_manual(values = genre_colours, name = "Genre (item j)") +
scale_alpha_manual(values = c("Same genre" = 1, "Cross genre" = 0.35),
guide = "none") +
scale_linewidth_manual(values = c("Same genre" = 1.2, "Cross genre" = 0.5),
guide = "none") +
facet_wrap(~ genre_i, ncol = 2,
labeller = labeller(genre_i = c(
"80s" = "Genre i: 80s", "Electronic" = "Genre i: Electronic",
"Jazz" = "Genre i: Jazz", "Metal" = "Genre i: Metal"))) +
labs(title = "Model 2: Context × Genre Estimated Marginal Means",
subtitle = "Faceted by genre_i; coloured by genre_j; thick lines = same-genre pairs",
x = "Context combination", y = "Cosine similarity (emmean)") +
theme_minimal(base_size = 10) +
theme(plot.title = element_text(face = "bold"),
axis.text.x = element_text(angle = 35, hjust = 1, size = 8),
strip.text = element_text(face = "bold", size = 10),
legend.position = "right",
panel.spacing = unit(1.2, "lines"))
print(p_emm2_int)Warning: Removed 12 rows containing missing values or values outside the scale range
(`geom_line()`).Warning: Removed 24 rows containing missing values or values outside the scale range
(`geom_pointrange()`).
ggsave(file.path(output_dir, "TFIDF_emmeans_model2_interaction_faceted.png"),
p_emm2_int, width = 14, height = 10, dpi = 300)Warning: Removed 12 rows containing missing values or values outside the scale range
(`geom_line()`).
Removed 24 rows containing missing values or values outside the scale range
(`geom_pointrange()`).cat("\n———Model 1 vs Model 2 (genre interaction):———\n")
———Model 1 vs Model 2 (genre interaction):———print(anova(model1, model2))Data: sim_df
Models:
model1: similarity ~ context_comb + genre_comb + (1 | clip_comb) + (1 | item_i) + (1 | item_j)
model2: similarity ~ context_comb * genre_comb + (1 | clip_comb) + (1 | item_i) + (1 | item_j)
npar AIC BIC logLik -2*log(L) Chisq Df Pr(>Chisq)
model1 29 -9512.0 -9349.3 4785.0 -9570.0
model2 140 -9497.4 -8712.2 4888.7 -9777.4 207.41 111 8.041e-08 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1cat("———ESTIMATED MARGINAL MEANS — Model 2 (interaction)———\n")———ESTIMATED MARGINAL MEANS — Model 2 (interaction)———cat("(pairwise context contrasts, Tukey adjustment)\n\n")(pairwise context contrasts, Tukey adjustment)emm2_ctx_pairs <- emmeans(model2, pairwise ~ context_comb, adjust = "tukey")fixed-effect model matrix is rank deficient so dropping 24 columns / coefficientsNOTE: Results may be misleading due to involvement in interactionsprint(summary(emm2_ctx_pairs, infer = c(TRUE, TRUE), level = 0.95))$emmeans
context_comb emmean SE df lower.CL upper.CL t.ratio p.value
bar_bar nonEst NA NA NA NA NA NA
bar_concert 0.150 0.00397 143 0.142 0.158 37.754 <0.0001
bar_movie 0.146 0.00397 144 0.138 0.153 36.617 <0.0001
bar_videogame 0.139 0.00397 144 0.131 0.147 35.000 <0.0001
concert_concert nonEst NA NA NA NA NA NA
concert_movie 0.140 0.00397 146 0.132 0.147 35.140 <0.0001
concert_videogame 0.133 0.00397 145 0.125 0.141 33.541 <0.0001
movie_movie nonEst NA NA NA NA NA NA
movie_videogame 0.140 0.00397 145 0.132 0.147 35.114 <0.0001
videogame_videogame nonEst NA NA NA NA NA NA
Results are averaged over the levels of: genre_comb
Degrees-of-freedom method: kenward-roger
Confidence level used: 0.95
$contrasts
contrast estimate SE df lower.CL
bar_bar - bar_concert nonEst NA NA NA
bar_bar - bar_movie nonEst NA NA NA
bar_bar - bar_videogame nonEst NA NA NA
bar_bar - concert_concert nonEst NA NA NA
bar_bar - concert_movie nonEst NA NA NA
bar_bar - concert_videogame nonEst NA NA NA
bar_bar - movie_movie nonEst NA NA NA
bar_bar - movie_videogame nonEst NA NA NA
bar_bar - videogame_videogame nonEst NA NA NA
bar_concert - bar_movie 0.004519 0.00456 78.7 -0.00881
bar_concert - bar_videogame 0.010946 0.00456 79.0 -0.00238
bar_concert - concert_concert nonEst NA NA NA
bar_concert - concert_movie 0.010387 0.00529 117.0 -0.00495
bar_concert - concert_videogame 0.016742 0.00529 117.2 0.00141
bar_concert - movie_movie nonEst NA NA NA
bar_concert - movie_videogame 0.010489 0.00529 118.5 -0.00484
bar_concert - videogame_videogame nonEst NA NA NA
bar_movie - bar_videogame 0.006428 0.00456 81.4 -0.00689
bar_movie - concert_concert nonEst NA NA NA
bar_movie - concert_movie 0.005868 0.00312 99.1 -0.00320
bar_movie - concert_videogame 0.012223 0.00529 118.3 -0.00311
bar_movie - movie_movie nonEst NA NA NA
bar_movie - movie_videogame 0.005971 0.00529 118.7 -0.00936
bar_movie - videogame_videogame nonEst NA NA NA
bar_videogame - concert_concert nonEst NA NA NA
bar_videogame - concert_movie -0.000559 0.00529 118.3 -0.01589
bar_videogame - concert_videogame 0.005796 0.00312 99.1 -0.00327
bar_videogame - movie_movie nonEst NA NA NA
bar_videogame - movie_videogame -0.000457 0.00312 91.2 -0.00954
bar_videogame - videogame_videogame nonEst NA NA NA
concert_concert - concert_movie nonEst NA NA NA
concert_concert - concert_videogame nonEst NA NA NA
concert_concert - movie_movie nonEst NA NA NA
concert_concert - movie_videogame nonEst NA NA NA
concert_concert - videogame_videogame nonEst NA NA NA
concert_movie - concert_videogame 0.006355 0.00456 81.4 -0.00696
concert_movie - movie_movie nonEst NA NA NA
concert_movie - movie_videogame 0.000103 0.00529 118.7 -0.01523
concert_movie - videogame_videogame nonEst NA NA NA
concert_videogame - movie_movie nonEst NA NA NA
concert_videogame - movie_videogame -0.006253 0.00312 90.8 -0.01533
concert_videogame - videogame_videogame nonEst NA NA NA
movie_movie - movie_videogame nonEst NA NA NA
movie_movie - videogame_videogame nonEst NA NA NA
movie_videogame - videogame_videogame nonEst NA NA NA
upper.CL t.ratio p.value
NA NA NA
NA NA NA
NA NA NA
NA NA NA
NA NA NA
NA NA NA
NA NA NA
NA NA NA
NA NA NA
0.01784 0.991 0.9196
0.02427 2.400 0.1690
NA NA NA
0.02572 1.963 0.3700
0.03208 3.164 0.0238
NA NA NA
0.02582 1.982 0.3588
NA NA NA
0.01974 1.409 0.7215
NA NA NA
0.01493 1.881 0.4197
0.02755 2.310 0.1984
NA NA NA
0.02130 1.128 0.8687
NA NA NA
NA NA NA
0.01477 -0.106 1.0000
0.01486 1.858 0.4339
NA NA NA
0.00862 -0.146 1.0000
NA NA NA
NA NA NA
NA NA NA
NA NA NA
NA NA NA
NA NA NA
0.01967 1.393 0.7309
NA NA NA
0.01543 0.019 1.0000
NA NA NA
NA NA NA
0.00283 -2.005 0.3479
NA NA NA
NA NA NA
NA NA NA
NA NA NA
Results are averaged over the levels of: genre_comb
Degrees-of-freedom method: kenward-roger
Confidence level used: 0.95
Conf-level adjustment: tukey method for comparing a family of 6 estimates
P value adjustment: tukey method for comparing a family of 6 estimates cat("\n———context × genre cell means (Model 2):———\n")
———context × genre cell means (Model 2):———print(summary(emmeans(model2, ~ context_comb | genre_comb),
infer = c(TRUE, FALSE), level = 0.95))fixed-effect model matrix is rank deficient so dropping 24 columns / coefficientsgenre_comb = 80s_80s:
context_comb emmean SE df lower.CL upper.CL
bar_bar 0.1703 0.01260 362 0.1456 0.195
bar_concert 0.1627 0.00999 351 0.1430 0.182
bar_movie 0.1573 0.00999 362 0.1377 0.177
bar_videogame 0.1518 0.00999 366 0.1322 0.171
concert_concert 0.1649 0.01190 698 0.1416 0.188
concert_movie 0.1624 0.00999 363 0.1428 0.182
concert_videogame 0.1498 0.00999 366 0.1302 0.169
movie_movie 0.1889 0.01190 714 0.1656 0.212
movie_videogame 0.1682 0.00999 364 0.1486 0.188
videogame_videogame 0.1669 0.01190 707 0.1435 0.190
genre_comb = 80s_Ele:
context_comb emmean SE df lower.CL upper.CL
bar_bar 0.1533 0.00999 310 0.1336 0.173
bar_concert 0.1411 0.00999 356 0.1215 0.161
bar_movie 0.1397 0.00999 365 0.1201 0.159
bar_videogame 0.1228 0.00999 368 0.1031 0.142
concert_concert 0.1411 0.00999 356 0.1215 0.161
concert_movie 0.1331 0.00999 365 0.1135 0.153
concert_videogame 0.1254 0.00999 368 0.1057 0.145
movie_movie 0.1442 0.00999 363 0.1246 0.164
movie_videogame 0.1243 0.00999 366 0.1047 0.144
videogame_videogame 0.1443 0.00999 359 0.1246 0.164
genre_comb = 80s_Jaz:
context_comb emmean SE df lower.CL upper.CL
bar_bar 0.1761 0.00999 334 0.1565 0.196
bar_concert 0.1571 0.00999 360 0.1374 0.177
bar_movie 0.1652 0.00999 366 0.1455 0.185
bar_videogame 0.1542 0.00999 369 0.1346 0.174
concert_concert 0.1527 0.00999 360 0.1331 0.172
concert_movie 0.1573 0.00999 366 0.1377 0.177
concert_videogame 0.1437 0.00999 369 0.1241 0.163
movie_movie 0.1625 0.00999 364 0.1429 0.182
movie_videogame 0.1425 0.00999 367 0.1228 0.162
videogame_videogame 0.1536 0.00999 360 0.1340 0.173
genre_comb = 80s_Met:
context_comb emmean SE df lower.CL upper.CL
bar_bar 0.1443 0.00999 346 0.1247 0.164
bar_concert 0.1333 0.00999 363 0.1137 0.153
bar_movie 0.1220 0.00999 368 0.1023 0.142
bar_videogame 0.1045 0.00999 370 0.0848 0.124
concert_concert 0.1420 0.00999 362 0.1224 0.162
concert_movie 0.1208 0.00999 367 0.1012 0.140
concert_videogame 0.1111 0.00999 370 0.0915 0.131
movie_movie 0.1256 0.00999 365 0.1059 0.145
movie_videogame 0.1048 0.00999 367 0.0852 0.124
videogame_videogame 0.1327 0.00999 360 0.1131 0.152
genre_comb = Ele_80s:
context_comb emmean SE df lower.CL upper.CL
bar_bar nonEst NA NA NA NA
bar_concert 0.1414 0.00999 352 0.1218 0.161
bar_movie 0.1346 0.00999 363 0.1150 0.154
bar_videogame 0.1456 0.00999 367 0.1260 0.165
concert_concert nonEst NA NA NA NA
concert_movie 0.1269 0.00999 362 0.1073 0.147
concert_videogame 0.1386 0.00999 366 0.1190 0.158
movie_movie nonEst NA NA NA NA
movie_videogame 0.1482 0.00999 364 0.1285 0.168
videogame_videogame nonEst NA NA NA NA
genre_comb = Ele_Ele:
context_comb emmean SE df lower.CL upper.CL
bar_bar 0.1667 0.01210 603 0.1429 0.190
bar_concert 0.1722 0.00999 357 0.1525 0.192
bar_movie 0.1586 0.00999 365 0.1389 0.178
bar_videogame 0.1567 0.00999 368 0.1371 0.176
concert_concert 0.1751 0.01190 707 0.1518 0.198
concert_movie 0.1538 0.00999 365 0.1342 0.173
concert_videogame 0.1572 0.00999 368 0.1376 0.177
movie_movie 0.1668 0.01190 717 0.1435 0.190
movie_videogame 0.1671 0.00999 365 0.1474 0.187
videogame_videogame 0.1506 0.01190 697 0.1272 0.174
genre_comb = Ele_Jaz:
context_comb emmean SE df lower.CL upper.CL
bar_bar 0.1526 0.00999 334 0.1329 0.172
bar_concert 0.1365 0.00999 360 0.1168 0.156
bar_movie 0.1389 0.00999 366 0.1193 0.159
bar_videogame 0.1299 0.00999 368 0.1103 0.150
concert_concert 0.1294 0.00999 359 0.1098 0.149
concert_movie 0.1272 0.00999 366 0.1075 0.147
concert_videogame 0.1257 0.00999 368 0.1060 0.145
movie_movie 0.1367 0.00999 363 0.1171 0.156
movie_videogame 0.1311 0.00999 365 0.1115 0.151
videogame_videogame 0.1204 0.00999 355 0.1007 0.140
genre_comb = Ele_Met:
context_comb emmean SE df lower.CL upper.CL
bar_bar 0.1432 0.00999 346 0.1235 0.163
bar_concert 0.1430 0.00999 363 0.1234 0.163
bar_movie 0.1339 0.00999 368 0.1142 0.154
bar_videogame 0.1161 0.00999 369 0.0965 0.136
concert_concert 0.1598 0.00999 362 0.1402 0.179
concert_movie 0.1348 0.00999 367 0.1152 0.154
concert_videogame 0.1283 0.00999 369 0.1087 0.148
movie_movie 0.1505 0.00999 364 0.1309 0.170
movie_videogame 0.1451 0.00999 366 0.1254 0.165
videogame_videogame 0.1502 0.00999 355 0.1305 0.170
genre_comb = Jaz_80s:
context_comb emmean SE df lower.CL upper.CL
bar_bar nonEst NA NA NA NA
bar_concert 0.1520 0.00999 352 0.1323 0.172
bar_movie 0.1452 0.00999 363 0.1256 0.165
bar_videogame 0.1570 0.00999 367 0.1373 0.177
concert_concert nonEst NA NA NA NA
concert_movie 0.1359 0.00999 362 0.1163 0.156
concert_videogame 0.1384 0.00999 366 0.1187 0.158
movie_movie nonEst NA NA NA NA
movie_videogame 0.1530 0.00999 362 0.1333 0.173
videogame_videogame nonEst NA NA NA NA
genre_comb = Jaz_Ele:
context_comb emmean SE df lower.CL upper.CL
bar_bar nonEst NA NA NA NA
bar_concert 0.1369 0.00999 357 0.1173 0.157
bar_movie 0.1397 0.00999 366 0.1200 0.159
bar_videogame 0.1255 0.00999 369 0.1059 0.145
concert_concert nonEst NA NA NA NA
concert_movie 0.1202 0.00999 365 0.1006 0.140
concert_videogame 0.1138 0.00999 368 0.0942 0.133
movie_movie nonEst NA NA NA NA
movie_videogame 0.1263 0.00999 364 0.1066 0.146
videogame_videogame nonEst NA NA NA NA
genre_comb = Jaz_Jaz:
context_comb emmean SE df lower.CL upper.CL
bar_bar 0.2020 0.01200 664 0.1785 0.226
bar_concert 0.1832 0.00999 360 0.1635 0.203
bar_movie 0.1962 0.00999 367 0.1766 0.216
bar_videogame 0.1828 0.00999 369 0.1631 0.202
concert_concert 0.1740 0.01190 712 0.1507 0.197
concert_movie 0.1778 0.00999 366 0.1581 0.197
concert_videogame 0.1642 0.00999 368 0.1445 0.184
movie_movie 0.1760 0.01190 717 0.1527 0.199
movie_videogame 0.1726 0.00999 364 0.1529 0.192
videogame_videogame 0.1454 0.01200 665 0.1218 0.169
genre_comb = Jaz_Met:
context_comb emmean SE df lower.CL upper.CL
bar_bar 0.1414 0.00999 346 0.1217 0.161
bar_concert 0.1338 0.00999 362 0.1142 0.153
bar_movie 0.1245 0.00999 367 0.1049 0.144
bar_videogame 0.1037 0.00999 369 0.0841 0.123
concert_concert 0.1473 0.00999 361 0.1277 0.167
concert_movie 0.1202 0.00999 366 0.1005 0.140
concert_videogame 0.0986 0.00999 368 0.0790 0.118
movie_movie 0.1222 0.00999 363 0.1025 0.142
movie_videogame 0.1097 0.00999 364 0.0901 0.129
videogame_videogame 0.1180 0.00999 346 0.0984 0.138
genre_comb = Met_80s:
context_comb emmean SE df lower.CL upper.CL
bar_bar nonEst NA NA NA NA
bar_concert 0.1389 0.00999 351 0.1192 0.159
bar_movie 0.1271 0.00999 362 0.1074 0.147
bar_videogame 0.1349 0.00999 366 0.1153 0.155
concert_concert nonEst NA NA NA NA
concert_movie 0.1279 0.00999 361 0.1082 0.148
concert_videogame 0.1253 0.00999 365 0.1056 0.145
movie_movie nonEst NA NA NA NA
movie_videogame 0.1293 0.00999 360 0.1097 0.149
videogame_videogame nonEst NA NA NA NA
genre_comb = Met_Ele:
context_comb emmean SE df lower.CL upper.CL
bar_bar nonEst NA NA NA NA
bar_concert 0.1386 0.00999 357 0.1190 0.158
bar_movie 0.1401 0.00999 365 0.1205 0.160
bar_videogame 0.1340 0.00999 368 0.1144 0.154
concert_concert nonEst NA NA NA NA
concert_movie 0.1350 0.00999 364 0.1154 0.155
concert_videogame 0.1295 0.00999 367 0.1098 0.149
movie_movie nonEst NA NA NA NA
movie_videogame 0.1311 0.00999 362 0.1115 0.151
videogame_videogame nonEst NA NA NA NA
genre_comb = Met_Jaz:
context_comb emmean SE df lower.CL upper.CL
bar_bar nonEst NA NA NA NA
bar_concert 0.1397 0.00999 360 0.1200 0.159
bar_movie 0.1318 0.00999 367 0.1122 0.151
bar_videogame 0.1281 0.00999 369 0.1085 0.148
concert_concert nonEst NA NA NA NA
concert_movie 0.1298 0.00999 365 0.1102 0.149
concert_videogame 0.1185 0.00999 368 0.0989 0.138
movie_movie nonEst NA NA NA NA
movie_videogame 0.1158 0.00999 363 0.0961 0.135
videogame_videogame nonEst NA NA NA NA
genre_comb = Met_Met:
context_comb emmean SE df lower.CL upper.CL
bar_bar 0.1923 0.01190 689 0.1689 0.216
bar_concert 0.1900 0.00999 362 0.1703 0.210
bar_movie 0.1733 0.00999 367 0.1537 0.193
bar_videogame 0.1776 0.00999 369 0.1580 0.197
concert_concert 0.2105 0.01190 715 0.1873 0.234
concert_movie 0.1710 0.00999 366 0.1513 0.191
concert_videogame 0.1644 0.00999 367 0.1447 0.184
movie_movie 0.1704 0.01190 714 0.1471 0.194
movie_videogame 0.1636 0.00999 363 0.1439 0.183
videogame_videogame 0.1935 0.01220 554 0.1697 0.217
Degrees-of-freedom method: kenward-roger
Confidence level used: 0.95 Logic: The clearest test of whether context cues do anything at all is to compare dc_SX_dg (different music, same context label) against dc_dx_dg (everything different — the dissimilarity floor). If context drives convergence, pairs sharing a context label should be more similar than pairs sharing nothing.
An omnibus Kruskal–Wallis test across all five conditions is run first to establish that condition membership explains variation before pairwise tests are interpreted.
OMNIBUS KRUSKAL–WALLIS TEST ACROSS ALL CONDITIONS============================================================ H(4) = 342.593, p = 6.9701e-73 *** Conditions tested: ['SC_dx', 'dc_SX_SG', 'dc_SX_dg', 'dc_dx_SG', 'dc_dx_dg']
Post-hoc pairwise Mann–Whitney U (Bonferroni corrected)------------------------------------------------------------ cond1 cond2 mean1 mean2 d p p_bonf sig_bonf
SC_dx dc_dx_dg 0.204137 0.132206 1.983544 2.547176e-46 2.547176e-45 ***
SC_dx dc_SX_dg 0.204137 0.143515 1.655360 3.600622e-34 3.600622e-33 ***
SC_dx dc_dx_SG 0.204137 0.156091 1.190881 6.587098e-21 6.587098e-20 ***
dc_SX_SG dc_dx_dg 0.165580 0.132206 1.015847 3.240886e-17 3.240886e-16 ***
SC_dx dc_SX_SG 0.204137 0.165580 0.963912 5.070595e-10 5.070595e-09 ***
dc_dx_SG dc_dx_dg 0.156091 0.132206 0.717875 2.308288e-23 2.308288e-22 ***
dc_SX_SG dc_SX_dg 0.165580 0.143515 0.663664 1.836084e-07 1.836084e-06 ***
dc_SX_dg dc_dx_dg 0.143515 0.132206 0.394785 2.266438e-11 2.266438e-10 ***
dc_SX_dg dc_dx_SG 0.143515 0.156091 -0.373620 6.668185e-05 6.668185e-04 ***
dc_SX_SG dc_dx_SG 0.165580 0.156091 0.254333 2.113173e-02 2.113173e-01 n.s.
Saved omnibus post-hoc tabledc_SX_dg) and joint alignment (dc_SX_SG)The dc_SX_dg condition previously collapsed same-context pairs regardless of whether the two items shared a genre. Splitting by genre gives two distinct tests: dc_SX_dg (different clip, same context, different genre) isolates the pure context effect with genre controlled out; dc_SX_SG (different clip, same context, same genre) captures joint context + genre alignment.
RQ1 — CONTEXT EFFECT (pure isolation: dc_SX_dg vs baseline)================================================================= dc_SX_dg (diff clip, same ctx, diff genre) M = 0.1435 (N = 384) dc_dx_dg (baseline) M = 0.1322 (N = 1152) Mean diff = +0.0113, 95% CI [0.0080, 0.0146] Welch's t(639.2) = 6.647, p = 0.0000 *** Cohen's d = 0.395 [note: d benchmarks not appropriate for pairwise data]
RQ1 — JOINT CONTEXT + GENRE ALIGNMENT (dc_SX_SG vs baseline)================================================================= dc_SX_SG (diff clip, same ctx, same genre) M = 0.1656 (N = 96) dc_dx_dg (baseline) M = 0.1322 (N = 1152) Mean diff = +0.0334, 95% CI [0.0244, 0.0424] Welch's t(104.4) = 8.646, p = 0.0000 *** Cohen's d = 1.016 [note: d benchmarks not appropriate for pairwise data]
Logic: Two complementary tests. Framing susceptibility: per genre, how much does sharing a context label raise similarity above the dissimilarity floor? Within-genre clip vs context dominance: within each genre, does clip or context drive more convergence? Model 2 (context × genre interaction) is presented as the model-based complement to these tests.
RQ2 — GENRE FRAMING SUSCEPTIBILITY======================================================================Framing gain = dc_SX_SG M − dc_dx_SG M (per genre)---------------------------------------------------------------------- 80S Ctx M=0.1632 Base M=0.1499 Gain=+0.0133 95% CI [0.0015, 0.0251] d=0.523 p=0.0327
ELE Ctx M=0.1480 Base M=0.1426 Gain=+0.0054 95% CI [-0.0096, 0.0204] d=0.166 p=0.4840
JAZ Ctx M=0.1669 Base M=0.1626 Gain=+0.0043 95% CI [-0.0179, 0.0265] d=0.089 p=0.7049
MET Ctx M=0.1841 Base M=0.1692 Gain=+0.0149 95% CI [0.0002, 0.0297] d=0.454 p=0.0537
Genres ranked by framing susceptibility (highest to lowest):
MET Gain=+0.0149 95% CI [0.0002, 0.0297] d=0.454 sig_bonf=n.s.
80S Gain=+0.0133 95% CI [0.0015, 0.0251] d=0.523 sig_bonf=n.s.
ELE Gain=+0.0054 95% CI [-0.0096, 0.0204] d=0.166 sig_bonf=n.s.
JAZ Gain=+0.0043 95% CI [-0.0179, 0.0265] d=0.089 sig_bonf=n.s.
Saved genre framing susceptibility tableRQ2 — WITHIN-GENRE CLIP vs CONTEXT DOMINANCE======================================================================Clip-driven: SC_dx (same clip, diff context, same genre)Context-driven: dc_SX_SG (diff clip, same context, same genre)---------------------------------------------------------------------- 80S Clip M=0.1851 Ctx M=0.1632 95% CI [0.0075, 0.0362] -> CLIP dominant d=0.863 p=0.0045
ELE Clip M=0.2160 Ctx M=0.1480 95% CI [0.0474, 0.0885] -> CLIP dominant d=1.872 p=0.0000
JAZ Clip M=0.2299 Ctx M=0.1669 95% CI [0.0353, 0.0907] -> CLIP dominant d=1.285 p=0.0001
MET Clip M=0.1855 Ctx M=0.1841 95% CI [-0.0175, 0.0203] -> CLIP dominant d=0.042 p=0.8845
Saved within-genre clip vs context moderator table<BarContainer object of 4 artists>
<matplotlib.lines.Line2D object at 0x17caa58b0>
[<matplotlib.axis.YTick object at 0x17ca43d60>, <matplotlib.axis.YTick object at 0x17ca45670>, <matplotlib.axis.YTick object at 0x177edb040>, <matplotlib.axis.YTick object at 0x17cab3f10>]
[Text(0, 0, 'JAZ'), Text(0, 1, 'ELE'), Text(0, 2, '80S'), Text(0, 3, 'MET')]
Text(0.5, 0, 'Framing Gain\n(dc_SX_SG − dc_dx_SG)')
Text(0.5, 1.0, 'A. Genre Framing Susceptibility\n(Higher = More Context-Sensitive)')
Text(0.006320898562834186, 0, 'n.s.')
Text(0.007406819589279799, 1, 'n.s.')
Text(0.015310712812802022, 2, 'n.s.')
Text(0.016917572568004163, 3, 'n.s.')
<BarContainer object of 4 artists>
<matplotlib.lines.Line2D object at 0x17cac0bb0>
<matplotlib.lines.Line2D object at 0x17ca64a00>
<matplotlib.lines.Line2D object at 0x17cacc2b0>
[<matplotlib.axis.YTick object at 0x17ca82880>, <matplotlib.axis.YTick object at 0x17ca82220>, <matplotlib.axis.YTick object at 0x17ca7a730>, <matplotlib.axis.YTick object at 0x17caccfd0>]
[Text(0, 0, 'MET'), Text(0, 1, '80S'), Text(0, 2, 'JAZ'), Text(0, 3, 'ELE')]
Text(0.5, 0, 'Clip Mean − Context Mean\n(+ve = Clip dominant; −ve = Context dominant)')
Text(0.5, 1.0, 'B. Clip vs Context Dominance by Genre\n(SC_dx vs dc_SX_SG)')
Text(0.0044064843955878815, 0, 'n.s.')
Text(0.024862657512534355, 1, '*')
Text(0.06600789123754344, 2, '***')
Text(0.07095048472289281, 3, '***')
<matplotlib.legend.Legend object at 0x17caeb2e0>
Text(0.5, 0.98, 'RQ2: Does Genre Moderate the Context Effect?')
Saved: RQ2 genre moderation plot
cat("\n———MODEL 3: + believability_pair (supplementary covariate)———\n")
———MODEL 3: + believability_pair (supplementary covariate)———model3 <- lmer(similarity ~ context_comb + genre_comb + believability_pair
+ (1 | clip_comb) + (1 | item_i) + (1 | item_j),
data = sim_df, REML = FALSE)
print(anova(model3))Type III Analysis of Variance Table with Satterthwaite's method
Sum Sq Mean Sq NumDF DenDF F value Pr(>F)
context_comb 0.021304 0.0023672 9 103.65 6.9433 8.769e-08 ***
genre_comb 0.059797 0.0039865 15 225.56 11.6930 < 2.2e-16 ***
believability_pair 0.009304 0.0093042 1 111.19 27.2906 8.254e-07 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1cat("\n———Omnibus fixed effects — no reference level———\n")
———Omnibus fixed effects — no reference level———print(car::Anova(model3, type = 3))Analysis of Deviance Table (Type III Wald chisquare tests)
Response: similarity
Chisq Df Pr(>Chisq)
(Intercept) 7.7375 1 0.005408 **
context_comb 62.4894 9 4.430e-10 ***
genre_comb 175.3948 15 < 2.2e-16 ***
believability_pair 27.2906 1 1.751e-07 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1cat("\n———Pseudo-R² (Nakagawa & Schielzeth), variance explained:———\n")
———Pseudo-R² (Nakagawa & Schielzeth), variance explained:———print(MuMIn::r.squaredGLMM(model3)) R2m R2c
[1,] 0.3096082 0.7387262cat("\n———EMMean, context combos with 95% CI———\n")
———EMMean, context combos with 95% CI———print(summary(emmeans(model3, ~ context_comb), infer = c(TRUE, FALSE), level = 0.95)) context_comb emmean SE df lower.CL upper.CL
bar_bar 0.161 0.00409 151 0.153 0.169
bar_concert 0.151 0.00363 123 0.144 0.158
bar_movie 0.146 0.00363 124 0.139 0.153
bar_videogame 0.137 0.00364 124 0.130 0.145
concert_concert 0.156 0.00394 168 0.148 0.163
concert_movie 0.141 0.00364 125 0.134 0.149
concert_videogame 0.133 0.00363 124 0.125 0.140
movie_movie 0.150 0.00393 169 0.142 0.158
movie_videogame 0.139 0.00363 124 0.131 0.146
videogame_videogame 0.143 0.00407 162 0.135 0.151
Results are averaged over the levels of: genre_comb
Degrees-of-freedom method: kenward-roger
Confidence level used: 0.95 cat("\n———EMMean, genre combos with 95% CI———\n")
———EMMean, genre combos with 95% CI———print(summary(emmeans(model3, ~ genre_comb), infer = c(TRUE, FALSE), level = 0.95)) genre_comb emmean SE df lower.CL upper.CL
80s_80s 0.164 0.00654 342 0.151 0.177
80s_Ele 0.139 0.00640 328 0.126 0.152
80s_Jaz 0.160 0.00640 328 0.147 0.173
80s_Met 0.123 0.00640 328 0.110 0.135
Ele_80s 0.144 0.00663 356 0.131 0.157
Ele_Ele 0.167 0.00650 347 0.154 0.180
Ele_Jaz 0.137 0.00641 329 0.124 0.149
Ele_Met 0.139 0.00640 328 0.127 0.152
Jaz_80s 0.150 0.00664 357 0.137 0.163
Jaz_Ele 0.132 0.00659 361 0.119 0.145
Jaz_Jaz 0.184 0.00652 347 0.171 0.197
Jaz_Met 0.120 0.00640 329 0.107 0.132
Met_80s 0.131 0.00668 358 0.118 0.144
Met_Ele 0.137 0.00662 361 0.124 0.150
Met_Jaz 0.131 0.00659 360 0.118 0.144
Met_Met 0.173 0.00664 348 0.160 0.186
Results are averaged over the levels of: context_comb
Degrees-of-freedom method: kenward-roger
Confidence level used: 0.95 cat("\n———random effects variance check———\n")
———random effects variance check———print(summary(model3)$varcor) Groups Name Std.Dev.
clip_comb (Intercept) 0.0204229
item_i (Intercept) 0.0068366
item_j (Intercept) 0.0098036
Residual 0.0184643cat("\n———Model comparison — Model 1 vs Model 3 (believability_pair):———\n")
———Model comparison — Model 1 vs Model 3 (believability_pair):———print(anova(model1, model3))Data: sim_df
Models:
model1: similarity ~ context_comb + genre_comb + (1 | clip_comb) + (1 | item_i) + (1 | item_j)
model3: similarity ~ context_comb + genre_comb + believability_pair + (1 | clip_comb) + (1 | item_i) + (1 | item_j)
npar AIC BIC logLik -2*log(L) Chisq Df Pr(>Chisq)
model1 29 -9512.0 -9349.3 4785.0 -9570.0
model3 30 -9535.7 -9367.4 4797.9 -9595.7 25.702 1 3.983e-07 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1Logic: SC_dx (same clip, different contexts) isolates the music contribution; dc_SX_dg (different clips, same context) isolates the context contribution.
TF-IDF note —
SC_SXunavailable: each clip × context cell is a single aggregated document and cannot be self-paired, soSC_SXyields zero pairs. The combined-alignment hypothesis (RQ3b) is therefore a MiniLM-only analysis.
RQ3a — CLIP vs CONTEXT============================================================ SC_dx (clip-driven): M = 0.2041 (N = 96) dc_SX_dg (context-driven): M = 0.1435 (N = 384) Mean difference = +0.0606, 95% CI [0.0516, 0.0697] Welch's t(117.8) = 13.127, p = 0.0000 *** Cohen's d = 1.655 [note: d benchmarks not appropriate for pairwise data]
-> CLIP drives greater MIMC similarity at the document levelSKIPPED in TF-IDF — MiniLM only. Each clip × context cell = one document; self-pairing is undefined.
SC_SXyields N = 0. Deferred to the MiniLM pipeline.
RQ3b — COMBINED ADVANTAGE [SKIPPED — TF-IDF level] SC_SX not available. Deferred to MiniLM pipeline.<BarContainer object of 2 artists><ErrorbarContainer object of 3 artists>[<matplotlib.axis.XTick object at 0x16db62550>, <matplotlib.axis.XTick object at 0x16db625b0>][Text(0, 0, 'Clip-Driven\n(Same Clip, Diff Ctx)'), Text(1, 0, 'Context-Driven\n(Diff Clip, Same Ctx)')]Text(0, 0.5, 'Mean Cosine Similarity')Text(0.5, 1.0, 'A. Clip vs Context (RQ3a)\n(95% CI error bars)')Text(1, 0.18382589412184927, '')Text(0.5, 0.19582589412184928, 'd = 1.655 ***\nΔ = +0.0606')(0.0, 0.2755852632855876)Text(0.5, 0.5, 'Combined alignment\nadvantage (RQ3b)\n\nNot available at\nTF-IDF level\n\n→ See MiniLM pipeline')Text(0.5, 1.0, 'B. Combined Advantage (RQ3b)\n[MiniLM-only — skipped here]')(np.float64(0.0), np.float64(1.0), np.float64(0.0), np.float64(1.0))Text(0.5, 0.98, 'RQ3: Does Music or Context Drive Greater Convergence?')
SUMMARY OF FINDINGS [TF-IDF — combMIMC level]======================================================================
RQ1 — Pure context effect (dc_SX_dg vs dc_dx_dg): dc_SX_dg M=0.1435 | Baseline M=0.1322 Mean diff=+0.0113 95% CI [0.0080, 0.0146] t(639.2)=6.647 p=0.0000 ***
RQ1 — Joint context+genre alignment (dc_SX_SG vs dc_dx_dg): dc_SX_SG M=0.1656 | Baseline M=0.1322 Mean diff=+0.0334 95% CI [0.0244, 0.0424] t(104.4)=8.646 p=0.0000 ***
RQ2 — Genre Framing Susceptibility (ranked highest to lowest):
MET Gain=+0.0149 d=0.454 sig_bonf=n.s.
80S Gain=+0.0133 d=0.523 sig_bonf=n.s.
ELE Gain=+0.0054 d=0.166 sig_bonf=n.s.
JAZ Gain=+0.0043 d=0.089 sig_bonf=n.s.
RQ2 — Within-Genre Clip vs Context Dominance:
80S -> CLIP dominant d=0.863 sig_bonf=*
ELE -> CLIP dominant d=1.872 sig_bonf=***
JAZ -> CLIP dominant d=1.285 sig_bonf=***
MET -> CLIP dominant d=0.042 sig_bonf=n.s.
RQ3a — Clip vs Context (SC_dx vs dc_SX_dg): Clip M=0.2041 | Ctx M=0.1435 Mean diff=+0.0606 95% CI [0.0516, 0.0697] CLIP dominant d=1.655 ***
RQ3b — Combined Advantage (SC_SX): SKIPPED — not available at TF-IDF level. -> Deferred to MiniLM pipeline.The analyses below do not map directly onto the three pre-specified research questions. They are retained as descriptive and methodological supplements that may be useful for characterising the data or for future exploratory work.
Word size reflects TF-IDF score; darker shading indicates a higher score. Useful for understanding the lexical character of each condition cell.
<wordcloud.wordcloud.WordCloud object at 0x17cbe0280>
<matplotlib.image.AxesImage object at 0x17cbbd790>
(np.float64(-0.5), np.float64(399.5), np.float64(299.5), np.float64(-0.5))
Text(0.5, 1.0, 'BAR')
Text(-0.1, 0.5, '80S')
<wordcloud.wordcloud.WordCloud object at 0x17cbe0220>
<matplotlib.image.AxesImage object at 0x17cbe0280>
(np.float64(-0.5), np.float64(399.5), np.float64(299.5), np.float64(-0.5))
Text(0.5, 1.0, 'CONCERT')
<wordcloud.wordcloud.WordCloud object at 0x17cbe0d90>
<matplotlib.image.AxesImage object at 0x17cbe0220>
(np.float64(-0.5), np.float64(399.5), np.float64(299.5), np.float64(-0.5))
Text(0.5, 1.0, 'MOVIE')
<wordcloud.wordcloud.WordCloud object at 0x17cbe0d00>
<matplotlib.image.AxesImage object at 0x17cbe0d90>
(np.float64(-0.5), np.float64(399.5), np.float64(299.5), np.float64(-0.5))
Text(0.5, 1.0, 'VIDEOGAME')
<wordcloud.wordcloud.WordCloud object at 0x17cbe0b50>
<matplotlib.image.AxesImage object at 0x17cbe0d00>
(np.float64(-0.5), np.float64(399.5), np.float64(299.5), np.float64(-0.5))
Text(-0.1, 0.5, 'ELECTRONIC')
<wordcloud.wordcloud.WordCloud object at 0x17cbe0880>
<matplotlib.image.AxesImage object at 0x17cbe0b50>
(np.float64(-0.5), np.float64(399.5), np.float64(299.5), np.float64(-0.5))
<wordcloud.wordcloud.WordCloud object at 0x17c4a23a0>
<matplotlib.image.AxesImage object at 0x17cbcc7c0>
(np.float64(-0.5), np.float64(399.5), np.float64(299.5), np.float64(-0.5))
<wordcloud.wordcloud.WordCloud object at 0x17cbe0d60>
<matplotlib.image.AxesImage object at 0x17ce57eb0>
(np.float64(-0.5), np.float64(399.5), np.float64(299.5), np.float64(-0.5))
<wordcloud.wordcloud.WordCloud object at 0x17c4a2e20>
<matplotlib.image.AxesImage object at 0x17ce570a0>
(np.float64(-0.5), np.float64(399.5), np.float64(299.5), np.float64(-0.5))
Text(-0.1, 0.5, 'JAZZ')
<wordcloud.wordcloud.WordCloud object at 0x17cbe0dc0>
<matplotlib.image.AxesImage object at 0x17ce57cd0>
(np.float64(-0.5), np.float64(399.5), np.float64(299.5), np.float64(-0.5))
<wordcloud.wordcloud.WordCloud object at 0x17c4a23a0>
<matplotlib.image.AxesImage object at 0x17cbe0dc0>
(np.float64(-0.5), np.float64(399.5), np.float64(299.5), np.float64(-0.5))
<wordcloud.wordcloud.WordCloud object at 0x17ce57f10>
<matplotlib.image.AxesImage object at 0x17ce48eb0>
(np.float64(-0.5), np.float64(399.5), np.float64(299.5), np.float64(-0.5))
<wordcloud.wordcloud.WordCloud object at 0x17c4a2e20>
<matplotlib.image.AxesImage object at 0x17ce48940>
(np.float64(-0.5), np.float64(399.5), np.float64(299.5), np.float64(-0.5))
Text(-0.1, 0.5, 'METAL')
<wordcloud.wordcloud.WordCloud object at 0x17cbe0850>
<matplotlib.image.AxesImage object at 0x17ce48df0>
(np.float64(-0.5), np.float64(399.5), np.float64(299.5), np.float64(-0.5))
<wordcloud.wordcloud.WordCloud object at 0x17c4a23a0>
<matplotlib.image.AxesImage object at 0x17ce43dc0>
(np.float64(-0.5), np.float64(399.5), np.float64(299.5), np.float64(-0.5))
<wordcloud.wordcloud.WordCloud object at 0x17cbe0880>
<matplotlib.image.AxesImage object at 0x17ce43490>
(np.float64(-0.5), np.float64(399.5), np.float64(299.5), np.float64(-0.5))Text(0.5, 0.995, 'Word Clouds by Genre × Context\n(Word size = TF-IDF score; Darker = higher value)')
Saved word cloudsTests whether specific context labels or genres differ from each other in how much convergence they produce. Not pre-specified in the hypotheses — retained as descriptive characterisation only.
PAIRWISE CONTEXT COMPARISONS (Bonferroni corrected)----------------------------------------------------------------------context1 context2 mean1 mean2 d p p_bonf sig_bonf
bar videogame 0.151812 0.136536 0.550182 0.000188 0.001130 **
bar movie 0.151812 0.140289 0.404074 0.005676 0.034056 *
concert videogame 0.145421 0.136536 0.309007 0.033575 0.201452 n.s.
bar concert 0.151812 0.145421 0.245188 0.091022 0.546129 n.s.
concert movie 0.145421 0.140289 0.174089 0.229306 1.000000 n.s.
movie videogame 0.140289 0.136536 0.121086 0.402577 1.000000 n.s.
PAIRWISE GENRE COMPARISONS (Bonferroni corrected)----------------------------------------------------------------------genre1 genre2 mean1 mean2 d p p_bonf sig_bonf
Ele Met 0.142604 0.169225 -0.788910 0.000005 0.000032 ***
80s Met 0.149922 0.169225 -0.635311 0.000212 0.001272 **
Ele Jaz 0.142604 0.162614 -0.481690 0.004545 0.027273 *
80s Jaz 0.149922 0.162614 -0.326604 0.052645 0.315870 n.s.
80s Ele 0.149922 0.142604 0.249654 0.136512 0.819075 n.s.
Jaz Met 0.162614 0.169225 -0.156283 0.350163 1.000000 n.s.Full within/between similarity for context and genre — useful as a descriptive data quality check and for visualising overall structure.
<Axes: >
Text(0.5, 1.0, 'A. Context Similarity Matrix\n(Diagonal = Within-Context)')
<Axes: >
Text(0.5, 1.0, 'B. Genre Similarity Matrix\n(Diagonal = Within-Genre)')Text(0.5, 0.98, 'Similarity Matrices: Context and Genre')
t-SNE projects the high-dimensional TF-IDF vectors into 2D space. Provides a visual check on whether documents with shared conditions cluster together — useful for data quality and interpretation, but not a hypothesis test.
SVD explained variance: 0.832 (50 components)<Axes: xlabel='TSNE1', ylabel='TSNE2'>Text(0.5, 1.0, 't-SNE: TF-IDF Document Embeddings (2D)')<matplotlib.legend.Legend object at 0x3036b27c0>
Saved: document t-SNE plotinstall.packages("renv")The following package(s) will be installed:
- renv [1.2.2]
These packages will be installed into "~/Library/Caches/org.R-project.R/R/renv/library/NLP_analyses-89bf1a08/macos/R-4.5/aarch64-apple-darwin20".
# Installing packages --------------------------------------------------------
[32m✔[0m renv 1.2.2 [linked from cache]
Successfully installed 1 package in 3.1 milliseconds.renv::init()
renv::snapshot()- The lockfile is already up to date.