Research Methods in Developmental Linguistics – Week 6

Dr Stefano Coretta

University of Edinburgh

Song 2020

• Data from Song 2020.

• Shallow morphological representation in L2 speakers.

• Lexical decision task: Is the word a real English word or not?

• Priming paradigm:

  • Prime: prolong (unrelated), unkind (constituent), kindness (non-constituent).

  • Target: unkindness ([un-kind]-ness, not un-[kind-ness]).

• English L1 and English L2/Cantonese L1 participants.

Song 2020: hypotheses

Vague hypothesis:

The Shallow Structure Hypothesis (SSH) allows the possibility for nonnative speakers to show strong priming effects from nonconstituent and constituent primes equally because the shallow processing route, which relies more heavily on semantic and surface-level information than on structural information.

Song 2020: the data

shallow <- read_csv("data/song2020/shallow.csv") |>
  filter(
    Critical_Filler == "Critical",
    Branching == "Left"
  ) |> 
  mutate(
    accuracy = ifelse(ACC == 0, "incorrect", "correct"),
    accuracy = factor(accuracy, levels = c("incorrect", "correct"))
  )
shallow

# A tibble: 1,170 × 12
   Group ID    List  Target         ACC    RT logRT Critical_Filler Word_Nonword
   <chr> <chr> <chr> <chr>        <dbl> <dbl> <dbl> <chr>           <chr>       
 1 L1    L1_01 A     unawareness      1   603  6.40 Critical        Word        
 2 L1    L1_01 A     unholiness       1   739  6.61 Critical        Word        
 3 L1    L1_01 A     unhappiness      1   370  5.91 Critical        Word        
 4 L1    L1_01 A     unsharpness      1   821  6.71 Critical        Word        
 5 L1    L1_01 A     unripeness       0  1035  6.94 Critical        Word        
 6 L1    L1_01 A     unkindness       1   498  6.21 Critical        Word        
 7 L1    L1_01 A     unwariness       1  1133  7.03 Critical        Word        
 8 L1    L1_01 A     unclearness      1   513  6.24 Critical        Word        
 9 L1    L1_01 A     reobtainable     0   964  6.87 Critical        Word        
10 L1    L1_01 A     resealable       1   709  6.56 Critical        Word        
# ℹ 1,160 more rows
# ℹ 3 more variables: Relation_type <chr>, Branching <chr>, accuracy <fct>

Song 2020: accuracy

Bernoulli regression (accuracy)

my_seed <- 4235

sha_bm_1 <- brm(
  accuracy ~ 0 + Group:Relation_type,
  family = bernoulli,
  data = shallow,
  cores = 4,
  seed = my_seed,
  file = "data/cache/sha_bm_1"
)

sha_bm_1

 Family: bernoulli 
  Links: mu = logit 
Formula: accuracy ~ 0 + Group:Relation_type 
   Data: shallow (Number of observations: 1170) 
  Draws: 4 chains, each with iter = 2000; warmup = 1000; thin = 1;
         total post-warmup draws = 4000

Regression Coefficients:
                                    Estimate Est.Error l-95% CI u-95% CI Rhat
GroupL1:Relation_typeConstituent        1.80      0.22     1.39     2.24 1.00
GroupL2:Relation_typeConstituent        1.24      0.16     0.94     1.55 1.00
GroupL1:Relation_typeNonConstituent     1.18      0.18     0.85     1.55 1.00
GroupL2:Relation_typeNonConstituent     1.08      0.16     0.78     1.40 1.00
GroupL1:Relation_typeUnrelated          1.00      0.17     0.67     1.35 1.00
GroupL2:Relation_typeUnrelated          0.69      0.14     0.42     0.97 1.00
                                    Bulk_ESS Tail_ESS
GroupL1:Relation_typeConstituent        5974     2708
GroupL2:Relation_typeConstituent        6326     3196
GroupL1:Relation_typeNonConstituent     5886     2697
GroupL2:Relation_typeNonConstituent     6350     2842
GroupL1:Relation_typeUnrelated          5117     3098
GroupL2:Relation_typeUnrelated          5487     2761

Draws were sampled using sampling(NUTS). For each parameter, Bulk_ESS
and Tail_ESS are effective sample size measures, and Rhat is the potential
scale reduction factor on split chains (at convergence, Rhat = 1).

Bernoulli regression: expected predictions

conditional_effects(sha_bm_1, "Group:Relation_type")

BUT…

• Multiple observations from different participants.

• Multiple observations from different target words.

• We need to include varying terms.

By-subject accuracy

By-target accuracy

By-subject and by-target varying terms

sha_bm_2 <- brm(
  ACC ~ 0 + Group:Relation_type +
    (0 + Relation_type | ID) +
    (0 + Group:Relation_type | Target),
  family = bernoulli,
  data = shallow,
  cores = 4,
  seed = my_seed,
  file = "data/cache/sha_bm_2"
)

summary(sha_bm_2)

 Family: bernoulli 
  Links: mu = logit 
Formula: ACC ~ 0 + Group:Relation_type + (0 + Relation_type | ID) + (0 + Group:Relation_type | Target) 
   Data: shallow (Number of observations: 1170) 
  Draws: 4 chains, each with iter = 2000; warmup = 1000; thin = 1;
         total post-warmup draws = 4000

Multilevel Hyperparameters:
~ID (Number of levels: 65) 
                                                          Estimate Est.Error
sd(Relation_typeConstituent)                                  1.31      0.29
sd(Relation_typeNonConstituent)                               1.18      0.24
sd(Relation_typeUnrelated)                                    0.84      0.20
cor(Relation_typeConstituent,Relation_typeNonConstituent)     0.88      0.09
cor(Relation_typeConstituent,Relation_typeUnrelated)          0.78      0.16
cor(Relation_typeNonConstituent,Relation_typeUnrelated)       0.79      0.15
                                                          l-95% CI u-95% CI
sd(Relation_typeConstituent)                                  0.82     1.94
sd(Relation_typeNonConstituent)                               0.75     1.68
sd(Relation_typeUnrelated)                                    0.46     1.26
cor(Relation_typeConstituent,Relation_typeNonConstituent)     0.65     0.99
cor(Relation_typeConstituent,Relation_typeUnrelated)          0.39     0.98
cor(Relation_typeNonConstituent,Relation_typeUnrelated)       0.42     0.98
                                                          Rhat Bulk_ESS
sd(Relation_typeConstituent)                              1.00     2191
sd(Relation_typeNonConstituent)                           1.00     2394
sd(Relation_typeUnrelated)                                1.00     2936
cor(Relation_typeConstituent,Relation_typeNonConstituent) 1.00     1871
cor(Relation_typeConstituent,Relation_typeUnrelated)      1.00     1938
cor(Relation_typeNonConstituent,Relation_typeUnrelated)   1.00     2379
                                                          Tail_ESS
sd(Relation_typeConstituent)                                  2725
sd(Relation_typeNonConstituent)                               2909
sd(Relation_typeUnrelated)                                    3220
cor(Relation_typeConstituent,Relation_typeNonConstituent)     2771
cor(Relation_typeConstituent,Relation_typeUnrelated)          2439
cor(Relation_typeNonConstituent,Relation_typeUnrelated)       3316

~Target (Number of levels: 18) 
                                                                             Estimate
sd(GroupL1:Relation_typeConstituent)                                             1.52
sd(GroupL2:Relation_typeConstituent)                                             1.43
sd(GroupL1:Relation_typeNonConstituent)                                          1.39
sd(GroupL2:Relation_typeNonConstituent)                                          0.87
sd(GroupL1:Relation_typeUnrelated)                                               1.24
sd(GroupL2:Relation_typeUnrelated)                                               1.23
cor(GroupL1:Relation_typeConstituent,GroupL2:Relation_typeConstituent)           0.23
cor(GroupL1:Relation_typeConstituent,GroupL1:Relation_typeNonConstituent)        0.34
cor(GroupL2:Relation_typeConstituent,GroupL1:Relation_typeNonConstituent)        0.31
cor(GroupL1:Relation_typeConstituent,GroupL2:Relation_typeNonConstituent)        0.28
cor(GroupL2:Relation_typeConstituent,GroupL2:Relation_typeNonConstituent)        0.58
cor(GroupL1:Relation_typeNonConstituent,GroupL2:Relation_typeNonConstituent)     0.37
cor(GroupL1:Relation_typeConstituent,GroupL1:Relation_typeUnrelated)             0.42
cor(GroupL2:Relation_typeConstituent,GroupL1:Relation_typeUnrelated)             0.56
cor(GroupL1:Relation_typeNonConstituent,GroupL1:Relation_typeUnrelated)          0.44
cor(GroupL2:Relation_typeNonConstituent,GroupL1:Relation_typeUnrelated)          0.43
cor(GroupL1:Relation_typeConstituent,GroupL2:Relation_typeUnrelated)             0.30
cor(GroupL2:Relation_typeConstituent,GroupL2:Relation_typeUnrelated)             0.64
cor(GroupL1:Relation_typeNonConstituent,GroupL2:Relation_typeUnrelated)          0.43
cor(GroupL2:Relation_typeNonConstituent,GroupL2:Relation_typeUnrelated)          0.49
cor(GroupL1:Relation_typeUnrelated,GroupL2:Relation_typeUnrelated)               0.57
                                                                             Est.Error
sd(GroupL1:Relation_typeConstituent)                                              0.64
sd(GroupL2:Relation_typeConstituent)                                              0.40
sd(GroupL1:Relation_typeNonConstituent)                                           0.45
sd(GroupL2:Relation_typeNonConstituent)                                           0.30
sd(GroupL1:Relation_typeUnrelated)                                                0.36
sd(GroupL2:Relation_typeUnrelated)                                                0.30
cor(GroupL1:Relation_typeConstituent,GroupL2:Relation_typeConstituent)            0.27
cor(GroupL1:Relation_typeConstituent,GroupL1:Relation_typeNonConstituent)         0.27
cor(GroupL2:Relation_typeConstituent,GroupL1:Relation_typeNonConstituent)         0.25
cor(GroupL1:Relation_typeConstituent,GroupL2:Relation_typeNonConstituent)         0.29
cor(GroupL2:Relation_typeConstituent,GroupL2:Relation_typeNonConstituent)         0.22
cor(GroupL1:Relation_typeNonConstituent,GroupL2:Relation_typeNonConstituent)      0.27
cor(GroupL1:Relation_typeConstituent,GroupL1:Relation_typeUnrelated)              0.25
cor(GroupL2:Relation_typeConstituent,GroupL1:Relation_typeUnrelated)              0.21
cor(GroupL1:Relation_typeNonConstituent,GroupL1:Relation_typeUnrelated)           0.25
cor(GroupL2:Relation_typeNonConstituent,GroupL1:Relation_typeUnrelated)           0.27
cor(GroupL1:Relation_typeConstituent,GroupL2:Relation_typeUnrelated)              0.25
cor(GroupL2:Relation_typeConstituent,GroupL2:Relation_typeUnrelated)              0.20
cor(GroupL1:Relation_typeNonConstituent,GroupL2:Relation_typeUnrelated)           0.24
cor(GroupL2:Relation_typeNonConstituent,GroupL2:Relation_typeUnrelated)           0.24
cor(GroupL1:Relation_typeUnrelated,GroupL2:Relation_typeUnrelated)                0.22
                                                                             l-95% CI
sd(GroupL1:Relation_typeConstituent)                                             0.53
sd(GroupL2:Relation_typeConstituent)                                             0.79
sd(GroupL1:Relation_typeNonConstituent)                                          0.67
sd(GroupL2:Relation_typeNonConstituent)                                          0.37
sd(GroupL1:Relation_typeUnrelated)                                               0.66
sd(GroupL2:Relation_typeUnrelated)                                               0.74
cor(GroupL1:Relation_typeConstituent,GroupL2:Relation_typeConstituent)          -0.31
cor(GroupL1:Relation_typeConstituent,GroupL1:Relation_typeNonConstituent)       -0.23
cor(GroupL2:Relation_typeConstituent,GroupL1:Relation_typeNonConstituent)       -0.21
cor(GroupL1:Relation_typeConstituent,GroupL2:Relation_typeNonConstituent)       -0.31
cor(GroupL2:Relation_typeConstituent,GroupL2:Relation_typeNonConstituent)        0.04
cor(GroupL1:Relation_typeNonConstituent,GroupL2:Relation_typeNonConstituent)    -0.19
cor(GroupL1:Relation_typeConstituent,GroupL1:Relation_typeUnrelated)            -0.13
cor(GroupL2:Relation_typeConstituent,GroupL1:Relation_typeUnrelated)             0.07
cor(GroupL1:Relation_typeNonConstituent,GroupL1:Relation_typeUnrelated)         -0.10
cor(GroupL2:Relation_typeNonConstituent,GroupL1:Relation_typeUnrelated)         -0.16
cor(GroupL1:Relation_typeConstituent,GroupL2:Relation_typeUnrelated)            -0.22
cor(GroupL2:Relation_typeConstituent,GroupL2:Relation_typeUnrelated)             0.18
cor(GroupL1:Relation_typeNonConstituent,GroupL2:Relation_typeUnrelated)         -0.08
cor(GroupL2:Relation_typeNonConstituent,GroupL2:Relation_typeUnrelated)         -0.04
cor(GroupL1:Relation_typeUnrelated,GroupL2:Relation_typeUnrelated)               0.08
                                                                             u-95% CI
sd(GroupL1:Relation_typeConstituent)                                             3.01
sd(GroupL2:Relation_typeConstituent)                                             2.36
sd(GroupL1:Relation_typeNonConstituent)                                          2.39
sd(GroupL2:Relation_typeNonConstituent)                                          1.55
sd(GroupL1:Relation_typeUnrelated)                                               2.07
sd(GroupL2:Relation_typeUnrelated)                                               1.89
cor(GroupL1:Relation_typeConstituent,GroupL2:Relation_typeConstituent)           0.72
cor(GroupL1:Relation_typeConstituent,GroupL1:Relation_typeNonConstituent)        0.81
cor(GroupL2:Relation_typeConstituent,GroupL1:Relation_typeNonConstituent)        0.76
cor(GroupL1:Relation_typeConstituent,GroupL2:Relation_typeNonConstituent)        0.78
cor(GroupL2:Relation_typeConstituent,GroupL2:Relation_typeNonConstituent)        0.91
cor(GroupL1:Relation_typeNonConstituent,GroupL2:Relation_typeNonConstituent)     0.82
cor(GroupL1:Relation_typeConstituent,GroupL1:Relation_typeUnrelated)             0.84
cor(GroupL2:Relation_typeConstituent,GroupL1:Relation_typeUnrelated)             0.89
cor(GroupL1:Relation_typeNonConstituent,GroupL1:Relation_typeUnrelated)          0.84
cor(GroupL2:Relation_typeNonConstituent,GroupL1:Relation_typeUnrelated)          0.86
cor(GroupL1:Relation_typeConstituent,GroupL2:Relation_typeUnrelated)             0.75
cor(GroupL2:Relation_typeConstituent,GroupL2:Relation_typeUnrelated)             0.92
cor(GroupL1:Relation_typeNonConstituent,GroupL2:Relation_typeUnrelated)          0.82
cor(GroupL2:Relation_typeNonConstituent,GroupL2:Relation_typeUnrelated)          0.88
cor(GroupL1:Relation_typeUnrelated,GroupL2:Relation_typeUnrelated)               0.90
                                                                             Rhat
sd(GroupL1:Relation_typeConstituent)                                         1.00
sd(GroupL2:Relation_typeConstituent)                                         1.00
sd(GroupL1:Relation_typeNonConstituent)                                      1.00
sd(GroupL2:Relation_typeNonConstituent)                                      1.00
sd(GroupL1:Relation_typeUnrelated)                                           1.00
sd(GroupL2:Relation_typeUnrelated)                                           1.00
cor(GroupL1:Relation_typeConstituent,GroupL2:Relation_typeConstituent)       1.00
cor(GroupL1:Relation_typeConstituent,GroupL1:Relation_typeNonConstituent)    1.00
cor(GroupL2:Relation_typeConstituent,GroupL1:Relation_typeNonConstituent)    1.00
cor(GroupL1:Relation_typeConstituent,GroupL2:Relation_typeNonConstituent)    1.00
cor(GroupL2:Relation_typeConstituent,GroupL2:Relation_typeNonConstituent)    1.00
cor(GroupL1:Relation_typeNonConstituent,GroupL2:Relation_typeNonConstituent) 1.00
cor(GroupL1:Relation_typeConstituent,GroupL1:Relation_typeUnrelated)         1.00
cor(GroupL2:Relation_typeConstituent,GroupL1:Relation_typeUnrelated)         1.00
cor(GroupL1:Relation_typeNonConstituent,GroupL1:Relation_typeUnrelated)      1.00
cor(GroupL2:Relation_typeNonConstituent,GroupL1:Relation_typeUnrelated)      1.00
cor(GroupL1:Relation_typeConstituent,GroupL2:Relation_typeUnrelated)         1.00
cor(GroupL2:Relation_typeConstituent,GroupL2:Relation_typeUnrelated)         1.00
cor(GroupL1:Relation_typeNonConstituent,GroupL2:Relation_typeUnrelated)      1.00
cor(GroupL2:Relation_typeNonConstituent,GroupL2:Relation_typeUnrelated)      1.00
cor(GroupL1:Relation_typeUnrelated,GroupL2:Relation_typeUnrelated)           1.00
                                                                             Bulk_ESS
sd(GroupL1:Relation_typeConstituent)                                             1795
sd(GroupL2:Relation_typeConstituent)                                             2425
sd(GroupL1:Relation_typeNonConstituent)                                          2709
sd(GroupL2:Relation_typeNonConstituent)                                          3228
sd(GroupL1:Relation_typeUnrelated)                                               3705
sd(GroupL2:Relation_typeUnrelated)                                               3833
cor(GroupL1:Relation_typeConstituent,GroupL2:Relation_typeConstituent)           1757
cor(GroupL1:Relation_typeConstituent,GroupL1:Relation_typeNonConstituent)        2678
cor(GroupL2:Relation_typeConstituent,GroupL1:Relation_typeNonConstituent)        3001
cor(GroupL1:Relation_typeConstituent,GroupL2:Relation_typeNonConstituent)        2560
cor(GroupL2:Relation_typeConstituent,GroupL2:Relation_typeNonConstituent)        3456
cor(GroupL1:Relation_typeNonConstituent,GroupL2:Relation_typeNonConstituent)     4272
cor(GroupL1:Relation_typeConstituent,GroupL1:Relation_typeUnrelated)             2504
cor(GroupL2:Relation_typeConstituent,GroupL1:Relation_typeUnrelated)             3901
cor(GroupL1:Relation_typeNonConstituent,GroupL1:Relation_typeUnrelated)          3867
cor(GroupL2:Relation_typeNonConstituent,GroupL1:Relation_typeUnrelated)          3674
cor(GroupL1:Relation_typeConstituent,GroupL2:Relation_typeUnrelated)             2401
cor(GroupL2:Relation_typeConstituent,GroupL2:Relation_typeUnrelated)             3356
cor(GroupL1:Relation_typeNonConstituent,GroupL2:Relation_typeUnrelated)          3950
cor(GroupL2:Relation_typeNonConstituent,GroupL2:Relation_typeUnrelated)          3507
cor(GroupL1:Relation_typeUnrelated,GroupL2:Relation_typeUnrelated)               3302
                                                                             Tail_ESS
sd(GroupL1:Relation_typeConstituent)                                             2343
sd(GroupL2:Relation_typeConstituent)                                             3228
sd(GroupL1:Relation_typeNonConstituent)                                          3029
sd(GroupL2:Relation_typeNonConstituent)                                          2684
sd(GroupL1:Relation_typeUnrelated)                                               3014
sd(GroupL2:Relation_typeUnrelated)                                               3401
cor(GroupL1:Relation_typeConstituent,GroupL2:Relation_typeConstituent)           2539
cor(GroupL1:Relation_typeConstituent,GroupL1:Relation_typeNonConstituent)        2900
cor(GroupL2:Relation_typeConstituent,GroupL1:Relation_typeNonConstituent)        3094
cor(GroupL1:Relation_typeConstituent,GroupL2:Relation_typeNonConstituent)        3065
cor(GroupL2:Relation_typeConstituent,GroupL2:Relation_typeNonConstituent)        3434
cor(GroupL1:Relation_typeNonConstituent,GroupL2:Relation_typeNonConstituent)     3507
cor(GroupL1:Relation_typeConstituent,GroupL1:Relation_typeUnrelated)             3101
cor(GroupL2:Relation_typeConstituent,GroupL1:Relation_typeUnrelated)             3625
cor(GroupL1:Relation_typeNonConstituent,GroupL1:Relation_typeUnrelated)          3383
cor(GroupL2:Relation_typeNonConstituent,GroupL1:Relation_typeUnrelated)          3577
cor(GroupL1:Relation_typeConstituent,GroupL2:Relation_typeUnrelated)             3297
cor(GroupL2:Relation_typeConstituent,GroupL2:Relation_typeUnrelated)             3393
cor(GroupL1:Relation_typeNonConstituent,GroupL2:Relation_typeUnrelated)          3652
cor(GroupL2:Relation_typeNonConstituent,GroupL2:Relation_typeUnrelated)          3410
cor(GroupL1:Relation_typeUnrelated,GroupL2:Relation_typeUnrelated)               3255

Regression Coefficients:
                                    Estimate Est.Error l-95% CI u-95% CI Rhat
GroupL1:Relation_typeConstituent        2.77      0.62     1.69     4.12 1.00
GroupL2:Relation_typeConstituent        2.01      0.50     1.12     3.08 1.00
GroupL1:Relation_typeNonConstituent     1.80      0.49     0.91     2.87 1.00
GroupL2:Relation_typeNonConstituent     1.52      0.36     0.86     2.30 1.00
GroupL1:Relation_typeUnrelated          1.45      0.42     0.68     2.34 1.00
GroupL2:Relation_typeUnrelated          0.99      0.38     0.28     1.77 1.00
                                    Bulk_ESS Tail_ESS
GroupL1:Relation_typeConstituent        2552     2703
GroupL2:Relation_typeConstituent        2519     2407
GroupL1:Relation_typeNonConstituent     2789     2924
GroupL2:Relation_typeNonConstituent     3126     2871
GroupL1:Relation_typeUnrelated          3313     2920
GroupL2:Relation_typeUnrelated          2723     2969

Draws were sampled using sampling(NUTS). For each parameter, Bulk_ESS
and Tail_ESS are effective sample size measures, and Rhat is the potential
scale reduction factor on split chains (at convergence, Rhat = 1).

By-subject and by-target model: expected predictions plot

Song 2020: hypotheses

Vague hypothesis:

The Shallow Structure Hypothesis (SSH) allows the possibility for nonnative speakers to show strong priming effects from nonconstituent and constituent primes equally because the shallow processing route, which relies more heavily on semantic and surface-level information than on structural information.

Difference between non/constituent in L2

sha_bm_2_draws <- as_draws_df(sha_bm_2, variable = "b_", regex = TRUE)

sha_bm_2_draws <- sha_bm_2_draws |> 
  mutate(
    diff_l2 = `b_GroupL2:Relation_typeNonConstituent` - `b_GroupL2:Relation_typeConstituent`
  )

sha_bm_2_draws

# A draws_df: 1000 iterations, 4 chains, and 7 variables
   b_GroupL1:Relation_typeConstituent b_GroupL2:Relation_typeConstituent
1                                 2.5                                2.1
2                                 2.7                                2.2
3                                 3.4                                1.4
4                                 2.8                                1.4
5                                 2.2                                1.6
6                                 2.6                                2.2
7                                 2.2                                2.1
8                                 2.3                                2.3
9                                 3.8                                2.1
10                                3.0                                3.0
   b_GroupL1:Relation_typeNonConstituent b_GroupL2:Relation_typeNonConstituent
1                                   1.63                                   1.5
2                                   1.59                                   2.0
3                                   1.20                                   1.2
4                                   1.99                                   1.5
5                                   1.63                                   1.0
6                                   1.52                                   1.4
7                                   0.88                                   1.8
8                                   1.14                                   1.8
9                                   2.87                                   2.1
10                                  3.42                                   1.7
   b_GroupL1:Relation_typeUnrelated b_GroupL2:Relation_typeUnrelated diff_l2
1                               1.5                             0.64  -0.670
2                               1.5                             1.19  -0.199
3                               1.1                             1.58  -0.272
4                               1.2                             0.79   0.079
5                               1.3                             0.58  -0.587
6                               1.4                             1.09  -0.799
7                               1.5                             1.30  -0.293
8                               1.0                             1.22  -0.554
9                               1.3                             1.42  -0.029
10                              1.5                             1.25  -1.266
# ... with 3990 more draws
# ... hidden reserved variables {'.chain', '.iteration', '.draw'}

Difference between non/constituent in L2: plot

Difference between non/constituent in L2: CrIs

sha_bm_2_draws |> 
  select(diff_l2) |> 
  summarise_draws(~quantile(.x, probs = c(0.05, 0.95, 0.15, 0.85))) |> 
  mutate(
    across(where(is.numeric), ~round(.x, digits = 2))
  )

# A tibble: 1 × 5
  variable  `5%` `95%` `15%` `85%`
  <chr>    <dbl> <dbl> <dbl> <dbl>
1 diff_l2   -1.3  0.26 -0.98 -0.01

Expected predictions by subject: get draws of constant terms

sha_bm_2_draws <- as_draws_df(sha_bm_2, variable = c("^b_.*"), regex = TRUE) |> 
  pivot_longer(
    `b_GroupL1:Relation_typeConstituent`:`b_GroupL2:Relation_typeUnrelated`,
    names_to = c("Group", "Relation_type"),
    names_pattern = "b_Group(.*):(.*)"
  )

sha_bm_2_draws

# A tibble: 24,000 × 6
   .chain .iteration .draw Group Relation_type               value
    <int>      <int> <int> <chr> <chr>                       <dbl>
 1      1          1     1 L1    Relation_typeConstituent    2.54 
 2      1          1     1 L2    Relation_typeConstituent    2.14 
 3      1          1     1 L1    Relation_typeNonConstituent 1.63 
 4      1          1     1 L2    Relation_typeNonConstituent 1.47 
 5      1          1     1 L1    Relation_typeUnrelated      1.49 
 6      1          1     1 L2    Relation_typeUnrelated      0.641
 7      1          2     2 L1    Relation_typeConstituent    2.68 
 8      1          2     2 L2    Relation_typeConstituent    2.21 
 9      1          2     2 L1    Relation_typeNonConstituent 1.59 
10      1          2     2 L2    Relation_typeNonConstituent 2.01 
# ℹ 23,990 more rows

Expected predictions by subject: get draws of varying terms

sha_bm_2_r <- spread_draws(sha_bm_2, r_ID[ID,Relation_type]) |> 
  mutate(
    Group = str_sub(ID, 1, 2)
  )

sha_bm_2_r

# A tibble: 780,000 × 7
# Groups:   ID, Relation_type [195]
   ID    Relation_type              r_ID .chain .iteration .draw Group
   <chr> <chr>                     <dbl>  <int>      <int> <int> <chr>
 1 L1_01 Relation_typeConstituent  0.994      1          1     1 L1   
 2 L1_01 Relation_typeConstituent  1.31       1          2     2 L1   
 3 L1_01 Relation_typeConstituent  0.167      1          3     3 L1   
 4 L1_01 Relation_typeConstituent  1.02       1          4     4 L1   
 5 L1_01 Relation_typeConstituent  0.463      1          5     5 L1   
 6 L1_01 Relation_typeConstituent -0.756      1          6     6 L1   
 7 L1_01 Relation_typeConstituent  1.47       1          7     7 L1   
 8 L1_01 Relation_typeConstituent  0.149      1          8     8 L1   
 9 L1_01 Relation_typeConstituent -0.919      1          9     9 L1   
10 L1_01 Relation_typeConstituent  1.66       1         10    10 L1   
# ℹ 779,990 more rows

Expected predictions by subject: join

sha_bm_2_draws_r <- left_join(sha_bm_2_draws, sha_bm_2_r) |> 
  mutate(
    epred = value + r_ID
  )

sha_bm_2_draws_r

# A tibble: 780,000 × 9
   .chain .iteration .draw Group Relation_type         value ID       r_ID epred
    <int>      <int> <int> <chr> <chr>                 <dbl> <chr>   <dbl> <dbl>
 1      1          1     1 L1    Relation_typeConstit…  2.54 L1_01  0.994   3.53
 2      1          1     1 L1    Relation_typeConstit…  2.54 L1_02 -0.965   1.57
 3      1          1     1 L1    Relation_typeConstit…  2.54 L1_03  2.62    5.16
 4      1          1     1 L1    Relation_typeConstit…  2.54 L1_05 -0.0306  2.51
 5      1          1     1 L1    Relation_typeConstit…  2.54 L1_06 -1.33    1.21
 6      1          1     1 L1    Relation_typeConstit…  2.54 L1_07  1.30    3.84
 7      1          1     1 L1    Relation_typeConstit…  2.54 L1_08  0.0710  2.61
 8      1          1     1 L1    Relation_typeConstit…  2.54 L1_09  1.73    4.27
 9      1          1     1 L1    Relation_typeConstit…  2.54 L1_10 -0.0995  2.44
10      1          1     1 L1    Relation_typeConstit…  2.54 L1_11 -0.416   2.12
# ℹ 779,990 more rows

Expected predictions by subject: plot

Difference between expected predictions by-subject

sha_bm_2_draws_r_wide <- sha_bm_2_draws_r |>
  select(-value, -r_ID) |> 
  pivot_wider(names_from = Relation_type, values_from = epred) |> 
  mutate(
    ncon_con = Relation_typeNonConstituent - Relation_typeConstituent
  )

sha_bm_2_draws_r_wide

# A tibble: 260,000 × 9
   .chain .iteration .draw Group ID    Relation_typeConstituent
    <int>      <int> <int> <chr> <chr>                    <dbl>
 1      1          1     1 L1    L1_01                     3.53
 2      1          1     1 L1    L1_02                     1.57
 3      1          1     1 L1    L1_03                     5.16
 4      1          1     1 L1    L1_05                     2.51
 5      1          1     1 L1    L1_06                     1.21
 6      1          1     1 L1    L1_07                     3.84
 7      1          1     1 L1    L1_08                     2.61
 8      1          1     1 L1    L1_09                     4.27
 9      1          1     1 L1    L1_10                     2.44
10      1          1     1 L1    L1_11                     2.12
# ℹ 259,990 more rows
# ℹ 3 more variables: Relation_typeNonConstituent <dbl>,
#   Relation_typeUnrelated <dbl>, ncon_con <dbl>

Difference between expected predictions by-subject: plot

Results overview

The Shallow Structure Hypothesis (SSH) allows the possibility for nonnative speakers to show strong priming effects from nonconstituent and constituent primes equally because the shallow processing route, which relies more heavily on semantic and surface-level information than on structural information.

• The 90% CrI of the difference between nonconstituent and constituent in L2 is [-1.3, 0.26].

• At 70% probability, the log-odds of the priming effect on accuracy between nonconstituent and constituent in L2 participants differs between -0.98 and -0.01.

• There is a lot of uncertainty in the expected predictions of individual participants, although there is a tendency for negative differences between nonconstituent and constituent.

Summary

• You should include varying terms if data is “hierarchical”, for example repeated measures from participants or items.

• Not including varying terms (wrongly) inflates posterior certainty.

• Frequentist regression models with lme4 often don’t converge and researchers simplify the hierarchical structure of the model.