This study replicated Experiment 2a in a between subjects-design (see details below). Another goal was to have more parallelized phrasings of the trigger and maintainer statements.

1 Demo Version of Experiment

To see a demo of the experiment in a separate browser window, click here.

To download the program files of the experiment, click here.

To download the program files for the animations, click here.

2 Test planning

The goal was to have \(n = 30\) subjects in each of the 8 theoretically relevant conditions (including the “side” counterbalancing factor, there were 16 conditions in total).

library(pwr)
## Warning: Paket 'pwr' wurde unter R Version 4.2.3 erstellt
pwr.t.test(n = 30, sig.level = 0.05, power = 0.8, type = c("two.sample"), alternative = "greater")
## 
##      Two-sample t test power calculation 
## 
##               n = 30
##               d = 0.649627
##       sig.level = 0.05
##           power = 0.8
##     alternative = greater
## 
## NOTE: n is number in *each* group

With \(n = 30\), the effect that can be detected with \(0.80\) power in a directed between-subjects t-test is \(d = 0.65\).

3 Data

3.1 Data set information

The data set contains the following columns:

  • subj_code: a random subject ID (anonymous)
  • condition: a numeric condition variable
  • desktop_conf: subjects’ confirmation of taking part via PC or Laptop
  • attent_conf: subjects’ confirmation to pay attention
  • intro_check: subjects answers to the video check question testing if subjects can see the clips
  • structure: condition variable coding whether causal structure was reversible or irreversible
  • first_object: counterbalancing variable coding which cause appeared first
  • test_statement_type: factor coding what kind of test statement subjects will rate
  • test_statement: the exact test statement that subjects rated in the respective conditions
  • learn_check_left: (dv) subjects’ responses to the learning video in which the left cause acts alone
  • learn_check_right: (dv) subjects’ responses to the learning video in which the right cause acts alone
  • statement_rating: (dv) subjects’ ratings of the causal test statement
  • timing_check: check question probing subjects’ understanding of the causal reversibility of the scenario
  • recorded_at: the date and time participants did the study
  • age: subjects’ age in year
  • gender: subjects’ gender
  • tech_issues: subjects’ report of potential technical problems
  • timing_check_correct: column coding whether subjects correctly remembered the order in which the causes acted in the test situation
  • learning_check_left_correct: codes whether subjects answered learning video question for left cause correctly
  • learning_check_right_correct: codes whether subjects answered learning video question for right cause correctly
  • learning_check_correct codes whether both learning checks (left and right) were answered correctly

5 Summary

5.1 Particiapnts

For details about the sample size demographics, see the analysis script here. The study employed the same inclusion and exclusion criteria as the previous experiments. 22 participants failed at least one of the control questions, which is why their data were excluded from all analyses.

5.2 Design, materials, and procedure

The experimental scenario and procedure was the same as in Experiment~2a. The only difference was that subjects this time evaluated only one test statement (using the same rating scale as in Experiment~2a). The different test statements were:

  • “The left male turned the female purple.” [trigger statement]
  • “The right male turned the female purple.” [trigger statement]
  • “The left male kept the female purple.” [maintainer statement]
  • “The right male kept the female purple.” [maintainer statement]

Whether the left or the right male served as first cause/second cause was counterbalanced between subjects.

5.3 Results and discussion

For details about the analysis (means, CIs, significance test results, etc.) see the analysis script here.

Subjects ratings are summarized in the Figure below, where it can be seen that the supplementary experiment replicated the findings of the main experiments. When the causal structure was irreversible, the second cause received low ratings; it was neither regarded as a trigger nor as a maintainer. This changed when the causal structure was reversible. In the reversible case, the second cause still was not regarded as a trigger, but it received high maintainer ratings. The study thus corroborates the Ross-Woodward hypothesis, just like the main studies. An unpredicted finding were the relative high maintainer ratings for the first cause under the irreversible structure. This had already been observed in other experiments of the paper. It is an open question why subjects have the intuition that the first cause maintains the effect even though they learned that the effect would not disappear again anyway.

The statistical analysis was analogous to the one of Experiment 2a (except that tests here were between-subjects tests; e.g., between-subjects Anova and contrasts instead of mixed Anova etc.). The exact codes and test results can be found in the analysis script here. The tests confirmed the predictions. In the ANOVA, there was a significant interaction effect between causal structure and test statement. Contrasts tested the difference between the trigger statements about the first and second cause in both causal structures. They were all significant. A final contrast tested if the maintainer ratings for the second cause were higher in the reversible case than in the irreversible case. This test was significant.

In sum, this supplementary study replicates the previous studies and confirms the Ross-Woodward hypothesis.

2024 Simon Stephan.