Research Field: Metacognition in Psychology
Project Title: Analyzing the effectiveness of smoothness of rhythm on people’s confidence and
accuracy in skilled performance
Project Description:
Dr. Carlson’s research lab conducted a study and is interested in conducting further studies to
test the impact of the smoothness of the performance rhythm (either on regular pace or irregular
pace) on people’s confidence and accuracy in counting. The research lab completed an
observational study of 90 participants who were asked to count the number of asterisks, which
flashed in sequence on a computer screen for 2/10ths of a second. The lab manipulated the pace
at which asterisks appeared. Participants were college-age students who voluntarily participated
from an Introduction to Psychology course and were randomly assigned into one of the three
groups by the experimenters. Groups are expected to have equal sizes. Each of the volunteers’
attempts was split into three sets (blocks) of 16 trials for a total of 48 trials. Each trial means a
sequence of 30-50 asterisks. Group one volunteers faced regular pacing to the asterisks for their
first and third blocks and irregular pacing for their second block. Group two volunteers faced
irregular pacing for their firsts and third blocks and regular pacing for their second block. Group
three volunteers faced regular pacing for all three blocks. At the end of each trial, volunteers
reported the number of asterisks they saw, as well as their confidence level (measured as a
percentage) about their accuracy in counting for that trial. As a result, 48 observations for both
counts and confidence level were reported from each volunteer. Researchers recorded the
accuracy variable as 0- incorrect or 1- correct depending on if the correct number of asterisks
were reported. Researchers assumed that the average number of asterisks volunteers were off by
when they were incorrect were about the same across groups.
The client provided raw data from the study collected by his research lab and approached us in
regards to analyzing the data in order to determine if the smoothness of rhythm has an impact on
people’s confidence and accuracy in counting. From the previous memory research, researchers
knew that people are expected to be more accurate and overconfident if the pace is regular even
when the average time for the asterisk is the same for an irregular pace. The client wants to know
Statistical Consulting Center
Department of Statistics
Eberly College of Science
Tel: (814) 863-0281
Fax: (814) 863-7114
The Pennsylvania State University
323 Thomas Building
University Park, Pa 16802
2
how confidence and accuracy compared, and if the expected outcome from the previous memory
research is correct. The client also wants to determine how confidence level and accuracy shifts
with the pacing, and if the effects of the pacing shift are different for confidence and accuracy.
We will try to answer these questions while analyzing the dada. This particular study will be
useful in helping the client determine if he should continue to investigate this claim with a more
expanded population. In the end, we will write up the results of our analysis on the data in a
report, as requested by the client, that will describe the statistical models used and the
interpretations of the results. Since the results will be used for publication, and replication of this
study is necessary, there will be enough detail included in the report such that the statistical
analysis are reproducible.
Questions to be addressed:
Question 1: How are people’s self-reported confidence and the observed accuracy compared with
each other?
Questions 2: Are people overconfident and more accurate with regular pace? And are the effects
of pacing shifts different for confidence and accuracy?
Second Meeting:
Dr. Carlson requests to get the results in three weeks. This is a reasonable turnaround time based
on the dataset and relevant hypotheses to test. Using this three-week window allows Dr. Carlson
to make a decision on whether or not he wants to continue researching these hypotheses in time
for spring break at PSU. Based on the results presented to him at the next meeting in three
weeks, Dr. Carlson will plan and pilot test another study which could then be implemented when
students return from spring break, enabling the study to be completed before the semester ends.
Thank you,