PublicVR Forum

Online discussions of everything having to do with PublicVR, especially current projects, tech support, educational practice and theory. See http://publicvr.org for basic information and send questions to jeff@publicvr.org
It is currently Sat Feb 24, 2018 7:09 am

All times are UTC - 5 hours




Post new topic Reply to topic  [ 3 posts ] 
Author Message
Unread postPosted: Thu May 29, 2008 1:08 pm 
Offline

Joined: Tue May 27, 2008 11:29 pm
Posts: 558
These are musings I sent to Kerry and Lynn in an e-mail.

Just to make sure we are on the "same page", I think the experimental design we propose for our study this fall can be graphed like this:

Code:
    Groups:      G1, G2, G3       Randomly assigned from test group. "C" 
                                          is the control group.

    Treatments:  T1, T2, T3     Egypt Hall, Egypt Lecture, Egypt in ET
    (conditions)

    Quiz (test)  Q1, Q2, Q3       Each quiz tests for approximately the
                                          same knowlege, but the specific
                                          questions are different.

          1st       2nd       3rd
          Hour      Hour      Hour

    G1    T1  Q1    T2  Q2    T3  Q3

    G2    T2  Q1    T3  Q2    T1  Q3

    G3    T3  Q1    T1  Q2    T2  Q3


One of the bibles of experimental design for education and social sciences is:

Campbell, C. T., Stanley, J. C. (1963). Experimental and
Quasi-Experimental Designs for Research: Houghton Mifflin Company.
0-395-30787-2

They describe our design as an example of a "Counterbalanced Design," which describes all experimental designs where all groups take all experimental treatments. This has been around for so long in educational research that it has several synonyms: Rotational Experiments, Crossover Designs, and Switch-Over Designs.

The purpose of the first round is to be able to compare the effect of each treatment with respect to the other two treatments.

the second round allows us to explore which media pairings work best. Is it better to get a lecture about something and then see it or is it better to see it then get a lecture? You'll notice that only three of the possible six combinations are represented -- we do have to stay within several budgets: time, money, and available test subjects.

The third round lets us see if there are any ordering effects. One would expect that once each group of students has seen each type of media presentation, they should all score the same on the final test. If we do, in fact, get any significant difference between either two groups or any group compared with the others, that would be really something. It would tell us that something important is going on in the way the media are interacting with the student to affect the learning process. It would not tell us WHAT is going on, just that something important is there.

Interestingly, if students do worse on Q3 than they do on Q2, it will tell us that they got so tired after the second hour that (1) they were so tired in the third hour that they can learn anything or (2) they were so tired after the third hour at the just didn't want to do the test enough to care! Either result, for an individual group and/or students as a whole would be informative on how much you can expect kids to do over three hours span. Actually I think we should give them breaks after the quizzes to rest a bit.

Our experiment would be made much stronger and better if we added a pretest, Q0, for all the students to take before they see anything. The pretest would let us know for sure whether any of the treatments are actually working, or of all three are mired in failure! Since students will usually learn *something* from almost any exposure to information, this is unlikely. In fact it will give us a good fallback position in case we don't find anything else.

The content for each of the three treatments should be different parts of a single lesson. The earth theater experience would focus on the Temple, obviously, the tour of Egypt Hall would be a subset of the items there, and the reading/lecture.

Ideally, each of the four quizzes (including the prestest) should be a general test of the students' knowledge of the whole lesson. That way, we expect test scores to rise from one test to the next. This may be difficult to design especially considering that the tests should take no more than 10 minutes. The alternative is to design three tests, each one testing for a different third of the knowledge we expect students to learn. Then, we test them on just what they have seen. This is not simple! We will have to think about it, a bit, and probably bring in a teacher to help us with the test design.


Top
 Profile  
 
Unread postPosted: Tue Jun 17, 2008 12:04 pm 
Offline

Joined: Tue May 27, 2008 11:29 pm
Posts: 558
We have three treatments, t1, t2 and t3. There are six possible orderings they are:

t1 t2 t3
t1 t3 t2
t2 t1 t3
t2 t3 t1
t3 t1 t2
t3 t2 t1

I propose that we have SIX test groups, one for each combination. That covers all possibilities for a full "counterbalanced design." We do not need control groups, because we can pretest the students. (A total of four quizzes.) If we do use control groups, we would need three, one for each quiz and a distractor task for them.

After each treatment we have a quiz and a break.


Top
 Profile  
 
Unread postPosted: Fri Aug 01, 2008 8:01 am 
Offline

Joined: Thu Jul 31, 2008 2:37 pm
Posts: 47
classes and tours can only take about 30 at a time.... so we can't fit more than 90 in a day - will breaking it into 6 groups make the groups to small (15 each?) or can we combine to answer different kinds of questions?

we need to think carefully about the time constraints - tours and classes are currently 60 minutes - with 15 for a test and 15 for a break/walk to the next section that is 90 minutes. With 45 minutes for lunch the day is over 5 hours - we will have to concentrate on very local schools or school that can give us an extended day..... or maybe even consider something like 3 boy scout groups who can come on a saturday

for content we should plan both from the top down (general questions like what should the museum teach in the context of egypt) and bottom up (what specifically works well in each format)

we need to keep the goal in mind - this is a stepping stone to a bigger project (and bigger question)


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 3 posts ] 

All times are UTC - 5 hours


Who is online

Users browsing this forum: No registered users and 1 guest


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
cron
Powered by phpBB® Forum Software © phpBB Group