helping with programming esp. randomization

Hi there,

I'm getting ready to program MediaLab for an upcoming experiment. I've actually tried a little - but it's not coming out how I'd like so I'm hoping that you can help me.

In our original experiment, we had videotaped participants during a 10 minute interaction. In our upcoming experiment, we'd like raters to look at clips from these videos and rate them on a number of dimensions. Because we don't want to tire out our participants from watching too much - we've split the 10 minute videos into 30 second clips (so there will be 20 of them for each interaction). We'd like participants to receive one random 30 second clip from the first 2.5 minutes, one from the second 2.5 minutes, one from the third 2.5 minutes, and one from the final 2.5 minutes. After each 30 second clip, they'll rate the video. The trick here is that obviously these random clips need to come from
the same 10 minute session.

After that, they would go on to rate another set of 30 second clips from a different interaction. We also would want to make sure that they wouldn't rate an interaction that they've already rated. We have about 40 or so interactions in total - but participants will probably only rate 15 of them or so.

Does this make sense? How would I go about setting this up? I was thinking that each interaction should probably be set up as a separate condition - but can you program medialab so that participants would randomly view 15 out of the 40 interactions? Also, what code would I use to randomize within each interaction?

Thanks!
Robin Freyberg

I would create 4 questionnaires--each with 5 clips. Randomize the order of the 5 clips within each questionnaire and then play only the first clip in each by using an unconditional skip value of "+4" on all 5 clips--so no matter which one they get, MediaLab will skip over the other four in that questionnaire. I think this may require 2006.2.40 or later.

Thanks so much for your response - it's definitely helpful.

I've been able to get it to work and managed to do a sample run with sets of videos from two separate participants by assigning each participant the same order value but assigning each participant a different value for within-subjects randomization within an experiment file.

But, the question remains about how to make sure that raters will not end up viewing the videos for all participants. As I mentioned before, there are about 40 different participant videos - there's no way that raters will be able to handle viewing all of them.

So, what would I need to do so that raters would view a random selection of 20 of the participants out of the 40?

Thanks for your help!

Can you zip your folder (without the videos) and attach it so I can see where you are with it?

Okay, so here's the sample that I worked on - you'll see that there are only two 30 second clips for each of two different participants.

Also, because there are sometimes two people in the video, I brought in an image of an arrow to indicate who the rater should focus on. What this means is that I set up the video portion as a backgroundvideo and the image of the arrow is the central feature of the questionnaire. I hope that makes sense.

So, I guess my question would be if there was a way that the rater would only end up randomnly rating 1 instead of both sets of video clips (but just realize that this will be expanded greatly for more participants, more clips, etc.).

Thanks for your help!

I missed that you were going to repeat the whole process 15 out of 40 possible times. That certainly makes it harder. Take a look at the simplified sample I've posted. The is the logic only--no filenames are mentioned.

Basically, it illustrates a situation where there are movies from three interactions--and each subject will see clips from two of those three. Further, each of the three movies is broken down into three sets of two clips. Each subject will view one of the two clips from each set and then answer a question about that clip.

See if the logic in the sample makes any sense. If so, you should be able to expand on the pattern to create your larger design. Let me know if you have any questions or if I've missed something.

Hi there,

Sorry it took so long to respond to your post - we were on break so I was away from the software. Regardless, I took a look at your attachment - and it looks like it will solve the problem. I haven't expanded the whole thing yet - but at least with the example you posted, it seems to work. So, thank you! I will let you know if any problems emerge as I expand the programming.

Thanks again!

Hi there,

If you remember, you were helping me with the randomization for all of those video files. So far, so good - except that it looks like I ran into the limit on a questionnaire file. I'm on the 2,000th row - and it looks like it won't go any further.

Is there anything I can do to fix this situation? Is there a way to extend the limit? Or should I start with a new questionnaire file and randomize between the different questionnaires? What do you think?

Thanks for your help!

You can try opening the file in Microsoft Access and editing it there. Before you do all that work though, I would test what you have so far--things might start getting slow once you have 2000+ items in a single questionnaire!

See also: [URL]http://www.empirisoft.com/support/showthread.php?t=663[/URL]

Thanks for your reply. I've tested it out a little and have no problems running the study. Things get a little slow when I'm copying and pasting in the questionnaire - but the actual running (so far) seems to be okay.

I'm a little nervous about using access since I've never worked with it before. Would it be possible to create several questionnaire files (let's say 4) with the video files equally broken up among them and then participants randomnly get the videos from that questionnaire file. How would I set up the experiment to randomnly choose one of the four questionnaire files?

Thanks for your help!

That would be a bit tougher because, although you can randomly order the questionnaire files within the greater experiment file, you can't set up skips in the exp file like you can in the que file.

With regard to Access, it really simple if all you are doing is editing the questionnaire information--it's like editing in Excel. Maybe make a copy of the file and just try it and see if you like it. I would definitely prefer it if I were working with so many items. But, hey, if it's working the way you're doing it, then maybe there is no need.

Another problem - I tried working with Access but the formatting looked strange when I brought the questionnaire file over. So, I've been sticking with splitting the experiment into 4 questionnaire files.

However, I think I may have run into another limit of the program. Do you know if there's a limit for the numbers that can be entered for WGR and BGR?

This happened in one of my previous files when I ran a mini experiment to make sure that everything worked, I got the error message "9subscript out of range". It seemed to work again once I changed the values for WGR and BGR.

Have you seen this before? Do you know what the limit on these files are for this value? Or is this referring to something else?

Thanks!

I recall one issue with this a while back. There may be a limit such that the specific BGR and/or WGR values that you choose can not exceed the actual number of groups you have in each case. For example, if you are randomizing 15 blocks, you may have trouble if you use a BGR value of 16 or higher for any of those blocks.

More questions! Surprise, surprise!!

When going to analyze the data, it looks like I've maxed out the excel file and when I open spss, I don't get the variable names - but a series of v0001, v002.

Do I need to get excel 2007 to see all of my variables? How do I get SPSS to show the name of the variable rather than the weird code? I've attached a data file as an example.

Also, I've been noticing some problems at the end of the study. It takes a very long while for the program to say "Please tell the experimenter that you are done". The program has been kind of crashing before that point - is there a way to stop that from happening? Everything else runs so smoothly even with all of the video files that the program is coordinating that it seems so odd to practically give up at the very end. If I do a control-alt-delete before getting that popup message, does the computer still hold onto the data or do you need to click "ok" for it to save the data?

for the excel limit issue, see:

[URL]http://www.empirisoft.com/support/showthread.php?t=498[/URL]

and see the section on "CSV vs TXT vs SAV Files" here regarding the variable naming issue:

[URL]http://www.empirisoft.com/medialab/help/data.htm[/URL]

let me know if that helps. regarding the stall at the end--how much time are we talking about?

Thanks! I will check those links out.

The last stall that I observed was about 2 minutes at least.

Is the stall just on this one experiment, or on all (e.g., even th sample experiment that comes with MediaLab)? If it is specific to this one experiment, do you have other programs involved with your study? e.g., DirectRT, Word, PowerPoint, etc.? What is the last item in your session?

I think it's just with the experiment in its completed form. When I even did a sample run with far fewer videos loaded, I never had a delay. I'm only using media lab for the study - coordinating videos with the questions.

The last thing they do is to complete a questionnaire after a video clip and then they get a thank you screen - there's a skip that takes them from the last questionnaire to the thank you screen.

I seemed to have stumbled upon another problem at the end today. The running of the actual experiment is fine - as usual.

Today, the problem seems to be with the skip. The participants got to the skip - but then it was supposed to go to the end - where they get a thank you screen. Today, the skip worked but then the program just closed - they didn't get the thank you screen and they didn't get the popup that says "please tell the experimenter that you are done" that's built into the system.

The only thing that I changed today was that I moved the skip earlier because the study was just too long. I've attached one of the questionnaire files and the experiment file so that you can take a look.

Sadly, when I open up the data folder, nothing is there which makes me think that I'm losing participants for today.

Please get back to me as soon as you can - otherwise, I'm afraid that I'm losing data.

Today, I've played around with a bunch of things - and I still can't get the skip to the end to work right. The skip works - but then it won't go to my final thank you screen - the whole program just closes (as if I did control-alt-delete).

It doesn't matter where I place the skip - the program closes even if I put the skip after the first set of videos. The same thing will happen - it will go to the skip but won't go to my thank you screen and the program closes and I don't get the popup saying that they should tell the experimenter that they are finished.

I don't understand why this is happening given that the only thing that I changed is the placement of the skip. Does the skip need to be closer to the end for it to work? That seems a little odd.

Is my data still saved somewhere?

Also, the sample experiment works fine so it's not the program as a whole...and as I mentioned, the experiment works right up until that point - there just seems to be a problem between it connecting from the skip to the end screen.

Thanks!

Let's try this. Right now you have 36 blocks. Make a copy and delete the last 26 blocks so that you have only 10. Run it and see if you have the same issue. If so, cut out another 5 and another 2 and so on until to see if you can get the same effect with just oine or two blocks. If so, then start deleting as many items within the blocks as you can so that the structure and skipping remain in tact but you have a bar minimum/skeleton version to problem solve with. If you can get the "minimalist" version of your que file to produce the problem, I'll go over it in detail to find where the problem might be coming from. Does that make sense?

Thanks for getting back to me - especially with Thanksgiving right around the corner.

So, I tried what you suggested. I just moved the skip after only 1 set of videos so that I wouldn't have to sit through 30+ minutes of videos.

Anyway, with only 10 blocks, I did get the thank you screen but then the screen went blank and no popup at the end occurred. I had to do a control-alt-delete to get out of the program. This is reminiscent of what was happening before when I had the skip after 25 blocks whereas for the study now it is after 15. At least it did not close out the program before the thank you screen.

So, then I tried it with 8 blocks, I got the thank you screen - it stalled for a little bit but then the popup finally occurred.

I'm attaching both questionnaire files so that you can take a look. The test1 file has 10 blocks - the test2 has 8.

Now what do you suggest?

Also, can you please tell me if there's a way to retrieve my data? It doesn't seem like the data saves if the participants don't get the thank you popup, but I feel like the data should be in there somewhere.

If you ctrl-alt-del before you see the final thank you pop-up, then MediaLab will not write the data because that is the point at which the data are saved.

If it works with 8 block but not 10--and that there is a long delay after 8, then I would be curious to know if 10 would work if given more time (e.g., go to lunch and come back kind of thing--again just curious). I have heard of delays happening after sessions (before the data are written) and I have some recollection that many videos have been involved in those cases. If so, it may be some kind of memory issue since videos can take up a lot of memory.

If you want to burn your experiment to a CD and send it to us, I'll try running it here and see if I can tell what's going on. In the mean time you could also send me a [I]dxdiag.txt[/I] file so I can check out your system specs and see if there is anything unusual about your computer setup.

See [URL="http://www.empirisoft.com/Support/showthread.php?t=75&highlight=dxdiag"]http://www.empirisoft.com/Support/showthread.php?t=75[/URL] for details on how to generate that file.

Thanks for getting back to me about this. I'll get that file to you, but I won't be able to send it today.

About the cd thing, would I send the videos too? If I send you the actual study with all of the videos, it won't fit on a cd - it won't fit on a dvd either. I'd have to send you several dvds - which I don't mind doing - it's just a warning. Should I send them to you? Where would I send them?

Also, I can try letting the 10 blocks go on and on and see what happens - but I'm concerned because it still doesn't get to the problem b/c the experiment chooses 15 random blocks out of 28.

I just wish I knew why it wasn't working - the whole experiment runs fine -except at the very end - it's something about that skip that's making it unhappy - even though it seemed to work before.

What version are you using i.e., 200x.x.x? OK< don;t send the whole thing if it's that big. But do try this.. try a copy of the questionnaire where you remove all the background videos. Keep the items and everything the same, just enter "none" in all the backvideo fields. That ought to tell us if it's a memory issue or a logic issue.

I don't have the software in front of me b/c it's a school, but I believe it was the most recent 2006 version. Based on your earlier recommendation (if you look at your very first response), I asked for 2006.240.

I should be able to send you the questionnaire file without the videos (and that other file you requested) on Monday.

I know this is probably unlikely, but b/c I'm located in NYC, is there any way that I could either pay you to make a housecall to the lab or that we could discuss this at your office? I feel like it might be more efficient esp. since we're probably only a few miles apart.

Hi there,

I'm attaching the experiment file along with the four questionnaires (if you remember, putting all of the videos in one questionnaire file was too much so we split the files evenly into four questionnaire files and participants randomnly get assigned to one of the four files). By the way, each of the four questionnaire files results in the same problem.

I'm also attaching that diagnostic file.

Please let me know if you have any solutions.

Please try running this (attached) revision all the way through and see if the issue persists. It eliminates the background videos and the delays are set to 2 seconds for the convenience of testing.

Thanks for getting back to me.

So, I was able to run the file - but unfortunately, the same problem occurred. The experiment ran fine until the end - I got to the skip and then there was no thank you screen and no popup - it just closed the program automatically.

I was really surprised by that b/c it means that it's not the videos that's causing the problem.