graphic cues superimposed on video sequences

[alvarom]

Hi, we are considering to buy Direct RT and I would like to know if it allows the presentation of visual cues (e.g. gif images) superimposed to short video sequences.

If this is not the case, is the solution proposed below correct?

If graphic cues couldn't be superimposed to video sequences these cues could be inserted in the video sequences (a rather tedious task) by editing single frames. Since I am interested in recording reaction times to these signals, should I keep the video frame rate the same as the monitor refresh rate? Please, keep in mind that I'm interested in measuring the reaction time to these graphic cues (participants would have to press a button when the cues appear on the screen). As far as I understand, if video frame rate and monitor refresh rate are the same, I should be able to get the reaction time to the graphic cues by substracting the reaction time mesured from the beginning of video presentation minus the cue presentation starting time.

Would this approach work?

Thanks in advance,

Alvaro

[jarvis24]

Hi Alvarom,

Awesome question. First off I think we *may* have added a function to superimpose graphics over video but I will need to check. For now, let's assume not and I'll try to answer your other question. In theory, absolutely, yes--if you have a 100Hz display which refreshes every 10ms, then a 10ms refresh rate would match nicely with a video encoded with 10ms frames. That sounds beautiful but in world of real things--I don't expect it would work that precisely. First, your encoder software would have to know you are really actually very serious about the 10ms frame rate--I don't know how much variability there is in that field. The other important variable I can not tell you is whether or not the frame rate would be matched perfectly with the screen refresh rate. In the vast majority of cases (I'm *guessing*) the frame rate would not be equal to the refresh rate (or equal to multiples of each other). Consequently, I can't see anybody having put effort into syncing them in the rare cases where they could be precisely coordinated.

So all of that is to say superimposing text over the video, or elsewhere on the screen (which you *certainly* can do) would be more accurate.

One thought--if there is any video codec out there that allows you to execute programmable events within the video you could take advantage of that. I have some recollection that .wmv (?) files many years ago would let you trigger events during the video. If you could embed text *and* send a TTL signal at the same time THEN you could know exactly when that text was presented.

Anyways, those are some initial thoughts. I will check through the code though and see if we got anywhere with text overlays in v2010. I wouldn't hold my breath but I do recall the effort to make it happen.

[jarvis24]

Looks like you can not overlay a text or image stimulus upon a video--even if you use a clear value of 0 for the text. The video will simply overwrite the space (the text or image will actually be behind it). *However* I tried playing a second video over top of the initial video after 1000ms delay. It had no problem paying on top. So if you could create your overlay stimulus/stimuli in any kind of video format (mpg, avi, animated gif) then you could locate and display it exactly as you like with as much precision as possible.

There was only significant problem I couldn't immediately work around. Videos do not automatically clear themselves from the screen. So the stimulus remained. Maybe this could be cured to some extent with an animated gif (or simply clearing the second video to a blank frame when it finishes) but you would still not see that region of the underlying video again. Some way to completely clear the second video without affecting the first would be necessary for this to be an easy method.

Some initial thoughts!

[alvarom]

Hi Jarvis,

Thanks a lot for your replies. I tried to do that -I've read in a different blog that the problem most likely arose for the text or graphic cues being still images. Unfortunatelly, when the videos were supposed to be shown, the experiment froze completely. Probably this was due to my lack of experience with Direct RT

I'll try to get both videos playing (most likely video and animated gif) and then try to find solution for clearing the 'cue video'.

I may come back to you in a couple of days.

Best,

Alvaro

[alvarom]

Hi again, Jarvis,

I still couldn't get to play the second video on top of the first one, but I did manage to play an animated gif on top of a video. However, I still faced some problems.

I thought that the best way to avoid the problem of the top video not clearing was to use an animated gif with a transparent frame showing for a long time. However, transparent areas/frames of animated gifs appear solid black. These areas appear transparent in Mozilla Firefox, MS PowerPoint and Open Office Impress.

Just for the record, there is an image distortion everytime a new loop starts.

I still couldn't manage to get a second video playing on top of a first one. When I try to do that, DRT freezes and get the following error message: "Empirisoft executeDRT has detected a problem and needs to close". At this point, my screen resolution changes from 1280x800 to 800x600.

Still, the most critical point would be to get rid of the black rectangle where the animated gif (or eventually the video) was showing.

Any ideas on how to proceed?

Best,

Alvaro

[jarvis24]

What version of DirectRT (20xx.x.x.x) are you running and on what version of Windows?

[alvarom]

Hi,

It's been quite a long time since my last post. In the last couple of weeks, we've started working again on this somehow forgotten project. I'd like to present the graphic cue on top of the video at random times, and measure the reaction time to this graphic signal (the subject would have to press a key).

The solution proposed by jarvis24 seemed very elegant (see above). On a few test experiments I got the desired graphic cue flashing over the clip. However the image presented by the exe file seemed to start much later than expected.

When trying different conditions, times, etc, I faced some problems which I couldn't solve. In some additional tests, I programmed the .exe file to start 10000 ms after the video has started (Video Stim - Loc: 0,01; Time: 10000; Exe Stim - Loc: 0,0,0; Time: rt:any) and had the following problems:

1. If I press any key after the exe file has been executed, the video closes down (the video clip is 2 min long).
2. If I don't press any keys, the video ends 10 seconds after the start. If I want to use random delays, I would have to use random times for the exe stimulus and the video would, most likely, end at those times.
3. In the log file, it's not really clear when the second stimuli (exe file) has started.
4. Are reaction times accurate when using exe files as stimuli?

Any ideas on how to overcome these problems?

Thanks in advance!

alvarom

[alvarom]

If I insert the graphic cues in the video clip itself, could I get a reaction time in relation to the beginning of the clip and still have the video play to the end?

This could be quite tedious, but -if possible- might be a way to circumvent the limitations...

Best,

alvarom

[alvarom]

After posting the last message I realized that we had already discussed about the possibility of inserting the graphic cues on the video itself. Since other solutions are not so straight forward, I guess I'll try check how accurately the video is reproduced. Would it make sense to record the experiment session (using some appropriate software), and then to analyze the frames of this recording? If I do this for a couple of video clips, would I be on the safe side?

Thanks again...

alvarom