Home|Software|Hardware|Store|Download|Support|Company
Psychology Experiment Software Reviews

See also: Personal Reviews

SPSP Psychology Software Summaries

Occasionally, the Society for Personality and Social Psychology list serve produces an informal survey of psychology experiment software. These results are extremely informal and biased in terms of self-selection. With that said, we post them here anyways because MediaLab and DirectRT usually come out looking pretty good. Where possible, we've added links to the websites of the other software programs. 

January 6th, 2007
July 7th, 2005
November 9th, 2004
January 8, 2001

DirectRT v2006 Review, Experimental Psychology
 
From Stahl, C. (2006). Software for generating psychological experiments. Experimental Psychology, 53, 218-232:
"Among the reviewed packages, DirectRT achieves the most complexity reduction. A simple experiment is most easily implemented with DirectRT's simple list structure syntax, essentially reducing the task to specifying the stimulus item, its location, and presentation time in a text editor. The other packages require generating longer and more complex definitions (Inquisit), or first becoming acquainted with a multiwindow visual user interface (E-Prime). Therefore, DirectRT would be the package of choice where assistance with standard paradigms and a quick start with experimenting is the major objective..." read article



From the SPSP Email Listserve, January 2nd, 2007:

Greetings: I've been conducting experiments using pencil/paper or programs designed specifically for a particular problem. But I would like to start using an all-purpose program (such as MediaLab) to build social psych experiments. The problem is that I know very little about what's out there. My primary concerns are 1) ease of use (I don't want to spend a lot of time "programming") and 2) generality (I'd like to be able to use the software for a wide range of experiments). Any thoughts on the pros and cons of existing programs would be very much appreciated! Thanks in advance. 

Brent Simpson
University of South Carolina

Summary Posted January 7th, 2007:

Hi Everyone: I received a LOT of helpful responses to my questions about software for building experiments. MediaLab came up most often but many responders made strong cases for other programs. I've pasted all the responses I received below, and attached two relevant documents. Thanks again to everyone who responded.

Brent

  • There is a product called SuperLab. It can do questionnaires, Stroops, subliminal studies. I use it and have found it to be relatively straightforward.
  • You might be interested in the attached manuscript (the pdf file) that appeared in Experimental Psychology in 2006.
  • Some reactions to your question: As a general rule, I have found that user-friendliness (or ease of use) and generality are negatively correlated. In other words, whereas the more advanced (and more difficult) packages or languages will allow you to do virtually anything, more user friendly programs will eventually always have certain limitations. This is a potential trade-off you will need to consider. Having said that, most 'basic' packages are capable of doing 80 to 90% of classic computerised tasks. I know there are many packages out there. But I have personally mostly used the following programs: (1) Inquisit (Millisecond software). I would describe this program as mid-level user-friendly. It is relatively easy to use, but it does require you to write the syntax by hand (which can be tricky). Inquisit has its limitations, but is great to quickly write straightforward programs, especially when millisecond accuracy is required (e.g. lexical decision, ...). (2) Visual Basic (Microsoft). This is not a experiment generator, but a programming language which makes it very powerful but also rather low in user-friendliness (and therefore not really suitable for your purpose).
  • I have recently done a review of the available packages: Stahl, C. (2006). Software for generating psychological experiments. Experimental Psychology, 53, 218-232. You can download it here: http://www.psychologie.uni-freiburg.de/Members/stahl/publications/expsoft/download
  • Can't tell you about other software, but I have been happy with MediaLab. Versatile and easy to use.
  • I have been using the Empirisoft suite (Medialab/DirectRT) for several years now, and I can say that together they have been able to fulfill all of my research needs. The tutorials are straightforward and quick, and teach you everything you need to know in a short period of time. Once you have been through the tutorials, you will be able to program a large variety of experiments in a short period of time.
  • MediaLab is very good and easy to use. For participant sign ups Sona Systems offers excellent software.
  • I love MediaLab. It's very easy to learn and use. Regarding generality, it has been getting better but there are still some drawbacks. If nothing has changed recently, then you can only have one response box (or one questionnaire item) on each screen. This may or may not be a limitation depending on what you want to do. Also, it's not as accurate as some may like if they want to measure reaction time and it probably wouldn't be useful for the subliminal presentation of stimuli. If you don't have those kinds of needs, then I highly recommend the program
  • The attached document (the MS Word doc) summarizes some reviews on experiment software. MediaLab/DirectRT is by far the most common and easy to use program.
  • [MediaLab has] excellent support. If you e-mail them with questions about how to do something, they are always very helpful and reply promptly. That's been a huge plus for me.
  • In response to Brent's question, I've used Medialab and Superlab and found them both to be user-friendly, with the exception of my stated dilemma, and require little programming. Medialab has helpful drop-down menus and the syntax is a simple spreadsheet like Excel. I've only run reaction time experiments and presented slide shows of stimuli (easy integration with PowerPoint), but I liked Medialab the best and found it easier to learn quickly.
  • I have been using RiddleMeThis for over a year now to great success. Check it out for free at loewald.com/RiddleMeThis/
  • CDC and e4xchange Inc built a survey program that incorporates factorial software so you can build experimental vignettes. It's very easy to use and allows amenities such as dropped cells in an otherwise fully factorial design (e.g., female + prostate should not be crossed in an experiment aimed at physicians). I was the project officer for CDC; Sam Addala, sam@e4xchange.com, is the PI. I should note it's pretty comprehensive (hosting, data storage), but not free. Feel free to look over www.e4xchange.com as reconnaissance! It ought to be just about operational.
  • The last two times this question was posted to the list, MediaLab came out the top favorite.. So just to add my vote -- I use MediaLab and DirectRT in my lab. They are easy, powerful, and flexible. And I have found the technical support from Blair Jarvis to be excellent. The company name is Empirisoft.
  • Over the past years, I've been working with ERTS, E-Prime, MediaLab, DirectRT, and Inquisit. Of these software packages, Inquisit stands out as the most versatile and easy to use. It allows you to program simple questionnaires AND sophisticated experimental tasks all within the same, relatively easy to learn programming language (spend 1 or 2 days with the tutorials, and you're good to go). An additional perk (although one that I don't use) is its ability to present experiments and questionnaires via the web. Besides, Sean Draine, the owner of Millisecond Software and mastermind behind Inquisit, is incredibly responsive when it comes to troubleshooting and incorporating features desired by users into new versions of Inquisit. This responsiveness is something I've come to appreciate over the years. For more info, check out the Inquisit website at: www.millisecond.com
  • You might check out the SPSP list archive, as this issue has come up before. In any case, check out http://psychexps.olemiss.edu/ which has a bunch of experiments run with the Authorware program. This is an open-source site (you can download the programs that run the experiments and tweak them to your own purposes). Plus the advantage is that you can run the programs on the web, which I don't think Medialab and other programs can do. Another advantage is that you can run your programs on multiple machines or on an intranet at no extra cost (you buy one copy of the program for about $600). Authorware does not meet your first criterion but it does the second. I think your 2 criterion are negatively correlated, so you can't satisfy both. The learning curve for Authorware is steep, but the psychexps folks have an interactive DVD that will teach it to you, and they used to offer workshops.
  • For the ease of use I'd go with Superlab. I've been using it for years and it is user friendly and can handle a wide range of experiments. However, if you're thinking about adding fMRI or ERP to your measures arsenal I'd recommend Presentation. See below for links describing other software and comparisons between different software options: www.socialpsychology.org/software.htm#psychology
    psych.hanover.edu/Krantz/software.html
    www.empirisoft.com/support/showthread.php?t=17 (a bit biased comparison)
    www.bc.edu/offices/help/meta-elements/doc/articles/html/RES-experimental.shtml
    xlab.berkeley.edu/news/software.htm
    www.psychologysoftwaredistribution.com/Catalogue/catalogue.html
    You should also search the SPSP listserve archive, as this question keeps popping up every now and then.
  • [this one is from us!] In case it helps, here is a link to some previous SPSP summaries on the issue: www.empirisoft.com/pressreleases.aspx. I'd be curious to know what kind of responses you get this time around! Here are some additional testimonials not from the SPSP list: www.empirisoft.com/support/showthread.php?t=46 Another package you might want to check out is Inquisit. Although a little trickier to learn than MediaLab and/or DirectRT, it's also a very powerful program and worth having in your software collection. My personal take on the differences between all the major programs can be found here: http://www.empirisoft.com/support/showthread.php?t=17
  • Sadly, there's almost inevitably a trade-off between ease of use of experiment-creating programs and their flexibility and power. I use an application called Authorware from Macromedia. It enables you to create standalone .exe programs, or web programs, that do just about everything. For example, the programs permits text, graphics, sound, and video to be flexibly handled, accepts variety of user inputs, records RTs with high resolution, and permits complete control over things like randomization and counterbalancing. For this, you pay the price of having to spend a number of weeks getting to grips with it, and developing a number of standard pieces of code. You can adapt pre-existing pieces of code however. You can see examples at http://psychexps.olemiss.edu/
  • MediaLab and DirectRT will be perfect for you. I've been using both since 1999 and have nothing but praise - extremely easy to use, relatively powerful in terms of what you can do with them.
  • Brent, this may be a minority view, but FWIW. I ran several experiments (including those for my dissertation) with FileMaker Pro, a database manager. Part of why I'd steer you in that direction is the incredible versatility of the application. What you learn to use it as a data-collection tool will then help you use it to solve a wide range of data-management issues. A somewhat recent improvement is the level of precision for timestamp functions. It captures 2 decimal places on seconds. I don't know if that's adequate to all tasks (I think some reaction-time processes may require more precision). FileMaker Pro Advanced also allows you to develop a free-standing solution that can be installed on lab computers, so you can create as many copies as you need. If you'd like I can give you some more details about the different kinds of things I've done w/ FMP.

Brent Simpson
University of South Carolina  



From the SPSP Email Listserve, July 7, 2005:

This is a long overdue summary of responses to a request I made some time ago concerning research software. We have also included our own assessment of the products we chose (in good part as a result of these responses). My original request was:

"Several years ago there was a request for suggestions concerning software for building social psychology experiments and collecting data on computers. At that time, people described programs like Superlab and Authorware. We are in the process of outfitting a new lab. Given the pace of change in these things, we wondered if people have new or different suggestions for programs of this type."

We had approximately 20 responses with some people commenting on more than one program:

Positive Responses

Negative Responses

MediaLab

13

1

DirectRT

7

E-Prime

2

1

RiddleMeThis

1

MatLab

1

Inquisit

1

Visual Basic

1

MouseLab

1


Clearly, the most commonly recommended program was MediaLab (by Empirisoft) with 13 people commenting positively. MediaLab was highly recommended for computer-based questionnaires since it allows for collecting data in a variety of formats (scale response, multiple response, thought listings, etc.). MediaLab, in general, was described as being very versatile, user-friendly, and powerful. In addition, a number of people praised the tech support (Blair Jarvis at www.empirisoft.com) received for the few problems/bugs encountered while working with the program. For reaction time measures, DirectRT (also by Empirisoft and integrates nicely with Medialab) and E-Prime were recommended with a clear nod to DirectRT. E-Prime was described as requiring more effort to become completely familiar. For those wanting more control than MediaLab and DirectRT can offer, MatLab was recommended (although this program was described as taking longer to learn and requires more programming knowledge). RiddleMeThis, Visual Basic, MouseLab, and Inquisit were other recommended programs. Our own computer technical support person preferred E-Prime (especially the most recent release), but admitted that learning the program was much more time consuming and E-Prime was not as user-friendly as MediaLab. We went with MediaLab & DirectRT, and have since used MediaLab for a number of studies in our lab. We have found MediaLab to be easy to use and pretty flexible. We have also encountered some issues. For example, only a single questionnaire item can be presented at a time when in some instances it would be preferable to present multiple items. In addition, formatting the questionnaire can be somewhat difficult. In order to make a single word or phase stand out (e.g., with color, italics…), for example, an alternate program is needed (e.g., Word, Powerpoint). All that being said, most problems can be solved if one is creative, and our research group has become pretty proficient in a relatively short time frame. We would add our names to the list of those recommending MediaLab .

Dr. Stephen Wright,
Simon Fraser University



From the SPSP Email Listserve, November 9th, 2004

Thanks for all the responses regarding RT software:

DirectRT, www.empirisoft.com, $475 w/21 day free trial
Inquisit, www.millisecond.com, $395 w/60 day free trial
DMDX, www.u.arizona.edu/~jforster/dmdx.htm, freeware
Affect, www.psy.kuleuven.ac.be/leerpsy/affect, freeware

By far the most recommended software was DirectRT, especially for it's ease of use. Inquisit was a close second, followed by two freeware programs. DMDX was developed by Jonathan Foster. Affect was developed by Hermans et al. (2002) [Hermans, D., Clarysse, J., Baeyens, F., & Spruyt, A. (2002). I hope this is helpful.

Best,
Tom Denson
University of Southern California



From the SPSP Email Listserve, January 8, 2001:

Dear Colleagues,
It has been a couple of weeks that I have asked you for experiences with lab software. Your responses were most informative. Many thanks for your help! Here is a short and impressionistic summary that is, of course, highly biased by the self-selection of the respondents: Over all, of the some twenty replies that we have received, there seems to be a strong preference for MediaLab and the corresponding RT tool named DirectRT. Only strong suggestions in favour of MediaLab were reported, literally every respondent who was using MediaLab seemed to be pleased with it. A different picture has emerged for other software: Unfortunately, very few respondents seemed to have first-hand experience with E-Prime. There still seem to be some problems in adapting old MEL programs. Supposedly, the programming code has not changed that much and still seems to be less intuitive than the easy-to-use, clickable tools of MediaLab. Nevertheless, the RT measuring tool of E-Prime seems to be comfortable and exact - something we do not know for Inquisit and Superlab, programs that seem to be less widely distributed. In sum, both E-Prime and MediaLab seem to have good RT-tools that work properly and yield accurate results in the "not so easy to handle" windows environment.

Fritz Strack
Universitaet Wuerzburg, Germany


Company

News

April 28th, 2020
Site licenses for v2020 now allow users to work from home--ask us for details!

April 21st, 2020
MediaLab and DirectRT v2020 are now available!

January 30th, 2020
We can now authorize site license machines by using your domain names--ask us for details!

April 21st, 2016
MediaLab v2016 is now available! Compatible with all versions of Windows. MediaLab v2016

April 21st, 2016
DirectRT v2016 is now available! Compatible with all versions of Windows. DirectRT v2016

February 21st, 2014
Official release of our brand new Rotary Controller. Gather continuous dial and dichotomous button input in one simple unit! Read more

March 16th, 2012
MediaLab v2012.4 is now available! Compatible with XP, Vista and Windows 7. MediaLab v2012

March 16th, 2012
DirectRT v2012.4 is now available! Compatible with XP, Vista and Windows 7. DirectRT v2012

January 1st, 2012
Release of the updated DirectIN v2012 button boxes. Same super-fast circuitry but now with remote firmware updates over the internet and full Windows 7 compatibility.Read more

March 16th, 2012
View and search the 2012 interactive user guides online: MediaLab and DirectRT

January 1th, 2012
The latest public software reviews from SPSP and elsewhere.

News Archive


MediaLab Research Software and DirectRT are trademarks of Empirisoft. All other products mentioned are registered trademarks or trademarks of their respective companies. Purchase inquiries should be directed toward our ever-friendly sales team. Frequently asked questions, how-to questions, suggestions, and troubleshooting inquires can be addressed through our searchable support forums located at http://support.empirisoft.com.

Copyright © 1997-Eternity+ Empirisoft Corporation. All rights reserved.
Last modified: April 21st, 2016.