recall list data misplaced (case 1747)

Hello,

I have created a questionnaire in which I use the "recall list" variable type, but there is a problem with data being misplaced in the CSV files. Details of questionnaire are as follows: participants are told to recall all that they remember of 4 types of cars. Each car name is presented to them, and they are asked to recall what they remember. And, they are told, "After you do not remember anymore attributes, you can press the "Esc" key in the upper left corner to move on to the next car. After 3 minutes, you will be automatically moved onto the next car whether you are finished or not." So, each car is presented in a random order using the randomize within feature set to 1 for each item, the n parameter is set to 12 (there were 12 attributes), and the duration parameter was set to 180 seconds. The write data and the RT options were checked for each item.

The "by questionnaire" CSV file is fine until the "Recall" questionnaire. At that point, responses start to be put into the questionnaires that followed the Recall questionnaire. This does not always occur, but it appears to do so, perhaps, after a participant has used the escape key, but I am unsure. Please see attached file for an example of this data misplacement. The problem begins on column DA.

Not seeing the attachment here. Could you also include a copy of the que and let me know what version of MediaLab you are using? i.e., 200x.x.x?

Here are the requested files. We are using ml2008.1.22.

Brandon,

I just ran through your que a few times with the duration values set to 10 seconds instead of 180. Each time, I would do a combination of responses--sometimes pressing escape to end the recall list early, sometimes letting the time pass, and other times, answering all 12 for a given item. In these tests, the data did not fail to write--everything looked ok.

I'm not sure if the greater context of your experiment might be having an impact though. Can you try running your que (the one you attached) in a similar manner and see if you can get it to replicate the result? If not, we may need to look at what precedes or follows this questionnaire.

My other other question (sorry, forgot)--is what you found the same in the case of every data file--i.e., the .txt, .csv, and spss .sav files--and in both the byVariable and byQuestionnaire folders?

And one last thought--in case it's a possible cause, you have an earlier release of v2008. If you are able to reproduce the problem and I can't--it might help to update to the final release of v2008 (v2008.1.33) found at [URL="http://www.empirisoft.com/download_old.aspx"]www.empirisoft.com/download_old.aspx[/URL].

I do get this problem in both the CSV and SPSS files for byQue, and byVariable (see attached zip file Au2010 NC...).

I also have tried to systematically replicate the problem using the Recall.que file in between the two questionnaires that it follows and precedes in the full experiment. However, I have not been able to replicate the problem using 30 different variations of responses (see attached zip file Au2010 test).

I will now try it with the updated final release of v2008.

looks like we are using the most up to date versions of v2008 (ML 2008.1.33, and DRT 2008.1.13) and have used them all of last semester when we were having the problems. I now don't believe the version is the problem.

So just to be clear here, I understand that it is affect all of your data files which is very useful to know. And that it happens with the final release of v2008. What I'm not clear on is this:
[QUOTE]I also have tried to systematically replicate the problem using the Recall.que file in between the two questionnaires that it follows and precedes in the full experiment. However, I have not been able to replicate the problem using 30 different variations of responses (see attached zip file Au2010 test).
[/QUOTE]
I like the "I have not been able to replicate the problem" part but I'm not clear on the conditions under which you are NOT replicating. I had been wondering of you would see the issue occur if you just ran the recall que alone--is this part of what you are describing here? If not, are you describing conditions under which you can see the issue or not see the issue (e.g., turn the problem "on" or "off") depending on what questionnaires you place before and after it (esp re 30 variations). I'm going through the attachment but these are my first thoughts/questions.

One other comment--if it is the case that you found you cant replicate the problem without the greater surrounding experiment--can you discover how much of that greater experimental context you *can* take away and still reliably experience the problem? That is, can you find the "tipping point" at which the problem will or will not occur depending on some critical change you can make or item/parameter you can take away. That sort of thing is akin to the holy grail of what you could offer us from a problem solving perspective.

See below for my response to your query about "the conditions under which you are NOT replicating.”
Conditions under which I am NOT replicating: Used the entire experiment, but skipped through questionnaires until the Choice.que. At that point, I answered the questions for the Choice.que, Recall.que, Weighting.que, and NeedCog.que. These are the questionnaires between which the Recall.que falls in the experiment file and in the data file (for Order by Que file). 15 of my 30 variations were in regard to the responses (i.e., answer all 12 recall items for each of the four cars in the Recall.que; or answer some and press escape; or answer none and let it time out; etc...). The data file for this test was the “Au2010 NC Presentation – Harriet ALTERED” file. In this file, subjects 1 through 13 are real participants, while 887 through 873 are my unsuccessful attempts to replicate problem.
My main problem is that I was never able to replicate the problem doing the procedure above or when I just sandwiched the Recall.que file in between the Choice.que and Weighting.que files (Au2010 test file uploaded onto forum) as it is ordered in the entire experiment.
Tested Recall.que alone? I have just run 15 variations of responses for with just the Recall.que and haven’t replicated the problem. I answered all the recall items for the first 6 variations, then I answered all for all except for the last car and either escaped out, or let it time out.
I will keep working on testing the Recall.que by itself, however, I have had to begin running another experiment with the Recall.que in it, and have run some participants, and the problem happened for the very first participant. I have attached the file (Su2011 NC). ???

So I just ran through su 2011 nc. To try to replicate, I didn't skip anything. I answered some recall questions fully and escaped on others. Followed through to completion and got full data set. I have a suspicion that it has to do with the timed events in the second and third que files (e.g., Cars Configural 2 sec Order 1.que, conscious.que). If you run ONLY those two questionnaires and then your recall que file--can you then produce the missing data? If so, try adding a simply untimed instruction item to the end of each que file. Something like "Click continue to proceed". i.e., at the end of any que file that ends with a timed item.

Another option would be to try the latest release of MediaLab v2010 just in case that is part of the issue.

Here is an example .exp file--wondering if you can get the missing data with this..

I've tried putting the Cars Configural 2 sec Order 1.que, Conscious.que, and Recall.que in consecutive order (Su2011 NC Testing Recall Problem), and with Choice.que and NCdebriefing.que interspersed (su2011b), but was not able to replicate the problem (note: even tried with and without randomization in the recall.que. See Su2011 NC Testing...). Not sure what to do as I don't have a license for the 2010 version.

Hello,

I have found a solution to my data organization problem with the "recall listing". If I change the "recall listing" to a "thought listing" I do not have a problem with faulty data organization within Media Lab 2008.

Recall list data [U]and[/U] thought listing data is [U]still[/U] being misplaced, even when using the newest Media Lab version (ml2012.4.130).

The above problem relates to a previous post from 1-3-2011 (pasted below). I thought changing from Recall List to Thought Listing had solved the problem, but when I upgraded to ML 2012, the problem still exists even when using the thought listing. I am getting this same problem with other experiments that use thought listings. In all of them, participants are told that they can press the escape key when they have listed all they want. Could the the thought/recall list problem be due to pressing escape?



Hello,

I have created a questionnaire in which I use the "recall list" variable type, but there is a problem with data being misplaced in the CSV files. Details of questionnaire are as follows: participants are told to recall all that they remember of 4 types of cars. Each car name is presented to them, and they are asked to recall what they remember. And, they are told, "After you do not remember anymore attributes, you can press the "Esc" key in the upper left corner to move on to the next car. After 3 minutes, you will be automatically moved onto the next car whether you are finished or not." So, each car is presented in a random order using the randomize within feature set to 1 for each item, the n parameter is set to 12 (there were 12 attributes), and the duration parameter was set to 180 seconds. The write data and the RT options were checked for each item.

The "by questionnaire" CSV file is fine until the "Recall" questionnaire. At that point, responses start to be put into the questionnaires that followed the Recall questionnaire. This does not always occur, but it appears to do so, perhaps, after a participant has used the escape key, but I am unsure. Please see attached file for an example of this data misplacement. The problem begins on column DA.

Hi bstewart,
Could you upload a copy of the data file that shows how this displacement looks?

Jason,

I've uploaded two different excel files from experiments that use the Thought Listing. In both, the Thought Listing is selected and also record RT, but write as text is not selected. The file called Thought Listing data misplaced has the incorrect placement of the data; this experiment was originally created and run using ML 2008, but then halfway through we upgraded to ML 2012. The data was misplaced when running in both. The file with NC on the end was originally created in ML 2008, but altered and run using ML 2012; there is no data displacement for that entire experiment.

Where exactly in the file is the data displacement. The two files have different variables, so it's difficult to compare the two.

In the "Thought Listing data misplaced" file, the displacement occurs after two of the thougth listing free responses. After CBfreeresponse ... & CBchoose, you will see that the 0 and 1 choices spill into the timing variables (tCBprompt, tCBfreeresponse, etc). This also occurs in the timing variables after the MCfreeresponse.

For the Thought Listing data misplaced-NC file, there is no displacement after for the Recall questionnaire.

I was reviewing the files that I sent to you and noticed that for the "Thought Listing data misplaced" data set, participants are not told about the escape key, though they are told about it in the "thought listing data misplaced - NC", which contrary to its name did not have data misplaced when using the ML 2012. The main difference between these two experiments (the non-NC one is brand new) is the amount that participants write. In the NC one, Ps write only a few words per thought listing, whereas in the non-NC one, they write much more (often several sentences). Could this be the cause of the data misplacement? In re-examining the data, it appears that some sentences are divided into separate variables.

Have you looked at the original text files in a program like Notepad instead of opening them directly in a spreadsheet like Excel? I wonder if the participants used commas in the free response items. Opening such a comma delimited file in Excel could lead to the misplaced data you see in the Thought Listing data misplaced files.

I have not opened them systematically in Notepad. I looked briefly at some data using Notepad and it "might" solve the problem, but I'd rather not have to import data for each participant manually. In the past, I have tried to use the SPSS syntax file, but have not liked the fact that the Syntax truncates all variable names at 8 characters. Is there another way to systematically import the data without experiencing the comma problem from the free response items?

Do you mean import the data into SPSS?

Yes, importing into SPSS using the syntax file, but I am interested in importing it into either SPSS or Excel if it alleviates the comma problem from the free response items.

Just checking in to see if you got your data into SPSS alright.

I can get it into SPSS, but I still need to manually fix the extra columns created by the comma in the Recall Listing (& Thought Listing) questions.

When I attempt to use the syntax file provided by ML to import the TXT data I receive the following error message:

"Error # 4135 in column 74. Text: -12 >You must not specify the column location for variables read in FREE or LIST >format. >Execution of this command stops."

I've attached a word document with the Syntax file I was using when I got this error code. I still haven't found a systematic way to import data from a Thought Listing where a participant may use a comma.

This particular error message may be due to using a dash in your column names. The Word doc you attached to your last post does show a lot of use of the terms "choose1-12" and "choose13-19." See what happens when you get rid of the dash from those column headings.