Is the randomization of blocks truly random?

[benwood23]

have been having some problem with randomization with the Medialab software. I have repeatedly looked over my program, but I can't find any way I could have caused this problem. Of course, randomization problems happen randomly and it could be that, but randomization problems have occurred in two separate experiments, and they are very robust with a p-value of less than .01.

In our first experiment, I had trouble with between group randomization mechanism. THe program was supposed to randomization which of two groups of questions appeared first. One group had 3 question and another group had 2 questions. However 61 percent of the people received the group of 3 questions first and 41 percent received the group of 2 questions first. This difference is significant at p<.01. I have attached the relevant medialab questionnaire. It is called pilot combined. The relevant lines of code are 55 to 59.

In a second experiment, we also experienced randomization problems of a different sort. In that experiment instead of having the computer randomly choose people into conditions, I did it manually using a random number generator. However, I still experienced randomization problems. I had two conditions, in one condition, two scales were correlated at .51 and in another condition they were correlated at -.11, which is p<.001. The problem is that the scales were asked before the manipulation in the experiment. They should have answered the same exact questions up to this point in the experiment. In addition this relationship only exists for one group of people - those who were born in the US. This question was asked during the demographics questionnaire which was placed earlier in the experiment than the two question scales that had randomization problems. (This is an important variable in our theory so that is why this relationship was checked.)

I have attached the code for the complete experiment file and also the single questionnaire (judicial institution) with the two question batteries that had randomization problems. One question battery was composed of the questions from line 14,15,17,18,19, and the other from the question on line 16. I can think of no possible way a programming error on my part or a program bug could conceivably create this relationship, and I have tried everything I could think of to why this relationship would occur. But since it is a 1 in a 1000 chance of it occurring randomly, it is hard to accept that it happened randomly.

I appreciate any help you can give me, and I hope I have been making a simple mistake of some kind and am not just the most unlucky man in the world.