Hm. To complete this task, your best bet may be to use DirectRT. DirectRT would allow for the equation to first be displayed. Just as you've noted, you can assign a key to bring up an answer box. If the user doesn't do so, the software can move to the next screen, which will display both the equation and the answer to the equation. From here, the user can press the 'response' key to bring up the pop-up textbox.

It is recommended that you go through each sample within the DirectRT samples folder to gain a better handle on how DirectRT works. Here's a few links that will provide you with functionality information that this experiment would most likely use:
Open Ended Responses - http://www.empirisoft.com/directrt/h..._responses.htm
Skipping - http://www.empirisoft.com/directrt/h...t_skipping.htm
Jumping - http://www.empirisoft.com/directrt/h...rt_jumping.htm

DirectRT Help Manual: http://www.empirisoft.com/directrt/help/index.html

Please let me know if you have any additional questions or if I can help in any way. Feel free to ask for help with coding any experiments you're working on as well.