Do give a penny for their thoughts
Turker experiment showing that there is substantial variability in inference judgments of the kind that RTE provided and asks us to train on. This suggests that a change in the RTE methodology would be welcome.
Zaenen, A. Do give a penny for their thoughts. To be published in Natural Language Engineering.