检查 Mechanical Turk 调查中的所有问题是否已得到解答
背景
我正在 Mechanical Turk 上设置数据注释任务。用户必须为句子对给出分数。每对句子都有一个带有 7 个单选按钮的表格。对于每个这样的表，用户必须给出答案。
问题
文档中有194个这样的表格，所以很容易错过一个。如果用户在点击提交时错过了一个或多个问题，则不会警告用户其中一些问题尚未得到解答，从而迫使我们因为简单的遗漏而拒绝他们的输入。
限制
不幸的是，我们无法选择将每个表作为单独的命中提交，因为：
a) 我们需要知道单个用户的完整数据集的分数是多少。
b) 我们有用户必须回答的质量控制问题，但他们无法得知这些问题是什么。
问题
是否有一种简单的方法可以确保用户回答每个问题：
a) 如果他们没有回答所有问题，则发出警告？如果是这样，这是如何完成的？
b) 在单独的页面上呈现每个表格，用户必须单击“下一步”才能获取下一个表格，在这种情况下，仍然有可能错过一个表格，但可能性较小（因为您不必向下滚动大页面） ？如果是这样，这是如何完成的？
预先感谢您提供解决此问题的任何帮助或解决方案。
Background
I am setting up a data annotation task on mechanical turk. The user has has to give scores for pairs of sentences. Each pair of sentences has a table with 7 radio buttons. For each such table, the user MUST give an answer.
Problem
There are 194 such tables in the document, so it's easy to miss one. The user is not warned that ones of the questions has not been answered if he/she missed one or more when they hit submit, forcing us to reject their input because of a simple omission.
Restrictions
Unfortunately, we do not have the option of submitting each table as a separate hit because:
a) We need to know what the scores are for the complete data set for a single user.
b) We have quality control questions which the user must answer, and they cannot be informed which ones these are.
Question
Is there a simple way to make sure that users answer each question either by:
a) Raising a warning if they have not answered all questions? If so, how is this done?
b) Presenting each table on a separate page with users having to click "next" to get the next table, in which case it is still possible to miss one but less likely (as you don't have to scroll down a large page)? If so, how is this done?
Thanks in advance for any help or solutions towards solving this problem.
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（1）
查看 mTurk API 文档，并具体查看 <一个href="http://docs.amazonwebservices.com/AWSMechTurk/2008-08-02/AWSMturkAPI/index.html?ApiReference_QuestionFormDataStructureArticle.html" rel="nofollow">QuestionFormStructure 部分。
如果您格式化您的问题并添加 IsRequired 标志，如果您没有明确设置它，则默认为 false。
但是，如果您在单个 HIT 中有 194 个这样的表，那么您最好将其作为外部问题来处理，这样您就可以添加一些 JavaScript 来帮助人们更快地导航到他们错过的部分。
Take a look at the mTurk API docs and specifically look at the QuestionFormStructure section.
If you format your Questions and add the IsRequired flag, which defaults to false if you don't explicitly set it.
However, if you have 194 of these tables in a single HIT, you may be better off doing it as an external question so you can add some javascript to help people navigate to the parts they missed more quickly.