Guidelines
The purpose of this experiment is to investigate how irony is expressed and recognized in online interactions.
In particular, we focus on short social media messages, where irony is often conveyed through contrast.
Irony is a complex linguistic phenomenon that often involves a contrast between what is explicitly stated and what is actually meant.
You will be presented with a series of short exchanges, each consisting of a post and two different corresponding replies. Each reply has been automatically generated by a different language model. For each question, the two replies are presented in random order.
Your task is to identify which of the two replies is ironic, or more ironic, according to your judgment. You may also select “Both are equally ironic” if you think the two replies are equally ironic, or “Neither” if you think neither reply is ironic.
In some questions, you will also be asked to identify which of the two replies better realizes a specific rhetorical figure. A rhetorical figure is a linguistic device that departs from ordinary language in order to create greater expressiveness, emphasis, or a particular meaning effect. The two rhetorical figures you will see are ANALOGY and RHETORICAL QUESTION.
ANALOGY
Irony is triggered by a literal or metaphorical comparison between two elements, or by a comparison between an element and an external context.
Example:
Post: He killed those people. You should ask for her resignation rather. She was not saying a word.
Reply: It’s like wanting a bicycle but asking for a car.
RHETORICAL QUESTION
A question is asked in order to make a point rather than to elicit an answer. The question may involve one or more elements of the exchange.
Example:
Post: It’s absolutely freezing outside.
Reply: Where is global warming when it is needed?
There are no right or wrong answers.
We are interested in your linguistic judgment and interpretation of ironic replies. This study aims to improve our understanding of the linguistic mechanisms underlying irony and to support the development of language models for research in natural language processing and computational linguistics.
Please note that the survey includes attention-check questions designed to ensure data quality. In these questions, you will be asked to select a specific answer as instructed. If these questions are not answered correctly, your response will not be considered valid and compensation will not be provided. Your responses will remain completely anonymous, and the data will be used exclusively for research purposes.