Computational Pun-derstanding

Liana Ermakova, Anne Gwenn-Bosser, Adam Jatowt, and Tristan Miller.
The JOKER Corpus: English–French parallel data for multilingual wordplay recognition.
In SIGIR '23: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, New York, NY, 2023. Association for Computing Machinery. ISBN 978-1-4503-9408-6. DOI: 10.1145/3539618.3591885. To appear.

Despite recent advances in information retrieval and natural language processing, rhetorical devices that exploit ambiguity or subvert linguistic rules remain a challenge for such systems. However, corpus-based analysis of wordplay has been a perennial topic of scholarship in the humanities, including literary criticism, language education, and translation studies. The immense data-gathering effort required for these studies points to the need for specialized text retrieval and classification technology, and consequently for appropriate test collections. In this paper, we introduce and analyze a new dataset for research and applications in the retrieval and processing of wordplay. Developed for the JOKER track at CLEF 2023, our annotated corpus extends and improves upon past English wordplay detection datasets in several ways. First, we introduce hundreds of additional positive examples; second, we provide French translations for the examples; and third, we provide negative examples with characteristics closely matching those of the positive examples. This last feature helps ensure that AI models learn to effectively distinguish wordplay from non-wordplay, and not simply texts differing in length, style, or vocabulary. Our test collection represents then a step towards wordplay-aware multilingual information retrieval.

@inproceedings{ermakova2023joker,

author = {Liana Ermakova and Anne Gwenn-Bosser and Adam Jatowt and Tristan Miller},

title = {The {JOKER} {Corpus}: {English}--{French} Parallel Data for Multilingual Wordplay Recognition},

booktitle = {{SIGIR} '23: Proceedings of the 46th {International} {ACM} {SIGIR} {Conference} on {Research} and {Development} in {Information} {Retrieval}},

year = {2023},

publisher = {Association for Computing Machinery},

address = {New York, NY},

isbn = {978-1-4503-9408-6},

doi = {10.1145/3539618.3591885},

note = {To appear},

}

Citation
Abstract
BibTeX

Waltraud Kolb and Tristan Miller.
La interacción entre el hombre y la máquina en la traducción de juegos de palabras [Human–computer interaction in pun translation].
In Laura Mejías-Climent and Julio de los Reyes Lozano, editors, La traducción audiovisual a través de la traducción automática y la posedición: prácticas actuales y futuras. Comares, Granada, 2023. Translated by Lorena Pérez Macías. To appear.

@incollection{kolb2023interaccion,

author = {Waltraud Kolb and Tristan Miller},

editor = {Laura Mejías-Climent and de los Reyes Lozano, Julio},

title = {La interacción entre el hombre y la máquina en la traducción de juegos de palabras [{Human}--Computer Interaction in Pun Translation]},

booktitle = {La traducción audiovisual a través de la traducción automática y la posedición: prácticas actuales y futuras},

year = {2023},

publisher = {Comares},

address = {Granada},

note = {Translated by Lorena Pérez Macías. To appear.},

}

Citation
Abstract
BibTeX

Liana Ermakova, Tristan Miller, Anne-Gwenn Bosser, Victor Manuel Palma Preciado, Grigori Sidorov, and Adam Jatowt.
Science for fun: The CLEF 2023 JOKER track on automatic wordplay analysis.
In Jaap Kamps, Lorraine Goeuriot, Fabio Crestani, Maria Maistro, Hideo Joho, Brian Davis, Cathal Gurrin, Udo Kruschwitz, and Annalina Caputo, editors, Advances in Information Retrieval: 45th European Conference on Information Retrieval, ECIR 2023, Dublin, Ireland, April 2–6, Proceedings, Part III, volume 13982 of Lecture Notes in Computer Science (ISSN 0302-9743), pages 546–556, Berlin, Heidelberg, April 2023. Springer. ISBN 978-3-031-28241-6. DOI: 10.1007/978-3-031-28241-6_63.

Understanding and translating humorous wordplay often requires recognition of implicit cultural references, knowledge of word formation processes, and discernment of double meanings – issues which pose challenges for humans and computers alike. This paper introduces the CLEF 2023 JOKER track, which takes an interdisciplinary approach to the creation of reusable test collections, evaluation metrics, and methods for the automatic processing of wordplay. We describe the track's interconnected shared tasks for the detection, location, interpretation, and translation of puns. We also describe associated data sets and evaluation methodologies, and invite contributions making further use of our data.

@inproceedings{ermakova2023science,

author = {Liana Ermakova and Tristan Miller and Anne-Gwenn Bosser and Victor Manuel Palma Preciado and Grigori Sidorov and Adam Jatowt},

editor = {Jaap Kamps and Lorraine Goeuriot and Fabio Crestani and Maria Maistro and Hideo Joho and Brian Davis and Cathal Gurrin and Udo Kruschwitz and Annalina Caputo},

title = {Science for Fun: The {CLEF} 2023 {JOKER} Track on Automatic Wordplay Analysis},

booktitle = {Advances in Information Retrieval: 45th {European} {Conference} on {Information} {Retrieval}, {ECIR} 2023, {Dublin}, {Ireland}, {April} 2–6, Proceedings, Part~{III}},

volume = {13982},

pages = {546--556},

series = {Lecture Notes in Computer Science},

month = apr,

year = {2023},

publisher = {Springer},

address = {Berlin, Heidelberg},

isbn = {978-3-031-28241-6},

issn = {0302-9743},

doi = {10.1007/978-3-031-28241-6_63},

}

Citation
Abstract
BibTeX

HTML
PDF

Tiansi Dong, Anthony Cohn, Christian Hempelmann, Kanishka Misra, Jens Lehmann, Alexander Mehler, Tristan Miller, Siba Mohsen, Roberto Navigli, Julia Rayz, Stefan Wrobel, Ron Sun, and Volker Tresp.
Towards a survey of meaning representation.
Dagstuhl Reports, 11(8):29, 2022. ISSN 2192-5283.

After the working group on “What is missing in ML&AI to understanding Jokes?”, we discussed the possibility to survey the expressiveness on existing models on meaning representation, contrasted by the forecast of existing theories in cognitive science about what is relevant cognitive activities and processes. Spatial stimuli activate the zoo of spatial cells in hippocampus, forming cognitive map or collage in the memory, producing spatial descriptions in languages. We need to survey existing models on Mental Spatial Representation (MSR) in the literature of cognitive psychology. On the other hand, we need to analyse vector embeddings of spatial entities and relations in the large-scaled pre-train world model, and find the gap between MSR and vector embedding via Machine Learning.

@article{dong2022towards,

author = {Tiansi Dong and Anthony Cohn and Christian Hempelmann and Kanishka Misra and Jens Lehmann and Alexander Mehler and Tristan Miller and Siba Mohsen and Roberto Navigli and Julia Rayz and Stefan Wrobel and Ron Sun and Volker Tresp},

title = {Towards a Survey of Meaning Representation},

journal = {Dagstuhl Reports},

volume = {11},

number = {8},

pages = {29},

year = {2022},

issn = {2192-5283},

}

Citation
Abstract
BibTeX

PDF

Liana Ermakova, Tristan Miller, Fabio Regattin, Anne-Gwenn Bosser, Claudine Borg, Élise Mathurin, Gaëlle Le Corre, Sílvia Araújo, Radia Hannachi, Julien Boccou, Albin Digue, Aurianne Damoy, and Benoît Jeanjean.
Overview of JOKER@CLEF 2022: Automatic wordplay and humour translation workshop.
In Alberto Barrón-Cedeño, Giovanni Da San Martino, Mirko Degli Esposti, Fabrizio Sebastiani, Craig Macdonald, Gabriella Pasi, Allan Hanbury, Martin Potthast, Guglielmo Faggioli, and Nicola Ferro, editors, Experimental IR Meets Multilinguality, Multimodality, and Interaction: Proceedings of the Thirteenth International Conference of the CLEF Association (CLEF 2022), volume 13390 of Lecture Notes in Computer Science (ISSN 0302-9743), pages 447–469, Cham, 2022. Springer. ISBN 978-3-031-13642-9. DOI: 10.1007/978-3-031-13643-6_27.

While humour and wordplay are among the most intensively studied problems in the field of translation studies, they have been almost completely ignored in machine translation. This is partly because most AI-based translation tools require a quality and quantity of training data (e.g., parallel corpora) that has historically been lacking for humour and wordplay. The goal of the JOKER@CLEF 2022 workshop was to bring together translators and computer scientists to work on an evaluation framework for wordplay, including data and metric development, and to foster work on automatic methods for wordplay translation. To this end, we defined three pilot tasks: (1) classify and explain instances of wordplay, (2) translate single terms containing wordplay, and (3) translate entire phrases containing wordplay (punning jokes). This paper describes and discusses each of these pilot tasks, as well as the participating systems and their results.

@inproceedings{ermakova2022overview,

author = {Liana Ermakova and Tristan Miller and Fabio Regattin and Anne-Gwenn Bosser and Claudine Borg and Élise Mathurin and Gaëlle Le Corre and Sílvia Araújo and Radia Hannachi and Julien Boccou and Albin Digue and Aurianne Damoy and Benoît Jeanjean},

editor = {Alberto Barrón-Cedeño and Giovanni Da San Martino and Mirko Degli Esposti and Fabrizio Sebastiani and Craig Macdonald and Gabriella Pasi and Allan Hanbury and Martin Potthast and Guglielmo Faggioli and Nicola Ferro},

title = {Overview of {JOKER@CLEF} 2022: Automatic Wordplay and Humour Translation Workshop},

booktitle = {Experimental {IR} Meets Multilinguality, Multimodality, and Interaction: Proceedings of the {Thirteenth} {International} {Conference} of the {CLEF} {Association} ({CLEF} 2022)},

volume = {13390},

pages = {447--469},

series = {Lecture Notes in Computer Science},

year = {2022},

publisher = {Springer},

address = {Cham},

isbn = {978-3-031-13642-9},

issn = {0302-9743},

doi = {10.1007/978-3-031-13643-6_27},

}

Citation
Abstract
BibTeX

HTML
PDF

Waltraud Kolb and Tristan Miller.
Human–computer interaction in pun translation.
In James Luke Hadley, Kristiina Taivalkoski-Shilov, Carlos S. C. Teixeira, and Antonio Toral, editors, Using Technologies for Creative-Text Translation, pages 66–88. Routledge, 2022. ISBN 9781003094159. DOI: 10.4324/9781003094159-4.

We present and evaluate PunCAT, an interactive electronic tool for the translation of puns. Following the strategies known to be applied in pun translation, PunCAT automatically translates each sense of the pun separately; it then allows the user to explore the semantic fields of these translations in order to help construct a plausible target-language solution that maximizes the semantic correspondence to the original. Our evaluation is based on an empirical pilot study in which the participants translated puns from a variety of published sources from English into German, with and without PunCAT. We aimed to answer the following questions: Does the tool support, improve, or constrain the translation process, and if so, in what ways? And what are the tool's main benefits and drawbacks as perceived and described by the participants? Our analysis of the translators' cognitive processes gives us insight into their decision-making strategies and how they interacted with the tool. We find clear evidence that PunCAT effectively supports the translation process in terms of stimulating brainstorming and broadening the translator's pool of solution candidates. We have also identified a number of directions in which the tool could be adapted to better suit translators' work processes.

@incollection{kolb2022human,

author = {Waltraud Kolb and Tristan Miller},

editor = {James Luke Hadley and Kristiina Taivalkoski-Shilov and Carlos S. C. Teixeira and Antonio Toral},

title = {Human--Computer Interaction in Pun Translation},

booktitle = {Using Technologies for Creative-Text Translation},

pages = {66--88},

year = {2022},

publisher = {Routledge},

isbn = {9781003094159},

doi = {10.4324/9781003094159-4},

}

Citation
Abstract
BibTeX

PDF
Code

Alexander Mehler, Tiansi Dong, Thomas Liebig, Tristan Miller, Siba Mohsen, and Sven Naumann.
What is missing in ML&AI to understand jokes?.
Dagstuhl Reports, 11(8):32, 2022. ISSN 2192-5283.

Why current Machine Learning and AI (ML&AI) techniques cannot understand jokes as we humans do? What is missing? The knowledge that is needed to understand jokes is neither in the joke texts, nor in the neural networks. Acquisition and reasoning with commonsense knowledge is still an open problem for Machine Learning and AI. The meaning representation based on embeddings is insufficient. We need meaning representation formats that are beyond vector representations. Vectors are only shadows. Information processing and meaning understanding are embodied. The discussion guides us to develop novel embodied ML&AI techniques to understand \emphSpatial Jokes first.

@article{mehler2022what,

author = {Alexander Mehler and Tiansi Dong and Thomas Liebig and Tristan Miller and Siba Mohsen and Sven Naumann},

title = {What Is Missing in {ML}\&{AI} to Understand Jokes?},

journal = {Dagstuhl Reports},

volume = {11},

number = {8},

pages = {32},

year = {2022},

issn = {2192-5283},

}

Citation
Abstract
BibTeX

PDF

Tristan Miller, Anthony Cohn, Tiansi Dong, Christian Hempelmann, Siba Mohsen, and Julia Rayz.
Can we diagram the understanding of humour?.
Dagstuhl Reports, 11(8):33, 2022. ISSN 2192-5283.

Cartoons can be understood without language. That is, a suitably arranged scene of simple objects, with no accompanying text, is often enough to make us laugh – evidence that thinking (mental activity) happens before language. This raises the question of non-linguistic diagrammatic representation of spatial humour, along with the mechanism of neural computation. In particular, we raise following questions: (1) How can we diagrammatically formalise spatial humour? (2) How can these diagrammatic formalisms be processed by neural networks? (3) How can this neural computation deliver high-level schema that are similar to the script-opposition semantic theory of humour? The spatial knowledge encoded in the scene can activate the necessary spatial and non- spatial knowledge. By what neural associative mechanism or process of reasoning do we put this all together to “get” the joke? During the seminar, we aimed to make some headway towards establishing (1) exactly what sort of scene-specific and common-sense knowledge is required to understand any given cartoon, (2) what part of this knowledge could in principle be acquired by existing machine learning (ML) techniques, and which could be acquired or encoded through symbolic structures, (3) what activation process acquires the rest of the knowledge required to interpret the humour, and (4) whether there is a unified representation that could represent this knowledge in a computer’s working memory.

@article{miller2022can,

author = {Tristan Miller and Anthony Cohn and Tiansi Dong and Christian Hempelmann and Siba Mohsen and Julia Rayz},

title = {Can We Diagram the Understanding of Humour?},

journal = {Dagstuhl Reports},

volume = {11},

number = {8},

pages = {33},

year = {2022},

issn = {2192-5283},

}

Citation
Abstract
BibTeX

PDF

Roberto Navigli, Tiansi Dong, Thomas Liebig, Yong Liu, Alexander Mehler, Tristan Miller, Siba Mohsen, and Sven Naumann.
Rotating sphere model for NLP.
Dagstuhl Reports, 11(8):29–30, 2022. ISSN 2192-5283.

The existing n-ball embedding approach can precisely encode a large symbolic tree structure into tree node embeddings. In this working group, we discussed how to apply the idea of n-ball to solve NLP tasks, in particular, the Word Sense Disambiguation (WSD). WSD is a fundamental task in Natural Language Processing (NLP), which impacts a variety of downstream NLP applications. WSD determines the intended meaning of words in a context. To tackle the WSD task, researchers have been investigating knowledge-based approaches, supervised, semi-supervised, and unsupervised machine learning. However, those methods encounter a number of limitations, besides their costly computation. We let n-ball rotate, and result in the Rotating Spheres Model (RSM). Using RSM, embeddings of word senses work like gestures of a word. Given a context, the word chooses the best gesture. The WSD is to determine the best rotating axis in a given context. Each rating axis represents a sense that in the predefined sense inventory.

@article{navigli2022rotating,

author = {Roberto Navigli and Tiansi Dong and Thomas Liebig and Yong Liu and Alexander Mehler and Tristan Miller and Siba Mohsen and Sven Naumann},

title = {Rotating Sphere Model for {NLP}},

journal = {Dagstuhl Reports},

volume = {11},

number = {8},

pages = {29--30},

year = {2022},

issn = {2192-5283},

}

Citation
Abstract
BibTeX

PDF

Liana Ermakova, Tristan Miller, Julien Boccou, Albin Digue, Aurianne Damoy, and Paul Campen.
Overview of the CLEF 2022 JOKER Task 2: Translate wordplay in named entities.
In Guglielmo Faggioli, Nicola Ferro, Allan Hanbury, and Martin Potthast, editors, Proceedings of the Working Notes of CLEF 2022 – Conference and Labs of the Evaluation Forum, Bologna, Italy, September 5th to 8th, 2022, volume 3180 of CEUR Workshop Proceedings (ISSN 1613-0073), pages 1666–1680, August 2022.

Onomastic wordplay has been widely used as a rhetorical device by novelists, poets, and playwrights, from character names in Shakespeare and other classic literature to named entities in Pokémon, Harry Potter, Asterix, and video games. The translation of such wordplay is problematic both for humans and algorithms due to its ambiguity and unorthodox morphology. In this paper, we present an overview of Pilot Task 2 of the JOKER@CLEF 2022 track, where participants had to translate wordplay in named entities from English into French. For this, we constructed a parallel corpus wordplay in named entities from movies, video games, advertising slogans, literature, etc. Five teams participated in the task. The methods employed by participants were based on the state-of-the-art transformer models, which have the advantage of subword tokenisation. The participants' models were pre-trained on large corpora and fine-tuned on the JOKER training set. We observed that in many cases the models provided the exact official translations, suggesting that they were pre-trained on the corpus containing the source texts used in the JOKER corpus. Those translations that differed from the official ones only rarely contained wordplay.

@inproceedings{ermakova2022overviewtask2,

author = {Liana Ermakova and Tristan Miller and Julien Boccou and Albin Digue and Aurianne Damoy and Paul Campen},

editor = {Guglielmo Faggioli and Nicola Ferro and Allan Hanbury and Martin Potthast},

title = {Overview of the {CLEF}~2022 {JOKER} {Task}~2: Translate Wordplay in Named Entities},

booktitle = {Proceedings of the {Working} {Notes} of {CLEF}~2022~-- {Conference} and {Labs} of the {Evaluation} {Forum}, {Bologna}, {Italy}, {September} 5th to 8th, 2022},

volume = {3180},

pages = {1666--1680},

series = {CEUR Workshop Proceedings},

month = aug,

year = {2022},

issn = {1613-0073},

}

Citation
Abstract
BibTeX

PDF

Liana Ermakova, Fabio Regattin, Tristan Miller, Anne-Gwenn Bosser, Sílvia Araújo, Claudine Borg, Gaëlle Le Corre, Julien Boccou, Albin Digue, Aurianne Damoy, Paul Campen, and Orlane Puchalski.
Overview of the CLEF 2022 JOKER Task 1: Classify and explain instances of wordplay.
In Guglielmo Faggioli, Nicola Ferro, Allan Hanbury, and Martin Potthast, editors, Proceedings of the Working Notes of CLEF 2022 – Conference and Labs of the Evaluation Forum, Bologna, Italy, September 5th to 8th, 2022, volume 3180 of CEUR Workshop Proceedings (ISSN 1613-0073), pages 1641–1665, August 2022.

As a multidisciplinary field of study, humour remains one of the most difficult aspects of intercultural communication. Understanding humour often involves understanding implicit cultural references and/or double meanings, which raises the questions of how to detect and classify instances of this complex phenomenon. This paper provides an overview of Pilot Task 1 of the CLEF 2022 JOKER track, where participants had to classify and explain instances of wordplay. We introduce a new classification of wordplay and a new annotation scheme for wordplay interpretation suitable both for phrase-based wordplay and wordplay in named entities. We describe the collection of our data, our task setup, and the evaluation procedure, and we give a brief overview of the participating teams' approaches and results.

@inproceedings{ermakova2022overviewtask1,

author = {Liana Ermakova and Fabio Regattin and Tristan Miller and Anne-Gwenn Bosser and Sílvia Araújo and Claudine Borg and Gaëlle Le Corre and Julien Boccou and Albin Digue and Aurianne Damoy and Paul Campen and Orlane Puchalski},

editor = {Guglielmo Faggioli and Nicola Ferro and Allan Hanbury and Martin Potthast},

title = {Overview of the {CLEF}~2022 {JOKER} {Task}~1: Classify and Explain Instances of Wordplay},

booktitle = {Proceedings of the {Working} {Notes} of {CLEF}~2022~-- {Conference} and {Labs} of the {Evaluation} {Forum}, {Bologna}, {Italy}, {September} 5th to 8th, 2022},

volume = {3180},

pages = {1641--1665},

series = {CEUR Workshop Proceedings},

month = aug,

year = {2022},

issn = {1613-0073},

}

Citation
Abstract
BibTeX

PDF

Liana Ermakova, Fabio Regattin, Tristan Miller, Anne-Gwenn Bosser, Claudine Borg, Benoît Jeanjean, Élise Mathurin, Gaëlle Le Corre, Radia Hannachi, Sílvia Araújo, Julien Boccou, Albin Digue, and Aurianne Damoy.
Overview of the CLEF 2022 JOKER Task 3: Pun translation from English into French.
In Guglielmo Faggioli, Nicola Ferro, Allan Hanbury, and Martin Potthast, editors, Proceedings of the Working Notes of CLEF 2022 – Conference and Labs of the Evaluation Forum, Bologna, Italy, September 5th to 8th, 2022, volume 3180 of CEUR Workshop Proceedings (ISSN 1613-0073), pages 1681–1700, August 2022.

The translation of the pun is one of the most challenging issues for translators and for this reason has become an intensively studied phenomenon in the field of translation studies. Translation technology aims to partially or even totally automate the translation process, but relatively little attention has been paid to the use of computers for the translation of wordplay. The CLEF 2022 JOKER track aims to build a multilingual corpus of wordplay and evaluation metrics in order to advance the automation of creative-language translation. This paper provides an overview of the track's Pilot Task 3, where the goal is to translate entire phrases containing wordplay (particularly puns). We describe the data collection, the task setup, the evaluation procedure, and the participants' results. We also cover a side product of our project, a homogeneous monolingual corpus for wordplay detection in French.

@inproceedings{ermakova2022overviewtask3,

author = {Liana Ermakova and Fabio Regattin and Tristan Miller and Anne-Gwenn Bosser and Claudine Borg and Benoît Jeanjean and Élise Mathurin and Gaëlle Le Corre and Radia Hannachi and Sílvia Araújo and Julien Boccou and Albin Digue and Aurianne Damoy},

editor = {Guglielmo Faggioli and Nicola Ferro and Allan Hanbury and Martin Potthast},

title = {Overview of the {CLEF}~2022 {JOKER} {Task}~3: Pun Translation from {English} into {French}},

booktitle = {Proceedings of the {Working} {Notes} of {CLEF}~2022~-- {Conference} and {Labs} of the {Evaluation} {Forum}, {Bologna}, {Italy}, {September} 5th to 8th, 2022},

volume = {3180},

pages = {1681--1700},

series = {CEUR Workshop Proceedings},

month = aug,

year = {2022},

issn = {1613-0073},

}

Citation
Abstract
BibTeX

PDF

Liana Ermakova, Tristan Miller, Orlane Puchalski, Fabio Regattin, Élise Mathurin, Sílvia Araújo, Anne-Gwenn Bosser, Claudine Borg, Monika Bokiniec, Gaelle Le Corre, Benoît Jeanjean, Radia Hannachi, Ġorġ Mallia, Gordan Matas, and Mohamed Saki.
CLEF Workshop JOKER: Automatic wordplay and humour translation.
In Matthias Hagen, Suzan Verberne, Craig Macdonald, Christin Seifert, Krisztian Balog, Kjetil Nørvåg, and Vinay Setty, editors, Advances in Information Retrieval: 44th European Conference on IR Research, ECIR 2022, Stavanger, Norway, April 10–14, 2022, Proceedings, Part II, Lecture Notes in Computer Science, pages 355–363, Berlin, Heidelberg, April 2022. Springer. ISBN 978-3-030-99738-0. DOI: 10.1007/978-3-030-99739-7_45.

Humour remains one of the most difficult aspects of intercultural communication: understanding humour often requires understanding implicit cultural references and/ or double meanings, and this raises the question of the (un)translatability of humour. Wordplay is a common source of humour in literature, journalism, and advertising due to its attention-getting, mnemonic, playful, and subversive character. The translation of humour and wordplay is therefore in high demand. Modern translation depends heavily on technological aids, yet few works have treated the automation of humour and wordplay translation and the creation of humour corpora. The goal of the JOKER workshop is to bring together translators and computer scientists to work on an evaluation framework for creative language, including data and metric development, and to foster work on automatic methods for wordplay translation. We propose three pilot tasks: (1) classify and explain instances of wordplay, (2) translate single words containing wordplay, and (3) translate entire phrases containing wordplay.

@inproceedings{ermakova2022clef,

author = {Liana Ermakova and Tristan Miller and Orlane Puchalski and Fabio Regattin and Élise Mathurin and Sílvia Araújo and Anne-Gwenn Bosser and Claudine Borg and Monika Bokiniec and Gaelle Le Corre and Benoît Jeanjean and Radia Hannachi and Ġorġ Mallia and Gordan Matas and Mohamed Saki},

editor = {Matthias Hagen and Suzan Verberne and Craig Macdonald and Christin Seifert and Krisztian Balog and Kjetil Nørvåg and Vinay Setty},

title = {{CLEF} {Workshop} {JOKER}: Automatic Wordplay and Humour Translation},

booktitle = {Advances in Information Retrieval: 44th European Conference on IR Research, ECIR 2022, Stavanger, Norway, April 10–14, 2022, Proceedings, Part II},

pages = {355--363},

series = {Lecture Notes in Computer Science},

month = apr,

year = {2022},

publisher = {Springer},

address = {Berlin, Heidelberg},

isbn = {978-3-030-99738-0},

issn = {0302-9743},

doi = {10.1007/978-3-030-99739-7_45},

}

Citation
Abstract
BibTeX

HTML
PDF

Jörg Wöckener, Thomas Haider, Tristan Miller, The-Khang Nguyen, Thanh Tung Linh Nguyen, Minh Vu Pham, Jonas Belouadi, and Steffen Eger.
End-to-end style-conditioned poetry generation: What does it take to learn from examples alone?.
In Proceedings of the 5th Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature (LaTeCH-CLfL 2021), pages 57–66, November 2021.

In this work, we design an end-to-end model for poetry generation based on conditioned recurrent neural network (RNN) language models whose goal is to learn stylistic features (poem length, sentiment, alliteration, and rhyming) from examples alone. We show this model successfully learns the ‘meaning' of length and sentiment, as we can control it to generate longer or shorter as well as more positive or more negative poems. However, the model does not grasp sound phenomena like alliteration and rhyming, but instead exploits low-level statistical cues. Possible reasons include the size of the training data, the relatively low frequency and difficulty of these sublexical phenomena as well as model biases. We show that more recent GPT-2 models also have problems learning sublexical phenomena such as rhyming from examples alone.

@inproceedings{woeckener2021end,

author = {J{\"{o}}rg W{\"{o}}ckener and Thomas Haider and Tristan Miller and The-Khang Nguyen and Thanh Tung Linh Nguyen and Minh Vu Pham and Jonas Belouadi and Steffen Eger},

title = {End-to-end Style-Conditioned Poetry Generation: {What} Does It Take to Learn from Examples Alone?},

booktitle = {Proceedings of the 5th Joint SIGHUM Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature (LaTeCH-CLfL 2021)},

pages = {57--66},

month = nov,

year = {2021},

}

Citation
Abstract
BibTeX

PDF

Alexandra Uma, Tommaso Fornaciari, Anca Dumitrache, Tristan Miller, Jon Chamberlain, Barbara Plank, Edwin Simpson, and Massimo Poesio.
SemEval-2021 Task 12: Learning with disagreements.
In Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 338–347, August 2021. ISBN 978-1-954085-70-1. DOI: 10.18653/v1/2021.semeval-1.41.

Disagreement between coders is ubiquitous in virtually all datasets annotated with human judgements in both natural language processing and computer vision. However, most supervised machine learning methods assume that a single preferred interpretation exists for each item, which is at best an idealization. The aim of the SemEval-2021 shared task on Learning with Disagreements (Le-wi-Di) was to provide a unified testing framework for methods for learning from data containing multiple and possibly contradictory annotations covering the best-known datasets containing information about disagreements for interpreting language and classifying images. In this paper we describe the shared task and its results.

@inproceedings{uma2021semeval,

author = {Alexandra Uma and Tommaso Fornaciari and Anca Dumitrache and Tristan Miller and Jon Chamberlain and Barbara Plank and Edwin Simpson and Massimo Poesio},

title = {{SemEval}-2021 {Task}~12: Learning with Disagreements},

booktitle = {Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021)},

pages = {338--347},

month = aug,

year = {2021},

isbn = {978-1-954085-70-1},

doi = {10.18653/v1/2021.semeval-1.41},

}

Citation
Abstract
BibTeX

PDF
Data

Tristan Miller.
Dmitri Borgmann's rotas square articles.
Notes and Queries, 67(3):431–432, September 2020. ISSN 0029-3970. DOI: 10.1093/notesj/gjaa113.

In 1979 and 1980, Word Ways: The Journal of Recreational Linguistics printed a series of articles on the early history, religious symbolism, and cultural significance of the rotas square, an ancient Latin-language palindromic word square. The articles were attributed to Dmitri A. Borgmann, the noted American writer on wordplay and former editor of Word Ways. While they attracted little attention at the time, some 35 years after their publication (and 29 years after Borgmann's death), questions began to be raised about their authorship. There is much internal and external evidence that, taken together, compellingly supports the notion that Borgmann did not write the articles himself. This paper surveys this evidence and solicits help in identifying the articles' original source.

@article{miller2020dmitri,

author = {Tristan Miller},

title = {{Dmitri Borgmann's} Rotas Square Articles},

journal = {Notes and Queries},

volume = {67},

number = {3},

pages = {431--432},

month = sep,

year = {2020},

issn = {0029-3970},

doi = {10.1093/notesj/gjaa113},

}

Citation
Abstract
BibTeX

PDF

Tristan Miller and Denis Auroux.
GPP, the generic preprocessor.
Journal of Open Source Software, 5(51), July 2020. ISSN 2475-9066. DOI: 10.21105/joss.02400.

In computer science, a preprocessor (or macro processor) is a tool that programatically alters its input, typically on the basis of inline annotations, to produce data that serves as input for another program. Preprocessors are used in software development and document processing workflows to translate or extend programming or markup languages, as well as for conditional or pattern-based generation of source code and text. Early preprocessors were relatively simple string replacement tools that were tied to specific programming languages and application domains, and while these have since given rise to more powerful, general-purpose tools, these often require the user to learn and use complex macro languages with their own syntactic conventions. In this paper, we present GPP, an extensible, general-purpose preprocessor whose principal advantage is that its syntax and behaviour can be customized to suit any given preprocessing task. This makes GPP of particular benefit to research applications, where it can be easily adapted for use with novel markup, programming, and control languages.

@article{miller2020gpp,

author = {Tristan Miller and Denis Auroux},

title = {{GPP}, the Generic Preprocessor},

journal = {Journal of Open Source Software},

volume = {5},

number = {51},

month = jul,

year = {2020},

issn = {2475-9066},

doi = {10.21105/joss.02400},

}

Citation
Abstract
BibTeX

PDF
Code

Tristan Miller.
Don't shun the pun: On the requirements and constraints for preserving ambiguity in the (machine) translation of humour.
In Mehrdad Sabetzadeh, Andreas Vogelsang, Sallam Abualhaija, Markus Borg, Fabiano Dalpiaz, Maya Daneva, Nelly C. Fernández, Xavier Franch, Davide Fucci, Vincenzo Gervasi, Eduard Groen, Renata Guizzardi, Andrea Herrmann, Jennifer Horkoff, Luisa Mich, Anna Perini, and Angelo Susi, editors, Joint Proceedings of REFSQ-2020 Workshops, Doctoral Symposium, Live Studies Track, and Poster Track co-located with the 26th International Conference on Requirements Engineering: Foundation for Software Quality (REFSQ 2020), volume 2584 of CEUR Workshop Proceedings (ISSN 1613-0073), March 2020.

How do we know when a translation is good? This seemingly simple question has long dogged human practitioners of translation, and has arguably taken on even greater importance in today’s world of fully automatic, end-to-end machine translation systems. Much of the difficulty in assessing translation quality is that different translations of the same text may be made for different purposes, each of which entails a unique set of requirements and constraints. This difficulty is compounded by ambiguities in the source text, which must be identified and then preserved or eliminated according to the needs of the translation and the (apparent) intent of the source text. In this talk, I survey the state of the art in linguistics, computational linguistics, translation, and machine translation as it relates to the notion of linguistic ambiguity in general, and intentional humorous ambiguity in particular. I describe the various constraints and requirements of different types of translations and provide examples of how various automatic and interactive techniques from natural language processing can be used to detect and then resolve or preserve linguistic ambiguities according to these constraints and requirements. In the vein of the “Translator’s Amanuensis” proposed by Martin Kay, I outline some specific proposals concerning how the hitherto disparate work in the aforementioned fields can be connected with a view to producing “machine-in-the-loop” computer-assisted translation (CAT) tools to assist human translators in selecting and implementing pun translation strategies in furtherance of the translation requirements. Throughout the talk, I will attempt to draw links with how this research relates to the requirements engineering community.

@inproceedings{miller2020dont,

author = {Tristan Miller},

editor = {Mehrdad Sabetzadeh and Andreas Vogelsang and Sallam Abualhaija and Markus Borg and Fabiano Dalpiaz and Maya Daneva and Nelly C. Fernández and Xavier Franch and Davide Fucci and Vincenzo Gervasi and Eduard Groen and Renata Guizzardi and Andrea Herrmann and Jennifer Horkoff and Luisa Mich and Anna Perini and Angelo Susi},

title = {Don't Shun the Pun: {On} the Requirements and Constraints for Preserving Ambiguity in the (Machine) Translation of Humour},

booktitle = {Joint Proceedings of REFSQ-2020 Workshops, Doctoral Symposium, Live Studies Track, and Poster Track co-located with the 26th International Conference on Requirements Engineering: Foundation for Software Quality (REFSQ 2020)},

volume = {2584},

series = {CEUR Workshop Proceedings},

month = mar,

year = {2020},

issn = {1613-0073},

}

Citation
Abstract
BibTeX

PDF

Tristan Miller, Erik-Lân Do Dinh, Edwin Simpson, and Iryna Gurevych.
Predicting the humorousness of tweets using Gaussian process preference learning.
Procesamiento del Lenguaje Natural, 64:37–44, March 2020. ISSN 1135-5948. DOI: 10.26342/2020-64-4.

Most humour processing systems to date make at best discrete, coarse-grained distinctions between the comical and the conventional, yet such notions are better conceptualized as a broad spectrum. In this paper, we present a probabilistic approach, a variant of Gaussian process preference learning (GPPL), that learns to rank and rate the humorousness of short texts by exploiting human preference judgments and automatically sourced linguistic annotations. We apply our system, which is similar to one that had previously shown good performance on English-language one-liners annotated with pairwise humorousness annotations, to the Spanish-language data set of the HAHA@IberLEF2019 evaluation campaign. We report system performance for the campaign's two subtasks, humour detection and funniness score prediction, and discuss some issues arising from the conversion between the numeric scores used in the HAHA@IberLEF2019 data and the pairwise judgment annotations required for our method.

@article{miller2020predicting,

author = {Tristan Miller and Do Dinh, Erik-L{\^{a}}n and Edwin Simpson and Iryna Gurevych},

title = {Predicting the Humorousness of Tweets Using {Gaussian} Process Preference Learning},

journal = {Procesamiento del Lenguaje Natural},

volume = {64},

pages = {37--44},

month = mar,

year = {2020},

issn = {1135-5948},

doi = {10.26342/2020-64-4},

}

Citation
Abstract
BibTeX

Tristan Miller.
Reinhold Aman, 1936–2019.
Humor: International Journal of Humor Research, 32(1):1–5, February 2020. ISSN 0933-1719. DOI: 10.1515/humor-2019-0085.

Reinhold Aman, the celebrated and controversial expert on vulgar and offensive language, died on March 2, 2019 at the age of 82. Aman was best known as the founder and editor of Maledicta: The International Journal of Verbal Aggression, whose notoriety played a small but important role in the founding of the International Society for Humor Studies.

@article{miller2020reinhold,

author = {Tristan Miller},

title = {Reinhold {Aman}, 1936--2019},

journal = {Humor: International Journal of Humor Research},

volume = {32},

number = {1},

pages = {1--5},

month = feb,

year = {2020},

issn = {0933-1719},

doi = {10.1515/humor-2019-0085},

}

Citation
Abstract
BibTeX

PDF

Tristan Miller.
Reinhold Aman (1936–2019).
The LINGUIST List, 30.4729, December 2019.

Reinhold Aman, the linguist known as ”the world's foremost authority on swearing”, died on March 2, 2019 at the age of 82. Aman was the founder and president of Maledicta: The International Research Center for the Study of Verbal Aggression, and served as the full-time editor of its eponymous journal. For nearly three decades, Maledicta published collections and studies of some of the most vile and vulgar, but oftentimes also the most creative, uses of human language. Aman himself penned many of the journal's articles, as well as a number of standalone books on wordplay and slang.

@article{miller2019reinhold,

author = {Tristan Miller},

title = {Reinhold {Aman} (1936--2019)},

journal = {The LINGUIST List},

volume = {30.4729},

month = dec,

year = {2019},

}

Citation
Abstract
BibTeX

HTML

Tristan Miller.
The punster's amanuensis: The proper place of humans and machines in the translation of wordplay.
In Proceedings of the Second Workshop on Human-Informed Translation and Interpreting Тechnology (HiT-IT 2019), pages 57–64, September 2019. DOI: 10.26615/issn.2683-0078.2019_007.

The translation of wordplay is one of the most extensively researched problems in translation studies, but it has attracted little attention in the fields of natural language processing and machine translation. This is because today's language technologies treat anomalies and ambiguities in the input as things that must be resolved in favour of a single “correct” interpretation, rather than preserved and interpreted in their own right. But if computers cannot yet process such creative language on their own, can they at least provide specialized support to translation professionals? In this paper, I survey the state of the art relevant to computational processing of humorous wordplay and put forth a vision of how existing theories, resources, and technologies could be adapted and extended to support interactive, computer-assisted translation.

@inproceedings{miller2019punsters,

author = {Tristan Miller},

title = {The Punster's Amanuensis: {The} Proper Place of Humans and Machines in the Translation of Wordplay},

booktitle = {Proceedings of the Second Workshop on Human-Informed Translation and Interpreting Тechnology (HiT-IT 2019)},

pages = {57--64},

month = sep,

year = {2019},

doi = {10.26615/issn.2683-0078.2019_007},

}

Citation
Abstract
BibTeX

PDF

Tristan Miller, Erik-Lân Do Dinh, Edwin Simpson, and Iryna Gurevych.
OFAI–UKP at HAHA@IberLEF2019: Predicting the humorousness of tweets using Gaussian process preference learning.
In Miguel Ángel García Cumbreras, Julio Gonzalo, Eugenio Martínez Cámara, Raquel Martínez Unanue, Paolo Rosso, Jorge Carrillo de Albornoz, Soto Montalvo, Luis Chiruzzo, Sandra Collovini, Yoan Guitiérrez, Salud Jiménez Zafra, Martin Krallinger, Manuel Montes y Gómez, Reynier Ortega-Bueno, and Aiala Rosá, editors, Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2019), volume 2421 of CEUR Workshop Proceedings (ISSN 1613-0073), pages 180–190, August 2019.

Most humour processing systems to date make at best discrete, coarse-grained distinctions between the comical and the conventional, yet such notions are better conceptualized as a broad spectrum. In this paper, we present a probabilistic approach, a variant of Gaussian process preference learning (GPPL), that learns to rank and rate the humorousness of short texts by exploiting human preference judgments and automatically sourced linguistic annotations. We apply our system, which had previously shown good performance on English-language one-liners annotated with pairwise humorousness annotations, to the Spanish-language data set of the HAHA@IberLEF2019 evaluation campaign. We report system performance for the campaign's two subtasks, humour detection and funniness score prediction, and discuss some issues arising from the conversion between the numeric scores used in the HAHA@IberLEF2019 data and the pairwise judgment annotations required for our method.

@inproceedings{miller2019ofaiukp,

author = {Tristan Miller and Do Dinh, Erik-L{\^{a}}n and Edwin Simpson and Iryna Gurevych},

editor = {García Cumbreras, Miguel Ángel and Julio Gonzalo and Martínez Cámara, Eugenio and Martínez Unanue, Raquel and Paolo Rosso and Jorge Carrillo-de-Albornoz and Soto Montalvo and Luis Chiruzzo and Sandra Collovini and Yoan Guitiérrez and Jiménez Zafra, Salud and Martin Krallinger and Manuel Montes-y-Gómez and Reynier Ortega-Bueno and Aiala Rosá},

title = {{OFAI}--{UKP} at {HAHA}@{IberLEF}2019: {Predicting} the Humorousness of Tweets Using {Gaussian} Process Preference Learning},

booktitle = {Proceedings of the Iberian Languages Evaluation Forum (IberLEF 2019)},

volume = {2421},

pages = {180--190},

series = {CEUR Workshop Proceedings},

month = aug,

year = {2019},

issn = {1613-0073},

}

Citation
Abstract
BibTeX

Edwin Simpson, Erik-Lân Do Dinh, Tristan Miller, and Iryna Gurevych.
Predicting humorousness and metaphor novelty with Gaussian process preference learning.
In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL 2019), pages 5716–5728, July 2019. ISBN 978-1-950737-48-2. DOI: 10.18653/v1/P19-1572.

The inability to quantify key aspects of creative language is a frequent obstacle to natural language understanding. To address this, we introduce novel tasks for evaluating the creativeness of language—namely, scoring and ranking text by humorousness and metaphor novelty. To sidestep the difficulty of assigning discrete labels or numeric scores, we learn from pairwise comparisons between texts. We introduce a Bayesian approach for predicting humorousness and metaphor novelty using Gaussian process preference learning (GPPL), which achieves a Spearman's $\rho$ of 0.56 against gold using word embeddings and linguistic features. Our experiments show that given sparse, crowdsourced annotation data, ranking using GPPL outperforms best–worst scaling. We release a new dataset for evaluating humor containing 28,210 pairwise comparisons of 4,030 texts, and make our software freely available.

@inproceedings{simpson2019predicting,

author = {Edwin Simpson and Do Dinh, Erik-L{\^{a}}n and Tristan Miller and Iryna Gurevych},

title = {Predicting Humorousness and Metaphor Novelty with {Gaussian} Process Preference Learning},

booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL 2019)},

pages = {5716--5728},

month = jul,

year = {2019},

isbn = {978-1-950737-48-2},

doi = {10.18653/v1/P19-1572},

}

Citation
Abstract
BibTeX

Computational Pun-derstanding

Computer-Assisted Translation of Humorous Wordplay

About the Project

Motivation

Methods

Innovation

Team

Principal Investigator

Research Staff

Cooperation Partners

Funding Agency

News

University of Konstanz talk

JOKER at ECIR 2023

JOKER papers published

Routledge chapter published

Two panels at ISHS 2022

JOKER at ECIR 2022

Successful Master's defence

Dagstuhl Seminar abstracts published

PunCAT at AV Translation Week

Paper accepted at LaTeCH-CLfL

Workshop at CLEF 2022

Abkhaz State University talk

Computational humour webinar

Routledge chapter accepted

Talk at CALT 2021

Will AI replace comedians?

Outstanding reviewer award

Shared task at SemEval-2021

Words/Machines invited talk

University of Huddersfield talk

Notes & Queries paper printed

Article published in JOSS

Welcome to Máté Lajkó

Job posting

Article published in PLN

Remembering Reinhold Aman

Keynote at NLP4RE'20

Panel on Humor and AI

Interview on Ö1's Digital.Leben

Comedy and AI lecture

Lecture at the University of Vienna

Outstanding reviewer award

Paper accepted at HiT-IT 2019

Paper accepted at IberLEF 2019

Second OFAI research talk

Paper accepted at ACL 2019

Research talk at OFAI

Plotting Poetry paper accepted

Two papers accepted at ISHS 2019

Publications