• Turin

EVALITA · Evaluation campaign

EVALITA is an initiative aimed at evaluating tools for the automatic processing of natural languages. The general objective of the campaign is to promote the development of language and speech technologies for Italian, providing a shared framework where different systems and approaches can be evaluated in a consistent manner.

The diffusion of shared evaluation practices is a critical step towards the development of resources and tools for natural language processing and speech science. The response to EVALITA, both in the number of participants and the quality of the results, has shown that it is worth pursuing such goals for the Italian language.

As a side benefit of the evaluation campaign, both training and test data are available to the scientific community as benchmarks for future improvements.

The 6th evaluation campaign EVALITA 2018 will be held in Turin, as a co-located event of the 5th Italian Conference on Computational Linguistics – CLiC-it 2018 (December 10-12, 2018).

The EVALITA website collects all the information about the evaluations till now.

Recent activities

The 6th evaluation campaign EVALITA 2018 (http://www.evalita.it/2018) has been organized along the following selected tasks :

  • Affect, Creativity and Style
    • ABSITA – Aspect-based Sentiment Analysis (V. Basile, P. Basile, D. Croce, M. Polignano)
    • ITAMoji – Italian Emoji Prediction (F. Barbieri, F. Ronzano, F. Chiusaroli, E.W. Pamungkas, V. Patti)
    • IronITA – Irony Detection in Twitter (A. Cignarella, S. Frenda, C. Bosco, V. Patti, P. Rosso)
    • GxG – Cross-Genre Gender Prediction (F. Dell’Orletta, M. Nissim)
  • Dialogue Systems
    • iLISTEN – itaLIan Speech acT labEliNg (N. Novielli, P. Basile)
    • IDIAL – Italian DIALogue systems evaluation (F. Cutugno, A. Origlia, M. di Maro, C. Tortora, B. Magnini, M. Guerini, S. Falcone)
  • Hate Speech
    • AMI – Automatic Misogyny Identification (M. Anzovino, E. Fersini, P. Rosso)
    • HaSpeeDe – Hate Speech Detection (C. Bosco, F. Dell’Orletta, M. Sanguinetti, F. Poletto,  M. Stranisci, M. Tesconi)
  • Semantics4AI
    • NLP4FUN – Solving language games (P. Basile. M. de Gemmis, G. Semeraro, L. Siciliani)
    • SUGAR – Spoken Utterances Guiding chef’s Assistant Robots (F. Cutugno, M. Di Maro, A. Origlia, C. Tortora)