WAT2021 English-Malayalam Multi-Modal Translation Task

After a positive response from the participants for the WAT 2019 and WAT2020 English-Hindi Multimodal Translation Tasks, the Workshop on Asian Translation 2021 (WAT2021) will continue the multimodal task with new languages. The task relies on our “Malayalam Visual Genome” (MVG), a multimodal dataset consisting of text and images suitable for English-Malayalam multimodal machine translation task and multimodal research.

Timeline

• April 26, 2021: Translations need to be submitted to the organizers
• May 17, 2021: System description paper submission deadline
• May 28, 2021: Review feedback for system description
• June 1, 2021: Camera-ready
• August 5-6, 2021: WAT2021 takes place

Task Description

The setup of the WAT2021 task is as follows:

• Inputs:
  • An image,
  • A rectangular region in that image,
  • A short English caption of the rectangular region.
• Output:
  • The caption translated to Malayalam.

Types of Submissions Expected

Participants of the WAT2021 task are welcome to submit their outputs in any subset of these submission types:

• Text-only translation (only the test used as input)
• Malayalam-only image captioning (only the image used as input)
• Multi-modal translation (uses both the image and the text)

Training Data

The Malayalam Visual Genome consists of:

• 29k training examples
• 1k dev set
• 1.6k evaluation set

Evaluation

WAT2021 English-Malayalam Multi-Modal Task will be evaluated on:

• 1.6k evaluation set of Malayalam Visual Genome
• 1.4k challenge set of Malayalam Visual Genome

Means of evaluation:

• Automatic metrics: BLEU, CHRF3, and others
• Manual evaluation, subject to the availability of Malayalam speakers

Participants of the task need to indicate which track their translations belong to:

• Text-only / Image-only / Multi-modal
  • see above
• Domain-Aware / Domain-Unaware
  • Whether the full (English) Visual Genome was used in the training or not.
• Constrained / Non-Constrained
  • 29k training segments from the Malayalam Visual Genome
  • (English-only) Visual Genome [making the submission a domain-aware run]
• Non-constrained submission may use other data, but need to specify what data was used.
     

Download Link

• Malayalam Visual Genome 1.0  

 

Submission Requirement

The system description should be a short report (4 to 6 pages) submitted to WAT 2021 describing the method(s).

Each participating team can submit at most 2 systems for each of the task (e.g. Text-only, Malayalam-only image captioning, multimodal translation using text and image). Please submit through the submission link available in the WAT2021 website and select the task for submission.   

Paper and References

Please refer to the below papers:

[paper] : https://www.cys.cic.ipn.mx/ojs/index.php/CyS/article/view/3294

[arxiv] : https://arxiv.org/abs/1907.08948

 

Organizers

• Shantipriya Parida (Idiap Research Institute, Switzerland)
• Ondřej Bojar (Charles University, Czech Republic)

Contact

email: wat-multimodal-task@ufal.mff.cuni.cz

License

The data is licensed under Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International.

Acknowledgement

This shared task is supported by the below projects/grants from Idiap Research Institute (Switzerland) and Charles University (Czech Republic).

• European Union, Project code: EC/H2020/833635, Project name: ROXANNE - Real time network, text, and speaker analytics for combating organized crime
• Grantová agentura České republiky, Project code: 19-26934X, Project name: Neural Representations in Multi-modal and Multi-lingual Modelling