SDP 2016: Broad-Coverage Semantic Dependency Parsing

Version 1.1; April 4, 2016


Background
==========

This directory contains data, tools, system results, and publications
associated with the 2014 and 2015 tasks on Broad-Coverage Semantic Dependency
Parsing (SDP), which were run in conjunction with the International Workshop on
Semantic Evaluation (SemEval).  Semantic dependency parsing is the task of
recovering sentence-internal predicate–argument relationships for all content
words.  The target representations, thus, are bilexical semantic dependency
graphs.

The SDP 2014 and 2015 data consists of graphs in three target representations:
MRS-Derived Semantic Dependencies (DM), Enju Predicate–Argument Structures
(PAS), and Prague Semantic Dependencies (PSD).  As a fourth, additional target
representation we also release a conversion of CCGbank to semantic dependency 
graphs (in the subdirectory ‘ccd’).  These graphs are aligned with the graphs
released in connection with SDP 2015 for English.

For additional information, please see the README files in each sub-directory,
a summary paper to be published in the 2016 Conference on Language Resources
and Evaluation ([3] below; included as ‘lrec16.pdf’), the original SemEval task
descriptions ([1] and [2] below; also included in PDF format in the ‘2014/’ and
‘2015/’ sub-directories, respectively), and the general SDP web pages at:

  http://sdp.delph-in.net/


Directory Structure
===================

The top-level directories of this data collection mirror the historic sequence
of events:

+ ‘2014/’ — the data, tools, and system results from Task 8 at SemEval 2014;
+ ‘2015/’ — the data, tools, and system results from Task 18 at SemEval 2015;
+ ‘ccd/’ — the new set of semantic dependency graphs derived from CCGbank.

In the 2014 and 2015 sub-directories, the file layout preserves the original
conventions used for data distribution to SemEval participants, so as to make
it easy to replicate published results.  Each sub-directory (including the new
‘ccd/’) provides its own file ‘README.txt’ with additional instructions.  In a
nutshell, the major components are as follows:

+ ‘2014/’ – training data: three ‘.sdp’ files, one per target representation;
+ ‘2014/companion/’ – syntactic companion analyses (for use in open track);
+ ‘2014/toolkit/’ – Java implementation of the SDP graph reader and scorer;
+ ‘2014/test/’ – test data: ‘.tt’ system input files and ‘.sdp’ gold standard;
+ ‘2014/submissions/’ – outputs from participating teams and official scores.

+ ‘2015/’ – training data: five ‘.sdp’ files, with Chinese PAS and Czech PSD;
+ ‘2015/companion/’ – companion analyses (for use in open and gold tracks);
+ ‘2015/test/’ – test data: ‘.tt’ system input files and ‘.sdp’ gold standard;
+ ‘2015/submissions/’ – outputs by teams, Python scorer, and official results;
+ ‘2015/pdf/’ – graphical renderings of the English DM, PAS, and PSD graphs;
+ ‘2015/eds/’ – DM background: EDS graphs in JSON serialization and AMR syntax;
+ ‘2015/treex/’ – PSD background: tectogrammatical trees in Treex (XML) syntax.

+ ‘ccd/’ – training and test data for CCD graphs; Python converter for CCGbank.


Contact
=======

For questions or comments, please do not hesitate to email the task organizers
at: ‘sdp-organizers@delph-in.net’.

Dan Flickinger
Jan Hajič
Angelina Ivanova
Marco Kuhlmann
Yusuke Miyao
Stephan Oepen
Yi Zhang
Daniel Zeman


Release History
===============

[Version 1.1; April 4, 2016]

+ Initial public release, for distribution via the Lingustic Data Consortium.


References
==========

[1] Stephan Oepen, Marco Kuhlmann, Yusuke Miyao, Daniel Zeman, Dan Flickinger,
    Jan Hajič, Angelina Ivanova, and Yi Zhang. SemEval 2014 Task 8.
    Broad-Coverage Semantic Dependency Parsing.  In Proceedings of the 8th
    International Workshop on Semantic Evaluation (SemEval 2014), pages 63–72,
    Dublin, Republic of Ireland, 2014.  http://aclweb.org/anthology/S14-2008

[2] Stephan Oepen, Marco Kuhlmann, Yusuke Miyao, Daniel Zeman, Silvie Cinková,
    Dan Flickinger, Jan Hajič, and Zdeňka Urešová.  SemEval 2015 Task 18.
    Broad-Coverage Semantic Dependency Parsing.  In Proceedings of the 9th
    International Workshop on Semantic Evaluation (SemEval 2015), pages
    915–926, Denver, CO, USA, 2015.  http://aclweb.org/anthology/S15-2153

[3] Stephan Oepen, Marco Kuhlmann, Yusuke Miyao, Daniel Zeman, Silvie Cinková,
    Dan Flickinger, Jan Hajič, Angelina Ivanova, and Zdeňka Urešová.  Towards
    Comparability of Linguistic Graph Banks for Semantic Parsing.  In
    Proceedings of the 10th International Conference on Language Resources and
    Evaluation (LREC 2016).  Portorož, Slovenia, 2016.