Docs: svalidation fails on new treebanks

[martinpopel]

http://universaldependencies.org/svalidation.html says
Traceback (most recent call last): File “docs-pages-source/gen_index/syn_validation_run.py”, line 267, in del lang_results[lw] KeyError: 'UD_English-ParTUT'

[dan-zeman]

Not only new treebanks. Same problem with UD_Polish.

[fginter]

:blush:

[sebschu]

There still seems to be some issue. English is now missing from the list and I know for a fact that we don't comply a 100% to the v2 guidelines yet ;)

[martinpopel]

@sebschu
Meanwhile you can check the English s-validation errors here http://ufallab.ms.mff.cuni.cz/~popel/en/dev-bugs.html
(and train-bugs.html, which has 37MiB)

I've generated this using Udapi ud.MarkBugs (which has few extra tests in addition to the svalidation.html)
with this command
udapy -HA ud.MarkBugs util.Filter keep_tree_if_node='node.misc["Bug"]' < en-ud-dev.conllu > dev-bugs.html

[sebschu]

@martinpopel Thanks! My browser isn't too happy about the 17M but this is really useful.

[martinpopel]

@sebschu Try

wget http://ufallab.ms.mff.cuni.cz/~popel/en/train-bugs.color.gz
zcat train-bugs.color.gz | less -R

This gz has just 1.7 MiB and I've created it with
udapy -TA ud.MarkBugs util.Filter keep_tree_if_node='node.misc["Bug"]' < en-ud-train.conllu | gzip > train-bugs.color.gz

[fginter]

@sebschu - ah, now I see it. Indexing into dep_search died on the empty nodes. I'll fix this and English will be back. @martinpopel - Is there somewhere a list of the tests you added to udapi? I'd like to include them into svalidation.html, since that gets regenerated on every push to dev for all languages (or will get, once I fix my bug :) The svalidation.html tests are here: https://github.com/UniversalDependencies/docs/blob/pages-source/gen_index/stests.yaml IIRC, the system will react on change of this file, so anyone can add new tests via this file.

[martinpopel]

list of the tests you added to udapi?

Yes, here.
The *-child test should be accompanied by the "DEBUGGING TEST..." notice (because of false alarms with head elision).

that gets regenerated on every push to dev for all languages

I know, and this is really very useful. This is why I recommended svalidation.html at the first place in my email.

On of my motivations for reimplementing it was that I wanted to test also the test sets which should not be pushed to GitHub.

stests.yaml

Once @dan-zeman told where this file is located I have inspected it. However, I don't know how to write e.g. the goeswith-gap test in SETS (while in Python+Udapi it is easy). Also I plan to add tests for the empty nodes and enhanced deps (this way I could solve the false alarms with head elision in *-child tests) and I am not sure this is possible in SETS.

[fginter]

@martinpopel thanks - I'll check it out and see if I can add the tests to svalidation.html so we keep everything in-sync. I will now also index nullnodes in dep_search and will add them a feature Nullnode=Nullnode that way it will be easy to look for them by simply saying Nullnode or something like VERB&Nullnode anywhere.

[fginter]

@sebschu English is back in svalidation.html. Thanks for letting me know! It is now possible to search for the null nodes http://bionlp-www.utu.fi/dep_search/query?search=Nullnode&db=UD_English-dev&case_sensitive=True&hits_per_page=10 -- I only yet need to upgrade the Brat visualizer in dep_search so it highlights the hits correctly. Won't take long, but is usable already now. Reopen this bug if more problems are spotted. I rocked the boat a bit now with some changes to the svalidation pipeline, sorry if this causes troubles. Just tell me if there's yet something, I'll fix.