Modernmt: Question about NMMT and Training Data

[mzeidhassan]

Hello MMT team,

I have a question regarding training data.

Most other neural MT solutions use data set for training, another for validation, and another for testing and you need to provide these to be able to train your model.

As far as I know, ModernMT needs only bi-text training data and you can also add monolingual data if you wish.

My question:

How does MMT handle the model training? Does it create validation and test data on the fly during model training? I searched the Wiki page, but couldn't find relevant info.

Thanks in advance for your support!

Kind regards,
Mohamed

[davidecaroselli]

Hi @mzeidhassan

very simple: by default MMT extract 1200 sentence pairs as test set and 1200 sentence pairs as develop/validation set. These are stored under ModernMT/engines/<your_engine>/data/dev and ModernMT/engines/<your_engine>/data/test.

These pairs are randomly extracted from the whole corpus during preprocessing and dev/validation set is used as Validation set in Neural Engine training (if not specified) or for development set for Phrase-Based tuning (if not specified). Test set is used for ./mmt evaluate if no --path option is specified.

It's a very quick and effective way to easily prepare and test your engine without worrying about messing up with parallel data ;)

Cheers,
Davide

[marcotrombetti]

Davide,

I think that also this is a good candidate for he FAQ.

Il giorno mar 21 nov 2017 alle 22:38 Davide Caroselli <
[email protected]> ha scritto:

Hi @mzeidhassan https://github.com/mzeidhassan

very simple: by default MMT extract 1200 sentence pairs as test set and
1200 sentence pairs as develop/validation set. These are stored under
ModernMT/engines//data/dev and
ModernMT/engines//data/test.

These pairs are randomly extracted from the whole corpus during
preprocessing and dev/validation set is used as Validation set in Neural
Engine training (if not specified) or for development set for Phrase-Based
tuning (if not specified. Test set is used for ./mmt evaluate if no --path
option is specified.

It's a very quick and effective way to easily prepare and test your engine
without worrying about messing up with parallel data ;)

Cheers,
Davide

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/ModernMT/MMT/issues/304#issuecomment-346169461, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AJcIE6YLyG5xBcu_u62lNbO_yuGtqt8Cks5s40LTgaJpZM4QmM5G
.

[mzeidhassan]

Thank you so much @davidecaroselli for your answer. This explains it.

One last question before closing this issue :-)

From your experiments, do you think 1200 sentences are enough for the neural decoder to provide good quality? I am asking because it seems that the standard is about 2000 validation sentences & 2000 test sentences in other solutions.

I like your approach by the way because I don't need to worry about preparation of validation and test data sets, but is there a way to increase the number of extracted data sets? Can I increase the number for example to 2000? If yes, where should I make such change? I know that this can be done manually, but I am asking if there is a way to make MMT go for 2000, instead of 1200 out of the box.

Thanks again for all your efforts, Davide!

Best regards,
Mohamed

[davidecaroselli]

Hi @mzeidhassan

the 1200 lines limit was chosen in the phrase-based era: we found it appropriate - the right balance between accuracy and time. It has remained untouched now with the Neural Engine.

Currently the 1200 value is hard-coded and it cannot be specified by command line. However if you want to force the change, it's not so difficult, just modify it here:

https://github.com/ModernMT/MMT/blob/master/src/core/src/main/java/eu/modernmt/facade/TrainingFacade.java#L34

and then run (from within your ModernMT root):

cd src
mvn clean install
cd ..

Let me know if this worked and if I can help you with some other issue.
Have a nice weekend!

@marcotrombetti Done! :)

[mzeidhassan]

Thanks @davidecaroselli for being always supportive and for taking the time to provide detailed and meaningful answers. I appreciate it. Please feel free to close this issue if you wish.

[davidecaroselli]

Thanks @mzeidhassan !