Rasa: Use new facebook duckling instead of deprecated version

[tmbo]

Currently we are using a wrapper around the clojure version of facebooks duckling (https://github.com/wit-ai/duckling_old). They created a new implementaion in haskell (https://github.com/facebookincubator/duckling).

We need a new wrapper to call it, either using native calls from
python (NLU) <-> haskell (duckling) or by using a webserver
python (NLU) <-> http <-> haskell (duckling).

[twerkmeister]

@FraBle can we expect a new wrapper? :smile:

[FraBle]

I'm seriously thinking about it ^^
Btw, I updated the current wrapper to version 1.6.0.
I guess I should start with a changelog 🤔

[erniedeferia]

Is the migration to use the haskell-based duckling on the roadmap? If so, when can we expect it be released?

[wrathagom]

@erniedeferia we need the python wrapper before we can use it in Rasa. Is there a particular feature you are after that's in the new version?

[erniedeferia]

@wrathagom not at all. looking to minimize runtime requirements (JVM).

[wrathagom]

I feel that pain, we run with docker and putting java in my containers is a bit terrifying for such a simple requirement.

[wrathagom]

@tmbo how would the http piece work? It's actually how we use it right now making http calls to the Haskell version in a docker container. I'm assuming you're meaning a hosted/public version someplace?

[erniedeferia]

Is this wrapper being developed by someone you work with?
https://github.com/sebastianmika/pyduckling

[wrathagom]

nice find.

[wrathagom]

@tmbo did you see 👆

[tmbo]

Yes, I already talked to the author. The only issue is that he doesn't have a lot of time to spent on the wrapper, but currently it only supports the time dimension (e.g. no number, date, ... extraction). It should be rather easy to extend, but we would need that before we can use it.

https://github.com/sebastianmika/pyduckling/issues/1

[pocman]

I'm currently using rasa only to use duckling (i'm planning on adding some ML after that).
I was not aware that rasa was using the previous, deprecated, version of duckling.
This is an issue since the deprecated version has a lot of missing extractors.
For instance, there is no way to extract email in de language :/

[wrathagom]

Are you a python and Haskell expert? It would be awesome for someone to take the above library and expand it. Until then your options are to use the deprecated version or use the server that is in the new duckling library and make API calls to it.

[tmbo]

It should be rather easy to build on the repository I mentioned. Most of the functionality is already there. What's missing though is a good way of installing that wrapper.

[wrathagom]

@tmbo I am going to close this since you have implemented the duckling http component.

[asauray]

@wrathagom @tmbo Could you give a link to the the implemented component ?

[fanglinchen]

I would also like to check out the implementation of the Haskell ducking http component. @wrathagom @tmbo could you guys point us to the related commit? Thanks.

[adamyi]

Same here. @wrathagom @tmbo

Thanks in advance ^_^

[tmbo]

Yes, so for some reason this component didn't make it on the docs page. But we are currently reworking the docs and we'll add it as part of that. You need to add

pipeline:
- name: "ner_duckling_http"
   url: "http://localhost:8000"

To your configuration. You also need to run the duckling docker container (or run the haskell duckling server natively on your machine). Running the duckling server can be done using:
docker run -p 8000:8000 rasa/duckling

[wrathagom]

I created an issue to update the docs #1157

[wrathagom]

Also my PR #1156 has a compose showing how to use the two side by side with Docker if you're interested.

[reza-ebrahimi]

@tmbo is ner_duckling component same as following repository?
https://github.com/wit-ai/duckling_old

which component is better to use? ner_duckling or ner_duckling_http?

[wrathagom]

ner_duckling uses a python wrapper of the repo you linked.

it is better to use ner_duckling_http support for ner_duckling will be removed in the next version.

[anaportela]

Hi! So now to use the duckling I have to have the docker running?

[naoko]

@anaportela , that's the easiest way to do.
if you are running rasa (or where you want to access duckling) is docker container then you want to create network

docker network create botnetwork

then you want to start rasa/duclking

docker run -p 8000:8000 -d --network botnetwork --name duckling rasa/duckling

if you are not using docker for your rasa, you can omit the --network botnetwork