Rasa: Removing projects

[tabergma]

We are going to remove projects in the next major release. E.g., your Rasa NLU server will not be able to handle multiple projects anymore. It will just have a single Rasa NLU model loaded at any time.

As we might introduce a similar logic at a later point again, we want to get some feedback from you:

• Are you using projects currently?
• How do you use them?

Please, feel free to comment on this issue with your feedback.

[wrathagom]

This would be a breaking change for Articulate, can I ask what the driving factor is?

[tmbo]

@wrathagom the reasons are that the usability for the every day user is hard:

• locally when training models you often only need a single project, nevertheless you need to specify it everywhere
• the project management parts of the codebase are quite fragile and lead to multiple out of memory and race condition issues at scale (e.g. models getting unloaded when someone is still using them)
• the implementation does not work well together with asyncio
• we need to unify model handling between core & nlu after the merge and core does not have the concept of projects

So the idea is not to remove projects from the place where they are now (somewhere in the middle, needed for all the commands) and add them in some different way for the use case they are needed (e.g. you need them to run a single server serving multiple models -> we can add this on top, without interfering with all the train & run commands). does that make sense?

[wrathagom]

@tmbo thanks for the reply. My actual biggest concern is this piece of it "It will just have a single Rasa NLU model loaded at any time." So to run A/B tests or live/dev models will require multiple instances of the Rasa NLU server?

[xumx]

We are using projects to run different development models within the same NLU Server. I have the same concerns as @wrathagom

[wrathagom]

I got a sneak peek of Rasa Platform from @alexweidauer last week and it looked like they had some methodology for running a dev and prod model. I wonder if they are using projects as well? or two entirely separate servers?

[ynouri]

Hi,

We are also using Rasa NLU server multiple model serving functionality in version 0.14.4. This is a critical feature for us for many reasons:

• our continuous deployment process is a bit heavy and managing only one or a few deployments of the NLU server per environment (dev -> test -> prod) is much simpler than managing N per environment (where N is a large number of models)
• it allows more efficient deployments in terms of allocated CPU and memory (consequence of previous observation)
• as suggested above by @xumx and @wrathagom it allows to switch easily between different versions of a given model by modifying the payload of the /parse POST. Switching model by switching the endpoint itself is much more involved (at least in our deployment model)
• it allows chatbot developers (who don't have deployment rights) to train a model through the /train endpoint and immediately access it through /parse

Finally, it's worth highlighting that Rasa NLU's value goes beyond the chatbot use case (although I understand this is Rasa's primary concern). We are currently using NLU Server standalone to serve NER models for non-chatbot purposes. The functionality of serving multiple NLP machine learning models from a single instance has value in itself. As far as I know, other ML Frameworks like MLFlow offer serving capabilities but no multi-model handling.

@tabergma , @tmbo , would there be a way to:

• stay on Rasa NLU server 0.14.x (or 0.15.x) to keep the multi model serving capability
• but upgrade to Rasa Core 1.0.x and Rasa X?

Thanks!

[tabergma]

@ynouri Thanks for your feedback.

Rasa NLU and Rasa Core are not separated anymore, they are both located inside one package, called Rasa. So, if you update to Rasa 1.0.x both modules (Core and NLU) are on version 1.0.x. Also the interface changed quite a bit, so Rasa 1.0.x is not compatible with Rasa NLU 0.14.x. Sorry, don't think what you are asking for is possible.

[ynouri]

Thanks @tabergma for your quick reply. We are currently deploying two services separately: a NLU service (running Rasa NLU server) and a Dialog Manager service (running Rasa Core). We could theoretically upgrade only the Dialog Manager service while leaving the NLU deployment untouched. But as you mention, if breaking API changes happened from 0.x to 1.x, our upgraded Dialog Manager service won't be able to function with the old 0.x NLU API. I'll try to look into wrapping NLU 1.x API around the 0.x one, hopefully it's not too difficult.

Regarding projects/multiple models handling feature, what do you think of the user feedback in this thread? I do understand their implementation represents a technical challenge, but they really have a lot of added value. Any plans to bring them back in 1.x?

Cheers

[tabergma]

We definitely consider implementing a similar logic in the future again. However, I cannot give you any timeline. The feedback here is very helpful as we did not know if and how the project logic was used before. We started this thread to get a better overview and to collect requirements. We will let you know, once we decide to implement something again.

[tabergma]

We also started a forum thread about this topic. Feel free to also give feedback over there: https://forum.rasa.com/t/removing-projects-for-rasa-nlu-server/8962

[jamesmf]

Agreed that projects were a vital feature. It is very helpful to be able to create focused, conversation-independent nlu models for different domains (and to re-use them across projects). It lets you keep your NER/NLU modular and reusable

[jamesmf]

Any update on this? It would be great to be able to use rasa 1.0+ and rasa x

I can confirm that it works to deploy an agent with rasa 1.0+ that has no nlu component and pair it with rasa-nlu 0.15.x for nlu so you can use projects. Whatever would normally hit the rasa endpoint just instead hits the rasa-nlu /parse endpoint first, posts the result to the /messages endpoint of the rasa agent, then hits the /predict endpoint.

[Polarisation]

I'll miss this when upgrading to v1.x. In 0.14.x I have been using projects to track different models for different environments (development / staging / production). Now I'd have to run a different Rasa server for each environment, which is frustrating because the RAM requirements for Rasa are fairly heavy (4 / 8 GB).

[Polarisation]

@TyDunn Any update on this issue? Are there plans to bring this feature back in some form?

[TyDunn]

Hi @Polarisation. This is the latest: https://forum.rasa.com/t/removing-projects-for-rasa-nlu-server/8962/17?u=tyd

[j-a-h-i-r]

I didn't get to use the projects feature in Rasa, but it seems that projects was perfect tool to serve multiple language models from the same endpoint. What are the current alternatives to enable such a feature without running two separate instances of Rasa?