Rasa: New docs: open API specification

[m-vdb]

⬆️ Relates to https://github.com/RasaHQ/growth/issues/1436

This ticket is about making the Open API specification work with the new doc system. The current API spec lives in docs/_static/spec/, and is made available at https://rasa.com/docs/rasa/api/http-api/

[tmbo]

oh right, yes did that - doesn't look pretty but it gets rendered as part of https://github.com/RasaHQ/rasa/pull/6043

[tmbo]

we need to decide design-wise what to do with this page, the content is pretty wide so it might make sense to remove the left-hand sidebar (we did this in the previous docs as well). we definitely need to reduce the whitespace though.

current in new docs:

previous:

@lunelson probably something that we need to put on your list :)

[lunelson]

@tmbo yep, it's in my scope, I'll dig in when I get back. Good to have this input. @eliseboyd and I need to develop a globally consistent responsive layout concept that will contain all of these scenarios

[m-vdb]

it's not working properly with a production build: make docs fails with ReferenceError: Buffer is not defined. Tried the workarounds in this issue but didn't work 🤷

[lunelson]

we need to decide design-wise what to do with this page, the content is pretty wide so it might make sense to remove the left-hand sidebar

So one thing that's tricky about this, is that ReDoc assumes a full-width layout and sets its own @media breakpoints accordingly, whereas our layout will be limited in width from a certain breakpoint, probably around 1400px max... is it out of the question to consider building the API docs to a different subdomain? This seems like a common scenario from what I've seen

[tmbo]

mhm, I think from a usability perspective it would be helpful to at least keep the header and its navbar - I think full width is fine though and I agree on something a lot of other documentation pages do as well. the way the framework evolved also made that API less important now than e.g. a year ago.

do you think keeping at least the header would be something we should/can do?

[lunelson]

do you think keeping at least the header would be something we should/can do?

@tmbo which header do you mean exactly here?

[lunelson]

@m-vdb I notice that the dependency "redoc": "^2.0.0-rc.31" is already on master; does this mean you are going with the 2.0 React-based version of redoc?

[m-vdb]

I think that's what it means :D but it's a very early implementation of the open API spec. We kept this issue open so if we want to do something different, we still can.

[tmbo]

@tmbo which header do you mean exactly here?

I meant the navbar (so logo + top level navigation)

[tmbo]

@m-vdb it doesn't seem to render properly though https://rasa.com/docs/rasa/next/api/http-api

we can def use a different library, I just added this one to make sure they API appears somewhere in some form until we figure out how to include it properly.

[lunelson]

I meant the navbar (so logo + top level navigation)

@tmbo OK so I would say since we're not going to be using any other top level features from Docusaurus i.e. a blog or other arbitrary pages, design-wise we will probably prefer to move the items there (title, version-dropdown, github link) in to the top of the sidebar. I haven't quite begun to dig in to it to figure out how to pull apart the code but AFAIK this should be possible as of the latest patch releases

—in other words, we're looking to unify the top-level navbar with the global nav of the website

[m-vdb]

@tmbo fixed here => https://github.com/RasaHQ/rasa/pull/6325

[lunelson]

@m-vdb I'm encountering something I didn't realize before: rasa and rasa-x _each_ have their own API documentation, so the solution we were thinking of, with linking just "API" in the header nav doesn't make sense because there isn't one single global API but two product-specific ones...is this right?

[m-vdb]

Yes this is right, these are 2 different APIs. There are also other APIs (that are not HTTP ones). Here are the current items under the "API" left menu on Rasa:

[lunelson]

@m-vdb OK how many of these need to use ReDoc to document them? Just the HTTP ones right?

[m-vdb]

none, only the HTTP ones

[lunelson]

@m-vdb OK @eliseboyd and I are working on the assumption there is _one HTTP API documentation per site_ (rasa, rasa-x), that will be using the ReDoc component 👍

[m-vdb]

and there is also the Action Server HTTP API: https://rasa.com/docs/rasa/next/running-action-server#action-server-http-api which is currently in Rasa docs, but should be moved to the Rasa SDK docs, if I'm not mistaken (cc @erohmensing to double check)

[eliseboyd]

Does something like this make sense then?

I agree we should keep the redoc full width, without our sidebar - but it still needs the top navbar to show you where you currently are in the context of the docs, and that you're away from the regular docs. It seems to be a common pattern to have API/SDK separated into a very visually different section. To make it even clearer, the left logo could also say "Rasa API/SDK"

[m-vdb]

yeah I was thinking about something like this. I don't think I should be the one deciding though :) WDYT @erohmensing @akelad @tmbo ?

[eliseboyd]

Just for more context, the issue I'm trying to solve is that currently when you go into the api docs, e.g. here, it's unclear what page you're on and how to get back to the docs / how this page relates to any other page. At _minimum_, these pages should have a title and "Rasa Open Source" or "Rasa X" should be selected in the top nav as the active link, depending on which section you're in.

The suggestion above goes a step further towards (hopefully) clearer information architecture, but value thoughts :)

We could also make those smaller improvements for now, and aim to tackle this section later.

[lunelson]

@erohmensing would the Rasa SDK docs be using a ReDoc style documentation then, or a Docusaurus style like the main docs? I had understood you were considering deploying another Docusaurus instance to handle this

[erohmensing]

The docs themselves would be docusaurus style, as there's lots of typical docs info about what actions are, how they should be used etc, that needs to go in there. Though I don't think a python autogeneration would hurt, as we're currently trying to figure out the easiest way to keep the user up to date with all of the tracker methods without using an autoclass like we did before(https://rasa.com/docs/rasa/api/rasa-sdk/#tracker)

The action server API also has a few endpoints that should get Open API specs (e.g. here: https://rasa.com/docs/rasa/api/action-server/ but there's actually at least 2 endpoints missing that we should document)

So basically the SDK package would ideally be as documented as the rasa open source package (docs, auto-gen python info, redoc-style api), whether that's embedded in the Rasa Open Source docs or as its own section.

[erohmensing]

the issue I'm trying to solve is that currently when you go into the api docs, e.g. here, it's unclear what page you're on and how to get back to the docs / how this page relates to any other page.

I totally agree with @eliseboyd, i've griped about this many times. I think the simplest way to handle this would just be for API specs to open in a new tab, if they don't have a sidebar to help you get back to where you came from, but definitely think @eliseboyd 's "at minimum" is very valid to knowing which API you're looking at

[lunelson]

OK @eliseboyd it sounds like this might be more complex than what you've pictured above; we'll have to try out some different solutions

[erohmensing]

Happy to workshop stuff together as well, we're still also not sure where the SDK/action server stuff really fits in, which makes it difficult

[eliseboyd]

Sounds good! If we think this might take a bit of time to figure out, @erohmensing's suggestion to open in a new tab sounds like a good first step too. In that case, I'd suggest these immediate small fixes:

1. Open any redoc links in new tab, add an 'external link' icon next to any of those links in the sidebar
2. Add a page title/description on any redoc page
3. Set Rasa / Rasa X as the active link in the top navbar

[m-vdb]

@lunelson @eliseboyd should we close this ticket or do we still need it?

[lunelson]

🙏 I'll close it for now, as it's technically done—though the actual UI/navigation should be improved in the future, but this is a theme issue. I'll create a follow-up there.