Monica: API for integrations

[adrum]

This issue serves the purpose of tracking the progress of the API development. I haven't found anything on this yet other than it needs to be done for mobile apps, so I figured we could track or even discuss its implementation here. I'm eagerly looking forward to contributing to the iOS version of the mobile app, so this is an issue I will be watching very closely.

[djaiss]

Thanks @adrum I'll initiate the discussion here as soon as I have the time to do so. The API is very, very high on the list of things we should do. I'll first focus on the bug fixes that I see popping on the server, then API, API, API.

[theBoatman]

I think the fastest way would be to implement CardDAV. Then you could use existing apps (like DAVdroid) to synchronize and don't need to implement new apps for a basic integration.

Of course this would be just a first step, but being able to sync at least names and numbers to mobile phones would increase interest in this project - at least mine ;-)

[degan6]

I agree that CardDav would be awesome. I think that plus a REST and we will be set.

Maybe we should get the contact to contact and custom fields or contact meta sorted out before we get started out first before we get started on the API though.

[djaiss]

This should not be the place to discuss CardDav because it would be two different purposes.

[issmirnov]

I strongly support a proper REST api. As long as that is designed properly the first time around, ideally you will only have to touch it once. Everyone else will be able to build up a CLI/iOS/Android client in parallel.

[virtadpt]

That also means that I could integrate a few of my bots into Monica!

[djaiss]

cc @webdistortion

Some discussions there https://github.com/monicahq/monica/pull/202 already

• Summary: we want to use Laravel Passport.
• We want to list all the routes we'll need before implement something.
• API needs to be rate limited.

Now the other items:

• We need to use the API key already set in the accounts table.
• Each method needs to be unit tested and documented before we code another method.
• The first API does not need to cover every aspect of the application.

Let's start a great foundation on which we can build upon, rather than doing this in 2 hours and suffer from the consequences later.

A perfect implementation of the wrong specification is worthless.

Source

Also, I'll need to create something like docs.monicahq.com or something to host the documentation of the API (and of the product in general). Any preferences for the name? I think either docs.monicahq.com or dev.monicahq.com would work.

I'll also create a repository for this documentation.

[webdistortion]

@djaiss - there's a great documentation project over here: https://github.com/lord/slate
See example docs: https://lord.github.io/slate/ which may help. Should be relatively easy to get up and running, and lets you modify documentation direct from pull requests to github. I'd go for docs. personally as dev looks like a dev instance for testing, but that's just me.

There is likely going to have to be some refactoring of controller code into repositories to get this in a tidy state of sharing methods between the API and main code base.

Perhaps we need a separate contribution guideline for the API i.e.

1) Method / Route has been agreed by community
2) Method has associated unit test
3) Method has REST documentation written on documentation repo
4) Database access code has been externalised to a repository where applicable and is now shared by the main codebase and API
5) anything else you think of.

You are right to slow this one down before speeding up, and I apologise if I've jumped the gun a bit on chasing it.

[djaiss]

Slate looks great!

To simplify things greatly, can we keep the API in the same codebase as the application for now? The API doc itself could have its own repository.

I agree about the necessity of providing a contribution guideline for the API.

[degan6]

@djaiss Laravel is more or less designed to have a web and api in the same app, it has prebuilt routes and middleware for it. I think everyone who is self hosting would like to do a get pull and have an API.

One vote for putting the API in the same app as the app.

[djaiss]

I'm starting the documentation of the API tomorrow. I needed to review PR and fix pressing bugs in the last two days.

[djaiss]

The documentation has been started. Check the GitHub repository or the site (this won't be the final URL).

• I need help with the authentication part (which is dummy content for now).
• I've documented all Contacts methods.
• I'll follow by Significant other, kids and notes.

I'd like to document everything before we start working on implementing the API.

[djaiss]

Added Significant others, Kids, Notes and Activities documentation.

[issmirnov]

Filed monicahq/docs.monicahq.com#2 - discussion about nested vs flat design. I vote for nested, in my experience it's been easier to work with for downstream consumers.

[djaiss]

Thanks, just read it. Will think of this. Also, as a sidenote, I think that on day 1, we should log API usage on the Monica instance. Both for the instance owner, and the users themselves.

[webdistortion]

Just to check back in on this.. I've been looking at trying to get this off the ground. What we need to get going as a starter for ten is an interface to allow clients to manage their own OAuth tokens. Out of the box, Laravel passport allows this: See under frontend quick start.

https://laravel.com/docs/5.4/passport#deploying-passport

However, you need to have the Frontend Vue library installed to use some of the built in controls, which to my knowledge Monica doesn't currently have. So two options:

1) We decide to try and get the Vue library up and running with Monica and a basic test stub for demoing it (Laravel provides some of these Vue components which take care of the interface for you) or 2) we decide to build that interface manually. Personally I'm not that familiar with Vue, but I can't imagine its that tough a library to get your head around.

This will allow a callback URL for clients consuming the API. As I mentioned previously, passport is built around OAuth so you'll need to manage client secrets etc. This would be stage one of the API build.

[djaiss]

Alright this is time to get back to this after a round of much requested features (after contact import and tags).

There is actually already VueJS in Monica and I use it in two very very small places. I have a local branch where I installed Passport and started to play with it - there are some bugs though.

A good friend of mine and I talked about the API and Passport at length and I think we are ready to build the first building block for the API. Tonight when I come back from work I'll explain how we'll do it in this issue so we can move forward with this.

[phrohdoh]

@djaiss Is progress on the API visible to the public somewhere?

[davisonio]

This would be a killer feature! Good luck

[djaiss]

Alright. So here is the status.
I've tried to implement Laravel Passport but setting it up is a bit too complex for me. This is clearly a lack of technical skills that I have. Laravel 5.2, which is the version Laravel started with, shipped with Vue.js and Passport uses it too, but for some reasons, the default views that are written in Vue.js in Passport don't play well with the current codebase. Because I can't make it work, I haven't been able to focus on the API as a whole (which is an error) and I worked on other cool stuff for the project that people really wanted (linking SO and child to real contacts for instance which took a long time).
Now I realize that we need an API as soon as possible.
I'm going to actually invest real dollars in the project by paying someone to implement the Oauth system that Passport provides to set the base of the API, then I'll be able to actually write the API once the authentication will be done. I'm planning to start the search of someone in the coming days and I assume that for a senior developer (which I'm not) it won't take too long to make it work. Why am I investing real money? Because I believe in the project, because I need it to grow it, because I believe this is what will make it grow.
Then I'll work on the API itself and I'll keep you posted.
Unless someone here wants to start with the authentication with Passport themselves. I'll take care of the rest :-D

[phrohdoh]

Then I'll work on the API itself and I'll keep you posted.

It really would be helpful to have a publicly accessible tracker for the API instead of having to poll for the status.

Would you be open to using the docs' repo as a place to track API implementation?

Edit:

What I am suggesting here is, for example, creating an issue for contact management with bullet points for methods that need to be written (create, read, update, delete, etc), their request / response objects, and any backing implementation details.

I'm very interested in creating integrations with Monica but unfortunately a lot of time has passed between writing prototypes and Monica still remains a box with no inputs or outputs.

[djaiss]

The work on the API has started 🎉

I've started the work on the authentication.
I didn't have to outsource the problem to a contractor to make it work. I'm now using Laravel Passport for Oauth and personal tokens. It's working fine locally, I'm fine tuning everything and plan to release a first part of the API in roughly two weeks, which will contain the Contacts, in a pull request.

Then we'll be able to follow the progress on the docs repository for the other parts.

I want to publish a first PR that I think is complete instead of showing the work in progress, because otherwise it won't be constructive and I'll spend my time answering questions instead of coding.

Thanks for your patience everyone. We'll get there.

[andreasjacobsen93]

Any updates on this? 😃

[djaiss]

Alright.
So.
I'm _nearly_ there for the API. The PR will be submitted very soon and the branch is there: https://github.com/monicahq/monica/tree/api (but it's super messy right now and it needs to be cleaned much more). I've been working a lot on it, iterating over and over, and I can't wait to publish the PR. I've also been super busy at work which slowed down the process, but the wait is almost over.
In this branch we'll also bump Laravel to Laravel 5.5.x, and fix some bugs that I found while I was writing the API. There is also the documentation that comes with the API of course, which takes a lot of time to write too. I've basically copied how Github does it.

[djaiss]

The first version of the API is done. It brings the Oauth authentication and everything about the contacts, as well as the journal.

What will remain to be done in another version is all about the account itself (creation, edition of data).

I'll make a final check tomorrow and in the next days, then I aim for the official v1.0.0-beta release next week. There is probably some stuff that don't work so far, I'll see.

I know it's a long time coming, and I'm sorry about it. Life gets in the way of projects sometimes 😀 But hopefully you'll like the work.

For sure I won't make the same mistake twice. I should have released the work on the API piece by piece instead of making one giant PR with a gazillion changes. I'm a bit scared of shipping this beast now 😀

Anyway, keep an eye on this thread in the coming days! Thanks, you are all awesome and supportive.

[djaiss]

The API has been shipped one month ago. Hope it's useful!

[luiseduardobrito]

Great work, specially with the documentation. Concise, yet descriptive and powerful. Now a little anxious to get it working with mobile apps.