Plots2: PLANNING ISSUE: v2 API development

[stefannibrasil]

1. About the project

@milaaraujo and I will be working part-time (20h per week) on the v2 API development this Rails Girls Summer of Code, starting July 2nd until September 28th. The entire description for the project can be found here.

We want to coordinate better as a team and also get feedback, so we can continue continue the work from #1449 (from which we have tried to set as base for our timeline).

2. Working hours (PDT timezone)

We will be working together at the following time:

18h-21h weekdays
14h-19h Saturdays

3. Timeline/Milestones

| Week(s) | Days | Tasks |
|------|------|-------|
| 1 | 02/July-6/July | Study Grape Entities and Swagger code and docs |
| 2-3 | 9/July-20/July | Refactor Search API #3070 |
| 4 | 23/July-27/July | Show most recently updated people on Search API #3069 |
| 5 | 30/July-03/Aug | Add Author and Tagged notes to Swagger interface |
| 6 | 06/Aug-10/Aug | Improve JSON tests (convert them to more standardized API calls) |
| 7 | 13/Aug-17/Aug | Improve the search results |
| 8 | 20/Aug-24/Aug | Work on the dynamic search #987 |
| 9 | 27/Aug-31/Aug | Ensure multiple format access: JSON, RSS, etc., for legacy endpoints that have these formats |
| 10 | 03/Sep-07/Sep | Write unit tests for the back-end to various APIs by writing Services like TypeAheadService and SearchService |
| 11 | 10/Sep-14/Sep | Check with the other contributors to see what API functions they need or want |
| 12 | 17/Sep-21/Sep | Ensure that v2 of Public Lab is fully working and solve possible bugs |
| 13 | 24/Sep-28/Sep | Finish our contribution by reviewing the docs and help planning the next steps for future contributors |

4. Real timeline (updated each week to compare with the one above)

| Week(s) | Days | Tasks |
|------|------|-------|
| 1 | 02/July-6/July | Studied Grape Entities and Swagger code and docs. Finished #2847 |
| 2-3 | 9/July-20/July | Refactored Search API #3070, Added some docs in #3068 and #3115 |
| 4 | 23/July-27/July | Show most recently updated people on Search API #3069 |
| 5 | 30/July-03/Aug | Working on #3069 |
| 6 | 06/Aug-10/Aug | #3069 and #3217 |
| 7 | 13/Aug-17/Aug | Improving /profiles endpoint on #3235 and adding unit tests to SearchService |
| 8 | 20/Aug-24/Aug | |
| 9 | 27/Aug-31/Aug | |
| 10 | 03/Sep-07/Sep | |
| 11 | 10/Sep-14/Sep | |
| 12 | 17/Sep-21/Sep | |
| 13 | 24/Sep-28/Sep | |

5. Questions

Is our timeline realistic and modular? What do you think we can remove or change?

6. Notes

We also plan to create first-timers-only issues along the way. Also, we didn't add to the timeline, but we'll be adding tests to all the changes and we have separated some days to add missing tests for the existing code. Another point is to ask regularly to the team if they need anything from the API.

@jywarren @publiclab/reviewers @publiclab/soc

[jywarren]

Love the table!!! I'll review soon, at a meeting right now. Thanks!!!!

[jywarren]

First glance this looks amazing. There are a lot of redundancies and hard to read sections in the current API. I think addressing these will be something to keep in mind as you go.

[jywarren]

Also, perhaps regularly checking in with other contributors to see what API functions they need or want, OR what of their work could be expressed through the API. A lot of JavaScript functionality could be redactores to go to through a cleaner and better documented API.

Great work!

[jywarren]

One redundancy example is between TypeaheadService and SearchService -- could be much simplified!

[grvsachdeva]

hi, @stefannibrasil there is also an open issue for API which requires some help. The basic API for fetching nodes (taking lat,lon input ) is built by me in https://github.com/publiclab/plots2/pull/1987 . I wonder if you would be interested in solving this one? Thanks.

[jywarren]

I have been seeing some JSON tests that probably correspond to different things we could convert to more standardized API calls. Could be worth searching through the tests for the string "json" to find some and try to improve them, when you reach the end of your plan?

[stefannibrasil]

That's great, thanks for the comments! I've edited the Timeline to address your points.

@Gauravano Yes, that would be great, I can try to help. Could you please guide me through the actionable tasks and explain how can I help? You need help with the part 3 from #1934, is it right? if so, we can discuss there. Thanks!

[grvsachdeva]

hmm, that part is also remaining but I guess @sagarpreet-chadha is on it. I want help on improving the first part of #1934 , In that part, I build a new API for fetching nearby nodes to a location, taking lat,lon as input, but for keeping it simple at first we decided to not include tags in the input. But as this API worked properly, we need to change that API or can build a new one (any way you are comfortable). So, the new API could accept the tags too in input URL and then process accordingly. That would solve the https://github.com/publiclab/plots2/issues/2254 issue. What do you think? Thanks.

[grvsachdeva]

Here's the current URL https://github.com/publiclab/plots2/blob/46f3d43a73f727a547f737a401ae76411d3c2ef6/app/api/srch/search.rb#L118-L119 . We want to pass tags too and then process the input accordingly in https://github.com/publiclab/plots2/blob/46f3d43a73f727a547f737a401ae76411d3c2ef6/app/services/search_service.rb#L188 to pass output nodes from there.

[stefannibrasil]

Alright, that looks good, I will start working on it tomorrow. Thanks for the info, @Gauravano I'll ask any questions that I might have on the Issue page :)

[jywarren]

Congrats on merging #2790!

I wanted to note that one of your milestones above - "Refactor/Simplify SearchService and Search endpoints" -- sounds particularly great and that you may after that want to look at #987 for some ways that we're hoping search will be refined to show really relevant results. It's one of the biggest usages of the API, and a place you could really see some highly visible improvements to how people are able to find things around the site! A great application of your code refactoring and reworking of the API.

Thanks and it was great meeting today!!! 🎉 🙌

[stefannibrasil]

Thanks, @jywarren
we added that to our Todo. We were discussing our next steps and we will start on those endpoints next week! Stay tuned for questions :laughing: just kidding, but we are planning to at least reduce the code duplication for now. Yeah, it was great talking with everyone together. Cheers!

[jywarren]

Here are some issues that could be broken out this week!

• [ ] I think we may want to remove the auto-added asterisk on this line, and treat "boolean search" as an expert search type: https://github.com/publiclab/plots2/blob/master/app/models/node.rb#L39
• [ ] add a link to more info on boolean searches like http://booleanblackbelt.com/2008/12/basic-boolean-search-operators-and-query-modifiers-explained/
• [ ] default ordering of profile search results
• [ ] table-styling on https://publiclab.org/search/profiles/balloons/ to match https://publiclab.org/people (https://getbootstrap.com/docs/3.3/)
• [ ] table-styling on https://publiclab.org/search/tags/balloons/ to match https://publiclab.org/tags (https://getbootstrap.com/docs/3.3/)
• [ ] more explanatory text for what the Tags option on the search page is actually searching/displaying: https://publiclab.org/search/tags/balloons/
• [ ] more explanatory text for what the Profiles option on the search page is actually searching/displaying: https://publiclab.org/search/profiles/balloons/ (i.e. does it search people by their tags? Could we break out another issue for this?)
• [ ] search optimization for this query: https://www.skylight.io/app/applications/GZDPChmcfm1Q/recent/12h/endpoints/Srch::Search%20%5BGET%5D%20srch%2Fall
• [ ] explore a combined display of Notes, Wikis, and Questions since they're all Node types

These could all be broken out into individual issues, but don't worry - we can also preserve them as a list here and add the break-me-up label for others to help break them out!

[milaaraujo]

Hey @jywarren, I moved all your points to individual issues here: https://github.com/publiclab/plots2/projects/6

[stefannibrasil]

Moving the API planning/discussion to #3520