Boostnote: Google drive Integration

[Rokt33r]

Google drive integration

Specs

• Convert current data api to Google drive api
• Data will be stored to Application data folder
• User can store multiple storages to a single google drive account.
• Storage can be detached even the integration still exists.
• Each note is saved as a file just like current way.
• App caches the data for offline support
• Written data in offline is applied automatically as soon as the user get back online.
  
  
  
  • Follow last-write-wins rule just google drive does.
  
  
• Boostnote server will provides some kind of MQTT service to broadcast the change event to other devices.
  
  
  
  • This will be reused Gist as a storage and other storage types and mobile app.
    
    

    Why no just putting the data into Google drive folder?

  
  

This is because Google drive desktop sucks(same to mobile app).
Its refresh detection is too slow often. Also, it doesn't work linux as well.

FIY: other planned targets

• Dropbox
• One Drive
• Box
• iCloud
• S3

[twhiting]

Looks good 👍 I'm personally using boost with Google Drive & OneDrive so these features would def help me.

[Rokt33r]

@twhiting
Thanks for reviewing.
Btw, I want you to join our slack. If it's okay, please send me a e-mail and I'll send an invitation. [email protected]

[tejado]

Hi, how are you planning to do this integration? I guess you will do this over some clean interface so the storage can be easily changed (from code point of view)?
I like to contribute to add further remote/cloud storage interfaces.
tjado

[Rokt33r]

@tejado I'm still wondering am I doing right.
Quiver's solution is quite simple and easy.
https://github.com/HappenApps/Quiver/wiki/Cloud-Syncing
But, my solution is quite hard way. Also, there might be a few people who want my solution.

What do you think? which one is better?
@twhiting I need your feedback also!

[tejado]

Yeah, the solution of Quiver is really simple.
But direct integration would be much better in my opinion. The first remote-storage integration would be tough as it has to solve all the caching/correct sync stuff.
I would start simple and abstracting the current data api, so replacing it with a different one can be done without (much) code modifications (sorry if this is already the case, didn't look deep into the code). Automatic detection & reload of file modifications (like Quiver) should be then the next step as this is always necessary, or? Then an additional data api can be developed (e.g. google drive... I would start with something different).

[Rokt33r]

I already made the abstract api and its test.
https://github.com/BoostIO/Boostnote/tree/master/browser/main/lib/dataApi

Could you glance over it? I'm actually not sure this api is the best.

Another option is making an adapter of couch DB for Google Drive.

[twhiting]

@Rokt33r I'm in favor of an integration as well. It would help you scale in the future as well (if a mobile app was developed, access all your boostnotes from a cloud api).

Maybe implement a short term solution like quiver now and then work towards a full integration as needed in the future?

[tejado]

I will try to look on this in the next days!

@twhiting Yeah, this is what I wanted to say with my last post! :)

[Rokt33r]

@tejado @twhiting
Thanks for your feedbacks.
I was afraid if quiver's way would be enough. But, as you said, we should do it eventually.

Also, about https://github.com/BoostIO/Boostnote/issues/153, I think it is good to stick with current way(each note is saved as a file).

[akush]

Currently using Boost with Google drive and OneDrive with workaround. I would love to see native sync support.

[obedm503]

just adding a folder inside the Google Drive synced folder worked, but making it official would be nice

[kirtan403]

Maybe time to bring this feature up?

I see third party integrations coming soon in the app. It's been already 9 months since the issue is created.

Btw, It's been an hour since downloaded the app. And loving it so far 😄

[farebord]

I second that, @kirtan403

[duyfken]

So the iOS and Android apps have dropped #831 but they're not as useful when you can't sync your notes... so any ETA on the mentioned 3rd party cloud integration (and hopefully included encryption #827) for Boostnote at this stage to make these new mobile apps more usable?

[itzsaga]

Might be worth looking at integrating https://github.com/remotestorage/remotestorage.js to achieve the goal. Looks like a lot of the heavy lifting is done already with that.

[patrick-elmquist]

Really like Boostnote but not being able to sync data between Windows and Ubuntu is deal breaker.

[kirtan403]

I have tried with new google drive sync software and it works great with selected folders. But the problem is it doesn't refresh automatically. You need to refresh in order to load changes done from other pc.

[joeypedicini92]

@kirtan403 I'm doing the same thing, would be nice if it could somehow auto refresh or at least put a notification somewhere saying the note has changed on the filesystem, would you like to refresh?

[patrick-elmquist]

@kirtan403 yeah but that only works with Windows and Mac right? AFAIK there's no native Google Drive client for Linux (?)

[itzsaga]

@patrick-iv you could use google-drive-ocamlfuse. It's actively developed.

[kirtan403]

@joeypedicini92 I totally agree. Something like auto refresh would be really nice.

@patrick-iv I'm not sure there is a native google drive sync for linux.

[Redsandro]

Google changed to a new api (v3) in september 2017. There are some options that are multiplatform.

• Using mono: https://developers.google.com/drive/v3/web/quickstart/dotnet
• Using python: https://developers.google.com/api-client-library/python/apis/drive/v3
• Using node: https://developers.google.com/drive/v3/web/quickstart/nodejs
• __And the winner is:__ https://github.com/google/google-api-nodejs-client

@Rokt33r said:

Boostnote server will provides some kind of MQTT service to broadcast the change event to other devices.

_Please don't use a central server._ Allow the app to remain totally independent from a central server.

• Centralized server is always the first step to conversion to commercial software. Even when said software claims "Free Forever".
• Centralized software is not in the spirit of the Free Software Foundation
• Privacy concerns
• Server down, software down

There are other solutions for figuring out files were updated.

1. Sync with service (e.g. google drive) every _n_ minutes when open
2. Sync once on startup
3. On click-to-edit, first check if version on service changed.

[timrossback]

I'm afraid I've got to disagree with you, @Redsandro. This project is overwhelmingly no where in the direction of becoming a Freemium product. There are nth applications out there that offer dedicated cloud storage that aren't using a Freemium or Premium business model.

I'm also not aware of any official statements from the FSF saying that having dedicated/centralised storage is a no? If they did, they'd be condemning pretty much every single storage platform.

And with the privacy, I don't get your point? When you host using a third-party network, you're exposed to the same risks that you would be if you used a dedicated network, if not more. Not only does using a dedicated and centralised storage system mean that the policies are designed for Boostnote users, but it also means that you won't have another service to deal with.

Regardless, a lot of the current third-party options cost the more you store. Having a dedicated and centralised networks means that the free quota for whatever third-party option is free for your other files, while the Boostnote storage is free for notes.

[Redsandro]

@ItsPugle said:

This project is overwhelmingly no where in the direction of becoming a Freemium product.

There are always similar words from a stranger prior to "the change". I believe in empirical evidence. My once facorite ExtJS. Or Media Browser (now Emby). Subsonic. Pushbullet. Plesk. Moving towards a centralized server is a tell. Please don't do it. It's not necessary.

I'm also not aware of any official statements from the FSF saying that having dedicated/centralised storage is a no?

SaaSS

If they did, they'd be condemning pretty much every single storage platform.

They do. They promote self-hosting. They gave it some thought.

And with the privacy, I don't get your point?

Are you actually being serious right now? Maybe someone at Google can tell you to use Google Keep. It has a lovely Android app with location-aware reminders.

Having a dedicated and centralised networks means that (...) the Boostnote storage is free for notes.

Who's gonna pay for that server? What if the user base multiplies by 100? Nothing in life is free.

a lot of the current third-party options cost the more you store.

Actually there are more free space options than you would need in a lifetime to store notes.

• Dropbox 2 - 8 GB
• Google Drive 15 GB
• OneDrive 15 GB
• MEGA 50 GB
• StackStorage 1000 GB

(Except ofcourse they make money by collecting data. Such as your location every hour of the day.)

And my personal favorites, approved by FSF:

• OwnCloud (self-hosted)
• LibreCloud (self-hosted)

But that was just to respond to your questions. The single most important reason:

It's not necessary. Except for hoarding personal data. Please don't go that road.

[timrossback]

In referencing to your article, GNU is independent from FSF. They're merely a principal sponsor. Not only that, but that article was originally published with the Boston Review if you had read the subtext.

Another thing: Boostnote is, for the most part, community-led. All source-code is open source, contributions are welcome and issues are being opened daily.

Not to mention that everything you've listed are exactly that (minus the self-hosting, but it's really unlikely that you're gonna have the whole user base pay for their own systems). They're all proprietary platforms. Basically, it's generic (not designed for Boostnote - everything you've listed) vs dedicated (designed for Boostnote, and most likely, open sourced).

As with Google Keep... I use it daily. It's governed by the Google Privacy Policy. The same policy that dictates Search, Maps, Gmail - all of that. Boostnotes intention is to make notetaking easier for developers, not to make money (as is that of Google). Any policies should reflect that.

As the user base grows, so does exposure. Exposure attracts sponsors. Sponsors pay. Also, as you implied, Boostnote notes don't take up much space, keeping prices low. It's not exactly like you're hosting images or files on Boostnote.

"It's not necessary" isn't a reason. If you always say that stuff isn't necessary, your product will never grow.

Have you thought about the positives of having a dedicated, Boostnote cloud storage network?

For the record, I also don't appreciate your overly condescending tone. I'm not trying to pick a fight - just have a discussion.

[Redsandro]

You're not hearing my objection. This is not productive. I have stated my opinion. It was not directed at you. I rather discuss with people making the relevant code contributions. I'll leave it at that.

[timrossback]

I do hear your objections, and I respect them, I just don't agree with them. With all do respect, I'd hope you'd understand that their is more to any application, any platform, any digital thing than purely the people who write the code.

[Redsandro]

I can agree to disagree. I'm all for features that make other people happy, as long as I can 100% opt-out of any home phoning for features that have alternative solutions.

[rsimp]

@ItsPugle What does it matter if it's open source unless everyone has access to the server? You have no guarantee what code a server is running. That's not really an argument against doing it, but open source doesn't really factor in here. By running servers you're choosing to give your trust to the server administrators, there's no way around that.

I'm not necessarily against having servers but as @Redsandro pointed out, they are definitely not free. It also feels weird to me to have a server and not use it for storage. That design seems half way in, half way out and will lead to some really bizarre design choices. Its like a distributed system where you don't own or control certain elements, and in fact only the client even has access to them. If we run a server in this context it really needs to be unnecessary, where the app can run better with a connection but doesn't require one. It seems to me this should be the very last thing to bring in because we'd still need all the same mechanisms as if we didn't have it.

Some other considerations:

• Google expects you to poll (they have webhooks, but no websockets, long polling, or SSE) but has limits on the app even across all users: https://stackoverflow.com/questions/10311969/what-is-the-limit-on-google-drive-api-usage . The user limit is very nice, but the app wide limit will hit a wall with enough active users. I think the google team will raise it if you talk to them, but only if they think you're not being excessive
• Changes require a full re-upload of the entire file, they don't allow you to upload deltas. This means we need a debounce period that considers a worse-case file size/bandwidth scenario, at least 15-30 seconds. It helps that markdown files don't contain assets which limits their size
• Orphaned images/assets may need to eventually be addressed
• Some sort of mature conflict resolution needs to be eventually implemented. I think we could start out with overwriting on conflict, but should move to the dropbox model of renaming conflicted files as _filename (conflicted copy).ext_
• Syncing the file on open/close sounds good, although we'd still need sync polling and dirty tracking
• Due to the way searching works with tags I think we'd have to basically download complete notebooks and cache them, instead of lazy loading individual notes on demand
• Dropbox helps with long-polling and no app wide quota, so maybe we should knock it out first

[rodtwo]

For Desktop, I think the workaround is pretty simple to do - specially for a "developers" note app. I'm using OneDrive and it works great for me.

However, I downloaded the Android app and found it very basic - apart from Dropbox, no other integration, neither with workaround is available. This is the only thing holding me to still use OneNote

[DFelten]

Same here. I would like to use Boostnote on my mobile phone, but since there is only Dropbox available, this is not possible for me. Also, on my company it's not possible to sync Google Drive with the file system. Here I always have to sync Boostnote manually.

[ghost]

That's a great idea! It would be very useful for most of people!

[Flexo013]

308