Tutanota: Conversation view

[ghost]

Gmail has conversation view.

I have been looking for a privacy-conscious email service that offers this
as well. I have been using OpenMailbox and RiseUp, however they both
use RoundCube software.

RoundCube has had a ticket open on this for 9 years so who knows when they
will implement this, if ever. Please add conversation view so I can dump Gmail
for good.

[mpfau]

Steven, thanks for your input!

This has already been prepared but not implemented on the client. Only the GUI is missing...

I posted your idea on our uservoice:
http://tutanota.uservoice.com/forums/237921-general/suggestions/6820765-conversation-view

You can vote for it!

[ghost]

It has been over a month, any updates?

[mpfau]

Sure. It has been upvoted and is one of the most wanted features. We are going to implement it soon.

[armhub]

The conversation view is not implemented yet, but I closed this issue as we track it here: http://tutanota.uservoice.com/forums/237921-general/suggestions/6820765-conversation-view

[mpfau]

@cup I will add it to our roadmap...

[vladimiry]

I was exploring the possibility to enable conversation view mode as a part of this issue https://github.com/vladimiry/email-securely-app/issues/32#issuecomment-420379320 and so far it looks feasible. Tutanota maintains the conversation entry and email message entities, which are linked with each other using one-one reference. The conversation entry entity is needed to construct the tree-like conversation view mode based on its previous/parent property. But the thing is that backend removes the conversation entry in along with removing the corresponding email message, which makes the conversation consistency broken. Building the desktop app I can get around that with a simple trick of never removing the conversation entries, just ignoring removing from the local database when Tutanota does that. @charlag would appreciate any other ideas for handling the possible conversation consistency breaking issue.

[charlag]

@vladimiry we've discussed conversation view yesterday, it shouldn't take too long on our side because backend is mostly there. Thanks for pointing out this issue, maybe we should make some changes on the backend for this to work, we will discuss it and will try to get back to you soon

[vladimiry]

I was not fully correct writing that backend just removes the conversation entries. Now I see that backend sets the entry.conversationType value to 3 if there are remaining messages in the conversation, and removes all the linked conversation entries only when the conversation gets empty. Such scenario allows preserving the conversation tree consistency, so there is no issue. I guess value 3 means removed? It's not yet listed here
https://github.com/tutao/tutanota/blob/b689218e6bae45bb38cfef7929494c708aa0f252/src/api/common/TutanotaConstants.js#L148

[charlag]

@vladimiry yup, that's just what I learned from the team
It is
NEW REPLY FORWARD UNKNOWN // a message for which no mail exists in Tutanota (unknown external mail or deleted mail)

we will update the definition

[vladimiry]

If someone wants to try it in action, the first beta of a desktop app with conversation view mode of the local database explorer has been just published.

[gabrielmip]

@vladimiry How independent is your work from the main Tutanota repo? Are you planning on submitting a PR here with the conversation view? Thanks!

[vladimiry]

@gabrielmip the code of the desktop app I've been building is not designed to be submitted to the Tutanota repo. Besides, regarding the desktop app, there is an essential piece of work that is happening on the backend, which in the case of Tutanota is not open-sourced (not a part of this repo). The backend task is preparing the data for displaying in the conversation mode and I guess Tutanota team will need to adjust the server-side code first, ie the API. If you are interested in trying the desktop app, I'd recommend taking the latest beta from this page, it works quite stable despite of the beta status.

[charlag]

@vladimiry would you mind sharing which part we would need to adjust? Conversations are recorded correctly afaik.

[vladimiry]

@charlag, this is what I've found out so far. There are two linked between each other entities, Mail and ConversationEntry. ConversationEntry is needed to construct the conversation tree. It's a common thing for the conversation view mode to initially display only the single most recent mail of each individual conversation with an option to expand the conversation. So there is a need in a sort of aggregation function that constructs the conversations based on the Mail and ConversationEntry entities, then it sorts the emails in each conversation and picks the most recent messages and sorts the final list based on these most recent messages. So the outcome of this function would be a list of the most recent emails in conversation plus some metainformation, like conversation size at least, so you know on the client side that conversation is expandable. Besides the final list will need to be reactively constructed and re-displayed on emails creating/removing events. It might be possible to do such kind of aggregation on the client side, but it would probably be more efficient to perform it at the server side. I might be missing something since I've explored only the Tutanota's client-side code and only to the extent needed to fulfill the desktop project building task.

Conversations are recorded correctly afaik.

Conversation view mode is supported for about 2 months already by the mentioned before desktop app (since v2.0.0-beta.1) and conversation is a default view mode since v2.0.0-beta.6. I have not noticed so far any issues with the conversation structure recording and so I confirm that conversations structure is well recorded.

[charlag]

@vladimiry thanks for sharing!
I think it's already does all the mapping. Mail has a reference to conversation entry (which is in a list). Conversation entry has a reference to the mail and to the previous conversation entry and there are some other parts I don't understand yet.

[vladimiry]

I think it's already does all the mapping.

It does the mapping, but the data convenient for displaying as the continuous feed is a different thing.

[vseshu]

I downloaded the AppImage for the linux Desktop client and I'm not seeing any option to activate conversation view? And my emails aren't showing up in conversation view? help?

[charlag]

@vseshu as you can see, the issue is open and on the roadmap. It means that it's only planned.

[vladimiry]

@vseshu, conversation view mode at the moment is only supported by the unofficial desktop app link to which was provided above in the thread.

[vseshu]

@vladimiry , thanks for the clarification. Between all of the different sources of news and info on this topic its been a bit confusing. Do you have a signature file for those app links? Is there a way to check the packages? Thanks!

[vladimiry]

Currently, the only way to verify installation packages consistency is comparing checksums as described in release notes of this release. The most recent release at the moment is v2.2.0. If you have more questions related to that app it would be better to raise them in the respective repository as it's irrelevant to discuss such stuff here.

[pukafever]

I've been looking forward seeing the conversation view implemented. It's now 5 years that this issue exists, do you plan to add it soon ?
To me, it's one of the most critical drawbacks.

[unrockbar]

Does not look like this feature will be released any time soon 🤔

[pukafever]

It seems, indeed. Time to look the code and figure out how to implement it. I think I'm going to give it a shot, submit a PR and hope the team accepts it!

[ptrwllrd]

This is what they said three days ago:

Separate window and conversation view are planned for the coming months.

https://www.reddit.com/r/tutanota/comments/e0459t/just_got_myself_a_protonmail_account_for/f8byydz/

Fingers crossed it's 1-2 months, not 9-11 ;)

[mpfau]

@Gaspacchio just let us know if you have any questions!

[pukafever]

It seems, indeed. Time to look the code and figure out how to implement it. I think I'm going to give it a shot, submit a PR and hope the team accepts it!
https://github.com/tutao/tutanota/issues/6#issuecomment-558574053

OK, so I got excited. Implementing conversation view is kind of a huge task I'm still figuring out what is responsible for creating the mail view and it will probably take some time. I'll submit a WIP PR to keep you posted about my progress as soon as possible.

[vladimiry]

@Gaspacchio if you are interested, conversation view is implemented here in local database/store view mode for more than a year. But Tutanota-related stuff is going to be dropped soon. Ask questions in respective repoitory if needed.

[charlag]

@vladimiry I think it could be extremely helpful if you linked to the place it is implemented in your app.

[vladimiry]

To present the mails as a conversations list there is a need to first group the mails by conversations, then sort mails in each conversation by mail.date and then the last mail of each conversation will be put in a list for displaying (so conversation.date = <newest mail in the conversation>.date). Then all the items in the conversations list need to be sorted by that conversation.date property. It's not so easy to do as Tutanota preserves the tree/parent-child-like conversation structure. So I handled the described stuff in the case of the desktop app as I have all the data loaded in memory but the web/in-browser version can't act the same way as the data is being loaded by portions (pagination/infinite scrolling). This is why I wrote before in this thread:

The backend task is preparing the data for displaying in the conversation mode and I guess Tutanota team will need to adjust the server-side code first, ie the API.

Attaching animated explanation with one conversation consisted of 5 mails

The app initially supported tree-like conversations items displaying for Tutanota but then such rendering got dropped, so just plain list:

@vladimiry I think it could be extremely helpful if you linked to the place it is implemented in your app.

Pointing to the code but it's not like a single code snippet and it's desktop-app specific:

• The app loads all the entities (mails/conversations/folders/contacts, bootstrap fetch first and then reactive syncing, details described here) and saves them in the local database and keeps it in memory. Data fetching code is here https://github.com/vladimiry/ElectronMail/blob/463ef7bb7b18aff58f8747ae35437dcbbe324347/src/electron-preload/webview/tutanota/api/build-db-patch.ts.
• Then there is logic that handles the described above stuff so it can be treaded as a backend API, located in https://github.com/vladimiry/ElectronMail/blob/463ef7bb7b18aff58f8747ae35437dcbbe324347/src/electron-main/api/endpoints-builders/database/folders-view.ts (forming virtual conversations cycle: https://github.com/vladimiry/ElectronMail/blob/463ef7bb7b18aff58f8747ae35437dcbbe324347/src/electron-main/api/endpoints-builders/database/folders-view.ts#L167-L192).
• And finally, there is a UI part https://github.com/vladimiry/ElectronMail/tree/463ef7bb7b18aff58f8747ae35437dcbbe324347/src/web/browser-window/app/_db-view

PS If someone wants to try the app in action, the local store feature doesn't work anymore in released version (you will get body must be a string validation error, which is expected to happen as data model got changed at the server/Tutanota side) and won't be as the functionality is deprecated and planned for dropping. But the link to the working installation package can be found in this message or the package can be compiled from the source code (master branch).

[HarlemSquirrel]

Hi all, is there any update on this? I've moved from Gmail and this would really be a nice feature to have.

[charlag]

I have a working prototype but it's not decided whether it will be the final version or not.