Mattermost-server: [Help Wanted] [MM-12464] [Bulk Export] Include DM/GM Channels and Their Posts in the Bulk Export

[esethna]

If you're interested please comment here and come join our "Contributors" community channel on our daily build server, where you can discuss questions with community members and the Mattermost core team. For technical advice or questions, please join our "Developers" community channel.

New contributors please see our Developer's Guide.

---

Notes: Jira ticket

This ticket forms part of the Bulk Export campaign. The bulk export feature (implementation in app/export.go and app/export_converters.go is intended to provide a CLI-based means to export the data in a Mattermost instance to a JSONL file. It is the inverse of the much more well developed Bulk Import feature, so a data file exported from the Bulk Export feature should be importable with the Bulk Import feature.

Currently the Bulk Export only includes channels, channel memberships and posts for Public and Private channel types. The objective for this ticket is also to include Group Message and Direct Message channel types. For inspiration on how to achieve this, take a look at how the bulk exporter currently works for Public/Private channels (app/export.go and app/exporter_converters.go are good starting points), and consider building out analogous functionality for the GM/DM channel types. Note also how the bulk importer treats these channel types, including the fact that they use different data structs to public/private channels (see app/import_types.go structs DirectChannelImportData and DirectPostImportData).

In order to keep this ticket manageable, you should ignore all but the most basic fields on the objects. You should ignore the FavoritedBy field on the channel and the FlaggedBy/Reactions/Attachments fields on the Posts. These will be handled separately in subsequent tickets.

[wejick]

I'll try to pick up this

one question for now
Is GM considered the same thing as DM? Quick check on import.go, couldn't find any reference to group.

update : nvm, I found there is member array in DirectChannelImportData

[wejick]

I stumbled upon this import validation.
https://github.com/mattermost/mattermost-server/blob/fe9a81208e4d8290df7b8d89bac2d880c045b84b/app/import_validators.go#L447

Here you can see that direct channel with less than 2 members will be rejected. But I found that we can have single member channel (the you channel)

How do we want it to be handled?

1. Don't export single member Direct channel
2. Change the import validation to support single member direct channel

[wejick]

After that I want to discuss about post export, the current code is to only export and import post from channel with team id.
I'd like to know how we prefer to handle this before I proceed to the implementation :
Export

1. Change the current query to export all post
2. Create new query to handle this

I prefer to use current query and change the INNER JOIN to LEFT JOIN for Teams table.

Import
Is it ok to remove this validation?
https://github.com/mattermost/mattermost-server/blob/fe9a81208e4d8290df7b8d89bac2d880c045b84b/app/import_validators.go#L403

[grundleborg]

@wejick That's an excellent question! When this was implemented, you could not DM with yourself, but now you can. I'm not sure immediately what the correct thing to do here is. Maybe it needs to be addressed in a separate ticket.

And as for you latest question, let me think for a minute then I will reply :)

[hanzei]

This is open for the public again.

[fcorrea]

I'll take this one

[hanzei]

Great, thanks @fcorrea! :tada:

[hanzei]

Hey @fcorrea,

I just want to check how your work on this going. Do you have any questions?

[fcorrea]

Hey @hanzei! Actually, this is almost merged https://github.com/mattermost/mattermost-server/pull/10421

[hanzei]

Got it. I've linked your PR to this issue. Hence it will be auto closed when your PR gets merged.