Kolibri: Unable to import channel when all content already exists in previously imported channels

[lauradanforth]

I created a new channel with one resource as a test and it was a resource I had already brought in via another channel. I was unable to import the new channel because it identified the resource as already being on my device. See picture

v0.11, Chrome

[lyw07]

@lauradanforth looks like the picture uploading failed. Could you please upload the picture again? Thank you!

[lauradanforth]

whoops, fixed the image!

[indirectlylit]

@khangmach @jtamiace any thoughts on what to do here?

[khangmach]

@indirectlylit

In Kolibri, there are currently instances of duplicate resources from multiple channels. How did we allow more than one copy to exist if the we don't allow them to be imported in the first place? (in Laura's original image above they're greyed out). Are duplicates treated as separate resources with distinct file sizes? Or are they copies/instances of one file?

For example, if there are two copies of an identical PDF in Kolibri, one from Sikana one from Khan Academy, is that PDF one file shared between the two channels? or imported as two separate PDFs?

Functionally, this doesn't seem like an issue to me, but if we absolutely want a duplicate to be imported, just with different channel source metadata, we could allow the import, but calculate the import size as (0mb)

[indirectlylit]

Are duplicates treated as separate resources with distinct file sizes? Or are they copies/instances of one file? For example, if there are two copies of an identical PDF in Kolibri, one from Sikana one from Khan Academy, is that PDF one file shared between the two channels?

Duplicate resources are currently treated as instances of a single resource. Under the hood, they share files.

From the admin's perspective, they don't care that it's a duplicate: they want their learners to see that content in _both_ channels.

Here's an interesting related issue that shows the opposite problem: #4082. In that case, an item exists multiple times in _a single channel_ (unlike this issue).

From the admin's perspective, they don't care that it's a duplicate: they want their learners to see it in _only one_ topic.

Functionally, this doesn't seem like an issue to me, but if we absolutely want a duplicate to be imported, just with different channel source metadata, we could allow the import, but calculate the import size as (0mb)

I agree. I think the fix here is to hide our 'under-the-hood' de-duplication optimization from the user. From the user's perspective, every 'instance' in the library should be distinct.

@rtibbles I think this strategy would also help us with the "mastery within a lesson" issue we're trying to resolve. If we track master on a resource instance basis instead of a resource ID basis, those will become inherently distinct. We originally made a decision to "share" progress tracking across all instances of a resource but I think we're seeing that is not what users expect.

[khangmach]

Let me know if any new user-facing design on this is needed for this. I think this just requires users to see the resource/topic as valid for import and for the interface to show a filesize value of 0?

So this behavior would only apply to:

• Duplicate topics/resources in a channel Y, even if they exist in another channel on your device, say, channel X

[indirectlylit]

Yes, I think that's nearly right. I would clarify:

Users see the resource as valid for import and see a file size value of 0 for duplicate resources that already exist in another channel or another topic in the same channel on their device. Topics are similarly shown as valid for import if they contain duplicate resources.

@rtibbles correct me if I'm wrong but I think that in technical terms the solution is:

When computing content availability, use node IDs instead of content IDs. When computing sizes, continue using content IDs.

[lauradanforth]

Tagging @laurenlichtman to add additional context after observing a related issue earlier this year

[rtibbles]

When computing content availability, use node IDs instead of content IDs. When computing sizes, continue using content IDs.

We calculate availability based on whether the files needed to display this content are available. They are, so we flag them as available. This has nothing to do with the content_id/node_id distinction. Even if the two nodes had different content_ids they would both be flagged as available because they rely on the same files to render.

[rtibbles]

Seems like this is probably best solved alongside other aspects of importability annotation.

[rtibbles]

After discussing this issue with @jtamiace, we think this is an extreme edge case of #4082 - where here it is the entire channel that is being effectively imported without the user's say so.

Unbeknownst to the user, once the metadata has been imported, then this channel is now available, even though they only went to have a look. Even if they click the X to exit, the channel will now be available on the device.

[laurenlichtman]

@rtibbles let me know if you need more examples of this. I had started writing out this issue at one point but I never added it to GH...

[rtibbles]

I think the issue is well described at this point, and the important shift in behaviour seems to be making the display of content entirely up to the user's volition, and not just based on the availability of files.

[indirectlylit]

this is being addressed in ongoing changes in 0.13.0