Wp-calypso: Media & VideoPress: Calypso Storage Space Indicator Breaking on Atomic Sites, VideoPress videos counting against storage

[JoshuaGoode]

Atomic sites that upload videos to VideoPress are having those videos count against their storage limit in Calypso. Eventually, the user may reach "100%" of 200GB while their site's wp-content directory is only a few GB. This doesn't prevent the user from uploading new videos, however.

Additionally, videos deleted from media libraries contribute to storage_used_bytes and linger (please see p9o2xV-Qo-p2 ).

Background

Recently, @jmdodd has reviewed issues with videos and VideoPress storage -- especially on Business sites and Atomic Business sites. This can be reviewed internally at p9o2xV-Qo-p2

My encounter with the issue is documented internally at p9F6qB-57O-p2

Specifically, a user's site is only using 2.1 GB on Atomic but Calypso reports "100%" utilization of their 200GB storage.

GET v1.1/sites/$site/media-storage returned the following:

max_storage_bytes: 214748364800
storage_used_bytes: 504458228723

A test video was then uploaded and deleted which increased storage_used_bytes but didn't decrease it when the video was deleted. This result was expected based on @jmdodd's findings on p9o2xV-Qo-p2

Steps to reproduce

To see the discrepancy between Calypso and Atomic, you can upload a video to an Atomic site via VideoPress, delete it, upload more..etc and compare the site's storage utilization under Site Health. Then, compare it with Calypso's storage_used_bytes.

To reproduce the situation where Calypso claims the user is out of storage space while they are not:

1. Create or observe a WordPress.com Atomic site
2. Upload 200GB worth of media to the site. Make sure a portion of that is VideoPress videos.
3. Check the site's storage utilizing in WP-Admin > Tools > Site Health > Info > Directories and Sizes or another method.
4. Check Calypso's storage utilization indicator under My Site > Site > Media
5. Use the Dev Console and run GET v1.1/sites/$site/media-storage to check the reported storage utilization that Calypso is using.
6. Delete video and check again. Review status of the video on VideoPress.

What I expected

• VideoPress videos that the user deletes are subtracted from the site's storage utilization.
• But, moreover, VideoPress videos to not count against the storage

What happened instead

• Calypso is counting VideoPress videos against the site's storage utilization.

• VideoPress videos deleted from the site's media library are not completely deleted and still attribute to storage_used_bytes

Context / Source

Internal:
p9o2xV-Qo-p2
p9F6qB-57O-p2
p1587666491353000-slack-atomic
2899651-zen

user-report #dogfooding

[lancewillett]

Removed the "High" label as this isn't a critical issue with security or data loss.

Would you be willing to clarify what you think is the next action to fix this? Is it changing a label in the UI, or something else in the backend?

[JoshuaGoode]

Thanks for checking in @lancewillett

Potential next steps:

• Adjust whatever is behind storage_used_bytes to subtract VideoPress videos from total so that the UI reflects actual usage.
• Or, don't count VideoPress uploads towards the bytes calculation at all on Atomic sites.

Another item, which may need its own issue, is regarding deletions. When a video is deleted from a media library, it should be subtracted from storage_used_bytes and completely removed.

Related internal notes:
p1588166113195000-slack-triage
p9o2xV-Qo-p2
p1HpG7-7W4-p2

[jmdodd]

I think it would be better to remove space-used indicators from Calypso for Atomic sites, as these do not differentiate between file space used pre-transfer and VideoPress space used post-transfer and are very confusing to the customer. WordPress.com-on-Atomic sites have accurate space indicators for their file space on Atomic in their site health pages; Jetpack sites do not seem to have an indicator for the amount of space used in VideoPress.