Community: Standardize + Publish Meeting Recordings

Created on 4 Sep 2019  ·  94Comments  ·  Source: open-telemetry/community

Currently, SIG meetings are taking place on a variety of accounts/platforms such as MS Teams, Zoom, etc. For meeting owners who are part of larger organizations, often these recordings are blocked from being shared outside of the org that created them, making it difficult for people to catch up.

We need to figure out a way to address this - it seems like maintainers/meeting hosts should have the ability to create meetings through a CNCF Zoom account, recordings of those meetings (or at least 'important' ones - spec, community, etc.) should be hosted somewhere stable like a YouTube/Vimeo channel, and so forth.

Most helpful comment

Hi all, just updated all videos to the format specified. Created some playlists based on the language to facilitate. Still have a lot of videos that have just the "meeting" title (still didn't have time to update all)

All 94 comments

@austinlparker I agree – this is not a good situation. AFAIK, we still don't have sharable credentials for a CNCF zoom account (per https://github.com/open-telemetry/community/issues/105)... that seems like the obvious first step to me.

I will ask about this on a governance committee call that happens to be scheduled for tomorrow AM.

We do have credentials for OpenTracing zoom via CNCF, we can use those.

@mtwo can we configure the OpenTel zoom to auto-record all meetings (at least by default)?

If you share the credentials with me I can take a look :)

(there will still be some manual effort involved with moving the recordings to a central location, as zoom only retains them for 1y)

Probably would be pretty straightforward to write something that pulls recordings out every day and uploads them somewhere.

I think that they're autorecorded by default but I haven't had a chance to play around with them yet. @bhs, I'll send the details over chat now.

@bhs sent you the details over email

I'll have to do the calendar transition tomorrow - turns out that testing zoom meetings over in-flight WiFi doesn't work out so well

Ok, I've moved over the meetings that don't overlap. The CNCF gave us additional zoom meeting links for overlapping meetings however these are currently bugged (they require a host to join but we don't have control over the host accounts). I've asked our CNCF reps to reconfigure the zoom links and will update the remaining meetings once this is complete.

Here is how zoom shows it. This recording will only be available for a short period:

https://zoom.us/recording/share/W13ypDWcmPtAeGdQAKNNWvXnFg2HWEpgnAqh03sJNkywIumekTziMw

We can upload to Youtube like this:

https://youtu.be/dPfUf-ky6Fw

I'd vote for youtube, let me know if anybody has concerns

I definitely prefer youtube if it's not too much hassle... I looked for
some sort of tool/plugin that automates that upload but didn't find
anything. :-/

On Tue, Sep 24, 2019 at 10:28 PM Sergey Kanzhelev notifications@github.com
wrote:

Here is how zoom shows it. This recording will only be available for a
short period:

https://zoom.us/recording/share/W13ypDWcmPtAeGdQAKNNWvXnFg2HWEpgnAqh03sJNkywIumekTziMw

We can upload to Youtube like this:

https://youtu.be/dPfUf-ky6Fw

I'd vote for youtube, let me know if anybody has concerns


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/open-telemetry/community/issues/169?email_source=notifications&email_token=AARFWIKK57Y754B5WDVFA3DQLLZHLA5CNFSM4ITRUUU2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD7QUYCY#issuecomment-534858763,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AARFWIPWNNX2GI4NUWU2TUDQLLZHLANCNFSM4ITRUUUQ
.

zoom also generate a transcript. Would it be useful to publish it? And if so - where? Looking for suggestions

@SergeyKanzhelev re

zoom also generate a transcript. Would it be useful to publish it? And if so - where? Looking for suggestions

I didn't realize that (I mean, that it generates a transcript)... have you ever looked at one? (I.e., is it usable???)

This whole issue (of published recordings) seems easy to describe but difficult to enforce. I think i would be pretty impressed if we can just get the videos to end up in the right place reliably; if we get that far, then tackle automatic transcripts? (I am just trying to avoid creating a lot of extra steps for SIGs)

It would probably be pretty straightforward to have a script that runs every day, grabs all the meeting recordings, and uploads them to youtube. I could whip something up in a day or two I reckon.

Even better, looks like Zoom has webhook support, so we could do something that'd trigger off a recording being completed and upload it in response to that. Let me poke at it for a bit.

@austinlparker we have some recordings when people just come and leave the room. Like under the minute visits. Ideally it should be published as a draft and then manually can be approved. Also it would be great to have a link to the meeting notes under the video. Can be taken from the meeting in google calendar

Great ideas! I'll incorporate those ideas.

@austinlparker any update on this? I'm uploading everything manually now and automation will save me a lot of effort =)

I still don't have access to the zoom account, let me bug some people.

@austinlparker I got the accounts.

Can you email me with the info? (austin at lightstep.com)

Sent details over the e-mail.

As a note for anyone following this thread - the issue is becoming rapidly complicated as it seems that our zoom accounts aren't able to turn on webhooks, and CNCF IT support is saying they're not able to enable it either.

Yeah, seems like the CNCF accounts don't have webhook access so I need to go back to the drawing board for a minute here.

On further investigation, it appears the CNCF accounts can't do _anything_ with the Zoom API, so we might be back to square one here.

@austinlparker I have a question - can you come up with the script that will do the upload part of the job. I can download a recording into a specific folder, it's name will be GMT20191010-165815_cncf-opent_1920x1028.mp4. It's a date first (Oct 10, 2019 09:58 AM Pacific Time (US and Canada)) than time and than some random things.

Script can pick up this file and upload to Yotube. Description can be taken from the public calendar based on date.

Sure - is the time the time of the meeting or the time of the download?

I believe it's a time of a meeting start.

I've been poking at this in the background for a month now and chatting with folks, just wanted to summarize where we're at for posterity's sake.

  • The core problem we're facing is that it seems extremely unlikely that the CNCF Zoom accounts will ever have API or Webhook access enabled. This isn't a problem for just us, it's literally every CNCF project.
  • I spoke with some folks in various k8s SIGs who record to YouTube - they actually have an interesting setup with a separate small PC that joins the meetings and handles recording/uploading/streaming, but it requires human intervention. Someone has to have the box, run some scripts, etc. I don't think this is terribly suitable for our purposes.

I think there's a solution, though. It should be possible to automatically download the recordings from the actual web page through browser automation/html scraping, and then process + upload them through that. It's kind of a brittle solution (if Zoom changes their page, then it breaks the scraper), but it might be the best one we've got that doesn't involve additional hardware or human intervention (i.e., a script that someone could feed the download link into).

@austinlparker it will be a great start to have a script that uploads videos from specific folder based on its name. Then it will be just a few minutes procedure to download videos. Not terrible at all. Html scrapping is a nice addition.

I kinda wonder if we should just pay for otel.zoom.us or whatever... this is all feeling a bit cumbersome, isn't it? I don't fully understand the T&C's for Zoom, but if it's allowed to share a certain number of dedicated meeting rooms and thus avoid having to pay a per-user fee for every SIG member (which would be too $$$$), I think it might be net better than using the CNCF zooms.

Thoughts???

As discussed in the governance meeting, I have the action to ask the CNCF about getting API access. If they don't have a way to provide this, we'll explore paying for our own Zoom account(s) that have API access.

Something I just discovered this morning while looking for a recording, it appears that all of the recordings are actually publicly viewable. You just need to log in to the recording and click 'Copy shareable link'.

image

As a workaround, could we give maintainers/SIG owners the credentials to whatever zoom account hosts their meetings, and they could have an AI to log in, rename the meeting recording appropriately, and put the shareable link in the meeting notes?

I like this solution as it avoids us transcoding all of the videos to Youtube, and I don't think that the discovery that comes with being listed on Youtube is something that we take much advantage of anyways.

@SergeyKanzhelev what do you think?

there is a limited time video will be available on Zoom. Better than nothing. But may not overweight the problem of credentials exposure and potential for something to be broken by it. Why not give Youtube permissions to everyone than for upload?

I believe the recordings are stored for six months on zoom.

@austinlparker did you check the size limits we have? I remember seeing those and they were quite small. Definitely wouldn't last for 6 month with our stream of meetings

I don't see a size limit anywhere, where did you see that at? From a quick glance, the first meeting recording I see on the 'main' zoom account is from 9/20/19.

It looks like we have two OpenTelemetry YouTube channel which videos have been uploaded to. I think it would be best if we could consolidate these into a single channel.
https://www.youtube.com/channel/UCHZDBZTIfdy94xMjMKz-_MA
https://www.youtube.com/channel/UCVF98rxLfV3sv8Oi4fcJ1GQ

Have there been any steps forward in automating uploads to YouTube, or at least a series of steps each SIG should follow to do this consistently?

Apparently the Kubernetes SIGs have a new solution to this. We need to take action on this, in addition to the two YouTube channels. I likely won't have time this week but I'll attempt to get on this next week.

@mtwo let me know if you need any help, I can maybe repurpose some things from when I was tinkering on this.

Adding this for my and Austin's benefit. Kubernetes is apparently using splain.io and are happy with it. The CNCF is willing to pay any bills incurred from this.

Works for me - I think all we'd need is for a CNCF admin to authorize the
application on our Zoom accounts?

On Thu, Apr 23, 2020 at 4:51 PM Morgan McLean notifications@github.com
wrote:

Adding this for my and Austin's benefit. Kubernetes is apparently using
splain.io and are happy with it. The CNCF is willing to pay any bills
incurred from this.


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/open-telemetry/community/issues/169#issuecomment-618662806,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AA7S5RCHAYR54BR5667UD2LROCS6TANCNFSM4ITRUUUQ
.

We're going to have to start this today

Checking in on this -
Which of the two OpenTelemetry accounts is the right one that we should be connecting with?
Happy to enable the zoom accounts to connect, just need clarification.

https://www.youtube.com/channel/UCHZDBZTIfdy94xMjMKz-_MA appears to have more subscribers, so let's use that. @lizthegrey I _think_ that you created this account? If not, do you know who has access to it?

If I can help get access to that, we can get up and running with Splain on this.

Kubernetes guidelines are here: https://github.com/kubernetes/community/blob/master/communication/youtube/youtube-guidelines.md#automation

@amye send me your gmail e-mail (or whatever I can give permissions to Youtube to).

Also there are couple questions:

  1. How Splain will know how to name the video? If it can take it from google calendar- it would be great. Otherwise, it will be great to have a name of a room AND timestamp as part of the name.
  2. There are a lot of 10 seconds videos when people connect to the meeting by mistake. Ideally those should not be uploaded.

We'll make everything that Splain uploads private, so that nothing gets published on accident.
I know we have a number of Zoom accounts for this project, which account should have Splain on it?

Is it possible to enable Splain with all three Zoom accounts? Our SIG meetings are spread across all of them, as some SIGs have overlapping meetings each week.


From: Amye Scavarda Perrin notifications@github.com
Sent: Thursday, April 30, 2020 2:26:54 PM
To: open-telemetry/community community@noreply.github.com
Cc: Morgan McLean morgan_mclean@hotmail.com; Mention mention@noreply.github.com
Subject: Re: [open-telemetry/community] Standardize + Publish Meeting Recordings (#169)

We'll make everything that Splain uploads private, so that nothing gets published on accident.
I know we have a number of Zoom accounts for this project, which account should have Splain on it?


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fopen-telemetry%2Fcommunity%2Fissues%2F169%23issuecomment-622123105&data=02%7C01%7C%7C66e88ee62b4c459da94408d7ed4d3471%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637238788148033561&sdata=BulIXwPzjj0hVwHnJmpSijq%2F1JpIWhlnYHwCUFiQsRA%3D&reserved=0, or unsubscribehttps://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAAIXLK3QW2GP4WBTIDYJCA3RPHUJ5ANCNFSM4ITRUUUQ&data=02%7C01%7C%7C66e88ee62b4c459da94408d7ed4d3471%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637238788148043549&sdata=kXzaCOFyKFBPzTbx5j987qD2pBzXQ5JQOhpnjgeVs3o%3D&reserved=0.

Splain is now enabled on all three accounts, but they are set to private upload.
Let's see how this goes for next week, we should start seeing auto-uploads now.

Ok, it does seems to be partially working. Problems:

  1. Video failed to upload, see screenshot below.
  2. Video captions doesn't have timestamp in a name. So we cannot match it to the actual meeting from the calendar.
  3. nit: perhaps those 25 second videos can be ommitted for upload

image

I believe the meeting failed to upload is this:

image

I've sent over the settings area for Splain, Splain shows that the video uploaded to YouTube. This error looks like a YouTube error.

This video worked fine:

image

I published it. I was guessing what meeting it was though. Let's see if that processing error will re-occur.

Out of two videos today - 1 is failed to be uploaded... again...

image

@amye do you have some way to file a support ticket with Splain?

In progress now. :)

On Tue, May 5, 2020 at 2:01 PM Sergey Kanzhelev notifications@github.com
wrote:

Out of two videos today - 1 is failed to be uploaded... again...

[image: image]
https://user-images.githubusercontent.com/9950081/81115612-da040c80-8ed8-11ea-9358-e799e002ae08.png

@amye https://github.com/amye do you have some way to file a support
ticket with Splain?


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/open-telemetry/community/issues/169#issuecomment-624304015,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAA2FGNDL2F575MSQGLAQODRQB5DLANCNFSM4ITRUUUQ
.

--
Amye Scavarda Perrin | Program Manager | [email protected]

It's getting worse:

image

This is really starting to look like a YouTube timeout, no response yet from Splain. @lucperkins, other ideas?

Has anyone experimented with the OSS tool?

Splain responded saying that there was a configuration issue but it should be fixed. Any better news, @SergeyKanzhelev?

still bunch of failures:

image

Splain reached out and said there's a known bug they're working on:

  • "Issues in uploading to Google (YouTube, Google Drive and Shared/Team
    Drive) because of the ‘time out’ error even after retries. We are trying to
    find out a workaround in out code. (mostly during week days)"

I will update this as I hear back from Splain

On Tue, May 12, 2020 at 3:28 PM Sergey Kanzhelev notifications@github.com
wrote:

still bunch of failures:

[image: image]
https://user-images.githubusercontent.com/9950081/81751901-3d57e680-9465-11ea-8ede-ab51e22ae506.png


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/open-telemetry/community/issues/169#issuecomment-627632598,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAA2FGIR3AUNIOY2BRBF6P3RRHER7ANCNFSM4ITRUUUQ
.

--
Amye Scavarda Perrin | Program Manager | [email protected]

Those red uploads keeps happening... Loosing a lot of recordings.

The recordings should still be in Zoom and can be manually uploaded.

On Tue, Jun 16, 2020 at 12:44 PM Sergey Kanzhelev notifications@github.com
wrote:

Those red uploads keeps happening... Loosing a lot of recordings.


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/open-telemetry/community/issues/169#issuecomment-644974919,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAA2FGKP26DKHZFIRS4ZP5DRW7DSRANCNFSM4ITRUUUQ
.

--
Amye Scavarda Perrin | Program Manager | [email protected]

@amye definitely. I just noted it FYI if you will consider Splain for other projects.

Perhaps I can post a message on readme suggesting that if specific meeting recording is needed so it can be specifically requested. It'a quite time consuming fishing for the meetings in zoom. Thank you!

Oh completely understood! I just didn't want you to think they were gone
forever!
Splain has said there are some current timeout issues going on, and I
suspect this is the outcome of that.

On Tue, Jun 16, 2020 at 12:59 PM Sergey Kanzhelev notifications@github.com
wrote:

@amye https://github.com/amye definitely. I just noted it FYI if you
will consider Splain for other projects.

Perhaps I can post a message on readme suggesting that if specific meeting
recording is needed so it can be specifically requested. It'a quite time
consuming fishing for the meetings in zoom. Thank you!


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/open-telemetry/community/issues/169#issuecomment-644981707,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAA2FGPYBV64KPMK2PVZ3HDRW7FL3ANCNFSM4ITRUUUQ
.

--
Amye Scavarda Perrin | Program Manager | [email protected]

Just wanted to point out that the videos being uploaded on the YouTube channel have unfriendly (meeting-date-only) titles since over a month now.

opentelemetry youtube videos screenshot

yeah, with the setup we have it's a bit of a manual labor to make proper titles. There is a request to Splain to include the recording timeframe, so at least some automation may be created. Also there is a percentage of recordings being lost due to some timeout issue between Zoom and Youtube by Splain. For now, c-span of OpenTelemetry is the best we managed to come up with. We are open for ideas.

Are we able to rename files that have been published? If not, I think we should go back to pending publish and create weekly task for each maintainer group go and update the video title & publish.

The current format of just having a meeting time isn't great and does not help people find these recordings. If anything, it harms the legitimacy of what we're trying to by doing it in a bad way. We certainly wouldn't want to advertise these recordings as they are.

Are we able to rename files that have been published?

Yes, easy to change the title. @andrewhsu helped before by commenting:

image

If not, I think we should go back to pending publish and create weekly task for each maintainer group go and update the video title & publish.

I believe this is how it works generally in k8s. I'm noticing that maintainers are not quite responsible all the time and videos got published with the huge delay in a batch. I have also asked couple times different set of people to help - not many agreed, nobody stayed for a long time. Happy to pass along this duty.

The current format of just having a meeting time isn't great and does not help people find these recordings. If anything, it harms the legitimacy of what we're trying to by doing it in a bad way. We certainly wouldn't want to advertise these recordings as they are.

I think this is a judgement call between having recordings at all and having them as they are today. So at governance committee meeting where I raised this issue we decided that having transparency and recordings is better than delays and no recordings at all. But we are definitely open for ideas how to improve the process

This is how the process looks like today:

  1. Splain published meetings to YouTube as private.
  2. Open YouTube studio and filter videos by Visibility: private.
    image
  3. Delete all videos that are shorter than a few minutes. Those are random connects to zoom meetings. As recording is automatic, we just record people joining and hanging up.
  4. For all the rest videos: rename to the <yyyy-mm-dd> meeting. If it's known what this meeting is about - rename to <yyyy-mm-dd> <Actual meeting name>.
  5. Change visibility for the meeting to public.

I think those steps can be automated so if anybody want to help - please let me know.

Hi @SergeyKanzhelev , i would like to help!

@eddynaka, I will make you an editor. Would you like to clean up the current backlog as a starting point?

@eddynaka, I will make you an editor. Would you like to clean up the current backlog as a starting point?

sure!

Hi all, just updated all videos to the format specified. Created some playlists based on the language to facilitate. Still have a lot of videos that have just the "meeting" title (still didn't have time to update all)

This is impressive how many videos you identified. It's a lot of work.

Ping me when you are ready to discuss the automation.

Hi all, do we have a calendar with all meetings? We have a lot of videos that I couldn't distinguish what SIG was that.

Thank you.

Probably a tough question: do we have a list of participants/maintainers, for example:

  • .NET SIG: cijo, reiley, me, others
  • Java SIG: a, b and c.

Each SIG maintains a list of approvers/maintainers on their respective repo's READMEs. The codeowners file is also accurate for approvers/maintainers.

There are also GitHub teams for both approvers & maintainers: https://github.com/orgs/open-telemetry/teams?query=maintainer

Update (idea from Jonah). Zoom now offers a native YouTube integration: https://support.zoom.us/hc/en-us/articles/360028478292-Streaming-a-Meeting-or-Webinar-on-YouTube-Live

This won't solve the meeting naming challenges (as those are inherent to Zoom), however this should avoid the missing uploads that happen when Splain drops a video. This will also allow us to stream meetings live.

However I'm not certain if we can enable this by default for all meetings, and it may need to be manually turned on during each call, which won't work.

@eddynaka would you like to check whether this feature would work for us?

@SergeyKanzhelev , already checking! 👍

looking at the tutorial, we can enable it. The issue: we won't be able to just join and live to youtube. For example, the owner of the zoom account has to start the meeting and enable the youtube live, then it would transfer automatically. If other member joins, he can't start the live.

It would be helpful IF the maintainers who lead the meetings login using the zoom accounts and enable it. Otherwise, it won't help us....

In that case let's stick with our current system, thanks for looking into this @eddynaka!

I'm curious. As of next Sunday, Zoom is going to require passwords on all
meetings (or a waiting room.)
If there's a password on the meeting, and we have host keys to the hosts,
can the hosts start the YT stream?

On Mon, Sep 21, 2020 at 12:34 PM Morgan McLean notifications@github.com
wrote:

In that case let's stick with our current system, thanks for looking into
this @eddynaka https://github.com/eddynaka!


You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/open-telemetry/community/issues/169#issuecomment-696326454,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAA2FGOK6JKVNPVA4MDDT2TSG6TDDANCNFSM4ITRUUUQ
.

--
Amye Scavarda Perrin | Program Manager | [email protected]

I'm curious. As of next Sunday, Zoom is going to require passwords on all meetings (or a waiting room.) If there's a password on the meeting, and we have host keys to the hosts, can the hosts start the YT stream?

This is a separate issue to address. In OTel we do not require maintainers to have zoom account creds, anybody can start a meeting. In order to solve the new requirement I'd suggest we go k8s route and set a passcode 77777 to all meetings

this is done correct? @SergeyKanzhelev feel free to reopen if anything needed

New video titles are being set to "unknown" for a couple of days now:

image

Hi @amitmahbubani , someone else is running this job. Mine adds the day + meeting.
I'm re-processing everything to fx that.
But, I don't know who else is doing this.

@amitmahbubani thank you for reporting!

@amye can we file the ticket with Splain to check if they have published videos on our channel without making them private first?

Also if anybody on this ticket knows the way to audit the channel (find out who uploaded video), it will be helpful

@amye replied over e-mail that Splain had issue and they are on it.

Was this page helpful?
0 / 5 - 0 ratings