Community: Creating Guidelines for using OpenTelemetry DevStats

[alolita]

CNCF DevStats dashboards (for example see https://opentelemetry.devstats.cncf.io/d/5/companies-table?orgId=1) are a popular way of viewing metrics for all CNCF projects including OpenTelemetry. The project metrics presented by these dashboards can include commits, contributors, companies, developer activities, GitHub events and more.

We have seen some project metrics, such as the contributions by each company, being used for promotional activities instead of for facilitating project contributions. We’d like to see dashboard data being used to encourage project growth rather than for promoting company bragging rights.

We propose guidelines that are part of a Code of Conduct for referencing DevStats metrics. The guidelines would emphasize responsible use of comparative and aggregate data that would preserve a level playing field for all contributors and would avoid being used to disadvantage one contributor over another. This would encourage true open source collaboration.

This issue proposes to develop a document that creates guidelines for all contributors to follow in their responsible use of DevStats metrics.

cc: @sarahnovotny @lizthegrey

[bhs]

Thanks for creating this issue, @alolita !

Re this particular topic, our "code of conduct" is more about stuff like harassment, derogatory language, etc – in particular, it's about the actions of individuals more than companies. I think we may be running up against a gap around how OpenTelemetry deals with the behavior of companies or organizations, or at least their marketing departments.

Along those lines, I would suggest – though of course I'm open to other opinions – that we broaden this issue to creating an initial set of guidelines for how OpenTelemetry enforces community expectations on companies that are involved with the project... i.e., even if we made the perfect rule about CNCF devstats, it wouldn't stop a vendor from looking at commit logs, creating their own dashboard, and making equivalent claims (which is really the immediate problem we discussed at the GC meeting last Thursday).

[SergeyKanzhelev]

Thank you for initiating this work!

+1 for @bhs suggestion that separation on expectations from individual vs. company would be useful. And likely those would be different documents.

Also a small nit:

encourage project growth rather than for promoting company bragging rights.

I believe the discussion was that certain language in marketing materials should be discouraged, not the bragging rights altogether. Examples were given: "top 5 contributors" ok while "the topmost" is not ok.

I agree with the statement overall, but needs to be careful with the language "bragging rights".

[tigrannajaryan]

I agree with the spirit of the suggestion and I think it is useful to have guidelines and recommendations.

One thing we need to be careful with: the people who brag are not necessarily the people who contribute to OpenTelemetry. We should not punish a contributor for what a marketer in the same company does.

[alolita]

@bhs @SergeyKanzhelev @tigrannajaryan Thanks for your feedback. Your thoughts are very helpful.

I will be working on a doc to provide guidelines for organizations for fair use of project metrics and other material. I will share a draft and then let’s iterate together on refining these guidelines.

[tedsuo]

I agree that individual contributors should not be punished for the sins of their marketing departments, but a "company code of conduct" would be very useful to define, as a starting point. It could then be leveraged as part of a corporate certification program.

[bhs]

@alolita per this comment of yours:

… I will be working on a doc to provide guidelines for organizations for fair use of project metrics and other material. I will share a draft and then let’s iterate together on refining these guidelines.

is there a rough timeline for when this doc ^^^ will be available for review? And/or if you're simply underwater, I'm happy to take a first pass. Generally, I'm just eager to make some tangible progress on this overarching issue. It would be great if we could have something basic in place by the end of the calendar year (if possible).

[alolita]

Hi @bhs I hope to share a draft for review before the GC meeting this week. Will share doc link with you. I agree we should have a discussion and tangible progress by no later than next week's GC meeting.

[bhs]

@alolita thanks and looking forward to reviewing!

Not to _further_ muddy the waters, but I'm also wondering whether we can create incentives for vendors to think about the diversity of their staff contributing to OTel given #371 (per this message, we have a long way to go :-/ ]. I'm happy to throw some of my ideas along those lines into the draft doc once it's available.

[alolita]

Hi all, Here is an initial draft for guidelines for using OTel project metadata including DevStats. This is a working draft so please feel free to add your comments and content.

https://docs.google.com/document/d/1pwYh_jilvoidvZ8qhe1iA8hXcrbdRFF2IKbJjXd7u3s/edit?usp=sharing

cc: @bhs

[bhs]

@alolita I made some "big" comments towards the top of the doc and also wanted to try to give some examples of the "Do's and Don'ts" approach I suggested at the meeting on Thurs. Rather than making a ton of nitpicks, I drafted an alternative approach to a slightly broader doc about OTel marketing guidelines in general. Take a look...

OpenTelemetry Marketing Guidelines for Contributing Organizations

[alolita]

@bhs Thanks for iterating on the doc. I added my comments to the alternative approach doc which reads well. We may want to add more specific examples in some sections.

[bhs]

Thanks, @alolita , great feedback and suggestions!

So do we want to move our attention to the "alternative version" doc? If so, I am happy to turn it into a PR against the community repo since there have been a few gdocs "LGTMs" and most of the comments at this point are minor.

[alolita]

Hi @bhs Yeah please do. I think your doc is well balanced. With some generic examples we should be good to post :-)