QuillHub
QuillHub
EN

Risk Metrics Model — Chaos working group notes

45:513 спикера
45:48
длительность
22
главы
3
спикеры
8
Термины
risk metricsgovernanceopen sourcedependencieslicensing riskecosystem criticality

Спикеры

3
Спикер A26%59 реплик
Спикер B58%62 реплики
Спикер C15%17 реплик

Основные идеи

6
  • A risk metrics model should be oriented by a small set of guiding questions (they limited to three questions) rather than many isolated metrics.
  • Governance transparency and structure are central to assessing project risk (e.g., preventing one company controlling a project).
  • Contributor concentration ('elephant factor') and licensing-change risk are high-risk indicators for open source projects.
  • Ecosystem context (dependencies, competing projects, criticality score) matters for project risk assessment.
  • Risk mitigation includes forks, maintaining local copies, building in-house expertise, and increasing participation in governance.

Важные моменты

5
  • Overlap noted between Chaos working group and OSPO/OSCO efforts (Verizon-led consumption-risk framework).
  • Discussion of governance risks including single-company influence and license changes (OpenSearch/Elasticsearch example).
  • Reference to existing metrics: best-practices badge, SBOM-like metrics, elephant factor, and criticality score.
  • Mention of research/data efforts: ZScore criticality dataset and MetaGov governance research community.
  • Practical mitigation advice: keep local copies, fork when necessary, and invest in in-house expertise to reduce dependence.

Решения

1
  • Structure the risk model around three orienting questions (rather than many isolated metrics).

Задачи

1
  • Share feedback on the model template with the metrics model working group

Термины

8
SBOM — Software Bill of Materials — a list of components/dependencies used by a project.
Elephant factor — metric indicating contributor concentration from a single company (risk of vendor domination).
Criticality score — an index that estimates a project's importance based on activity, contributors, orgs, and dependencies.
DORA — DevOps research program (Google Cloud) focused on developer/DevOps metrics and benchmarking.
MetaGov — community/research initiative studying governance models, often across disciplines including open source.

Основное содержание

22 главы · 45:48
0:00

1. Introduction and Meeting Logistics

Спикер A0:00

Okay, so hopefully this is going to the cloud. I— the meet— I know we didn't record last time because nobody had the login. Um, and the time before that, I, I know I hit record, but I can't find it. It's not in the Chaos Cloud; it's not in my cloud. I like— I don't know where it recorded to.

Спикер B0:19

Um, so we basically— we talked about the risk metrics, like building a metrics model for risk, and chatted about the template, as well as say what kind of questions or cases could we orient a metrics model around in terms of just selecting a couple of themes or metrics.

And you can kind of— you can see our notes where we went. I think there was one action item for me that I haven't done yet, which was share some of the feedback on the model template with the metrics model working group, just because that was— we talked about that the whole time, but I kind of moved it on to make sure that we got to actually thinking about what would go into a risk metrics model.

0:58

2. Related Work: OSPO and Consumption Risk

Спикер B0:58

But what's been fun is, in addition to this conversation, the OSPO working group has also been talking about a— not necessarily a risk model, but looking at the consumption risk. So if you were a company, in this case, it's been led by an individual from Verizon who's trying to build a framework for assessing packages that they want to ingest into their company.

And so it is somewhat of a risk model, but he's looking at it very comprehensively. So maybe 4 or 5 metrics for each category.

Спикер A1:31

Yeah, I was in that meeting. I was in that meeting.

Спикер B1:34

So I feel like there's a little bit of overlap there, at least in terms of which groups are talking about the different kinds of risks. I'm just—this is mostly for Victor's benefit, who probably hasn't been in all these different meetings.

1:44

3. Working Group Scope and Framing Questions vs. Metrics

Спикер B1:44

In this working group, I feel like we try to be a little bit more general about risk or various different kinds of risk, but in other working groups, risk always comes up because it's kind of—it's a popular topic everywhere. So I feel like the OSPE working group is another popular area where this kind of theme comes up.

Yeah, I agree. Trying to manage or measure specific elements of risk. So that was my, like, precursor. Do you want to talk through what we had discussed there or proposed and get any feedback on that?

Спикер A2:19

Yeah, I'm interested in that. I laid out like what I think I pulled out. Like, I was trying to look for the metrics that were needed, needing to be developed for this model that we didn't already have. But I don't know if I pulled out the metrics or the questions.

Спикер B2:42

Um, those are the questions, I would say, but it's also like the question will hopefully orient us on the ideal metric.

Спикер A2:50

Yeah.

Спикер B2:50

And then the details below were specific examples of what could be measured as part of that question. So we didn't really limit it to three metrics, but we limited it to three questions.

3:03

4. Existing Metrics: SBOM, Best Practices, and Governance

Спикер A3:03

Yeah, that's—I mean, I think that's fine. And some of these questions, like, I don't know, like, okay, governance, SBOM, and best practices badge.

We have, I know we have a best practices badge metric, and I think we have an, I don't know if we have an SBOM. I think we have something like an SBOM metric, but we didn't call it SBOM at the time.

And I don't know if there's a governance one off the top of my head.

Спикер B3:40

Well, governance, you're mostly thinking about, like, is the governance model published? Is that something that's accessible and available?

Спикер C3:49

Yeah, I actually created an issue in the OSCO book about—I think it's one of the risks, actually, is when open source becomes closed source.

Yeah, that seems to be a major problem for companies that are using an open source technology and then just cannot do that anymore.

Yeah, I don't know if it's a matrix or if it's just one of the use cases, I guess.

Спикер B4:26

Yeah, I feel like that kind of fits in most with this question in terms of how transparently it is governed, managed, and progressing in the community, and whether or not what's implied is that it's community-driven.

4:40

5. Company Influence and Governance Concerns

Спикер B4:40

But I think in your example, Victor, it would be a case where the community is one company that is driving it toward their needs. And so I guess that could also be part of it, because I think there—I mean, I think this is not a secret. I work at Google. I mean, it certainly happens when you have a large portion of the contributing population inside one company.

And so there are often conversations and discussions happening before they hit the community. And depending on— and I think within our own company, we rely heavily on the structure of the governance model to ensure that there is no pull from one company. So when we do want to, say, bring something to the project, there is a process around proposing some sort of technical proposal or whatever that is or whatever that change is.

And that gets brought to the governing board and decision-making processes in the project. And so ideally, we— like, that's always going to happen, I think, in terms of, like, if you have a big team working on a thing, like, it makes sense for you to discuss and get on the same page before you bring it to the community.

But I think maybe in terms of the project going in a direction closer to a company, I feel like that's a little bit more like the elephant factor, I would think, in terms of existing metrics, which I would say is slightly different than transparency. But I do agree, Victor, that that is an issue.

Sean and I have talked about it from a different perspective, like what was the risk of licensing changes, say? Is one example of— Mm-hmm. If there's a group that's really put— puts all their eggs in one project and they want more control over how people are operating and/or using it, then changing the license is one way they could do that.

But that's— we've seen examples of that be highly problematic for the project and seeing the community fracture as a result. So we were talking about that as a potential metric as well, but that was specifically looking at sort of change in licensing. But I think changing governance and change in sort of company influence in governance could also be part of that, like, same kind of conversation.

Sean, do you think that's something that, like, when you think about transparency, does your mind go there, or do you see that as a separate topic?

Спикер A6:55

No, I—and I think where it came in the discussion before is that we have examples of a single company taking over a governing board effectively and changing the license of a project.

So I don't actually know if our governance document for Chaos prohibits that, but I think it does. I think that's part of the LF template that no one firm can have a controlling number of seats on the board because that's where, like—

And the example I have is the most prominent example in recent history that I recall anyway is Open Elasticsearch basically doing that, and a lot of people having to switch to a fork of that project called OpenSearch, which, you know, caused a lot of people in the industry to experience pain.

Yeah.

Спикер B8:02

So I'm just thinking in terms of risk. It is—I feel like that is somewhat in this metrics model. It could be implied in governance, um, because I think ideally if the governance model is designed well, it can discourage that behavior, um, by at least like limiting a single company's control in it.

But that's kind of dependent on the governance model, so that could be part of the nuance of it. Like, I think I just know, for example, like Istio, that's written in their governing model that the basically like decision-making and leadership roles, you get a seat depending on how much your company is contributing and ensure that all the companies that are contributing in the upper echelon are represented in that decision-making body.

And so that's written into the—but they designed it that way, um, where I don't, I don't know if I've seen that in other projects.

Спикер A9:00

Yeah, I— Chaos is the only project within which I've had a governing role at all. So, we're a pretty friendly place.

9:14

6. High-Risk Scenarios: Open→Closed and Licensing Changes

Спикер C9:14

So, for example, that open source to change to closed source model, I guess what metrics can be used to measure the risk beforehand? It could be that the project is doing very well. Actually, that's—I mean, it's being accepted, growing adoption, and all that regular open source community health indicators is all flashing green and everything doing well. That's when this risk becomes high, right?

So, I guess coming back to the risk, what is the highest risk? I think getting open source to closed source is one risk. Licensing is another. Any other high-risk possibilities?

Спикер B10:07

Sorry, I should be writing these down. I'm going to start taking notes. I like this idea of thinking, and I don't want to lose it.

Спикер C10:18

So far, the other Chaos Matrix, oh nice, useful. Those are all primarily for the health of the project. It might just influence how the, you know, OSCO try to, you know, make the developer contribute better or not, right?

But in the end, it's all for individual organizations, whether it's a company or government agencies, to adopt a particular open source project and decide to use it and use it for long term.

So yeah, that's why I think when I see this title risk, I think, yeah, this actually is probably interesting. Because what if you do all that work and end up in limbo?

11:04

7. Measuring community openness and participation

Спикер B11:04

Yeah, I think in the past we've looked at it, at least in Chaos, from that perspective of participating company and affiliation, if you can measure that, and just sort of looking at the percentage of contributions by various companies.

If you can see more into the work distribution, like, is it— who's actually reviewing and merging, and is that also from a diverse population, or is that less? Not from this perspective, but I think it could apply to this case too, is looking at sort of the responsivity to contribution from those outside of those affiliations.

So, say you're another company and you're contributing to a project and you're relatively new to the project or community. If you submit a pull request or a bug fix or something like that, how quickly is that actually picked up, and is it picked up at all? Because we have seen some projects that look neutral from a broad participation standpoint, but then pull requests from outsiders are not really considered at the same rate as pull requests from insiders, um, sort of existing members of the community.

And so that can kind of be a single indicator that it's more of a closed community versus an open community.

Спикер A12:23

Mm-hmm.

Спикер B12:24

Um, necessarily the same as closed source, but as in terms of like trying to extrapolate the possible trajectory of this project, I think that would be an indicator that it is more closed community, which could be an indicator of more closed source.

So I don't, I don't think that's come up yet in any metrics model, like any metric itself. I think, again, elephant factor is probably the most direct one, but I— at least those are the 2 things that come to mind in my view.

But I'd be curious if you have any other thoughts on what else you would want to— what other indicators you could possibly look at.

13:11

8. Governance, transparency, and DEI metrics

Спикер A13:11

Yeah, I think, I mean, from my perspective, the governance model and how it sort of protects against that kind of behavior is important. And I do think that's effectively characterized, I guess, transparency. I mean, that makes sense to me. Mm-hmm.

And I think I could look at the— I guess we need to look and see which of these— I would say I definitely know that we have licensing. I definitely know we have something SBOM-esque. I don't know about governance, if that exists or not.

Спикер B14:12

It might exist in another working group. I'm curious. Yeah, I can do a quick search on our website because I feel like that might have come up in another space, like common, um, worth, because that is not necessarily a risk metric.

It's a, do you have a governance framework, do you not? Is that available to people to see, um, as something that's— it's harder to scan for metric-wise. Like, do governance files have similar naming criteria on GitHub? Like, I don't— I can't say that comfortably.

Спикер A14:45

So, DEI has, I can share this. I downloaded this from the old website because I found I'm actually finding it difficult to find our metrics on the new website, comparatively speaking.

But, so there's a focus area for diversity, equity, and inclusion, and one of their focus areas is governance. These get at board council diversity and the presence of a code of conduct.

Спикер B15:21

Mm-hmm.

Спикер A15:22

And there's also, I think, possibly some related metrics, um, inclusive leadership, mentorship, sponsorship.

But, um, board council diversity, I'm a little bit curious if that gets at organizational diversity. Probably not.

Спикер B15:42

I would guess it's more demographic related than organizational. Yeah.

Спикер A15:56

Yeah, I mean, yeah, demographics is mentioned. I mean, I would— I certainly think demographic diversity is a critical factor. I think the diversity we're talking about is partly that, but also partly organizational participation diversity.

Спикер B16:21

Yeah, and that we do again have a metric for.

Спикер A16:21

Organizational participation diversity?

Спикер B16:32

Okay, please go ahead and paste the text you would like me to process.

16:37

9. Elephant factor and corporate concentration risk

Спикер A16:37

I actually, I think, I think we do. We have an elephant factor metric.

Спикер B16:41

We do have an elephant factor.

Спикер A16:44

Elephant factor, elephant factor is that—

Спикер B16:49

Oh yeah, the elephant factor seems like it's more of a specific ratio. If 50% of the community members are employed by the same company, whereas the metric that I put in the chat is more generalized and looks at a number of different metrics that are also representing participation levels from an individual company.

Спикер A17:13

Ah.

Спикер C17:23

I guess there's also risk mitigation, right? So, even with the best governance model in place, it is possible for a startup that just got $100 million plus funding and just hired all the best maintainers in that project.

So, regardless of the governance model, it could effectively become a monopoly in that technical area. How should a company mitigate that scenario?

Спикер A18:19

So if it's— I mean, I think the character— it's interesting if the characteristics of the project— I mean, I think good open source projects are ones that function well. Yeah, open source company.

Спикер C18:37

Yeah, I've seen that even just the past few months in the open source community. There are some projects that begin with a lot of participation. And then one of the companies, it doesn't have to be a startup, just any company with a lot of cash, decides they want to dominate in that area and then just hires everyone from that community. And so, even if it's still an open source project, it effectively becomes a kind of a monopoly.

Спикер A19:09

Yeah.

Спикер C19:09

So, I mean, it's okay, actually, as long as the project continues. It's just at that point, what if a company— if a user adopts that project, what kind of risk mitigation needs to be, you know, need to be considered?

Спикер B19:27

Interesting. We don't typically cover risk mitigation, but I have ideas.

19:33

10. Mitigation strategies: engagement, forking, and maintenance trade-offs

Спикер B19:33

Well, I mean, there's always like the consultation. There's the suggestion of, is it possible for you to become more involved, where you could have more of an active role in the community and eventual direction steering of the project as another company?

Like, I think there's always an element of how much weight do you put in, but it's not—you're not always able to grow into it. So it's always like it's somewhat up to the individual and up to the community whether or not they're accepting of that as part of that role.

And the other aspect is just, like, I don't know, make sure that you're not pulling directly from upstream. Always have a fork.

Спикер A20:07

Yeah.

Спикер B20:08

Like, I think there's natural ways to sort of break that, and you want to keep track of sort of your distance from upstream and distance to the latest version and whether or not—how easy it is for you to continue to upgrade toward the upstream version.

But, like, having a version that's working well for you now that you can always fork and take away from if there is that need, if there is some—because there is natural divergence in some communities and forks become their own thing.

I've also seen the opposite case where projects come together because there was too much overlap. It's not always a clean process, but it is possible.

And so I think it's like a company—that is one way that we mitigate that risk is by ensuring that we always have the current copy available to us locally without having to depend on pulling from third-party package managers with every build.

So that's like a basic mitigation, but I don't know.

Спикер C21:04

Yeah, I agree. Yeah, fork is probably the best way. Fork and building in-house expertise is probably the only way if they become a critical part of the business roadmap.

Спикер B21:19

It does, it does make maintenance more complex. Like, I think there's always sort of that, like, the trade-offs of pulling always from upstream current versions or latest stable versions to taking your own version off another path.

Then, maintainability and ability to recognize whether or not new vulnerabilities are impacting your version sometimes gets a little hairy because then you need to know what version you forked from and how much you've diverged, and whether or not you can apply upstream patches or whether or not the vulnerability even impacts you. Maybe you've changed enough that it doesn't.

So I think then for the ability to automatically detect whether or not— because you can see vulnerabilities reported in whatever the available packages externally, but if your internal version looks different enough, then it gets— it mostly ends up being a flag, and then you have to go remediate individually.

So we spent a bit of time on transparency and governance. How would you react to this sort of relationship to the broader ecosystem? I know, like, makes you think of, like, somewhat like criticality score. Actually, that's not even in here. But it's also— it's a little bit— what we've written is a little bit more like subjective qualities, not necessarily like they're harder to measure.

22:49

11. Dependency Chains and Limits

Спикер B22:49

So I'm curious if there are things that come to mind when you see this without seeing those descriptions that would either be in line with them or divergent from them.

Спикер A23:05

So, would that be related to the ecosystem question? Mm-hmm. Yeah, I mean, I think dependency chains are a real thing.

I mean, I've heard organizations starting to talk about limit, you know, somewhat arbitrary or at least calibrated limitations on the number of metrics in a project, or the number of dependencies for a project that would be considered acceptable.

And the people I've talked about that with say that they recognize it's arbitrary, but if a team is— if a project is left unchecked against something, then the number of— the amount of dependencies that creep in over time can become unbearable to maintain.

Спикер B24:04

That's sort of interesting because I'm curious how you would even do that. I feel like they're—like you look at all the dependencies, but if you do the full transitive property, doesn't it just like keep going?

Спикер A24:17

Yeah, it's turtles all the way down, as I think we've said before.

Спикер B24:24

That's interesting, though. I haven't heard someone take that stance. I mean, I've heard it from the perspective of individual projects in terms of whether or not they choose to import something or build it themselves.

And I know some projects that basically don't want any dependencies. They want to be completely self-sustained in their own code base. And so, because of that, they are building a lot of things that they don't necessarily need to build, but it's an active choice.

Спикер A24:50

Yeah.

Спикер B24:50

So that there is no—like, then they don't have any dependencies, and they don't have to worry about it.

24:55

12. Choosing to Build vs. Use Dependencies

Спикер B24:55

So, like, you can make that choice, which is kind of the most extreme way to do it, but it keeps your dependencies under wraps because you don't have any.

Спикер A25:05

It's like choosing to build a car from entirely hand-manufactured parts, as opposed to an assembly line.

Спикер B25:15

You used to have to do that, Sean.

Спикер A25:17

I know, I know, I know, but thanks to Henry Ford, we don't. Um, I mean, yeah, it's, it's like, uh, gosh, it's almost like a— I almost want to say it sounds very Brooklyn, you know? It's very hipster craft. It's like craft open source where every piece is handcrafted by the makers. I'm having a hard time even imagining a world where I would have a motivation to do that.

Спикер B25:58

Oh, remember that time before you could use the internet?

Спикер A26:04

Yeah.

Спикер B26:07

I mean, I was like 10, but—

Спикер A26:10

Yeah, no, no, I always asked—

Спикер B26:13

I was not writing code then.

Спикер A26:14

Yeah, I was asked what I thought of the internet in 1994, and my answer was, "It's a fad. Everything I need I can get off of CompuServe." I was wrong.

26:28

13. Project Criticality and Metrics

Спикер B26:28

Um, interesting. So I looked at the criticality score, not what we want. Uh, it says a project's critical score defines the influence and importance of a project, but they're only looking at.

Спикер A26:44

So basically, the downstream dependencies that a

Спикер B26:46

Project has. No, there's no dependencies at all. It looks at how, like, basically when did you start it, how long has it been since it's been updated, how many people are working on it, how many organizations are working on it, how many commits per week is it getting, releases per year, issues closed, issues updated, comments per issue, number of project mentions.

Oh, here, there's a dependency count and then dependencies. So most of it is just like, is it still being worked on by how many people, by how many orgs, and then how many dependencies it has.

So I don't, I don't love it. I think that's actually what I remember talking about. Do you remember the ZScore conversation?

Спикер A27:28

Yeah.

Спикер B27:28

That was from Victor, if you're familiar. The researchers from Harvard partnered with the Linux Foundation researchers to look at the relative criticality of packages across multiple, like, popular package managers, using both things like download and contributor count, as well as pulling in usage information from—I don't remember how many organizations, but they were trying to characterize basically how many people, or how many teams, how many orgs were using it.

And so it's a somewhat opaque calculation because it could share all the data and the exact methodology, but they were trying to factor in more things than just this. I think, I guess, looking at the criticality metric, it's because those are things that are readily available to you.

But I don't know, I feel like for me, this sort of broader ecosystem points more at understanding the role of this project inside it, like a dependency chain, to your point. Like, I think that that's where my head goes as well, which is.

Спикер C28:32

Yeah, there are more systematic studies, for sure. Actually, there is actually— I actually— there's actually another— not research, but a researcher did some information gathering. And let me find that.

28:51

14. Ecosystem Boundaries and Competing Projects

Спикер C28:51

There's a lot of statistics about the open source community. But there's no, there's no actually conclusion on that.

Спикер B29:05

Okay. Yeah. I guess the who competes with it, I don't know who's highlighting that. Is that it's also like maybe we'd say what competes with it, because sometimes there are competing projects, and that could also depend. It sort of.

Спикер A29:25

Yeah, I was trying to decide about the—how competing things, whether or not those would be part of the same ecosystem in some cases.

Like, um, I'm trying to think of competing projects that share whatever an ecosystem is. I have an example.

Yeah, yeah, give me an example.

Спикер B29:53

The multiple service mesh projects under the CNCF.

Спикер A29:56

The multiple what?

Спикер B29:58

Service mesh projects: Linkerd, Istio, and Aspen Mesh, all under the CNCF. Actually, is Aspen Mesh? Sorry, I have to check that one. But basically, there's been multiple service mesh projects and initiatives in the same space.

Спикер A30:20

Um, yeah, like, so like, I know for example there's like 6 or 7 products in the Zephyr space, and Amazon is the leading one in terms of usage, and Zephyr is the second leading one, and then there's 3 or 4 others.

So they're part—I suppose they're part of a competitive, you know, a competing project, product, community framework, but people contribute. You know, there's not a lot of overlap in the individual or organizational contributors to either one.

30:58

15. Service mesh ecosystem and competition

Спикер A30:58

Um, and, but, but they are, they do exist in the same consumption ecosystem where you can choose to use one or the other for these, you know, end-of-the-wire devices.

Спикер B31:11

I'm sorry, I was incorrect. There seems to be 1, 2, 3, 4, 5, 6, 7, 14, 21, 22 service mesh projects in the CNCF.

Спикер A31:25

Wow.

Спикер B31:25

But I think, to your point, Sean, you can't assume plug and play with them. Like, I think it's not like you can swap one out for another. I think there's enough difference in them.

Weird to me is that before CIO came into the CNCF, there was an initiative to create the service mesh interface to try and create more conformance, more performance layer across different service meshes. So, in theory, if you use that, then you could use any of them. But then that went away.

Спикер A32:02

Yeah, because for-profit entities really like, um, that to be—to have their customers locked into them.

Спикер B32:10

Um, maybe. I mean, it's all—it's like, it's clear that there are many ways to do this. And so I think because there is no one single project, it is going to be interesting to see how that space evolves.

Like, I think we saw that happen with Kubernetes early on, but then Kubernetes became the clear winner, and then this entire ecosystem developed around it where the other ones didn't. Like Mesosphere ended up adopting Kubernetes later on.

Спикер A32:41

Yeah.

Спикер B32:41

So it's one of those things where, like,

Спикер A32:46

winner

Спикер B32:46

Takes all, possibly. But like, in the case of understanding its relationship with the broader ecosystem, looking at any of these projects, you would look at them and say, hey, in theory, they're all competing with each other in both mindshare of user and potential contributor.

If someone's like, I want to work on a service mesh, or they typically will work on the one that they use, maybe. I don't know how people pick what they want to work on. I think it's a little bit of a combination of both. But I think eventually there is going to be some consolidation because— or like, I don't know, because it's just like— or maybe they'll go in different directions. I'm not really sure.

But in terms of how it relates to the risk of that individual project, I would say it is riskier for any of these because they have 22 competitors, or 21 competitors, already in the same space that support service mesh architectures on top of Kubernetes/container-based application design.

Yeah. I would say that is inherently riskier because there are more things that are competing with it. It versus if there's only 1 or 2, then it has a greater chance that all of them continue or one of them ends up being the one that is preferred.

Спикер A34:01

Right.

34:02

16. Open-source research dataset and MetaGov introduction

Спикер C34:02

I posted that link for that research. I thought there was some conclusion about open source. It actually turned out to be just a data collection. So, there's no conclusions.

Спикер B34:18

Actually, I know about this project. I hadn't read the paper yet, probably because it came out in April. Thank you, Victor.

Спикер A34:23

It's not a—I guess it's not a paper, but a dataset, you said?

Спикер C34:27

Yeah, that's my understanding.

Спикер B34:29

I, yeah, I met with some of these folks, um, Sean, back in San Francisco, um, at the meeting, and they were talking about this project because there was a couple of, um, Victor, there was a couple of different researchers that were attempting to build more datasets around governance models, um, just to be able to research them in terms of the impact of a governance model and type of model on the trajectory of a community and project, and whether or not they could identify various attributes, traits, design principles that led to a more favorable or negative trajectory in a project.

So that is—thank you for sharing this because I didn't know it had been published and I want to.

Спикер C35:14

Yeah, if you don't know that, that's coming from another community called MetaGov, which focuses on governance models. That's why I also join those meetings sometimes.

Спикер A35:27

What is it called?

Спикер C35:28

MetaGov, M-E-T-A-G-O-V. And they focus on a lot of things, but in general, there are two aspects. One is W3C blockchain-related technology. The other side is more soft, like governance model.

35:58

17. Meeting interruptions and personal updates (travel and AQI)

Спикер B35:58

Sean, can you put that in the new—

Спикер A36:01

Oh, geez, am I down on the wrong—oh my God.

Спикер B36:07

I'm all for revisionist history, but, uh, no.

Спикер A36:09

no, sorry.

Спикер B36:11

I'm actually not.

Спикер A36:12

But I lost track of— oops.

Спикер B36:16

It's okay.

Спикер A36:17

I'm also losing track of where the buttons are on my keyboard.

Спикер B36:22

Am I sending my terrible AQI through the interwebs? I was so out of it yesterday, for the record. I had to cancel meetings, just like... turns out I can't think straight if I'm not breathing well.

Спикер A36:34

Yeah, no, I've been— my return from Japan has been rough. Like, usually Asia's rough, but I don't know why this time it's really, really rough.

Спикер B36:47

Actually, I had a great time coming back from Japan. I actually was a morning person for like a whole week.

Спикер A36:54

Yeah.

36:57

18. MetaGov and Governance Research

Спикер B36:57

I tried to ride it out as long as possible. I'm sorry. So, MetaGov, is this a community or is this a research group?

Спикер C37:05

Yeah, if you search, yeah, so they have a lot of PhDs, actually, very theoretical sometimes into what is governance model, even like what is democracy. For example, what is the, yeah, all kinds of research related to some type of that area.

Yeah, that's why the reason I attend some of the meetings is related to governance model. Like, what is the right— is there a real— what's a perfect governance model? Does that even exist? I guess that's the question.

Спикер B37:44

I would say there isn't, but that's my personal opinion. I think that there's a lot of people that are thinking about this, Victor. And I think at least in that meeting back in March, there was a lot of people coming back to the sort of model of the commons.

I'm trying to remember—I need to remember the exact name of it, but there was a researcher that was proposing this model of the commons and then people that were applying those principles to governance models. And let me see if I can pull up my notes from that, because then I can be smarter about it in real time.

And I can also help you individually if you're interested in speaking with some of the folks that are thinking about this. I think places—it sounds like MetaGov is a good place to do that, as well as any folks that are on that UC Davis paper. I know Seth Frey and Vladimir are on that.

Yeah, I guess I've met—

Спикер A38:35

Yeah, I'm curious where they submitted this for publication.

Спикер B38:41

Yeah.

Спикер A38:41

Because it's, it's the open source literature in the academy is so fractured across so many different discourse communities.

38:58

19. Ostrom and the Commons: Applying Principles to Open Source

Спикер B38:58

Yeah, so there's *Governing the Commons* by Elinor Ostrom.

Спикер A39:02

Oh yeah, that's a classic.

Спикер B39:04

Yeah, so that was, that was brought up a lot in terms of how to apply the principles she outlines in this book to governing open source. And I think it looks like Sustain wrote up a blog about that.

Спикер A39:20

Yeah, I mean, the core nugget of Ostrom's book is, imagine a common grazing area that is overgrazed, and that's her tragedy of the commons. Where everybody shares the same resources, but it—there's an incentive to have more goats because you get more goat milk, and then it just becomes overgrazed.

Спикер C39:48

Just in case you're interested, I posted the Medigov meeting calendar.

39:56

20. Organizational Governance: Company Roles and Updating the Chaos Model

Спикер B39:56

Nice.

Спикер A39:56

Did you post it in the?

Спикер B39:59

Yep.

Спикер A39:59

It's

Спикер B40:00

In the chat. Well, thanks for sharing this. I didn't know this community existed, so it's great to know because I think, um, we as a company generally have these conversations with, like, say there's a new project that's thinking about releasing itself as a project into the community. Um, that's often where our team is brought in for consultation on to say, like, how are you going to make decisions? How are you going to allow people to work on this with you? What requirements do you have?

And basically working through those project leads on creating at least some basic elements of governance, as well as, like, a starting point. But it also does need to evolve over time. I think I love following the progression of the Kubernetes organization just because it's gone from, I don't want to say like hundreds to tens of thousands of people working on it. And there are some examples where we're starting to see where the models are working well or not working well because of the sheer scale of the project.

And essentially confirming that— I guess this is sort of my it depends answer for governance models. I think there are key things that all governance models seem to have, but there does seem to be some element of reflecting the needs of the community in your governance model and acknowledging that the size, structure, makeup, and organizing principles within your community should guide what your governance model looks like.

I think something we've been talking about, Victor, in our Chaos weekly calls is actually updating the Chaos governance model. So that might also be interesting for you to see. And just in terms of how at least Don's been working on trying to update it and ensure that everyone in the community, as well as the governing board, is on board with everything that's in there.

And it's not just about how rules are made, but also, say, outlining how individuals are promoted up into maintainer leadership or governing board positions in the community, and how we think about growth, development, and expectation of those positions. And it goes beyond just decision-making and sort of structure of the project to understanding how to work with us, and if you want to do more in the community, how you do that with us.

So I think it's one of those things where it is sort of a choose your own adventure, but I am curious to see how research in this space can help us better design these things. Like, I think, like, I'm curious to see what this metagov space is like. And thank you for sharing it because I'm going to pass it off to some of my colleagues that are in those roles, so they're actively meeting with our project leadership on governance structure and model.

42:46

21. DORA: DevOps Metrics and Their Relevance to Open Source

Спикер B42:46

And so, I think it might be interesting for them to see that these conversations are happening in other places.

Спикер C42:58

Google has done a lot of good things for the open source community, for sure.

Спикер A43:04

Yeah.

Спикер C43:04

Including the research, such as— I know, is Dora at Google? A Google project? Is DORA a DevOps report? Is that a Google report, or is it just on Google?

Спикер B43:18

Yeah, so it was another company that was acquired by Google, so now it's part of our research portfolio, but it's distinct from open source. DORA focuses a lot on DevOps metrics and benchmarking for developer teams within companies.

So, actually, this is one of the presentations that I loved. A couple of years ago, I did a presentation with one of the leads on DORA, sort of offering a view of how to evolve DORA for open source, recognizing that that sort of basic assumption of a team and shared incentives as team members working for the same company, those assumptions go away in the community, and sort of thinking about how that would impact the types of things that you want to measure to achieve the same goals around understanding participation, engagement, and sustainability, reactivity, quality of release, and things like that.

So, DORA isn't an open-source project per se, but it is a research community within Google.

Спикер A44:21

Is there a link for DORA?

Спикер B44:23

We can link DORA. It's not as applicable to chaos things. Beyond their interests, but let's see.

Спикер A44:31

And I think we're at the— we're over time a little bit, I'll point out. So, it flies when you're having fun.

Спикер B44:41

Can I find our DORA DevOps page?

Спикер A44:47

We don't need to find it. We can throw it in later.

Спикер B44:50

Oh, this is bothering me. We should have. Well, it's like this whole thing called search that also comes out of our company.

Спикер A44:59

Does Google do search? Uh,

Спикер B45:07

finding other—

Спикер A45:09

Yeah, dora.dev looks like the most promising. Does this look familiar?

Спикер B45:20

Yeah, uh, yeah, I think that's it, dora.dev, because at the end it does say DORA is a program run by Google Cloud. So this is—I think it's the official DORA website.

45:30

22. Meeting Wrap-up and Next Steps

Спикер A45:30

All right, there we go. It's in the notes. And I bid you all farewell until two weeks from now.

Спикер B45:38

Thank you. And thank you, Victor, for showing up and participating. It's always nice to have new folks. Yeah, thank you.

Спикер C45:45

Have a good day.

Спикер A45:46

You too. Bye.

Спикер B45:48

Bye.