Feedback on Stability and Release Practices Proposal #3098
Replies: 3 comments 2 replies
-
|
There is a lot of good stuff in here @austinlparker thanks for putting it together. How do you plan to enforce this with so many distributions out there, is the objective to have a certification for distributions? I assume these crtiera would allow it to be certified in some way? Have you considered LTS versus "epoch" releases? In Kubernetes they always avoided this as it creates more work for the maintiners to patch/merge/build multiple versions. They expected the vendors to implement this which most of them have done and handle themselves. |
Beta Was this translation helpful? Give feedback.
-
|
These proposals are only focused on the things we can control; e.g., java agent, collector, etc. I can see a future state where there's ways for other people to release distributions that align with the upstream strategy though. To your LTS point -- we, effectively, already have LTS releases (see https://github.com/open-telemetry/opentelemetry-specification/blob/d5cfbdf78edd7a81a14397eeea5a93b586c46326/oteps/0232-maturity-of-otel.md and https://github.com/open-telemetry/opentelemetry-specification/blob/d5cfbdf78edd7a81a14397eeea5a93b586c46326/oteps/0143-versioning-and-stability.md), or at least something close to it. The problem is that each individual sig, effectively, gets to pick how they release (and in many cases those sigs are mixing stable and unstable features/code together in ways that aren't easy for end users to understand). The idea with epochs is that we can provide a stable, documented, and tested integration point across all SIGs in order for end users to consume updates in a more structured way. |
Beta Was this translation helpful? Give feedback.
-
|
This is probably already in the plans, but I think it'll be necessary in the OTEPs that come out of this proposal to lay out the rollout plan to get to that "stable by default" state. My main concern is around instrumentation libraries and collector components. The fact is that a lot of current end-user experiences and workflows are built on components that are not stable, and having a breaking change of this scale (to disable them) will create confusion for end-users and toil for those supporting those experiences (i.e. observability platform teams and vendors getting customers reporting broken experiences). For instance, if we were to roll out a standard feature flag option to control disabling unstable instrumentations/components, some folks will understand that they're building that on unstable components and enable it. However, those that don't read release docs, (or SDK/collector startup logs) will be the ones that probably fail to set that flag when upgrading, and then get frustrated. Effectively, the way I interpret the effort of "unstable is disabled by default" is, in a way, an added layer of stability to what observability teams and vendors have been doing with their own distros and consolidated configs. E.g. in the Java agent were all instrumentations are enabled by default, some of these teams have been taking the approach of disabling all of them and enabling specific ones that they have "vetted" as "stable for their own purposes, in a particular version". I think these teams will appreciate the extra layer of "stable as identified by maintainers". It will make their job easier. So, the main pain will be for those end-users that don't have that team doing this assessment, or a vendor distribution that controls what's enabled and what not for them. IMO a good first option would be to start with new components and instrumentation libs. These, as new components, can be immediately put behind this "stable by default" behaviour, and only enabled when the pertinent feature flag (or whatever mechanism) is in place. This would create immediate value with no breaking changes. For existing components/libs, I think we should think of specific timelines that either (1) seem achievable to move critical components to stablisation, or prompt action to do so (which could be a good call to action) and (2) give enough time to communicate that they will be disabled in future major versions. |
Beta Was this translation helpful? Give feedback.
-
|
On a separate topic, and more related to semconv. Having a clear way to communicate that a certain instrumentation package can be stable, but the semconv of the data it produces might not be, and what happens to component versioning when that semconv changes, will be crucial. If we just say "this instrumentation lib is stable" most users would immediately understand that the telemetry it produces will be based on stable semconv, which may not be the case. So I think that, for instrumentation libs, the status of one aspect cannot be reported without the other. For instance, at the moment Collector Contrib receivers have well-documented stability levels, but no link to the semconv they're based on (if applicable). I know the relationship between these will be covered in OTEPs and spec work, but we should ensure that it reaches the end user that doesn't (shouldn't have to) read any of those (registry 2.0?) |
Beta Was this translation helpful? Give feedback.
-
|
The core problem (well stated) Today, when a user sees: “This instrumentation library is stable” they will reasonably infer: “The telemetry shape (attributes, metrics, spans) it produces follows stable semantic conventions and won’t change in breaking ways.” But that inference is not always true, because: This creates silent breaking change risk. ⸻ Why “status of one aspect cannot be reported without the other” is exactly right Instrumentation has two independent stability axes: they will reasonably infer: “The telemetry shape (attributes, metrics, spans) it produces follows stable semantic conventions and won’t change in breaking ways.” But that inference is not always true, because: This creates silent breaking change risk. ⸻ Why “status of one aspect cannot be reported without the other” is exactly right Instrumentation has two independent stability axes: Right now we mostly report only the first, but users mostly care about the second. That mismatch is the root of the confusion. ⸻ What end users actually need to know (without reading specs) For any instrumentation component, a user should be able to answer at a glance: If the answer to any of these is unclear, we’ve failed the UX. ⸻ A concrete mental model that does work Treat semconv as a versioned contract, not an implementation detail Instead of saying: “This instrumentation is stable” we need to say something closer to: “This instrumentation is stable and emits telemetry conforming to SemConv X.Y (stability: experimental/stable)” That makes semconv: ⸻ Proposal: Dual stability reporting (library × semconv)
Every instrumentation component should declare both: Instrumentation Stability: Stable Stability: This avoids the misleading shorthand of a single “stable” label.
For example, in: There should be a direct link to: telemetry: This is the part users fear most, and rightly so. We need a clear rule, such as: If a stable instrumentation upgrades from: Then one of the following must be true (and documented): If none of these are present, the component is not truly “stable” from a user perspective. ⸻
You’re absolutely right here: “Collector Contrib receivers have well-documented stability levels, but no link to the semconv they’re based on” This leads to: Receiver Stability: Beta
Registry 2.0 is exactly the right place for this You’re spot on: this should not live in OTEPs alone. Registry 2.0 can act as the end-user contract surface, not just a spec backend. What Registry 2.0 should expose: Example UX: That single sentence prevents countless surprises. ⸻ Key takeaway (and I think you already believe this) “Stable instrumentation” without semconv context is a misleading claim. Instrumentation stability is two-dimensional, and unless both dimensions are visible together, users will assume the safer interpretation—and get burned. You’re not asking for more spec complexity; you’re asking for honest communication. |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
This is a space to discuss our stability proposals.
To summarize and recap the blog, these are our goals that should be addressed:
We plan to introduce these changes via OTEPs in the specification, but would like to use this space to collect feedback from the community, end-users, maintainers, and others about the goals and objectives, as well as how they're achieved.
Beta Was this translation helpful? Give feedback.
All reactions