New "Acceptance Testing" section for RFCs? #3236
Comments
|
In the interest of continuing to leave a trail for things related to this effort, see also The case for a new release channel: testing, Survey of "High Impact" Features, RFC: Preview for Unstable Features, and ultimately this comment. I love the idea of a "call for testing" in TWiR. My main concern continues to be the ability to get widespread testing when it requires nightly. |
|
@epage Interesting thoughts. Would it help if the section were titled something more "call-to-action" oriented e.g. "Call for Testing"? To that end, a section with a call to action and clear instructions (or a link to clear instructions) that would evolve as the feature stabilization process does would allow us to move forward (if we elected to) quickly, while still preserving the option to evolve the stabilization process. Definitely something to discuss with @nellshamrell and the rest of the TWiR editors, but if either of you have a draft of what you'd like to see the section look like please share it here, otherwise let us know. I'm happy bring up the topic for discussion with the other editors. -- My own 2¢ on the topic is to start with Nightly, moving to having a mechanism to ungate the to-be-stablilized features individually and as a group. From there, move the capability to Beta, and we can learn about what issues there are with exercising the features using Beta vs feature-gated on Stable. At some point beyond this, if deemed appropriate, we could move the to-be-stabilized feature gates to stable (although I think the time-lag synchronizing with Stable could be a source of vocal objections). Along with the technical implementation of the feature gates, key would be having clear instructions on how to exercise each of the to-be-stabilized features, as most people will be unfamiliar with the nuances of the feature and will not be inclined to read all the historical context to get up-to speed. |
|
Draft: comes before "Final Comment Period" ### Call For Testing
An important step for RFC implementation is for people to experiment with the implementation and give feedback, especially before stablization. The following RFCs are at point where user testing is needed before moving forward:
Pre-Stablization
- [RFC 2906 Tracking Issue](https://github.com/rust-lang/cargo/issues/8415): Testing steps **LINKME** |
|
Having a summary of the changes might be nice as well. I know for |
|
Adding this section sounds good to me - and it's a great way to increase participation! |
…epage pre-stabilization documentation for workspace inheritance This is adding documentation for how we would like users to test workspace inheritance. This came about from a discussion between `@epage` and I on better ways to document "pre-stabilization" features that are looking for people to test them. One of the ideas was to add some of the documentation to `unstable.md` so that it is all in one area. Having it in one area allows us to link to it so there are testing notes and documentation in one place. It also helps when posting in various places looking for testers as we can link to the nightly docs as needed. One idea was to post in TWiR [under a new table](rust-lang/this-week-in-rust#3236) and this also helps with this. The new documentation covers - What we are looking for from testers - Where to give feedback - How to test this feature - An example port as a guide r? `@epage`
|
Unless there is more to discuss on how the future looks for this section I think #3249 closes this issue |
|
Thanks everyone. The new section is in the template. I am happy to fill out this section on an ongoing basis as a function of my FCP/RFC editor role--but if there is anyone on the TWiR editorial team who had their heart set on owning this section, I am flexible--just let me know! I will leave this issue open a bit longer as we currently lack specific inclusion criteria for the line items to be included in this section. I will do a run-through for this week's TWiR-443 and will publish what I come up with here as a draft for feedback and adjustments. If what we come up with works for folks, I will then close this issue. As always, thoughts/feedback/ideas welcome! |
|
In working through this, I think the simplest way forward is to create a new label (e.g. (For future consideration, I remain convinced that we will get better uptake when the line items come with specific testing instructions; this is something we can iterate on over time.) I have added the two RFCs explicitly called out in this thread RFC 3243 and TI 8415 to this week's TWiR and will keep an eye out for feedback. @epage, please note I have omitted the "Pre-stabilization" and "Intermediate Design" categories at least initially, since there are only two items in this weeks TWiR. I am happy to discuss how and what we should include as we get a sense of volume and how this information will be used. **Going forward, I propose building the section based on new results from this query after having added a If you have suggestions on any of this please share, otherwise if this looks good I'll figure out how to create a new Edit: removed "is-open" criteria from proposed weekly query. |
That query would miss things like RFC 2906 as it is "closed". I think just looking for the label |
|
@Muscraft Great catch. I've edited the proposed query to search only for the label, per your suggestion. I have added the Label: call-for-testing |
I think it would make sense to have these expire automatically so as not to dominate the newsletter with aging content. If I remove the tag when I add the content each week, the issue owner could add it again when it is time for another test run (or if they simply wanted to be listed for another consecutive week). What do you think of that system? |
I can't help with this, I do not have permission to add labels
I think it will be perfect to start with and we can tweak it as we go. I can see why it is better to have to opt-in each week or every time there is a need for more testers |
|
Great, I agree. We'll start with this process as described here and will evolve it as the need arises. |
Add notes about pre-stabilization to contributor unstable docs This PR is meant to add more direction for contributors on the path to stabilization for unstable features. It adds a section titled `Pre-Stabilization` to the unstable contributor docs. The idea for this [came out of the discussion](https://rust-lang.zulipchat.com/#narrow/stream/246057-t-cargo/topic/workspace.20inheritance.20stabilization/near/281856280) about when and how to stabilize workspace inheritance. The notes that are being added were derived from the above comment as well as the [the adding of the `Call for Testing`](rust-lang/this-week-in-rust#3236) section to TWiR. [This comment](rust-lang/this-week-in-rust#3260 (comment)) gives more information as well. As for the requirement of testing notes, [there is still discussion about if they are needed](rust-lang/this-week-in-rust#3260 (comment)). While what was added is not comprehensive it is meant as a guide for what to do as each feature has different requirements for stabilization r? `@epage`
Some times it can be a challenge to get testers for an RFC implementation to get the feedback to know if its ready for stablization (see Mechanism for beta testing unstable features).
What if we had a new section "Acceptance Testing" where we queued up RFCs that are are at stage of needing general feedback, either as part of an intermediate design milestone (e.g. rust-lang/rfcs#3243) or in prep for stablization (e.g. rust-lang/cargo#8415 (comment))? They'd need to link out to a test plan (defining what you are looking, link to how to use the feature, link to how to use it with nightly, where to apply feedback, etc).
The text was updated successfully, but these errors were encountered: