Knowledge Sharing Platform - idea sources

[matthiazzz]

Testhub
https://docs.google.com/document/d/1-IJbPRcaWvN_Iy9pVqT9DTKJ1wBvYtRMTZTR_5NcOmg/edit

A centralized repository of community contributed, peer reviewed, hand-written requirements and test cases
Citation: Eric Milliman’s idea :)
Standardized interchange language for use cases, test descriptions, and test code
Requirement
Plain language test case
Test code
Test environment
Approach for grabbing an a-la-carte set of requirements, test cases, and test code to put into a template document form (e.g. an internal validation summary document template)
User interface would provide package searchers with a natural language search for a task they hope to complete (e.g. specific regression task, data manipulation, database connectivity, etc).
UI would return similar requirements that a given package has been validated for in the past
If user’s task is found, user can adopt existing test case and code for their internal validation reporting needs
If not, user is prompted to contribute
Test code could live anywhere and be mirrored in to the repository
Has a developer already written a very specific test as a part of the package? Write the Requirement and Test Case information and store a reference to the relevant code. Pull in relevant code at validation summary report compile time.
Can some of it be automated with LLMs :)

Pharma "Views"
https://docs.google.com/document/d/1KZsCKqIlXeGo7Xiucgkotaer9wOlxUULvc0FtJdRcDY/edit

Having curated lists of packages and real-world use case scenarios to standardize on a consistent set of packages

**Scaling Trust **
https://docs.google.com/document/d/1G-shBNI0uq-LVrECM1u6OenJRkBKEYWsdocUtfhBWCM/edit

Scaling Trust in R Packages – A Hybrid Approach
1 The Problem
The key to maximise value from R is leveraging the extensive open-source package ecosystem. In a regulated environment, where enterprises are required to demonstrate quality of the software that they employ in an auditable way, using open-source packages poses a challenge. Currently, no standard for communicating quality assessments of R packages in an auditable way exists. This precludes effective sharing and combining of validation efforts – both free and for-profit. Relying on proprietary validation frameworks and protocols leads to lock-in risks on the customer side and increased marketing/adoption costs for vendors of package validation services.
2 Proposed Solution
We believe that no single organisation can effectively manage R package validation.
Instead, we propose to create an open, auditable standard for R package qualification and validation together with an open protocol that allows the standardised exchange of R package quality information, containing detailed information about

1.  the package version (+ sha256 hash of sources)
   

2.  sub-package (function) level resolution information
   

3.  known bug warnings
   

4.  automated tests and reports to be run as integration tests on the recipient systems
   

5.  metadata about the testing environment such as versions of dependencies and operating system or container.
   

An important part of the protocol is the built-in capability to optionally share relevant validation documents only after authentication. This allows business models where the result of an assessment is shared publicly but the auditable information is only available for paying customers. Given the large amount of resources required to thoroughly quality check statistical packages, allowing business models to build on the open ecosystem is key.
The protocol makes information about package quality sharable in a federated network and auditable evidence can be generated directly on the target system.
The network should allow queries for individual packages and specific versions thereof as well as sets of packages respecting interdependencies to obtain validation status and bug reports as well as optionally additional paywalled data. This enables to ‘mix-and-match’ validation information from different vendors and the open-source community to compose an auditable R package library in a flexible manner.
3 Proposed MVP
The WG would specify the validation standard, the communication protocol, and would demonstrate the technical viability by setting up two servers publishing validation information using the protocol, one of which requiring additional authentication for access to validation documents and unit tests.
The data is then integrated in a ‘validation pipeline’ that demonstrates how this information can be used to create a ‘reliable’ R Package repository on posit package manager.