Ideation

[kkmann]

I roughly see three components

1. A file specification (md document) that describes the structure of the files that we use to exchange ‘knowledge’ (json?)
   a. Which protocol (version of) used for quality assessment
   b. Which package / version / md5
   c. Specification of functionality (plot xyz) as markdown compatible text
   d. topics / keywords -> enables task-view like connections
   e. (optionally) test code to run + required loaded packages
   f. (optionally) renv specification of test environment
   g. (optionally) docker container to run in (rocker?)
   h. ...
2. Some (3+) example files
3. A small shiny app that can be used to visualize and browse the data / demonstrates the potential queries

[matthiazzz]

That is a great start, Kevin. I think you proposal makes a lot of sense.
I think we can skip 3) for the POC.

Regarding 1
There are several documents mentioned. I assume the first one would refer to technical specifications outlining the general idea and implementation of the individual package assessment? Something we would start with first, I assume?

Perhaps it would be interesting to explore the idea that the "knowledge sharing" is done via custom test cases following a naming convention i.e. for dosefinding we would create rmes_dosefinding test and knowlege sharing (either within the rmes packages or as an additional package)? Organizing of these assessments could be done by the main package.
Content could be flexible, like any kind of documentation as a md/quarto file, additional test-cases by means of unit test, a test pipeline for CI/CD integration, etc. High level info of this could be place in something like a description file wich which could be parsed by rmes and provided as arbitraty output format (static, dynamic, in memory, as file, ...)

We could also automatically generate documentation for storage of assessment results, either again a md/quarto document, or something more technical like a JSON type report file.

This approach would store ALL knowledge in one single packages, rmes. Any contributor could - in theory - add test cases, new packages, etc by a sub-branch, cloning,... and merging to the main branch (after testing). Vignettes would serve as main documentation to the general approach of how rmes is assessing quality and reliability of a package.

[Stefan-Doering-BI]

That is a great start, Kevin. I think you proposal makes a lot of sense.
I think we can skip 3) for the POC.

Agree with the good start and to skip 3). Assumption is, one user-groups that mostly wants to contribute. These are Developers/Validators -group who want add or review/approve validation documentation. For this user-group is probably more ok with a backend heavy solution, as this anyway involved writing and understanding unit tests.
Second group are quality people, they expect a report in a certain format (pdf is not uncommon), which can be imported into a DMS.

(either within the rmes packages or as an additional package)
I think this topic was quickly brought up by Doug. Good practise would be to make it part of the main package unless there are good reasons not to do so. One good reason against that we can't influence the release process of the package. Matthias, unless your idea is to fork the package and a validated version on our platform?