Proposal: Memory Safety Scorecard Checks

[nellshamrell]

Memory Safety Scorecard Proposal

The larger discussion on this is in Scorecard #3736.

However, I'd like to start an issue within this repo to discuss this as a SIG and come up with a proposal to bring to Scorecard.

Context

Scorecard consists of checks and probes.

A check has many probes. A probe may or may not be associated with a check.

Proposal

Add a Memory Safety Check to Scorecard. Add a probe for each language ecosystem (starting with only one or two and iterating).

The closest equivalent check to our potential memory safety check (one that addresses multiple ecosystems and tools) is the fuzzing check. Currently the fuzzing check has its logic for different ecosystems in the check, rather than in probes. I discussed this in the scorecard Slack channel and it makes more sense to implement a memory safety check as general logic in the check, and language/ecosystem specific logic in individual probes.

Before we discuss a technical implementation of this (and there is some guidance in the scorecard repo on how to implement checks and probes), I'd like to work out a process for submitting memory safety probes and maintaining them (maintaining is always the key).

I have reached out to the scorecard team asking how they currently maintain existing checks. I will update this issue when I get an answer.

UPDATE:

Scorecard does not have anything structured with re: to maintainers for probes. It was discussed in the past, but was not pursued.

[balteravishay]

Update

Scorecard Memory Safety checks are added as probes that are specific to behaviours that are being checked, and cover all the ecosystems that support that behaviour.

Unsafe code block

This probe checks for unsafe code blocks, or allowing for unsafe code blocks in the code.

Ecosystems that have unsafe code blocks features

• go - supported by the probe
• .net c# - supported by the probe
• rust - not currently supported
• java - not currently supported
• swift - not currently supported

Data race detection

This probe is not currently implemented. implementation should support go data race detector and check if any other ecosystems should be supported.