Test compatibility with other tools

[irongut]

Feature Request

CCS has been tested with Coverlet and gcov, test with other tools that output Cobertura format.

• coverage.py
• Istanbul
• Maven
• simplecov
• AltCover

Expected Behaviour

Obtain sample Coberura output from as many testing tools as possible, CCS should parse them correctly.

Additional Context

Linked To

#20 What does complexity mean? (issue with coverage.py)
#22 Allow hiding some output values

[irongut]

coverage.py Sample

<coverage branch-rate="0" branches-covered="0" branches-valid="0" complexity="0" line-rate="0.6500" lines-covered="65" lines-valid="100" timestamp="1635484070186" version="6.0.2">
    <packages>
        <package branch-rate="0" complexity="0" line-rate="0.6500" name="one" />
        <package branch-rate="0" complexity="0" line-rate="0.6000" name="two" />
        <package branch-rate="0" complexity="0" line-rate="0.7000" name="three" />
    </packages>
</coverage>

(Details not relevant to CCS omitted.)

[pboling]

Tried it with simplecov & simplecov-cobertura, and it failed, but that may be caused by the options I have enabled not being supported by simplecov? If that's the case this error should be handled more gracefully or avoided altogether.

Here's the log:

Run irongut/CodeCoverageSummary@v1.[2](https://github.com/oauth-xx/oauth2/runs/5226726966?check_suite_focus=true#step:9:2).0
  with:
    filename: ./coverage/coverage.xml
    badge: true
    fail_below_min: true
    format: markdown
    hide_branch_rate: false
    hide_complexity: true
    indicators: true
    output: both
    thresholds: 95 97
  env:
    CC_TEST_REPORTER_ID: ***
    GIT_COMMIT_SHA: 5e7798ac8bfdc4f[3](https://github.com/oauth-xx/oauth2/runs/5226726966?check_suite_focus=true#step:9:3)d8537d[4](https://github.com/oauth-xx/oauth2/runs/5226726966?check_suite_focus=true#step:9:4)23af23be[5](https://github.com/oauth-xx/oauth2/runs/5226726966?check_suite_focus=true#step:9:5)f33ec[6](https://github.com/oauth-xx/oauth2/runs/5226726966?check_suite_focus=true#step:9:6)81
    GIT_BRANCH: 1-4-stable
/usr/bin/docker run --name ghcrioirongutcodecoveragesummaryv120_db1[7](https://github.com/oauth-xx/oauth2/runs/5226726966?check_suite_focus=true#step:9:7)64 --label 7db2d7 --workdir /github/workspace --rm -e CC_TEST_REPORTER_ID -e GIT_COMMIT_SHA -e GIT_BRANCH -e INPUT_FILENAME -e INPUT_BADGE -e INPUT_FAIL_BELOW_MIN -e INPUT_FORMAT -e INPUT_HIDE_BRANCH_RATE -e INPUT_HIDE_COMPLEXITY -e INPUT_INDICATORS -e INPUT_OUTPUT -e INPUT_THRESHOLDS -e HOME -e GITHUB_JOB -e GITHUB_REF -e GITHUB_SHA -e GITHUB_REPOSITORY -e GITHUB_REPOSITORY_OWNER -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RETENTION_DAYS -e GITHUB_RUN_ATTEMPT -e GITHUB_ACTOR -e GITHUB_WORKFLOW -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GITHUB_EVENT_NAME -e GITHUB_SERVER_URL -e GITHUB_API_URL -e GITHUB_GRAPHQL_URL -e GITHUB_REF_NAME -e GITHUB_REF_PROTECTED -e GITHUB_REF_TYPE -e GITHUB_WORKSPACE -e GITHUB_ACTION -e GITHUB_EVENT_PATH -e GITHUB_ACTION_REPOSITORY -e GITHUB_ACTION_REF -e GITHUB_PATH -e GITHUB_ENV -e RUNNER_OS -e RUNNER_ARCH -e RUNNER_NAME -e RUNNER_TOOL_CACHE -e RUNNER_TEMP -e RUNNER_WORKSPACE -e ACTIONS_RUNTIME_URL -e ACTIONS_RUNTIME_TOKEN -e ACTIONS_CACHE_URL -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" -v "/home/runner/work/_temp/_runner_file_commands":"/github/file_commands" -v "/home/runner/work/oauth2/oauth2":"/github/workspace" ghcr.io/irongut/codecoveragesummary:v1.2.0  "--files" "./coverage/coverage.xml" "--badge" "true" "--fail" "true" "--format" "markdown" "--hidebranch" "false" "--hidecomplexity" "true" "--indicators" "true" "--output" "both" "--thresholds" "[9](https://github.com/oauth-xx/oauth2/runs/5226726966?check_suite_focus=true#step:9:9)5 97"
Code Coverage File: ./coverage/coverage.xml
Parse Error: Sequence contains no elements
Error: System.NullReferenceException - Object reference not set to an instance of an object.

Going to tweak options and see if I can get it working.

[pboling]

Simplecov Sample (Ruby, branch coverage not enabled)

<?xml version='1.0'?> 
<!DOCTYPE coverage SYSTEM "http://cobertura.sourceforge.net/xml/coverage-04.dtd">
<!-- Generated by simplecov-cobertura version 2.1.0 (https://github.com/dashingrocket/simplecov-cobertura) -->
<coverage line-rate="0.97" lines-covered="350" lines-valid="362" complexity="0" version="0" timestamp="1645060052">
  <sources>
    <source>/Users/pboling/src/my/oauth2</source>
  </sources>
  <packages>
    <package name="oauth2" line-rate="0.97" complexity="0">
      <classes>
        <class name="oauth2" filename="lib/oauth2.rb" line-rate="1.0" complexity="0">
          <methods/>
          <lines>
            <line number="1" hits="1"/>
            <line number="2" hits="1"/>
            <line number="3" hits="1"/>
            <line number="4" hits="1"/>
            <line number="5" hits="1"/>
            <line number="6" hits="1"/>
            <line number="7" hits="1"/>
            <line number="8" hits="1"/>
            <line number="9" hits="1"/>
            <line number="10" hits="1"/>
            <line number="11" hits="1"/>
            <line number="12" hits="1"/>
          </lines>
        </class>
        <class name="access_token" filename="lib/oauth2/access_token.rb" line-rate="0.96" complexity="0">
          <methods/>
          <lines>
            <line number="1" hits="1"/>
            <line number="2" hits="1"/>
            <line number="3" hits="1"/>
            <line number="4" hits="1"/>
            <line number="7" hits="1"/>
            <line number="13" hits="1"/>
            <line number="14" hits="7"/>
            <line number="15" hits="7"/>
            <line number="23" hits="1"/>
            <line number="24" hits="1"/>
            <line number="41" hits="1"/>
            <line number="42" hits="151"/>
            <line number="43" hits="151"/>
            <line number="44" hits="151"/>
            <line number="45" hits="151"/>
            <line number="46" hits="453"/>
            <line number="48" hits="151"/>
            <line number="49" hits="151"/>
            <line number="50" hits="151"/>
            <line number="51" hits="151"/>
            <line number="52" hits="151"/>
            <line number="55" hits="151"/>
            <line number="61" hits="1"/>
            <line number="62" hits="6"/>
            <line number="68" hits="1"/>
            <line number="69" hits="11"/>
            <line number="75" hits="1"/>
            <line number="76" hits="3"/>
            <line number="83" hits="1"/>
            <line number="84" hits="2"/>
            <line number="86" hits="2"/>
            <line number="87" hits="2"/>
            <line number="88" hits="2"/>
            <line number="89" hits="2"/>
            <line number="90" hits="2"/>
            <line number="91" hits="2"/>
            <line number="97" hits="1"/>
            <line number="98" hits="1"/>
            <line number="107" hits="1"/>
            <line number="108" hits="20"/>
            <line number="109" hits="20"/>
            <line number="115" hits="1"/>
            <line number="116" hits="2"/>
            <line number="122" hits="1"/>
            <line number="123" hits="18"/>
            <line number="129" hits="1"/>
            <line number="130" hits="2"/>
            <line number="136" hits="1"/>
            <line number="137" hits="0"/>
            <line number="143" hits="1"/>
            <line number="144" hits="2"/>
            <line number="148" hits="1"/>
            <line number="149" hits="8"/>
            <line number="152" hits="1"/>
            <line number="154" hits="1"/>
            <line number="155" hits="20"/>
            <line number="157" hits="8"/>
            <line number="158" hits="8"/>
            <line number="160" hits="8"/>
            <line number="161" hits="8"/>
            <line number="163" hits="4"/>
            <line number="164" hits="4"/>
            <line number="165" hits="4"/>
            <line number="167" hits="0"/>
            <line number="171" hits="0"/>
            <line number="175" hits="1"/>
            <line number="176" hits="13"/>
            <line number="178" hits="12"/>
          </lines>
        </class>
        <class name="authenticator" filename="lib/oauth2/authenticator.rb" line-rate="1.0" complexity="0">
          <methods/>
          <lines>
            <line number="1" hits="1"/>
            <line number="3" hits="1"/>
            <line number="4" hits="1"/>
            <line number="5" hits="1"/>
            <line number="7" hits="1"/>
            <line number="8" hits="106"/>
            <line number="9" hits="106"/>
            <line number="10" hits="106"/>
            <line number="22" hits="1"/>
            <line number="23" hits="106"/>
            <line number="25" hits="14"/>
            <line number="27" hits="89"/>
            <line number="29" hits="1"/>
            <line number="31" hits="1"/>
            <line number="33" hits="1"/>
            <line number="37" hits="1"/>
            <line number="38" hits="15"/>
            <line number="41" hits="1"/>
            <line number="45" hits="1"/>
            <line number="46" hits="89"/>
            <line number="51" hits="1"/>
            <line number="52" hits="1"/>
            <line number="57" hits="1"/>
            <line number="58" hits="14"/>
            <line number="59" hits="14"/>
            <line number="60" hits="14"/>
            <line number="64" hits="1"/>
            <line number="65" hits="14"/>
          </lines>
        </class>
        <class name="client" filename="lib/oauth2/client.rb" line-rate="0.95" complexity="0">
          <methods/>
          <lines>
            <line number="1" hits="1"/>
            <line number="2" hits="1"/>
            <line number="4" hits="1"/>
            <line number="6" hits="1"/>
            <line number="7" hits="1"/>
            <line number="9" hits="1"/>
            <line number="10" hits="1"/>
            <line number="11" hits="1"/>
            <line number="31" hits="1"/>
            <line number="32" hits="204"/>
            <line number="33" hits="204"/>
            <line number="34" hits="204"/>
            <line number="35" hits="204"/>
            <line number="36" hits="204"/>
            <line number="39" hits="204"/>
            <line number="49" hits="204"/>
            <line number="55" hits="1"/>
            <line number="56" hits="0"/>
            <line number="57" hits="0"/>
            <line number="61" hits="1"/>
            <line number="62" hits="423"/>
            <line number="63" hits="158"/>
            <line number="64" hits="158"/>
            <line number="65" hits="134"/>
            <line number="66" hits="134"/>
            <line number="69" hits="158"/>
            <line number="76" hits="1"/>
            <line number="77" hits="11"/>
            <line number="78" hits="11"/>
            <line number="84" hits="1"/>
            <line number="85" hits="100"/>
            <line number="100" hits="1"/>
            <line number="101" hits="141"/>
            <line number="103" hits="141"/>
            <line number="105" hits="141"/>
            <line number="106" hits="141"/>
            <line number="107" hits="141"/>
            <line number="109" hits="141"/>
            <line number="111" hits="141"/>
            <line number="113" hits="3"/>
            <line number="114" hits="3"/>
            <line number="115" hits="3"/>
            <line number="117" hits="2"/>
            <line number="118" hits="1"/>
            <line number="119" hits="1"/>
            <line number="121" hits="2"/>
            <line number="124" hits="127"/>
            <line number="126" hits="11"/>
            <line number="127" hits="11"/>
            <line number="129" hits="2"/>
            <line number="130" hits="2"/>
            <line number="132" hits="0"/>
            <line number="133" hits="0"/>
            <line number="143" hits="1"/>
            <line number="144" hits="97"/>
            <line number="145" hits="182"/>
            <line number="146" hits="2"/>
            <line number="148" hits="180"/>
            <line number="151" hits="97"/>
            <line number="153" hits="97"/>
            <line number="154" hits="97"/>
            <line number="155" hits="97"/>
            <line number="156" hits="97"/>
            <line number="157" hits="79"/>
            <line number="158" hits="79"/>
            <line number="160" hits="18"/>
            <line number="161" hits="18"/>
            <line number="163" hits="97"/>
            <line number="164" hits="97"/>
            <line number="167" hits="97"/>
            <line number="169" hits="0"/>
            <line number="172" hits="97"/>
            <line number="173" hits="3"/>
            <line number="174" hits="3"/>
            <line number="176" hits="94"/>
            <line number="182" hits="1"/>
            <line number="183" hits="48"/>
            <line number="189" hits="1"/>
            <line number="190" hits="5"/>
            <line number="196" hits="1"/>
            <line number="197" hits="11"/>
            <line number="203" hits="1"/>
            <line number="204" hits="21"/>
            <line number="207" hits="1"/>
            <line number="208" hits="9"/>
            <line number="227" hits="1"/>
            <line number="228" hits="55"/>
            <line number="229" hits="2"/>
            <line number="231" hits="53"/>
            <line number="235" hits="1"/>
            <line number="236" hits="91"/>
            <line number="237" hits="91"/>
            <line number="240" hits="1"/>
            <line number="242" hits="1"/>
            <line number="243" hits="97"/>
            <line number="244" hits="97"/>
            <line number="246" hits="93"/>
            <line number="250" hits="93"/>
            <line number="251" hits="1"/>
            <line number="253" hits="92"/>
          </lines>
        </class>
        <class name="error" filename="lib/oauth2/error.rb" line-rate="1.0" complexity="0">
          <methods/>
          <lines>
            <line number="1" hits="1"/>
            <line number="2" hits="1"/>
            <line number="3" hits="1"/>
            <line number="7" hits="1"/>
            <line number="8" hits="14"/>
            <line number="9" hits="14"/>
            <line number="11" hits="14"/>
            <line number="12" hits="7"/>
            <line number="13" hits="7"/>
            <line number="14" hits="7"/>
            <line number="17" hits="14"/>
            <line number="23" hits="1"/>
            <line number="24" hits="14"/>
            <line number="26" hits="14"/>
            <line number="28" hits="14"/>
            <line number="29" hits="7"/>
            <line number="30" hits="7"/>
            <line number="32" hits="7"/>
            <line number="35" hits="14"/>
            <line number="37" hits="14"/>
          </lines>
        </class>
        <class name="mac_token" filename="lib/oauth2/mac_token.rb" line-rate="0.95" complexity="0">
          <methods/>
          <lines>
            <line number="1" hits="1"/>
            <line number="2" hits="1"/>
            <line number="3" hits="1"/>
            <line number="4" hits="1"/>
            <line number="6" hits="1"/>
            <line number="7" hits="1"/>
            <line number="14" hits="1"/>
            <line number="15" hits="3"/>
            <line number="18" hits="1"/>
            <line number="30" hits="1"/>
            <line number="31" hits="19"/>
            <line number="32" hits="19"/>
            <line number="34" hits="18"/>
            <line number="43" hits="1"/>
            <line number="44" hits="4"/>
            <line number="46" hits="4"/>
            <line number="47" hits="4"/>
            <line number="49" hits="4"/>
            <line number="53" hits="1"/>
            <line number="54" hits="1"/>
            <line number="61" hits="1"/>
            <line number="62" hits="9"/>
            <line number="63" hits="9"/>
            <line number="65" hits="9"/>
            <line number="67" hits="8"/>
            <line number="69" hits="7"/>
            <line number="71" hits="7"/>
            <line number="80" hits="1"/>
            <line number="82" hits="8"/>
            <line number="91" hits="8"/>
            <line number="97" hits="1"/>
            <line number="98" hits="19"/>
            <line number="101" hits="1"/>
            <line number="103" hits="0"/>
            <line number="107" hits="17"/>
            <line number="109" hits="0"/>
            <line number="112" hits="1"/>
            <line number="116" hits="1"/>
            <line number="120" hits="1"/>
            <line number="124" hits="1"/>
            <line number="125" hits="8"/>
          </lines>
        </class>
        <class name="response" filename="lib/oauth2/response.rb" line-rate="1.0" complexity="0">
          <methods/>
          <lines>
            <line number="1" hits="1"/>
            <line number="2" hits="1"/>
            <line number="3" hits="1"/>
            <line number="5" hits="1"/>
            <line number="7" hits="1"/>
            <line number="8" hits="1"/>
            <line number="9" hits="1"/>
            <line number="14" hits="62"/>
            <line number="15" hits="44"/>
            <line number="16" hits="3"/>
            <line number="20" hits="1"/>
            <line number="32" hits="1"/>
            <line number="33" hits="3"/>
            <line number="34" hits="3"/>
            <line number="35" hits="3"/>
            <line number="36" hits="6"/>
            <line number="46" hits="1"/>
            <line number="47" hits="147"/>
            <line number="48" hits="147"/>
            <line number="52" hits="1"/>
            <line number="53" hits="6"/>
            <line number="57" hits="1"/>
            <line number="58" hits="150"/>
            <line number="62" hits="1"/>
            <line number="63" hits="154"/>
            <line number="69" hits="1"/>
            <line number="70" hits="134"/>
            <line number="72" hits="131"/>
            <line number="76" hits="1"/>
            <line number="77" hits="244"/>
            <line number="81" hits="1"/>
            <line number="82" hits="244"/>
            <line number="84" hits="244"/>
            <line number="89" hits="1"/>
            <line number="90" hits="1"/>
          </lines>
        </class>
        <class name="assertion" filename="lib/oauth2/strategy/assertion.rb" line-rate="0.89" complexity="0">
          <methods/>
          <lines>
            <line number="1" hits="1"/>
            <line number="3" hits="1"/>
            <line number="4" hits="1"/>
            <line number="23" hits="1"/>
            <line number="27" hits="1"/>
            <line number="28" hits="1"/>
            <line number="45" hits="1"/>
            <line number="46" hits="8"/>
            <line number="47" hits="8"/>
            <line number="50" hits="1"/>
            <line number="51" hits="8"/>
            <line number="53" hits="8"/>
            <line number="60" hits="1"/>
            <line number="62" hits="8"/>
            <line number="67" hits="8"/>
            <line number="68" hits="8"/>
            <line number="69" hits="0"/>
            <line number="70" hits="0"/>
          </lines>
        </class>
        <class name="auth_code" filename="lib/oauth2/strategy/auth_code.rb" line-rate="1.0" complexity="0">
          <methods/>
          <lines>
            <line number="1" hits="1"/>
            <line number="2" hits="1"/>
            <line number="6" hits="1"/>
            <line number="10" hits="1"/>
            <line number="11" hits="3"/>
            <line number="17" hits="1"/>
            <line number="18" hits="3"/>
            <line number="27" hits="1"/>
            <line number="28" hits="44"/>
            <line number="30" hits="44"/>
          </lines>
        </class>
        <class name="base" filename="lib/oauth2/strategy/base.rb" line-rate="1.0" complexity="0">
          <methods/>
          <lines>
            <line number="1" hits="1"/>
            <line number="2" hits="1"/>
            <line number="3" hits="1"/>
            <line number="4" hits="1"/>
            <line number="5" hits="95"/>
          </lines>
        </class>
        <class name="client_credentials" filename="lib/oauth2/strategy/client_credentials.rb" line-rate="1.0" complexity="0">
          <methods/>
          <lines>
            <line number="1" hits="1"/>
            <line number="2" hits="1"/>
            <line number="6" hits="1"/>
            <line number="10" hits="1"/>
            <line number="11" hits="1"/>
            <line number="18" hits="1"/>
            <line number="19" hits="20"/>
            <line number="20" hits="20"/>
          </lines>
        </class>
        <class name="implicit" filename="lib/oauth2/strategy/implicit.rb" line-rate="1.0" complexity="0">
          <methods/>
          <lines>
            <line number="1" hits="1"/>
            <line number="2" hits="1"/>
            <line number="6" hits="1"/>
            <line number="10" hits="1"/>
            <line number="11" hits="3"/>
            <line number="17" hits="1"/>
            <line number="18" hits="3"/>
            <line number="24" hits="1"/>
            <line number="25" hits="1"/>
          </lines>
        </class>
        <class name="password" filename="lib/oauth2/strategy/password.rb" line-rate="1.0" complexity="0">
          <methods/>
          <lines>
            <line number="1" hits="1"/>
            <line number="2" hits="1"/>
            <line number="6" hits="1"/>
            <line number="10" hits="1"/>
            <line number="11" hits="1"/>
            <line number="19" hits="1"/>
            <line number="20" hits="10"/>
            <line number="23" hits="10"/>
          </lines>
        </class>
      </classes>
    </package>
  </packages>
</coverage>

[irongut]

Hi @pboling thanks for the sample, that's always useful.

Looking at the sample there are no branch metrics on coverage or package nodes. Unfortunately the hide_branch_rate option only hides it from the report, the action still attempts to calculate branch metrics.

It's funny how many different ways different tools can output what is supposed to be the same format. I'll need to make some changes to prevent attempting to calculate metrics that don't exist in the data.

[pboling]

@irongut Branch coverage can be turned on, but I've never tried it. I'll see what that generates now. But, yeah, definitely good to support the lack of any specific feature. Always assume the null case :)

[irongut]

@pboling Looking at the docs, enable branch coverage but leave primary coverage set to the default of line. That should be compatible with the current version of CCS.

I'll add a note to the CCS docs and include a fix for missing branch rate in the next version.

[github-actions]

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this issue will be closed in 30 days.