Skip to content

Expose Issue severity and isFailure as API #1075

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

suzannaratcliff
Copy link
Contributor

@suzannaratcliff suzannaratcliff commented Apr 14, 2025

Expose Issue Severity and isFailure as API

Motivation:

To allow users to create issues on their tests without causing the tests to fail, it would be helpful to support creating issues with a warning severity.

Test Warnings:
Severity:

  public enum Severity: Sendable {
    /// The severity level for an issue which should be noted but is not
    /// necessarily an error.
    ///
    /// An issue with warning severity does not cause the test it's associated
    /// with to be marked as a failure, but is noted in the results.
    case warning

    /// The severity level for an issue which represents an error in a test.
    ///
    /// An issue with error severity causes the test it's associated with to be
    /// marked as a failure.
    case error
  }

This will allow users to create issues but they will not have to fail the test

Example Usage:

Issue.record("My comment", severity: .warning)

Later on users can inspect the severity of their issue with the severity property.

/// The severity of the issue.
public var severity: Severity
  /// Whether or not this issue should cause the test it's associated with to be
  /// considered a failure.
  ///
  /// The value of this property is `true` for issues which have a severity level of
  /// ``Issue/Severity/error`` or greater and are not known issues via
  /// ``withKnownIssue(_:isIntermittent:sourceLocation:_:when:matching:)``.
  /// Otherwise, the value of this property is `false.`
  ///
  /// Use this property to determine if an issue should be considered a failure, instead of
  /// directly comparing the value of the ``severity`` property.
public var isFailure: Bool

This API will be useful for teams who would like to inspect if their issue is a failing issue
https://developer.apple.com/documentation/xctest/xctestcase/record(_:)

Modifications:

  • Add isFailure variable
  • Remove experimental spi from Severity

Checklist:

  • Code and documentation should follow the style of the Style Guide.
  • If public symbols are renamed or modified, DocC references should be updated.

public var severity: Severity

/// If the issues is a failing issue.
public var isFailure: Bool {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this member is necessary; whether or not a warning is a failure might depend on external configuration (some equivalent to -Werror perhaps.)

@grynspan grynspan added this to the Swift 6.x milestone Apr 14, 2025
@grynspan grynspan added enhancement New feature or request public-api Affects public API api-proposal API proposal PRs (documentation only) issue-handling Related to Issue handling within the testing library labels Apr 14, 2025
@suzannaratcliff suzannaratcliff force-pushed the suzannaratcliff/expose-severity-as-api branch from c383628 to 7616fc7 Compare April 15, 2025 18:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api-proposal API proposal PRs (documentation only) enhancement New feature or request issue-handling Related to Issue handling within the testing library public-api Affects public API
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants