[🚀 Feature]: Selenium 4 Grid Slot Matching RFC

[sbabcoc]

Selenium 4 Grid Slot Matching RFC

Background

A core feature of Selenium 4 Grid is slot matching – the process of pairing a client “new session” request with a node that can satisfy the request. This process involves evaluating the client’s desired capabilities against the stereotypes of available nodes to determine which (if any) matches the request. The default implementation is defined in a class named DefaultSlotMatcher.

Implementation Details of DefaultSlotMatcher

• Step 1: Initial match – evaluate request against defined stereotype values:
  • Filter out extension capabilities (names that contain “:”)
  • Filter out platformName capability
  • Evaluate stereotype value against desired capability value:
    • If desired value is a string, perform case-insensitive comparison…
    • … else if desired value is defined, perform object comparison…
    • … else ignore capability with undefined desired value
• Step 2: Evaluate managed download capability:
  • Match if desired value is undefined or ‘false’…
  • … else match if stereotype supports managed downloads
• Step 3: Evaluate platformVersion capability:
  • Evaluate desired capabilities with names containing platformVersion
  • Match if a stereotype capability with identical name has the same value
• Step 4: Evaluate request against defined extension capability values:
  • Filter out capabilities containing goog:, moz:, ms:, or se:
  • Evaluate stereotype value against desired capability value:
    • If desired value is a string, perform case-insensitive comparison…
    • … else if desired value is defined, perform object comparison…
    • … else ignore capability with undefined desired value
• Step 5: Evaluate browserName capability:
  • Match if desired browserName value is undefined or empty
  • Match if stereotype browserName capability has the desired value
• Step 6: Evaluate browserVersion capability:
  • Match if desired browserVersion value is undefined, empty, or “stable”
  • Match if desired value is equivalent to stereotype browserVersion
• Step 7: Evaluate platformName capability:
  • Match if desired platformName value is undefined
  • Match if stereotype platformName capability has the desired value
  • Match if desired value is equivalent to stereotype platformName

Current Issue – Extension Capability Handling

As currently implemented, DefaultSlotMatcher has one obvious issue at Step 4 in its handling of extension capabilities. It ignores the extension capabilities of three vendors (Google, Mozilla, and Microsoft), but considers the extension capabilities of every other vendor (including Appium and Apple). This produces inconsistent slot matching behavior, with Appium extension capabilities causing slot matching to fail while similar Google extension capabilities would have no ill effect.

Solution – Capability Type Differentiation

The existing handling of extension capabilities for the three “special” vendors suggests that capabilities might be divided into two distinct categories: “identity” values that are considered for slot matching and configuration options that are transmitted to the matched node without analysis.

Patterns from Current Usage

In current usage, values defined within “options” extension capability objects should be ignored for slot matching (i.e. – configuration options). Two additional extension capabilities are also currently ignored: <prefix>:debuggerAddress and <prefix>:loggingPrefs. Every other defined capability for the three “special” vendors is treated as an “identity” value. However, these categories are merely incidental, not formally defined within the specifications. Consequently, there’s no defined mechanism for vendors to add new “identity” values or configuration options.

Proposal: Formally Define Capability Type Differentiation

Formalizing the current implied categorization of capabilities into “identity” values and configuration options will provide two distinct benefits:

1. Consistent treatment of extension capabilities will provide consistent behavior for all vendors.
2. Defining explicit patterns for “identity” values and configuration options enables each vendor to select the desired treatment for their extension capabilities, allowing them to add new capabilities and options as needed and get the expected handling without the need for custom slot matchers.

Definition of configuration options

The proposed definition for configuration options, which will be ignored for purposes of slot matching:

• Any extension capability with prefix se: (Selenium).
• Any extension capability with one of the following suffixes:
  • options
  • Options
  • loggingPrefs
  • debuggerAddress

Definition of “identity” values

All capabilities that don’t conform with the definition of configuration options as defined above will be treated as “identity” values, which will be considered for purposes of slot matching. This includes extension capabilities with “special” prefixes that would previously have been ignored. This change is merely academic, though, given that there are no currently defined “special” extension capabilities that are affected by the revised behavior.

Written with StackEdit.

Usage example

This feature will be implemented in DefaultSlotMatcher. Consequently, anyone using Selenium 4 Grid without explicitly configuring a custom slot matcher will encounter the revised behavior.

[github-actions]

@sbabcoc, thank you for creating this issue. We will troubleshoot it as soon as we can.

---

Info for maintainers

Triage this issue by using labels.

If information is missing, add a helpful comment and then I-issue-template label.

If the issue is a question, add the I-question label.

If the issue is valid but there is no time to troubleshoot it, consider adding the help wanted label.

If the issue requires changes or fixes from an external project (e.g., ChromeDriver, GeckoDriver, MSEdgeDriver, W3C), add the applicable G-* label, and it will provide the correct link and auto-close the issue.

After troubleshooting the issue, please add the R-awaiting answer label.

Thank you!

[titusfortner]

Wait, how do we match on platformVersion if that isn't even a valid capability?

If we filter out platformName in step 1 how are we matching it in step 7?

I don't like the idea of artificially declaring some values to be identity and some to be configuration. It's an arbitrary distinction that isn't in the spec.

I'd like to go the other direction and not treat any capabilities as special. Everything that I know of implements platformName, but Appium sessions don't always have browserName and browserVersion value is completely ignored by chromedriver regardless of what you put.

My question is: why do we need to treat any of these keys as special if the user isn't going to manually put them the stereotype? Is there some place where a more complicated stereotype is automatically generated?

[sbabcoc]

@titusfortner The check for platformVersion is performed in the existing DefaultSlotMatcher implementation. This isn't a new feature being proposed in this RFC, but it may have been added for the benefit of a specific vendor. As described in the Current Issues section, the existing implementation is inconsistent, treating extension capabilities with "special" prefixes differently than it treats extension capabilities with any other prefix.

As stated in this RFC, the purpose of capability differentiation is to enable vendors to decide which capabilities should be considered for slot matching, and which should be passed on to selected end nodes without analysis. This is currently only possible via custom slot matchers, which are mutually exclusive per hub. Another possible mechanism to enable vendor-specific behavior would be to add another method to WebDriverInfo to enable vendors to extend or override the matching performed by DefaultSlotMatcher.

Regarding the concept of formally defining two distinct capability categories versus how things are defined in the specification, I'm unaware of any detailed specifications for Selenium Grid, let alone the esoteric details of slot matching. As such, the implementation is the spec. The W3C WebDriver spec provides a lot of wiggle room regarding the handling of extension capabilities. It even presents the option of defining routing-specific capabilities to control how intermediate nodes match "new session" requests with compatible end nodes. This feature could potentially introduce an additional capability category (routing options), but I don't think this is strictly necessary.

[titusfortner]

I know I'm trying to understand why the existing implementation is like that. (and we probably need to wait until after the conference for the Grid experts to have time to answer) 😄

[sbabcoc]

Regarding the handling of platformName, each step is able to evaluate the entire set of defined capabilities. Filtering reduces the number of capabilities that must be evaluated on each step by eliminating irrelevant values.

The goal of this RFC is to replace the current arbitrary disparities in how extension capabilities are handled with a defined pattern to differentiate "identity" values from configuration options. This will enable vendors to explicitly select the treatment they need, and it will remove the ambiguities currently faced by Grid creators and Grid users.

[VietND96]

In DefaultSlotMatcher, there is a comment for platformVersion

private Boolean platformVersionMatch(Capabilities stereotype, Capabilities capabilities) {
/*
This platform version match is not W3C compliant but users can add Appium servers as
Nodes, so we avoid delaying the match until the Slot, which makes the whole matching
process faster.
*/

When you are reading Appium v2 docs. You will see 2 ways to pass the platformVersion

1. { "appium:platformVersion": "14" }
2. { "appium:options": { "platformVersion": "14" } }

I think we should remove this platformVersionMatch to avoid confusion. Let users define explicitly as above.

[VietND96]

Filter out capabilities containing goog:, moz:, ms:, or se:

This is valid. For example, if you consider the case of Apple, we can add its vendor prefix too.
Moreover, I think we had a long discussion in this PR #14485. Where we propose that skip capability something like $cloud:options, $vendor:options from slot matcher. Whether users set via node stereotypes or request capabilities, those options will be merged and sent to particular vendor only.

[VietND96]

Another thing that I can see is if you are referring to the Appium test through Grid using Relay Node. The matching is involved another implementation in RelaySessionFactory.java. So, if the request passes DefaultSlotMatcher and is assigned to Relay Node, if it isn't matching the logic in test(), you also can see the error SessionNotCreatedException("New session request capabilities do not " + "match the stereotype.")); from RelaySessionFactory

[sbabcoc]

@VietND96 The platformVersion capability should be treated as an "identity" value, which means that it must be declared as appium:platformVersion under the specifications defined by this RFC. There would be no need for Step 3, so the platformVersionMatch method would be eliminated.

To treat all prefixes uniformly, all extension capabilities except those beginning with se: will be treated as configuration options, which will not be considered for slot matching. Extension capabilities ending with options, Options, loggingPrefs, or debuggerAddress will also be treated as configuration options.

The matching performed by RelaySessionFactory will also be revised to remove Appium-specific implementation. The updated matching performed by DefaultSlotMatcher will no longer require this implementation.

[VietND96]

I am not sure how you can test with platformVersion. Immediately, Grid will throw the error

selenium.common.exceptions.WebDriverException: Message: Illegal key values seen in w3c capabilities: [platformVersion]
Stacktrace:
java.lang.IllegalArgumentException: Illegal key values seen in w3c capabilities: [platformVersion]
at org.openqa.selenium.remote.NewSessionPayload.lambda$validate$5(NewSessionPayload.java:163)

Here is the Appium tests through Grid that I tried locally, for both hybrid browser session and native app, it works - https://github.com/NDViet/test-grid-relay-appium

[sbabcoc]

The extension capability appium:platformVersion is totally W3C compliant. This value cannot be specified without its appium: prefix. Since this capability will be treated as an "identity" value, it will be evaluated against available slot stereotypes to determine which node matches the desired capabilities.

I have an open PR that implements the patterns of this RFC here: #14485

[sbabcoc]

@VietND96 Appium works with Grid though Relay, but the way we've implemented extension capability support is inconsistent and inflexible. My local-grid-parent project is able to stand up Grid instances that support all flavors of Appium engines. Configuration of such instances is not trivial, and much of the complexity is rooted in the handling of extension capabilities.

[titusfortner]

I agree on removing platformVersionMatch and have Appium users namespace their values with appium:

I still am confused about why we need to treat any of these keywords as "non-identity" values. Aren't we iterating over the stereotype, not the capabilities? In which case if it isn't in the stereotype it doesn't matter?