Testcafe Browser Tools.app tries to execute even when there is no interaction with a local browser

[pterofractal]

What is your Scenario?

Hi,

I am trying to run my testcafe tests inside of AWS Device Farm so that I can run my tests on real mobile devices, specifically iOS Safari.

AWS Device Farm uses Appium to handle the communication and send instructions to the remote browser.

I have a test working on my Mac where I am running Appium and using this selenium browser provider my test gets executed and in iOS Simulator I can see the test run and behave correctly.

This is not the case inside of AWS Device Farm.

This problem might be related to this one about System Integrity Protection (SIP) on Mac OS but I can't seem to figure out if this is just a limitation of testcafe and I am out of luck or if there is some workaround. I saw this answer in Stack overflow but I can't use this since the devices in Device farm only have access to the public internet and can't communicate with the server running Appium or a remote testcafe session.

What is the Current behavior?

In AWS Device Farm, when calling createTestCafe I see the following exception being thrown

NativeBinaryHasFailedError: The find-window process failed with the 1 exit code.
The application /Users/device-farm/.testcafe-browser-tools/TestCafe Browser Tools.app/Contents/MacOS/testcafe-browser-tools cannot be opened for an unexpected reason, error=Error Domain=NSOSStatusErrorDomain Code=-10826 "kLSNoLaunchPermissionErr: User doesn't have permission to launch the app (managed networks)" UserInfo={_LSFunction=_LSLaunchWithRunningboard, _LSLine=2508, NSUnderlyingError=0x7f9d7ed2c260 {Error Domain=RBSRequestErrorDomain Code=5 "Launch failed." UserInfo={NSLocalizedFailureReason=Launch failed., NSUnderlyingError=0x7f9d7ed2af50 {Error Domain=OSLaunchdErrorDomain Code=125 "Domain does not support specified action" UserInfo={NSLocalizedFailureReason=Domain does not support specified action}}}}}
    at ChildProcess.<anonymous> (/private/tmp/scratch0ockPu.scratch/test-package2ya78f/node_modules/MangaWebReaderTestCafe/node_modules/testcafe-browser-tools/src/utils/exec.js:74:24)
    at ChildProcess.emit (events.js:314:20)
    at ChildProcess.EventEmitter.emit (domain.js:483:12)
    at Process.ChildProcess._handle.onexit (internal/child_process.js:276:12)
    at Process.callbackTrampoline (internal/async_hooks.js:126:14)

What is the Expected behavior?

I would expect the test to attempt to execute since it shouldn't need to interact with any local browser. The docs on npm for testcafe-browser-tools give me the impression that the application is only needed when trying to interact with a browser installed locally on the platform testcafe is executing on.

In the case of a remote browser, I would expect this application to not run at all

What is your public website URL? (or attach your complete example)

https://www.google.com

What is your TestCafe test code?

import { Selector } from 'testcafe';

fixture`Test Google`.page("https://www.google.com");
test.only('Check Title', async (t) => {
  await t.expect(Selector('#main').visible).eql(true);
});

Your complete configuration file

// Detailed explanation of settings can be found at
// https://devexpress.github.io/testcafe/documentation/using-testcafe/configuration-file.html
{
  "concurrency": 1,
  "skipJsErrors": true,
  "skipUncaughtErrors": true,
  "pageLoadTimeout": 15000,
  "selectorTimeout": 5000,
  "assertionTimeout": 3000,
  "quarantineMode": true,
  "reporter": [
    {
      "name": "spec"
    }
  ],
  "src": "src/tests/**/*.ts"
}

Your complete test report

npm run device-farm-test
> testcafe selenium:safari:iOS

Try to use selenium browser
BrowserProfile: [safari]
browser: [safari]
version: []
platform: [iOS]
proxy: []
platform_name: [undefined]
device_name: [undefined]
os_version: [undefined]
 Running tests in:
 - Safari 15.4 / iOS 15.4

 Test Google
 ✓ Check Title


 1 passed (3s)
BUILD SUCCEEDED

Screenshots

N/A

Steps to Reproduce

1. Install XCode
2. Install Appium npm install -g appium
3. Run Appium on port 4444 appium -p 4444
4. Execute testcafe using selenium browser testcafe selenium:safari:iOS
5. Validate test runs and passes on your local mac
6. Taking the same tests and deploying it to AWS Device Farm fails with the above errors

Excluding steps involved in setting up device farm since I don't think it is necessary. I can add detailed instructions though if you need them.

TestCafe version

1.18.5

Node.js version

v12.20.0

Command-line arguments

testcafe selenium:safari:iOS

Browser name(s) and version(s)

Safari

Platform(s) and version(s)

iOS 15.02

Other

No response

[pterofractal]

Not sure if this is also related to #7028

[pterofractal]

Skipping the permissions check like what is done in this other issue seems to resolve the error.

The tests still fail but that seems to be because the devices can't connect to the mac that is orchestrating the tests. Looking at the video provided by device farm I can see it try to connect to 192.168.1.100 and then fail.

I will close this task since the root cause is the same as #7028. The remaining issue of actually getting tests running on DeviceFarm Mobile at this time isn't due to any issue in TestCafe