Merge pull request #599 from brave/laurenwags-macos-update

Update desktop manual passes for macOS

WikiTemplate/Desktop/Minor_CR_Bump/wikitemplate-minorCRbump-macOS-x64.md

@@ -0,0 +1,62 @@
+### Installer
+
+- [ ] Ensured that the following executables work as expected
+   - [ ] `Brave-Browser-x64.dmg`
+      - [ ] Check executable size, should be `~290mb`
+      - [ ]  Check signature: If OS Run `spctl --assess --verbose` for the installed version and make sure it returns `accepted`.
+   - [ ] `Brave-Browser-universal.dmg`
+       - [ ] Check executable size, should be `~500mb`
+       - [ ]  Check signature: If OS Run `spctl --assess --verbose` for the installed version and make sure it returns `accepted`.
+
+### Widevine
+
+- [ ]  Using `x64` build:
+   - [ ]  Verify `Widevine Notification` is shown when you visit Netflix for the first time
+   - [ ]  Test that you can stream on Netflix on a fresh profile after installing Widevine 
+- [ ]  Using `universal` build:
+   - [ ]  Verify `Widevine Notification` is shown when you visit Netflix for the first time
+   - [ ]  Test that you can stream on Netflix on a fresh profile after installing Widevine 
+
+#### Upgrade - `Brave-Browser-x64.dmg`
+
+- [ ] Make sure that data from the last version appears in the new version OK
+- [ ] Ensure that `brave://version` lists the expected Brave & Chromium versions
+- [ ] With data from the last version, verify that
+  - [ ] Bookmarks on the bookmark toolbar and bookmark folders can be opened
+  - [ ] Cookies are preserved
+  - [ ] Installed extensions are retained and work correctly
+  - [ ] Opened tabs can be reloaded
+  - [ ] Stored passwords are preserved
+  - [ ] Sync chain created in previous version is retained 
+  - [ ] Social media blocking buttons changes are retained
+  - [ ] Custom filters under brave://settings/shields/filters are retained
+  - [ ] Custom lists under brave://settings/shields/filters are retained
+  - [ ] Rewards
+    - [ ] BAT balance is retained
+    - [ ] Auto-contribute list is retained
+    - [ ] Both Tips and Monthly Contributions are retained
+    - [ ] Panel transactions list is retained
+    - [ ] Changes to rewards settings are retained
+    - [ ] Ensure that Auto Contribute is not being enabled when upgrading to a new version if AC was disabled
+  - [ ] Ads
+    - [ ] Both `Estimated pending rewards` & `Ad notifications received this month` are retained
+    - [ ] Changes to ads settings are retained
+    - [ ] Ensure that ads are not being enabled when upgrading to a new version if they were disabled
+    - [ ] Ensure that ads are not disabled when upgrading to a new version if they were enabled 
+
+#### Upgrade - `Brave-Browser-universal.dmg`
+
+Pre-requisite: Make sure that the previous version is installed using the universal build
+- [ ] Make sure that after upgrade, the universal build is upgraded to the appropriate architecture specific version
+- [ ] Confirm that the .app file size has decreased as a result of the upgrade
+- [ ] Make sure that data from the last version appears in the new version OK
+- [ ] Ensure that `brave://version` lists the expected Brave & Chromium versions
+- [ ] With data from the last version, verify that
+  - [ ] Bookmarks on the bookmark toolbar and bookmark folders can be opened
+  - [ ] Cookies are preserved
+  - [ ] Opened tabs can be reloaded
+  - [ ] Stored passwords are preserved
+  - [ ] Sync chain created in previous version is retained 
+  - [ ] Social media blocking buttons changes are retained 
+  - [ ] Custom filters under brave://settings/shields/filters are retained
+  - [ ] Custom lists under brave://settings/shields/filters are retained

WikiTemplate/Desktop/Minor_CR_Bump/wikitemplate-minorCRbumpDesktop.md

@@ -1,15 +1,15 @@
 ### Installer
 
 - [ ]  Check signature: 
-  - [ ] If macOS, using x64 binary run `spctl --assess --verbose` for the installed version and make sure it returns `accepted` 
-  - [ ] If macOS, using universal binary run `spctl --assess --verbose` for the installed version and make sure it returns `accepted` 
+  - [ ] If macOS, using `arm64` binary run `spctl --assess --verbose` for the installed version and make sure it returns `accepted` 
+  - [ ] If macOS, using `universal` binary run `spctl --assess --verbose` for the installed version and make sure it returns `accepted` 
   - [ ] If Windows right click on the `brave_installer-x64.exe` and go to Properties, go to the Digital Signatures tab and double click on the signature.  Make sure it says "The digital signature is OK" in the popup window
 
 ### Widevine
 
 - [ ]  Verify `Widevine Notification` is shown when you visit Netflix for the first time
 - [ ]  Test that you can stream on Netflix on a fresh profile after installing Widevine
-- [ ]  If macOS, run the above Widevine tests for both `x64` and `universal` builds
+- [ ]  If macOS, run the above Widevine tests for both `arm64` and `universal` builds
 
 ### Rewards
 

WikiTemplate/Desktop/wikitemplate-macOS-x64.md

@@ -0,0 +1,65 @@
+### Installer
+
+- [ ] Ensured that the following executables work as expected
+   - [ ] `Brave-Browser-x64.dmg`
+      - [ ] Check executable size, should be `~290mb`
+      - [ ]  Check signature: If OS Run `spctl --assess --verbose` for the installed version and make sure it returns `accepted`.
+   - [ ] `Brave-Browser-universal.dmg`
+       - [ ] Check executable size, should be `~500mb`
+       - [ ]  Check signature: If OS Run `spctl --assess --verbose` for the installed version and make sure it returns `accepted`.
+   - [ ] `Brave-Browser-universal.pkg`
+       - [ ] Check executable size, should be `~500mb`
+       - [ ]  Check signature: If OS Run `spctl --assess --verbose` for the installed version and make sure it returns `accepted`.
+
+### Widevine
+
+- [ ]  Using `x64` build:
+   - [ ]  Verify `Widevine Notification` is shown when you visit Netflix for the first time
+   - [ ]  Test that you can stream on Netflix on a fresh profile after installing Widevine 
+- [ ]  Using `universal` build:
+   - [ ]  Verify `Widevine Notification` is shown when you visit Netflix for the first time
+   - [ ]  Test that you can stream on Netflix on a fresh profile after installing Widevine 
+
+#### Upgrade - `Brave-Browser-x64.dmg`
+
+- [ ] Make sure that data from the last version appears in the new version OK
+- [ ] Ensure that `brave://version` lists the expected Brave & Chromium versions
+- [ ] With data from the last version, verify that
+  - [ ] Bookmarks on the bookmark toolbar and bookmark folders can be opened
+  - [ ] Cookies are preserved
+  - [ ] Installed extensions are retained and work correctly
+  - [ ] Opened tabs can be reloaded
+  - [ ] Stored passwords are preserved
+  - [ ] Sync chain created in previous version is retained 
+  - [ ] Social media blocking buttons changes are retained
+  - [ ] Custom filters under brave://settings/shields/filters are retained
+  - [ ] Custom lists under brave://settings/shields/filters are retained
+  - [ ] Rewards
+    - [ ] BAT balance is retained
+    - [ ] Auto-contribute list is retained
+    - [ ] Both Tips and Monthly Contributions are retained
+    - [ ] Panel transactions list is retained
+    - [ ] Changes to rewards settings are retained
+    - [ ] Ensure that Auto Contribute is not being enabled when upgrading to a new version if AC was disabled
+  - [ ] Ads
+    - [ ] Both `Estimated pending rewards` & `Ad notifications received this month` are retained
+    - [ ] Changes to ads settings are retained
+    - [ ] Ensure that ads are not being enabled when upgrading to a new version if they were disabled
+    - [ ] Ensure that ads are not disabled when upgrading to a new version if they were enabled 
+
+#### Upgrade - `Brave-Browser-universal.dmg`
+
+Pre-requisite: Make sure that the previous version is installed using the universal build
+- [ ] Make sure that after upgrade, the universal build is upgraded to the appropriate architecture specific version
+- [ ] Confirm that the .app file size has decreased as a result of the upgrade
+- [ ] Make sure that data from the last version appears in the new version OK
+- [ ] Ensure that `brave://version` lists the expected Brave & Chromium versions
+- [ ] With data from the last version, verify that
+  - [ ] Bookmarks on the bookmark toolbar and bookmark folders can be opened
+  - [ ] Cookies are preserved
+  - [ ] Opened tabs can be reloaded
+  - [ ] Stored passwords are preserved
+  - [ ] Sync chain created in previous version is retained 
+  - [ ] Social media blocking buttons changes are retained 
+  - [ ] Custom filters under brave://settings/shields/filters are retained
+  - [ ] Custom lists under brave://settings/shields/filters are retained

WikiTemplate/Desktop/wikitemplate.md

@@ -2,8 +2,8 @@
 
 - [ ] Check the installer is close to the size of the last release
 - [ ]  Check signature: 
-  - [ ] If macOS, using x64 binary run `spctl --assess --verbose` for the installed version and make sure it returns `accepted` 
-  - [ ] If macOS, using universal binary run `spctl --assess --verbose` for the installed version and make sure it returns `accepted` 
+  - [ ] If macOS, using `arm64` binary run `spctl --assess --verbose` for the installed version and make sure it returns `accepted` 
+  - [ ] If macOS, using `universal` binary run `spctl --assess --verbose` for the installed version and make sure it returns `accepted` 
   - [ ] If Windows right click on the `brave_installer-x64.exe` and go to Properties, go to the Digital Signatures tab and double click on the signature.  Make sure it says "The digital signature is OK" in the popup window
 
 ### About pages
@@ -69,7 +69,7 @@
 - [ ] Test that you can stream on Netflix on a fresh profile after installing Widevine
 - [ ] Verify `Widevine Notification` is shown when you visit HBO Max for the first time
 - [ ] Test that you can stream on HBO Max on a fresh profile after installing Widevine
-- [ ] If macOS, run the above Widevine tests for both `x64` and `universal` builds
+- [ ] If macOS, run the above Widevine tests for both `arm64` and `universal` builds
 
 ### Geolocation
 

brave_testrun_generator.py

@@ -50,8 +50,8 @@ def laptop_testruns(milestonever):
 
     wiki_laptop_file = open("WikiTemplate/Desktop/wikitemplate.md", "r")
     laptop_template = wiki_laptop_file.read()
-    wiki_macOS_arm = open("WikiTemplate/Desktop/wikitemplate-macOS-arm64.md", "r")
-    macOS_arm64 = wiki_macOS_arm.read()
+    wiki_macOS_intel = open("WikiTemplate/Desktop/wikitemplate-macOS-x64.md", "r")
+    macOS_x64 = wiki_macOS_intel.read()
 
     for issue in bc_repo.get_issues(
         milestone=bc_milestone[milestonever], sort="created",
@@ -81,12 +81,12 @@ def laptop_testruns(milestonever):
                 output_line = " - [ ] " + issue_title + ".([#" +\
                         str(issue.number) + "](" + issue.html_url + "))"
                 checklist.append(output_line)
-                if("QA Pass-macOS" not in label_names and
+                if("QA Pass-macOS-arm64" not in label_names and
                         "OS/Windows" not in label_names and
                         "OS/Linux" not in label_names and
                         "QA/No" not in label_names and
                         "tests" not in label_names):
-                    mac_checklist.append(output_line)
+                    macarm64_checklist.append(output_line)
 
                 if("QA Pass-Win64" not in label_names and
                         "OS/macOS" not in label_names and
@@ -112,39 +112,39 @@ def laptop_testruns(milestonever):
         print(line)
     print("")
 
-    print("\nMac Checklist (Intel):")
+    print("\nMac Checklist (arm64):")
     print(laptop_template)
     print("")
-    macTitle = "Manual test run on macOS (Intel) for " + milestonever
-    macList = ["OS/macOS",
+    macarm64Title = "Manual test run on macOS (arm64) for " + milestonever
+    macarm64List = ["OS/macOS-arm64",
                "release-notes/exclude",
                "tests",
                "QA/Yes",
                "OS/Desktop"]
 
     if args.test is None:
-        bc_repo.create_issue(title=macTitle,
+        bc_repo.create_issue(title=macarm64Title,
                                  body=laptop_template,
                                  milestone=bc_milestone[milestonever],
-                                 labels=macList)
+                                 labels=macarm64List)
 
     print("--------------------------------------------------------\n")
 
-    print("Mac Checklist (arm64):")
-    print(macOS_arm64)
+    print("Mac Checklist (x64):")
+    print(macOS_x64)
     print("")
-    macarm64Title = "Manual test run on macOS (arm64) for " + milestonever
-    macarm64List = ["OS/macOS-arm64",
+    macx64Title = "Manual test run on macOS (x64) for " + milestonever
+    macx64List = ["OS/macOS",
                "release-notes/exclude",
                "tests",
                "QA/Yes",
                "OS/Desktop"]
 
     if args.test is None:
-        bc_repo.create_issue(title=macarm64Title,
-                                 body=macOS_arm64,
+        bc_repo.create_issue(title=macTitle,
+                                 body=macOS_x64,
                                  milestone=bc_milestone[milestonever],
-                                 labels=macarm64List)
+                                 labels=macx64List)
 
     print("--------------------------------------------------------\n")
 
@@ -188,8 +188,8 @@ def laptop_CRminor_testruns(milestonever):
 
     wiki_laptop_CRminor = open("WikiTemplate/Desktop/Minor_CR_Bump/wikitemplate-minorCRbumpDesktop.md", "r")
     laptop_CRminor_template = wiki_laptop_CRminor.read()
-    wiki_macOS_arm = open("WikiTemplate/Desktop/Minor_CR_Bump/wikitemplate-minorCRbump-macOS-arm64.md", "r")
-    macOS_arm64 = wiki_macOS_arm.read()
+    wiki_macOS_intel = open("WikiTemplate/Desktop/Minor_CR_Bump/wikitemplate-minorCRbump-macOS-x64.md", "r")
+    macOS_x64 = wiki_macOS_intel.read()
 
     for issue in bc_repo.get_issues(
         milestone=bc_milestone[milestonever], sort="created",
@@ -219,12 +219,12 @@ def laptop_CRminor_testruns(milestonever):
                 output_line = " - [ ] " + issue_title + ".([#" +\
                         str(issue.number) + "](" + issue.html_url + "))"
                 checklist.append(output_line)
-                if("QA Pass-macOS" not in label_names and
+                if("QA Pass-macOS-arm64" not in label_names and
                         "OS/Windows" not in label_names and
                         "OS/Linux" not in label_names and
                         "QA/No" not in label_names and
                         "tests" not in label_names):
-                    mac_checklist.append(output_line)
+                    macarm64_checklist.append(output_line)
 
                 if("QA Pass-Win64" not in label_names and
                         "OS/macOS" not in label_names and
@@ -238,39 +238,39 @@ def laptop_CRminor_testruns(milestonever):
         print(line)
     print("")
 
-    print("\nMac Checklist (Intel):")
+    print("\nMac Checklist (arm64):")
     print(laptop_CRminor_template)
     print("")
-    macTitle = "Manual test run on macOS (Intel) for " + milestonever
-    macList = ["OS/macOS",
+    macarm64Title = "Manual test run on macOS (arm64) for " + milestonever
+    macarm64List = ["OS/macOS-arm64",
                "release-notes/exclude",
                "tests",
                "QA/Yes",
                "OS/Desktop"]
 
     if args.test is None:
-        bc_repo.create_issue(title=macTitle,
+        bc_repo.create_issue(title=macarm64Title,
                                  body=laptop_CRminor_template,
                                  milestone=bc_milestone[milestonever],
-                                 labels=macList)
+                                 labels=macarm64List)
 
     print("--------------------------------------------------------\n")
 
-    print("Mac Checklist(arm64):")
-    print(macOS_arm64)
+    print("Mac Checklist(x64):")
+    print(macOS_x64)
     print("")
-    macarm64Title = "Manual test run on macOS (arm64) for " + milestonever
-    macarm64List = ["OS/macOS-arm64",
+    macx64Title = "Manual test run on macOS (x64) for " + milestonever
+    macx64List = ["OS/macOS",
                "release-notes/exclude",
                "tests",
                "QA/Yes",
                "OS/Desktop"]
 
     if args.test is None:
-        bc_repo.create_issue(title=macarm64Title,
-                                 body=macOS_arm64,
+        bc_repo.create_issue(title=macx64Title,
+                                 body=macOS_x64,
                                  milestone=bc_milestone[milestonever],
-                                 labels=macarm64List)
+                                 labels=macx64List)
 
     print("--------------------------------------------------------\n")