HIVE-2911: Add Microsoft Entra Workload ID cluster installation procedure #2772
Conversation
openshift-ci-robot
added
the
jira/valid-reference
|
@huangmingxia: This pull request references HIVE-2911 which is a valid jira issue. In response to this: Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: huangmingxia The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
|
/hold |
openshift-ci
bot
added
the
do-not-merge/hold
huangmingxia
force-pushed
the
HIVE-2911
branch
6 times, most recently
from
5ec579b to
bf506a9
Compare
|
@huangmingxia: This pull request references HIVE-2911 which is a valid jira issue. In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
/unhold |
openshift-ci
bot
removed
the
do-not-merge/hold
|
Hi @suhanime Could you take a look at this doc change when you have a chance? Thanks! |
suhanime
left a comment
suhanime
left a comment
There was a problem hiding this comment.
@huangmingxia I haven't been able to successfully install the cluster so cannot verify Step 7+, but these are my notes so far.
| [ | ||
| { | ||
| "cloudName": "AzureCloud", | ||
| "homeTenantId": "REDACTED", | ||
| "id": "REDACTED", | ||
| "isDefault": true, | ||
| "managedByTenants": [], | ||
| "name": "REDACTED", | ||
| "state": "Enabled", | ||
| "tenantId": "REDACTED", | ||
| "user": { | ||
| "name": "REDACTED", | ||
| "type": "REDACTED" | ||
| } | ||
| } | ||
| ] |
There was a problem hiding this comment.
Interesting, I do not get this output on az login, but need to do az account show to see this.
I simply get the subscription and tenant info.
$ az login
A web browser has been opened at https://login.microsoftonline.com/organizations/oauth2/v2.0/authorize. Please continue the login in the web browser. If no web browser is available or if the web browser fails to open, use device code flow with `az login --use-device-code`.
Retrieving tenants and subscriptions for the selection...
The following tenants don't contain accessible subscriptions. Use `az login --allow-no-subscriptions` to have tenant level access.
<redacted> 'Red Hat, Inc'
<redacted> 'PnT Cloud Ops Dev/Test'
[Tenant and subscription selection]
No Subscription name Subscription ID Tenant
----- ------------------- ------------------------------------ -------------
[1] * OpenShift Team Hive <redacted> OpenShift Dev
The default is marked with an *; the default tenant is 'OpenShift Dev' and subscription is 'OpenShift Team Hive' (<redacted>).
Select a subscription and tenant (Type a number or Enter for no changes):
Tenant: OpenShift Dev
Subscription: OpenShift Team Hive (<redacted>)
[Announcements]
With the new Azure CLI login experience, you can select the subscription you want to use more easily. Learn more about it and its configuration at https://go.microsoft.com/fwlink/?linkid=2271236
If you encounter any problem, please open an issue at https://aka.ms/azclibug
[Warning] The login output has been updated. Please be aware that it no longer displays the full list of available subscriptions by default.
| $ cat <azure_credentials_file_path>/osServicePrincipal.json | ||
| { | ||
| "subscriptionId": "REDACTED", | ||
| "clientId": "REDACTED", | ||
| "clientSecret": "REDACTED", | ||
| "tenantId": "REDACTED" | ||
| } |
There was a problem hiding this comment.
If I understand it correctly, osServicePrincipal.json doesn't exist by default or simply on az login, rather created as a part of installation attempt via installer - at least that's how it seems from the docs. I double checked on azure docs as well, there is no mention of this file, it is used only by OpenShift or OKD installation. Maybe include a note here to use the info from az account show if the json file doesn't exist.
There was a problem hiding this comment.
Later: confirmed - since my creds needed to be refreshed - I deleted the osServicePrincipal file (as instructed here), and it was recreated as a part of running openshift install.
It needed subscription and tenant id provided via az account show, as well as clientId (app Id) and password that were received after setting up my service principal.
There was a problem hiding this comment.
I have content about creating this file in my to-be-resurrected azure readme. Big picture, we'll have that document as well as this one, so we can cross-reference.
There was a problem hiding this comment.
Thanks both! I’ve added a note above that the Azure credentials file can be referenced in Eric’s PR in Azure.md.
| installed: false | ||
| platform: | ||
| azure: | ||
| baseDomainResourceGroupName: xxx |
There was a problem hiding this comment.
I don't know if this is important to mention, but I used the baseDomainResourceGroupName mentioned in my generated install config
There was a problem hiding this comment.
I think it's important that they match, or machinepools will splode.
|
|
||
| #### 6.4 Deploy Cluster | ||
|
|
||
| Deploy the cluster: |
There was a problem hiding this comment.
You might want to include creation of global-pull-secrets, or include the hiveutil option for the same - as it is unlikely to work without that - it didn't for me.
There was a problem hiding this comment.
Thank you. I remember that global-pull-secret no longer needs to be configured. I haven’t used this setting for a long time. I’m looking into the issue you mentioned.
There was a problem hiding this comment.
I think the reason I was able to create clusters without configuring a global pull secret or specifying the pull-secret parameters for hiveutil is that I had my pull secret stored in ~/.pull-secret, and hiveutil automatically picked it up from the default location. I’ve added this note to hiveutil.md. Thanks!
| $ oc apply -f clustermanifests.yaml | ||
| ``` | ||
|
|
||
| ### Step 7-8: Same as [Procedure 1: Microsoft Entra Workload ID Cluster Installation](#procedure-1-microsoft-entra-workload-id-cluster-installation) |
There was a problem hiding this comment.
Slightly unfortunate that if a provision attempt wasn't successful - or simply that it timed out, cleanup also removes the resource-group that we generated earlier, so if the installattemptlimit isn't set to 1, it will fail every time after.
There was a problem hiding this comment.
cleanup also removes the resource-group that we generated earlier
This is expected behavior, as this resource group contains all the Azure resources required by the cluster and is automatically deleted when the cluster is deleted. Note that OIDC-related resources are stored in a separate OIDC resource group, which requires manual cleanup using ccoctl.
I’ve updated the ClusterDeployment to include installAttemptsLimit: 1 and added the --install-once parameter to hiveutil.
When the cluster installation fails, redeploying the cluster requires manually restoring the deleted installation resource group. Recovery steps after a failed provision have been added.
huangmingxia
force-pushed
the
HIVE-2911
branch
3 times, most recently
from
ff8fec8 to
c37a141
Compare
| ``` | ||
|
|
||
| #### 2.2 Log in to Azure CLI | ||
| For more details on creating and managing Azure credentials files, refer to the [Azure credentials documentation in hive-sops](https://github.com/2uasimojo/hive-sops/blob/0d9784bd4b691c2c5251275a537615f4665966c0/sop/Azure.md) |
There was a problem hiding this comment.
Hi @2uasimojo @suhanime How about we update the link URL here after Eric’s PR is merged? We can leave it like this for now.
huangmingxia
force-pushed
the
HIVE-2911
branch
4 times, most recently
from
4ea639b to
330aee5
Compare
|
/hold |
openshift-ci
bot
added
the
do-not-merge/hold
huangmingxia
force-pushed
the
HIVE-2911
branch
2 times, most recently
from
0b4c2f6 to
4801a6d
Compare
huangmingxia
force-pushed
the
HIVE-2911
branch
from
4801a6d to
a388512
Compare
|
/unhold |
openshift-ci
bot
removed
the
do-not-merge/hold
4 participants
This PR adds comprehensive documentation for installing OpenShift clusters using Microsoft Entra Workload Identity through Hive.
docs/microsoft_entra_workload_id.mdwith step-by-step procedures for deploying Microsoft Entra Workload ID clusters using HiveREADME.md,docs/install.md, anddocs/aws-sts-provisioning.mdto reference the new documentation