Detect and handle HSMs in DEGRADED state

[ckamps]

Address scenario in which an HSM enters the "degraded" state during creation of the HSM.

During testing of our IaC, we have seen cases in which, during cluster creation, the first HSM to be created doesn't enter the ACTIVE state but enters a degraded state. With the current code, this state is not caught. Eventually, the create operation times out and an auto rollback of the stack is attempted.

When additional HSMs are created beyond the first HSM, any of the create actions could result in an HSM entering the degraded state.

An HSM in a degraded state can be deleted. i.e. the HSM won't automatically transition to an ACTIVE state later of its own accord.

This is what the state of such an HSM looks like:

            "Hsms": [
                {
                    "AvailabilityZone": "us-east-2a",
                    "ClusterId": "cluster-quhwuyosn7k",
                    "SubnetId": "subnet-04a76758b58c05023",
                    "EniId": "eni-09422fd6133d93b07",
                    "EniIp": "10.4.14.83",
                    "HsmId": "hsm-n5yjka6nfos",
                    "State": "DEGRADED",
                    "StateMessage": "HSM creation failed. Please delete this HSM and try again."
                }
            ],