DNS-01 Challenge Support

README.md

@@ -17,12 +17,20 @@ Many ESXi servers are accessible over the Internet and use self-signed X.509 cer
 
 ## Prerequisites
 
-Before installing `w2c-letsencrypt-esxi`, ensure the following preconditions are met:
+Before installing `w2c-letsencrypt-esxi`, ensure the following preconditions are met.
 
+- A _Fully Qualified Domain Name (FQDN)_ must be set in ESXi. Something like `localhost.localdomain` will not work.
+
+Additional requirements depend on the challenge type you plan to use:
+
+### HTTP-01 Challenges
 - Your server is publicly reachable over the Internet
-- A _Fully Qualified Domain Name (FQDN)_ is set in ESXi. Something like `localhost.localdomain` will not work
 - The hostname you specified can be resolved via A and/or AAAA records in the corresponding DNS zone
 
+### DNS-01 Challenges
+- Your server _does not_ need to be publicly reachable over the Internet; this method also allows wildcard certificates
+- You must be able to manage DNS records for your domain (API credentials for supported providers, or manual access)
+
 **Note:** As soon as you install this software, any existing, non Let's Encrypt certificate gets replaced!
 
 ## Install
@@ -59,20 +67,44 @@ $ cat /var/log/syslog.log | grep w2c
 3. _Manage -> Packages:_ Switch to the list of installed packages, click on _Install update_ and enter the absolute path on the datastore where your just uploaded VIB file resides.
 4. While the VIB is installed, ESXi requests a certificate from Let's Encrypt. If you reload the Web UI afterwards, the newly requested certificate should already be active. If not, see the [Wiki](https://github.com/w2c/letsencrypt-esxi/wiki) for troubleshooting.
 
-### Optional Configuration
-
-If you want to try out the script before putting it into production, you may want to test against the [staging environment](https://letsencrypt.org/docs/staging-environment/) of Let's Encrypt. Probably, you also do not wish to renew certificates once in 30 days but in longer or shorter intervals. Most variables of `renew.sh` can be adjusted by creating a `renew.cfg` file with your overwritten values.
+### Configuration
 
-`vi /opt/w2c-letsencrypt/renew.cfg`
+To customize certificate renewal, copy the example configuration file and edit it:
 
 ```bash
-# Request a certificate from the staging environment
-DIRECTORY_URL="https://acme-staging-v02.api.letsencrypt.org/directory"
-# Set the renewal interval to 15 days
-RENEW_DAYS=15
+cp /opt/w2c-letsencrypt/renew.cfg.example /opt/w2c-letsencrypt/renew.cfg
+vi /opt/w2c-letsencrypt/renew.cfg
 ```
 
-To apply your modifications, run `/etc/init.d/w2c-letsencrypt start`
+Most options can be set in `renew.cfg`. See [`renew.cfg.example`](renew.cfg.example) for all available settings and DNS provider variables.
+
+#### Common configuration examples
+
+- **Use Let's Encrypt staging environment and change the renewal interval:**
+
+    ```bash
+    DIRECTORY_URL="https://acme-staging-v02.api.letsencrypt.org/directory"
+    RENEW_DAYS=15
+    ```
+
+- **Enable DNS-01 challenge (Cloudflare):**
+
+    ```bash
+    CHALLENGE_TYPE="dns-01"
+    DNS_PROVIDER="cloudflare"
+    CF_API_TOKEN="your-cloudflare-api-token"
+    ```
+
+- **Enable DNS-01 challenge (manual):**
+
+    ```bash
+    CHALLENGE_TYPE="dns-01"
+    DNS_PROVIDER="manual"
+    ```
+
+    You will be prompted to create and remove DNS TXT records interactively. Certificates obtained this way cannot be renewed automatically, as manual intervention is always required.
+
+**Note:** Automated renewal is only supported for providers with API support (e.g., Cloudflare).
 
 ## Uninstall
 
@@ -92,7 +124,7 @@ This action will purge `w2c-letsencrypt-esxi`, undo any changes to system files
 
 ## Usage
 
-Usually, fully-automated. No interaction required.
+For HTTP-01 and DNS-01 with a supported API provider, operation is fully automated and requires no user interaction. For manual DNS-01, as you must interactively create and remove DNS TXT records each time, certificates cannot be renewed automatically.
 
 ### Hostname Change
 

acme_tiny.py

@@ -13,7 +13,7 @@
 LOGGER.addHandler(logging.StreamHandler())
 LOGGER.setLevel(logging.INFO)
 
-def get_crt(account_key, csr, acme_dir, log=LOGGER, CA=DEFAULT_CA, disable_check=False, directory_url=DEFAULT_DIRECTORY_URL, contact=None, check_port=None):
+def get_crt(account_key, csr, acme_dir, log=LOGGER, CA=DEFAULT_CA, disable_check=False, directory_url=DEFAULT_DIRECTORY_URL, contact=None, check_port=None, challenge_type="http-01", timeout=30):
     directory, acct_headers, alg, jwk = None, None, None, None # global variables
 
     # helper functions - base64 encode for jose spec
@@ -70,6 +70,28 @@ def _poll_until_not(url, pending_statuses, err_msg):
             result, _, _ = _send_signed_request(url, None, err_msg)
         return result
 
+    # helper function - execute DNS API actions
+    def _execute_dns_api(action, domain, token, key_auth=None):
+        api_dir = os.path.join(os.path.dirname(os.path.abspath(__file__)), "dnsapi")
+        api_script = os.path.join(api_dir, "dns_api.sh")
+        if not os.path.exists(api_script) or not os.access(api_script, os.X_OK):
+            raise ValueError("DNS API script not found or not executable: {0}".format(api_script))
+        env = os.environ.copy()
+        env.update({
+            'ACME_CHALLENGE_TYPE': challenge_type,
+            'ACME_DOMAIN': domain,
+            'ACME_TOKEN': token,
+            'ACME_KEY_AUTH': key_auth or '',
+            'DNS_PROVIDER': os.environ.get('DNS_PROVIDER', '')
+        })
+        cmd = ["/bin/sh", api_script, action, domain, token, key_auth or '']
+        proc = subprocess.Popen(cmd, env=env, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+        stdout, stderr = proc.communicate()
+        if proc.returncode != 0:
+            error_msg = stderr.decode('utf8', errors='replace')
+            raise IOError("DNS API failed: {0}".format(error_msg))
+        return stdout.decode('utf8', errors='replace')
+
     # parse account key to get public key
     log.info("Parsing account key...")
     out = _cmd(["openssl", "rsa", "-in", account_key, "-noout", "-text"], err_msg="OpenSSL Error")
@@ -130,28 +152,40 @@ def _poll_until_not(url, pending_statuses, err_msg):
             log.info("Already verified: {0}, skipping...".format(domain))
             continue
         log.info("Verifying {0}...".format(domain))
-
-        # find the http-01 challenge and write the challenge file
-        challenge = [c for c in authorization['challenges'] if c['type'] == "http-01"][0]
-        token = re.sub(r"[^A-Za-z0-9_\-]", "_", challenge['token'])
-        keyauthorization = "{0}.{1}".format(token, thumbprint)
-        wellknown_path = os.path.join(acme_dir, token)
-        with open(wellknown_path, "w") as wellknown_file:
-            wellknown_file.write(keyauthorization)
-
-        # check that the file is in place
-        try:
-            wellknown_url = "http://{0}{1}/.well-known/acme-challenge/{2}".format(domain, "" if check_port is None else ":{0}".format(check_port), token)
-            assert (disable_check or _do_request(wellknown_url)[0] == keyauthorization)
-        except (AssertionError, ValueError) as e:
-            raise ValueError("Wrote file to {0}, but couldn't download {1}: {2}".format(wellknown_path, wellknown_url, e))
-
+        if challenge_type == "http-01":
+            challenge = [c for c in authorization['challenges'] if c['type'] == "http-01"][0]
+            token = re.sub(r"[^A-Za-z0-9_\-]", "_", challenge['token'])
+            keyauthorization = "{0}.{1}".format(token, thumbprint)
+            wellknown_path = os.path.join(acme_dir, token)
+            with open(wellknown_path, "w") as wellknown_file:
+                wellknown_file.write(keyauthorization)
+
+            # check that the file is in place
+            try:
+                wellknown_url = "http://{0}{1}/.well-known/acme-challenge/{2}".format(domain, "" if check_port is None else ":{0}".format(check_port), token)
+                assert (disable_check or _do_request(wellknown_url)[0] == keyauthorization)
+            except (AssertionError, ValueError) as e:
+                raise ValueError("Wrote file to {0}, but couldn't download {1}: {2}".format(wellknown_path, wellknown_url, e))
+        elif challenge_type == "dns-01":
+            challenge = [c for c in authorization['challenges'] if c['type'] == "dns-01"][0]
+            token = challenge['token']
+            keyauthorization = "{0}.{1}".format(token, thumbprint)
+            log.info("Setting up DNS challenge for {0}...".format(domain))
+            _execute_dns_api("add", domain, token, keyauthorization)
+            log.info("DNS challenge setup complete. Waiting for propagation...")
+            wait_time = int(os.getenv('DNS_PROPAGATION_WAIT', '30'))
+            time.sleep(wait_time)
         # say the challenge is done
         _send_signed_request(challenge['url'], {}, "Error submitting challenges: {0}".format(domain))
         authorization = _poll_until_not(auth_url, ["pending"], "Error checking challenge status for {0}".format(domain))
+        # cleanup
+        if challenge_type == "http-01":
+            os.remove(wellknown_path)
+        elif challenge_type == "dns-01":
+            log.info("Cleaning up DNS challenge for {0}...".format(domain))
+            _execute_dns_api("rm", domain, token, keyauthorization)
         if authorization['status'] != "valid":
             raise ValueError("Challenge did not pass for {0}: {1}".format(domain, authorization))
-        os.remove(wellknown_path)
         log.info("{0} verified!".format(domain))
 
     # finalize the order with the csr
@@ -182,17 +216,31 @@ def main(argv=None):
     )
     parser.add_argument("--account-key", required=True, help="path to your Let's Encrypt account private key")
     parser.add_argument("--csr", required=True, help="path to your certificate signing request")
-    parser.add_argument("--acme-dir", required=True, help="path to the .well-known/acme-challenge/ directory")
+    parser.add_argument("--acme-dir", required=False, help="path to the .well-known/acme-challenge/ directory")
     parser.add_argument("--quiet", action="store_const", const=logging.ERROR, help="suppress output except for errors")
     parser.add_argument("--disable-check", default=False, action="store_true", help="disable checking if the challenge file is hosted correctly before telling the CA")
     parser.add_argument("--directory-url", default=DEFAULT_DIRECTORY_URL, help="certificate authority directory url, default is Let's Encrypt")
     parser.add_argument("--ca", default=DEFAULT_CA, help="DEPRECATED! USE --directory-url INSTEAD!")
     parser.add_argument("--contact", metavar="CONTACT", default=None, nargs="*", help="Contact details (e.g. mailto:[email protected]) for your account-key")
     parser.add_argument("--check-port", metavar="PORT", default=None, help="what port to use when self-checking the challenge file, default is port 80")
+    parser.add_argument("--challenge-type", default="http-01", choices=["http-01", "dns-01"], help="ACME challenge type to use (http-01 or dns-01)")
+    parser.add_argument("--timeout", type=int, default=30, help="Timeout for DNS propagation wait (seconds)")
 
     args = parser.parse_args(argv)
     LOGGER.setLevel(args.quiet or LOGGER.level)
-    signed_crt = get_crt(args.account_key, args.csr, args.acme_dir, log=LOGGER, CA=args.ca, disable_check=args.disable_check, directory_url=args.directory_url, contact=args.contact, check_port=args.check_port)
+    signed_crt = get_crt(
+        args.account_key,
+        args.csr,
+        args.acme_dir,
+        log=LOGGER,
+        CA=args.ca,
+        disable_check=args.disable_check,
+        directory_url=args.directory_url,
+        contact=args.contact,
+        check_port=args.check_port,
+        challenge_type=args.challenge_type,
+        timeout=args.timeout
+    )
     sys.stdout.write(signed_crt)
 
 if __name__ == "__main__": # pragma: no cover

build/create_vib.sh

@@ -41,11 +41,11 @@ INIT_DIR=${VIB_PAYLOAD_DIR}/etc/init.d
 mkdir -p ${BIN_DIR} ${INIT_DIR}
 
 # Copy files to the corresponding locations
-cp ../* ${BIN_DIR} 2>/dev/null
+cp -r ../* ${BIN_DIR} 2>/dev/null
 cp ../w2c-letsencrypt ${INIT_DIR}
 
 # Ensure that shell scripts are executable
-chmod +x ${INIT_DIR}/w2c-letsencrypt ${BIN_DIR}/renew.sh
+chmod +x ${INIT_DIR}/w2c-letsencrypt ${BIN_DIR}/renew.sh ${BIN_DIR}/dnsapi/dns_api.sh
 
 # Create tgz with payload
 tar czf ${TEMP_DIR}/payload1 -C ${VIB_PAYLOAD_DIR} etc opt