diff --git a/README.md b/README.md index b14ca5f..21282a3 100644 --- a/README.md +++ b/README.md @@ -17,12 +17,20 @@ Many ESXi servers are accessible over the Internet and use self-signed X.509 cer ## Prerequisites -Before installing `w2c-letsencrypt-esxi`, ensure the following preconditions are met: +Before installing `w2c-letsencrypt-esxi`, ensure the following preconditions are met. +- A _Fully Qualified Domain Name (FQDN)_ must be set in ESXi. Something like `localhost.localdomain` will not work. + +Additional requirements depend on the challenge type you plan to use: + +### HTTP-01 Challenges - Your server is publicly reachable over the Internet -- A _Fully Qualified Domain Name (FQDN)_ is set in ESXi. Something like `localhost.localdomain` will not work - The hostname you specified can be resolved via A and/or AAAA records in the corresponding DNS zone +### DNS-01 Challenges +- Your server _does not_ need to be publicly reachable over the Internet; this method also allows wildcard certificates +- You must be able to manage DNS records for your domain (API credentials for supported providers, or manual access) + **Note:** As soon as you install this software, any existing, non Let's Encrypt certificate gets replaced! ## Install @@ -59,20 +67,44 @@ $ cat /var/log/syslog.log | grep w2c 3. _Manage -> Packages:_ Switch to the list of installed packages, click on _Install update_ and enter the absolute path on the datastore where your just uploaded VIB file resides. 4. While the VIB is installed, ESXi requests a certificate from Let's Encrypt. If you reload the Web UI afterwards, the newly requested certificate should already be active. If not, see the [Wiki](https://github.com/w2c/letsencrypt-esxi/wiki) for troubleshooting. -### Optional Configuration - -If you want to try out the script before putting it into production, you may want to test against the [staging environment](https://letsencrypt.org/docs/staging-environment/) of Let's Encrypt. Probably, you also do not wish to renew certificates once in 30 days but in longer or shorter intervals. Most variables of `renew.sh` can be adjusted by creating a `renew.cfg` file with your overwritten values. +### Configuration -`vi /opt/w2c-letsencrypt/renew.cfg` +To customize certificate renewal, copy the example configuration file and edit it: ```bash -# Request a certificate from the staging environment -DIRECTORY_URL="https://acme-staging-v02.api.letsencrypt.org/directory" -# Set the renewal interval to 15 days -RENEW_DAYS=15 +cp /opt/w2c-letsencrypt/renew.cfg.example /opt/w2c-letsencrypt/renew.cfg +vi /opt/w2c-letsencrypt/renew.cfg ``` -To apply your modifications, run `/etc/init.d/w2c-letsencrypt start` +Most options can be set in `renew.cfg`. See [`renew.cfg.example`](renew.cfg.example) for all available settings and DNS provider variables. + +#### Common configuration examples + +- **Use Let's Encrypt staging environment and change the renewal interval:** + + ```bash + DIRECTORY_URL="https://acme-staging-v02.api.letsencrypt.org/directory" + RENEW_DAYS=15 + ``` + +- **Enable DNS-01 challenge (Cloudflare):** + + ```bash + CHALLENGE_TYPE="dns-01" + DNS_PROVIDER="cloudflare" + CF_API_TOKEN="your-cloudflare-api-token" + ``` + +- **Enable DNS-01 challenge (manual):** + + ```bash + CHALLENGE_TYPE="dns-01" + DNS_PROVIDER="manual" + ``` + + You will be prompted to create and remove DNS TXT records interactively. Certificates obtained this way cannot be renewed automatically, as manual intervention is always required. + +**Note:** Automated renewal is only supported for providers with API support (e.g., Cloudflare). ## Uninstall @@ -92,7 +124,7 @@ This action will purge `w2c-letsencrypt-esxi`, undo any changes to system files ## Usage -Usually, fully-automated. No interaction required. +For HTTP-01 and DNS-01 with a supported API provider, operation is fully automated and requires no user interaction. For manual DNS-01, as you must interactively create and remove DNS TXT records each time, certificates cannot be renewed automatically. ### Hostname Change diff --git a/acme_tiny.py b/acme_tiny.py index d992d02..1a500e7 100644 --- a/acme_tiny.py +++ b/acme_tiny.py @@ -13,7 +13,7 @@ LOGGER.addHandler(logging.StreamHandler()) LOGGER.setLevel(logging.INFO) -def get_crt(account_key, csr, acme_dir, log=LOGGER, CA=DEFAULT_CA, disable_check=False, directory_url=DEFAULT_DIRECTORY_URL, contact=None, check_port=None): +def get_crt(account_key, csr, acme_dir, log=LOGGER, CA=DEFAULT_CA, disable_check=False, directory_url=DEFAULT_DIRECTORY_URL, contact=None, check_port=None, challenge_type="http-01", timeout=30): directory, acct_headers, alg, jwk = None, None, None, None # global variables # helper functions - base64 encode for jose spec @@ -70,6 +70,28 @@ def _poll_until_not(url, pending_statuses, err_msg): result, _, _ = _send_signed_request(url, None, err_msg) return result + # helper function - execute DNS API actions + def _execute_dns_api(action, domain, token, key_auth=None): + api_dir = os.path.join(os.path.dirname(os.path.abspath(__file__)), "dnsapi") + api_script = os.path.join(api_dir, "dns_api.sh") + if not os.path.exists(api_script) or not os.access(api_script, os.X_OK): + raise ValueError("DNS API script not found or not executable: {0}".format(api_script)) + env = os.environ.copy() + env.update({ + 'ACME_CHALLENGE_TYPE': challenge_type, + 'ACME_DOMAIN': domain, + 'ACME_TOKEN': token, + 'ACME_KEY_AUTH': key_auth or '', + 'DNS_PROVIDER': os.environ.get('DNS_PROVIDER', '') + }) + cmd = ["/bin/sh", api_script, action, domain, token, key_auth or ''] + proc = subprocess.Popen(cmd, env=env, stdout=subprocess.PIPE, stderr=subprocess.PIPE) + stdout, stderr = proc.communicate() + if proc.returncode != 0: + error_msg = stderr.decode('utf8', errors='replace') + raise IOError("DNS API failed: {0}".format(error_msg)) + return stdout.decode('utf8', errors='replace') + # parse account key to get public key log.info("Parsing account key...") out = _cmd(["openssl", "rsa", "-in", account_key, "-noout", "-text"], err_msg="OpenSSL Error") @@ -130,28 +152,40 @@ def _poll_until_not(url, pending_statuses, err_msg): log.info("Already verified: {0}, skipping...".format(domain)) continue log.info("Verifying {0}...".format(domain)) - - # find the http-01 challenge and write the challenge file - challenge = [c for c in authorization['challenges'] if c['type'] == "http-01"][0] - token = re.sub(r"[^A-Za-z0-9_\-]", "_", challenge['token']) - keyauthorization = "{0}.{1}".format(token, thumbprint) - wellknown_path = os.path.join(acme_dir, token) - with open(wellknown_path, "w") as wellknown_file: - wellknown_file.write(keyauthorization) - - # check that the file is in place - try: - wellknown_url = "http://{0}{1}/.well-known/acme-challenge/{2}".format(domain, "" if check_port is None else ":{0}".format(check_port), token) - assert (disable_check or _do_request(wellknown_url)[0] == keyauthorization) - except (AssertionError, ValueError) as e: - raise ValueError("Wrote file to {0}, but couldn't download {1}: {2}".format(wellknown_path, wellknown_url, e)) - + if challenge_type == "http-01": + challenge = [c for c in authorization['challenges'] if c['type'] == "http-01"][0] + token = re.sub(r"[^A-Za-z0-9_\-]", "_", challenge['token']) + keyauthorization = "{0}.{1}".format(token, thumbprint) + wellknown_path = os.path.join(acme_dir, token) + with open(wellknown_path, "w") as wellknown_file: + wellknown_file.write(keyauthorization) + + # check that the file is in place + try: + wellknown_url = "http://{0}{1}/.well-known/acme-challenge/{2}".format(domain, "" if check_port is None else ":{0}".format(check_port), token) + assert (disable_check or _do_request(wellknown_url)[0] == keyauthorization) + except (AssertionError, ValueError) as e: + raise ValueError("Wrote file to {0}, but couldn't download {1}: {2}".format(wellknown_path, wellknown_url, e)) + elif challenge_type == "dns-01": + challenge = [c for c in authorization['challenges'] if c['type'] == "dns-01"][0] + token = challenge['token'] + keyauthorization = "{0}.{1}".format(token, thumbprint) + log.info("Setting up DNS challenge for {0}...".format(domain)) + _execute_dns_api("add", domain, token, keyauthorization) + log.info("DNS challenge setup complete. Waiting for propagation...") + wait_time = int(os.getenv('DNS_PROPAGATION_WAIT', '30')) + time.sleep(wait_time) # say the challenge is done _send_signed_request(challenge['url'], {}, "Error submitting challenges: {0}".format(domain)) authorization = _poll_until_not(auth_url, ["pending"], "Error checking challenge status for {0}".format(domain)) + # cleanup + if challenge_type == "http-01": + os.remove(wellknown_path) + elif challenge_type == "dns-01": + log.info("Cleaning up DNS challenge for {0}...".format(domain)) + _execute_dns_api("rm", domain, token, keyauthorization) if authorization['status'] != "valid": raise ValueError("Challenge did not pass for {0}: {1}".format(domain, authorization)) - os.remove(wellknown_path) log.info("{0} verified!".format(domain)) # finalize the order with the csr @@ -182,17 +216,31 @@ def main(argv=None): ) parser.add_argument("--account-key", required=True, help="path to your Let's Encrypt account private key") parser.add_argument("--csr", required=True, help="path to your certificate signing request") - parser.add_argument("--acme-dir", required=True, help="path to the .well-known/acme-challenge/ directory") + parser.add_argument("--acme-dir", required=False, help="path to the .well-known/acme-challenge/ directory") parser.add_argument("--quiet", action="store_const", const=logging.ERROR, help="suppress output except for errors") parser.add_argument("--disable-check", default=False, action="store_true", help="disable checking if the challenge file is hosted correctly before telling the CA") parser.add_argument("--directory-url", default=DEFAULT_DIRECTORY_URL, help="certificate authority directory url, default is Let's Encrypt") parser.add_argument("--ca", default=DEFAULT_CA, help="DEPRECATED! USE --directory-url INSTEAD!") parser.add_argument("--contact", metavar="CONTACT", default=None, nargs="*", help="Contact details (e.g. mailto:aaa@bbb.com) for your account-key") parser.add_argument("--check-port", metavar="PORT", default=None, help="what port to use when self-checking the challenge file, default is port 80") + parser.add_argument("--challenge-type", default="http-01", choices=["http-01", "dns-01"], help="ACME challenge type to use (http-01 or dns-01)") + parser.add_argument("--timeout", type=int, default=30, help="Timeout for DNS propagation wait (seconds)") args = parser.parse_args(argv) LOGGER.setLevel(args.quiet or LOGGER.level) - signed_crt = get_crt(args.account_key, args.csr, args.acme_dir, log=LOGGER, CA=args.ca, disable_check=args.disable_check, directory_url=args.directory_url, contact=args.contact, check_port=args.check_port) + signed_crt = get_crt( + args.account_key, + args.csr, + args.acme_dir, + log=LOGGER, + CA=args.ca, + disable_check=args.disable_check, + directory_url=args.directory_url, + contact=args.contact, + check_port=args.check_port, + challenge_type=args.challenge_type, + timeout=args.timeout + ) sys.stdout.write(signed_crt) if __name__ == "__main__": # pragma: no cover diff --git a/build/create_vib.sh b/build/create_vib.sh index 4e2e1d1..6e40f0c 100644 --- a/build/create_vib.sh +++ b/build/create_vib.sh @@ -41,11 +41,11 @@ INIT_DIR=${VIB_PAYLOAD_DIR}/etc/init.d mkdir -p ${BIN_DIR} ${INIT_DIR} # Copy files to the corresponding locations -cp ../* ${BIN_DIR} 2>/dev/null +cp -r ../* ${BIN_DIR} 2>/dev/null cp ../w2c-letsencrypt ${INIT_DIR} # Ensure that shell scripts are executable -chmod +x ${INIT_DIR}/w2c-letsencrypt ${BIN_DIR}/renew.sh +chmod +x ${INIT_DIR}/w2c-letsencrypt ${BIN_DIR}/renew.sh ${BIN_DIR}/dnsapi/dns_api.sh # Create tgz with payload tar czf ${TEMP_DIR}/payload1 -C ${VIB_PAYLOAD_DIR} etc opt diff --git a/dnsapi/dns_api.sh b/dnsapi/dns_api.sh new file mode 100644 index 0000000..5a196c8 --- /dev/null +++ b/dnsapi/dns_api.sh @@ -0,0 +1,998 @@ +#!/bin/sh +# +# DNS API Framework - Core functionality for DNS providers +# Main entry point for ACME DNS-01 challenges +# Provides standardized interface and common utilities for all DNS providers +# +# Usage: dns_api.sh [txt_value] +# Commands: add, rm, info, list, test +# + +DNSAPIDIR=$(dirname "$(readlink -f "$0")") +LOCALDIR="$DNSAPIDIR/.." + +# Parse command line arguments +COMMAND="$1" +DOMAIN="$2" +TOKEN="$3" +KEY_AUTH="$4" + +# Calculate TXT value from key authorization for DNS-01 challenges +calculate_txt_value() { + key_auth="$1" + if [ -z "$key_auth" ]; then + return 1 + fi + + # Use the same calculation as acme_tiny.py: base64(sha256(key_auth)) + if which python3 >/dev/null 2>&1; then + echo -n "$key_auth" | python3 -c " +import sys, hashlib, base64 +data = sys.stdin.read().encode('utf8') +hash_digest = hashlib.sha256(data).digest() +result = base64.urlsafe_b64encode(hash_digest).decode('utf8').replace('=', '') +print(result) +" + elif which python >/dev/null 2>&1; then + echo -n "$key_auth" | python -c " +import sys, hashlib, base64 +data = sys.stdin.read().encode('utf8') +hash_digest = hashlib.sha256(data).digest() +result = base64.urlsafe_b64encode(hash_digest).decode('utf8').replace('=', '') +print(result) +" + else + # Fallback using openssl (ESXi compatible) + echo -n "$key_auth" | openssl dgst -sha256 -binary | openssl base64 -A | \ + sed 's/=//g' | sed 'y/\/+/_-/' + fi +} + +# For DNS-01 challenges, calculate TXT value from key authorization +if [ "$COMMAND" = "add" ] || [ "$COMMAND" = "rm" ]; then + if [ -n "$KEY_AUTH" ]; then + TXT_VALUE=$(calculate_txt_value "$KEY_AUTH") + if [ -z "$TXT_VALUE" ]; then + echo "Error: Failed to calculate TXT value from key authorization" >&2 + exit 1 + fi + elif [ -n "$ACME_KEY_AUTH" ]; then + # Fallback to environment variable + TXT_VALUE=$(calculate_txt_value "$ACME_KEY_AUTH") + if [ -z "$TXT_VALUE" ]; then + echo "Error: Failed to calculate TXT value from ACME_KEY_AUTH" >&2 + exit 1 + fi + else + # Legacy: assume third parameter is already the TXT value + TXT_VALUE="$TOKEN" + fi +fi + +# Load configuration from renew.cfg +if [ -r "$LOCALDIR/renew.cfg" ]; then + . "$LOCALDIR/renew.cfg" +elif [ -r "$DNSAPIDIR/../renew.cfg" ]; then + . "$DNSAPIDIR/../renew.cfg" +fi + +# DNS API version +DNS_API_VERSION="1.2.0" + +# Default settings that providers can override +DEFAULT_DNS_TIMEOUT=${DNS_TIMEOUT:-30} +DEFAULT_TTL=${DNS_TTL:-120} +DEFAULT_PROPAGATION_WAIT=${DNS_PROPAGATION_WAIT:-120} +DEFAULT_MAX_RETRIES=${MAX_RETRIES:-3} +DEFAULT_RETRY_DELAY=${RETRY_DELAY:-5} + +# Logging functions +dns_log_debug() { + if [ "${DEBUG:-0}" = "1" ]; then + echo "[DNS-DEBUG] $(date '+%Y-%m-%d %H:%M:%S') $*" >&2 + fi +} + +dns_log_info() { + echo "[DNS-INFO] $(date '+%Y-%m-%d %H:%M:%S') $*" +} + +dns_log_warn() { + echo "[DNS-WARN] $(date '+%Y-%m-%d %H:%M:%S') $*" >&2 +} + +dns_log_error() { + echo "[DNS-ERROR] $(date '+%Y-%m-%d %H:%M:%S') $*" >&2 +} + +dns_log_debug "DNS_PROVIDER is '$DNS_PROVIDER'" +dns_log_debug "CF_API_TOKEN is '$CF_API_TOKEN'" + +# Validation functions +dns_validate_domain() { + domain="$1" + if [ -z "$domain" ]; then + dns_log_error "Domain cannot be empty" + return 1 + fi + + # Basic domain validation + if ! echo "$domain" | grep -qE '^[a-zA-Z0-9]([a-zA-Z0-9\-]*[a-zA-Z0-9])?(\.[a-zA-Z0-9]([a-zA-Z0-9\-]*[a-zA-Z0-9])?)*$'; then + dns_log_error "Invalid domain format: $domain" + return 1 + fi + + return 0 +} + +dns_validate_txt_value() { + txt_value="$1" + if [ -z "$txt_value" ]; then + dns_log_error "TXT value cannot be empty" + return 1 + fi + + # Validate base64-like encoding (basic check) + if [ ${#txt_value} -lt 40 ]; then + dns_log_error "TXT value seems to short (${#txt_value} chars)" + return 1 + fi + + return 0 +} + +# DNS zone detection utilities +dns_get_zone() { + domain="$1" + provider="$2" + + # Try different zone detection strategies + + # Strategy 1: Direct domain match + if dns_zone_exists "$domain" "$provider"; then + echo "$domain" + return 0 + fi + + # Strategy 2: Parent domains + parent_domain="$domain" + while [ "$(echo "$parent_domain" | awk -F'.' '{print NF}')" -gt 2 ]; do + parent_domain=$(echo "$parent_domain" | cut -d. -f2-) + if dns_zone_exists "$parent_domain" "$provider"; then + echo "$parent_domain" + return 0 + fi + done + + # Strategy 3: Common patterns + base_domain=$(echo "$domain" | awk -F. '{if(NF>=2) print $(NF-1)"."$NF; else print $0}') + if dns_zone_exists "$base_domain" "$provider"; then + echo "$base_domain" + return 0 + fi + + dns_log_error "Could not determine DNS zone for domain: $domain" + return 1 +} + +# BusyBox-only HTTP GET utility +dns_http_get() { + url="$1" + headers="$2" + timeout="${3:-$DEFAULT_DNS_TIMEOUT}" + max_redirects="${4:-5}" + + # Ensure timeout is a bare integer (BusyBox compatible) + timeout="$(echo "$timeout" | sed 's/[a-zA-Z]//g')" + + dns_log_debug "HTTP GET: $url (timeout: ${timeout}s)" + + if ! which wget >/dev/null 2>&1; then + dns_log_error "No HTTP client available (wget required)" + return 127 + fi + + set -- -qO- --no-check-certificate + if [ -n "$headers" ]; then + OLD_IFS="$IFS" + IFS=' +' + for header in $headers; do + set -- "$@" --header="$header" + done + IFS="$OLD_IFS" + fi + set -- "$@" "$url" + if [ "${DEBUG:-0}" = "1" ]; then + echo "[DNS-DEBUG] $(date '+%Y-%m-%d %H:%M:%S') Final wget command: wget $*" >&2 + fi + if which timeout >/dev/null 2>&1; then + response=$(timeout -t $timeout wget "$@" 2>&1) + if [ "${DEBUG:-0}" = "1" ]; then + echo "[DNS-DEBUG] $(date '+%Y-%m-%d %H:%M:%S') Raw HTTP GET response:" >&2 + echo "$response" >&2 + fi + exit_code=$? + if [ $exit_code -eq 124 ]; then + dns_log_error "wget timed out after ${timeout}s" + return 124 + fi + else + response=$(wget "$@" 2>&1) + if [ "${DEBUG:-0}" = "1" ]; then + echo "[DNS-DEBUG] $(date '+%Y-%m-%d %H:%M:%S') Raw HTTP GET response:" >&2 + echo "$response" >&2 + fi + exit_code=$? + fi + if [ $exit_code -eq 0 ]; then + dns_log_debug "HTTP GET response: $response" + echo "$response" + return 0 + else + dns_log_debug "wget failed with exit code $exit_code: $response" + return $exit_code + fi +} + +# BusyBox-only HTTP POST utility (not supported) +dns_http_post() { + url="$1" + data="$2" + headers="$3" + timeout="${4:-$DEFAULT_DNS_TIMEOUT}" + + # Ensure timeout is a bare integer (BusyBox compatible) + timeout="$(echo "$timeout" | sed 's/[a-zA-Z]//g')" + + dns_log_debug "HTTP POST: $url (timeout: ${timeout}s)" + + if ! which wget >/dev/null 2>&1; then + dns_log_error "No HTTP client available (wget required)" + return 127 + fi + + # Compact JSON data: remove all newlines, carriage returns, tabs, and literal \n, \t; collapse spaces + compact_data=$(echo "$data" \ + | sed 's/\\n//g; s/\\t//g' \ + | sed 's/[\r\n\t]//g' \ + | sed 's/ */ /g') + + set -- -qO- --no-check-certificate --post-data="$compact_data" + if [ -n "$headers" ]; then + OLD_IFS="$IFS" + IFS=' +' + for header in $headers; do + set -- "$@" --header="$header" + done + IFS="$OLD_IFS" + fi + set -- "$@" "$url" + if [ "${DEBUG:-0}" = "1" ]; then + echo "[DNS-DEBUG] $(date '+%Y-%m-%d %H:%M:%S') Final wget POST command: wget $*" >&2 + fi + if which timeout >/dev/null 2>&1; then + response=$(timeout -t $timeout wget "$@" 2>&1) + if [ "${DEBUG:-0}" = "1" ]; then + echo "[DNS-DEBUG] $(date '+%Y-%m-%d %H:%M:%S') Raw HTTP POST response:" >&2 + echo "$response" >&2 + fi + exit_code=$? + if [ $exit_code -eq 124 ]; then + dns_log_error "wget timed out after ${timeout}s" + return 124 + fi + else + response=$(wget "$@" 2>&1) + if [ "${DEBUG:-0}" = "1" ]; then + echo "[DNS-DEBUG] $(date '+%Y-%m-%d %H:%M:%S') Raw HTTP POST response:" >&2 + echo "$response" >&2 + fi + exit_code=$? + fi + if [ $exit_code -eq 0 ]; then + dns_log_debug "HTTP POST response: $response" + echo "$response" + return 0 + else + dns_log_debug "wget POST failed with exit code $exit_code: $response" + return $exit_code + fi +} + +dns_http_delete() { + url="$1" + headers="$2" + timeout="${3:-$DEFAULT_DNS_TIMEOUT}" + + dns_log_debug "HTTP DELETE: $url (timeout: ${timeout}s)" + + # DELETE not supported with wget + dns_log_warn "DELETE method not supported with wget, record may not be cleaned up" + return 1 +} + +# URL encoding utility (ESXi-compatible) +dns_url_encode() { + string="$1" + encoded="" + char="" + + # Process each character + while [ -n "$string" ]; do + char="${string%"${string#?}"}" # Get first character + string="${string#?}" # Remove first character + + case "$char" in + [a-zA-Z0-9._~-]) + encoded="$encoded$char" + ;; + *) + # Convert to hex (ESXi compatible method) + if which printf >/dev/null 2>&1; then + encoded="$encoded$(printf '%%%02X' "'$char")" + else + # Fallback for limited environments + encoded="$encoded%$(echo -n "$char" | od -An -tx1 | sed 's/ //g')" + fi + ;; + esac + done + + echo "$encoded" +} + +# Enhanced JSON utilities with better error handling +dns_json_get() { + json="$1" + path="$2" + + # Validate input + if [ -z "$json" ] || [ -z "$path" ]; then + dns_log_debug "Invalid JSON or path provided" + return 1 + fi + + # Use python if available for robust JSON parsing + if which python >/dev/null 2>&1; then + echo "$json" | python -c " +import sys, json +try: + data = json.load(sys.stdin) + path = '$path'.split('.') + result = data + for key in path: + if key.isdigit(): + result = result[int(key)] + elif key in result: + result = result[key] + else: + print('') + sys.exit(0) + if result is not None: + if isinstance(result, (str, int, float, bool)): + print(result) + else: + print(json.dumps(result)) + else: + print('') +except (KeyError, IndexError, TypeError, ValueError) as e: + print('') +except Exception as e: + print('') + sys.exit(1) +" + elif which python3 >/dev/null 2>&1; then + echo "$json" | python3 -c " +import sys, json +try: + data = json.load(sys.stdin) + path = '$path'.split('.') + result = data + for key in path: + if key.isdigit(): + result = result[int(key)] + elif key in result: + result = result[key] + else: + print('') + sys.exit(0) + if result is not None: + if isinstance(result, (str, int, float, bool)): + print(result) + else: + print(json.dumps(result)) + else: + print('') +except (KeyError, IndexError, TypeError, ValueError) as e: + print('') +except Exception as e: + print('') + sys.exit(1) +" + else + # Enhanced fallback using sed/awk for ESXi compatibility + dns_log_debug "Using fallback JSON parser" + + # Handle simple cases with sed/grep + case "$path" in + *.*) + # Complex path - not well supported in fallback + echo "$json" | sed -n "s/.*\"$(echo "$path" | cut -d. -f1)\"[[:space:]]*:[[:space:]]*\"\([^\"]*\)\".*/\1/p" | head -1 + ;; + *) + # Simple key lookup + echo "$json" | sed -n "s/.*\"$path\"[[:space:]]*:[[:space:]]*\"\([^\"]*\)\".*/\1/p" | head -1 + ;; + esac + fi +} + +# JSON validation utility +dns_json_validate() { + json="$1" + + if which python >/dev/null 2>&1; then + echo "$json" | python -c " +import sys, json +try: + json.load(sys.stdin) + sys.exit(0) +except: + sys.exit(1) +" 2>/dev/null + elif which python3 >/dev/null 2>&1; then + echo "$json" | python3 -c " +import sys, json +try: + json.load(sys.stdin) + sys.exit(0) +except: + sys.exit(1) +" 2>/dev/null + else + # Basic validation - check for balanced braces + open_braces="" + close_braces="" + open_braces=$(echo "$json" | sed 's/[^\{]//g' | wc -c) + close_braces=$(echo "$json" | sed 's/[^\}]//g' | wc -c) + [ "$open_braces" -eq "$close_braces" ] + fi +} + +# Extract error messages from API responses +dns_extract_error() { + response="$1" + provider="$2" + + if [ -z "$response" ]; then + echo "Empty response from API" + return 1 + fi + + # Try to validate and parse JSON response + if dns_json_validate "$response"; then + # Provider-specific error extraction + case "$provider" in + "cloudflare") + error_msg=$(dns_json_get "$response" "errors.0.message") + [ -n "$error_msg" ] && echo "$error_msg" && return 0 + ;; + "route53") + error_msg=$(dns_json_get "$response" "Error.Message") + [ -n "$error_msg" ] && echo "$error_msg" && return 0 + ;; + "digitalocean") + error_msg=$(dns_json_get "$response" "message") + [ -n "$error_msg" ] && echo "$error_msg" && return 0 + ;; + esac + + # Generic error field extraction + for field in "error" "message" "error_description" "detail"; do + error_msg=$(dns_json_get "$response" "$field") + if [ -n "$error_msg" ]; then + echo "$error_msg" + return 0 + fi + done + fi + + # Fallback to basic text extraction + if echo "$response" | grep -qi "error\|failed\|invalid"; then + echo "$response" | head -3 | awk '{printf "%s ", $0}' + return 0 + fi + + echo "Unknown API error" + return 1 +} + +# Enhanced DNS propagation checking with multiple strategies +dns_check_propagation() { + domain="$1" + expected_value="$2" + max_wait="${3:-$DEFAULT_PROPAGATION_WAIT}" + check_interval="${4:-10}" + + dns_log_info "Checking DNS propagation for _acme-challenge.$domain" + + waited=0 + # Multiple resolver sets for comprehensive checking + public_resolvers="8.8.8.8 1.1.1.1 208.67.222.222 9.9.9.9" + backup_resolvers="8.8.4.4 1.0.0.1 208.67.220.220 149.112.112.112" + + # Start with authoritative nameserver check if available + auth_ns="" + if which dig >/dev/null 2>&1; then + auth_ns=$(dig +short NS "$domain" 2>/dev/null | head -1) + if [ -n "$auth_ns" ]; then + # Remove trailing dot + auth_ns=$(echo "$auth_ns" | sed 's/\.$//') + dns_log_debug "Found authoritative nameserver: $auth_ns" + fi + fi + + while [ $waited -lt $max_wait ]; do + found=0 + total_resolvers=0 + resolvers_to_check="$public_resolvers" + + # Check authoritative nameserver first if available + if [ -n "$auth_ns" ]; then + total_resolvers=$((total_resolvers + 1)) + if dns_query_resolver "$auth_ns" "$domain" "$expected_value"; then + found=$((found + 1)) + dns_log_debug "Found expected value on authoritative NS: $auth_ns" + fi + fi + + # Check public resolvers + for resolver in $resolvers_to_check; do + total_resolvers=$((total_resolvers + 1)) + if dns_query_resolver "$resolver" "$domain" "$expected_value"; then + found=$((found + 1)) + dns_log_debug "Found expected value on resolver: $resolver" + fi + done + + # If not enough resolvers agree, try backup resolvers + required=$((total_resolvers / 2 + 1)) + if [ $found -lt $required ] && [ $waited -gt $((max_wait / 2)) ]; then + dns_log_debug "Trying backup resolvers for additional confirmation" + for resolver in $backup_resolvers; do + total_resolvers=$((total_resolvers + 1)) + if dns_query_resolver "$resolver" "$domain" "$expected_value"; then + found=$((found + 1)) + dns_log_debug "Found expected value on backup resolver: $resolver" + fi + done + required=$((total_resolvers / 2 + 1)) + fi + + if [ $found -ge $required ]; then + dns_log_info "DNS propagation confirmed ($found/$total_resolvers resolvers)" + return 0 + fi + + dns_log_debug "DNS propagation incomplete ($found/$total_resolvers), waiting $check_interval seconds..." + sleep $check_interval + waited=$((waited + check_interval)) + done + + dns_log_error "DNS propagation check failed after $waited seconds" + return 1 +} + +# Helper function to query a specific resolver +dns_query_resolver() { + resolver="$1" + domain="$2" + expected_value="$3" + + result="" + + # Use dig if available, fallback to nslookup + if which dig >/dev/null 2>&1; then + result=$(dig @"$resolver" TXT "_acme-challenge.$domain" +short +timeout=5 +tries=1 2>/dev/null | sed 's/"//g' | head -1) + elif which nslookup >/dev/null 2>&1; then + # nslookup with timeout (ESXi compatible) + result=$(timeout 10 nslookup -type=TXT "_acme-challenge.$domain" "$resolver" 2>/dev/null | grep -o '"[^\"]*"' | sed 's/"//g' | head -1) + else + dns_log_error "No DNS query tool available (dig or nslookup)" + return 1 + fi + + [ "$result" = "$expected_value" ] +} + +# DNS cache busting - force fresh queries +dns_flush_cache() { + domain="$1" + + dns_log_debug "Attempting to flush DNS cache for $domain" + + # Try various cache-busting techniques + if which systemd-resolve >/dev/null 2>&1; then + systemd-resolve --flush-caches 2>/dev/null || true + elif which resolvectl >/dev/null 2>&1; then + resolvectl flush-caches 2>/dev/null || true + elif [ -f /etc/init.d/nscd ]; then + /etc/init.d/nscd restart 2>/dev/null || true + fi + + # Add random query to bust caches + random_subdomain="cache-bust-$(date +%s)" + if which dig >/dev/null 2>&1; then + dig "$random_subdomain.$domain" +short >/dev/null 2>&1 || true + fi +} + +# ESXi Environment Initialization (ESXi 6.5+ Only) +# This project is designed exclusively for ESXi environments +dns_init_esxi_environment() { + dns_log_debug "Initializing ESXi 6.5+ environment" + + # ESXi-optimized defaults (memory-based caching only) + DNS_CACHE_TTL=${DNS_CACHE_TTL:-120} # 2 minutes - conservative for ESXi + DNS_USE_FILE_CACHE=0 # Always disabled for ESXi read-only filesystem + + # Check memory constraints specific to ESXi + free_mem="" + if which free >/dev/null 2>&1; then + free_mem=$(free 2>/dev/null | awk '/^Mem:/ {print $4}' 2>/dev/null) + if [ -n "$free_mem" ] && [ "$free_mem" -lt 100000 ]; then # Less than ~100MB + dns_log_debug "ESXi memory constrained (${free_mem}K) - using shorter cache TTL" + DNS_CACHE_TTL=60 # 1 minute for memory-constrained ESXi hosts + fi + fi + + dns_log_debug "ESXi environment initialized: TTL=${DNS_CACHE_TTL}s, Memory cache only" +} + +# Supported DNS providers +SUPPORTED_PROVIDERS="cloudflare route53 digitalocean namecheap godaddy powerdns duckdns ns1 gcloud azure manual" + +# Provider loading and validation +dns_load_provider() { + provider="$1" + + if [ -z "$provider" ]; then + dns_log_error "No DNS provider specified" + return 1 + fi + + # Check if provider is supported + supported=false + for p in $SUPPORTED_PROVIDERS; do + if [ "$p" = "$provider" ]; then + supported=true + break + fi + done + + if [ "$supported" != "true" ]; then + dns_log_error "Unsupported DNS provider: $provider" + dns_log_info "Supported providers: $SUPPORTED_PROVIDERS" + return 1 + fi + + # Load provider script + provider_script="$DNSAPIDIR/dns_${provider}.sh" + + # Debug: Check provider script permissions and type + ls -l "$provider_script" >&2 + if [ -x "$provider_script" ]; then + echo "[DNS-WARN] Provider script $provider_script is executable. It should NOT be executable; it is meant to be sourced, not run directly." >&2 + fi + + dns_log_debug "Checking for provider script: $provider_script" + if [ ! -f "$provider_script" ]; then + dns_log_error "Provider script not found: $provider_script" + ls -l "$DNSAPIDIR" >&2 + return 1 + fi + if [ ! -r "$provider_script" ]; then + dns_log_error "Provider script is not readable: $provider_script" + ls -l "$provider_script" >&2 + return 1 + fi + + dns_log_debug "Loading DNS provider: $provider from $provider_script" + . "$provider_script" + source_status=$? + if [ $source_status -ne 0 ]; then + dns_log_error "Failed to source provider script: $provider_script (exit code $source_status)" + return 1 + fi + + dns_log_debug "Provider $provider loaded (function checks skipped)." + return 0 +} + +# Provider wrapper functions with retry logic +dns_provider_add() { + provider="$1" + domain="$2" + txt_value="$3" + retries=0 + func="dns_${provider}_add" + while [ $retries -lt $DEFAULT_MAX_RETRIES ]; do + if $func "$domain" "$txt_value"; then + return 0 + fi + retries=$((retries + 1)) + if [ $retries -lt $DEFAULT_MAX_RETRIES ]; then + dns_log_warn "DNS add attempt $retries failed, retrying in $DEFAULT_RETRY_DELAY seconds..." + sleep $DEFAULT_RETRY_DELAY + fi + done + dns_log_error "Failed to add DNS record after $DEFAULT_MAX_RETRIES attempts" + return 1 +} + +dns_provider_rm() { + provider="$1" + domain="$2" + txt_value="$3" + retries=0 + func="dns_${provider}_rm" + while [ $retries -lt $DEFAULT_MAX_RETRIES ]; do + if $func "$domain" "$txt_value"; then + return 0 + fi + retries=$((retries + 1)) + if [ $retries -lt $DEFAULT_MAX_RETRIES ]; then + dns_log_warn "DNS remove attempt $retries failed, retrying in $DEFAULT_RETRY_DELAY seconds..." + sleep $DEFAULT_RETRY_DELAY + fi + done + dns_log_error "Failed to remove DNS record after $DEFAULT_MAX_RETRIES attempts" + return 1 +} + +dns_provider_test() { + provider="$1" + func="dns_${provider}_test" + # Call the function and handle if not defined + if type "$func" 2>/dev/null | grep -q 'function'; then + $func + else + dns_log_warn "Provider $provider does not support testing" + return 0 + fi +} + +dns_provider_info() { + provider="$1" + func="dns_${provider}_info" + # Call the function and handle if not defined + if type "$func" 2>/dev/null | grep -q 'function'; then + $func + else + echo "DNS Provider: $provider" + echo "No additional information available" + fi +} + +# Command handlers +dns_cmd_add() { + domain="$1" + txt_value="$2" + + if ! dns_validate_domain "$domain"; then + return 1 + fi + + if ! dns_validate_txt_value "$txt_value"; then + return 1 + fi + + if [ -z "$DNS_PROVIDER" ]; then + dns_log_error "DNS_PROVIDER not set in configuration" + return 1 + fi + + if ! dns_load_provider "$DNS_PROVIDER"; then + return 1 + fi + + dns_log_info "Adding DNS TXT record for $domain using $DNS_PROVIDER" + dns_log_debug "[GLOBAL] Entering provider add logic for $domain" + + DNS_ADD_TIMEOUT="${DNS_ADD_TIMEOUT:-120}" + add_exit_code=1 + + # Always run in current shell for function scope + start_time=$(date +%s) + dns_provider_add "$DNS_PROVIDER" "$domain" "$txt_value" + add_exit_code=$? + end_time=$(date +%s) + elapsed=$((end_time - start_time)) + if [ $elapsed -gt "$DNS_ADD_TIMEOUT" ]; then + dns_log_warn "Provider add operation exceeded timeout of ${DNS_ADD_TIMEOUT}s (ran ${elapsed}s)" + fi + + dns_log_debug "[GLOBAL] Exited provider add logic for $domain with exit code $add_exit_code" + + if [ $add_exit_code -eq 0 ]; then + dns_log_info "DNS record added successfully" + + # Wait for propagation if configured + if [ "${DNS_PROPAGATION_WAIT:-0}" -gt 0 ]; then + dns_log_info "Waiting ${DNS_PROPAGATION_WAIT}s for DNS propagation..." + if dns_check_propagation "$domain" "$txt_value" "$DNS_PROPAGATION_WAIT"; then + dns_log_info "DNS propagation verified" + else + dns_log_warn "DNS propagation could not be verified, but record was added" + fi + fi + + return 0 + else + dns_log_error "Failed to add DNS record" + return 1 + fi +} + +dns_cmd_rm() { + domain="$1" + txt_value="$2" + + if ! dns_validate_domain "$domain"; then + return 1 + fi + + if [ -z "$DNS_PROVIDER" ]; then + dns_log_error "DNS_PROVIDER not set in configuration" + return 1 + fi + + if ! dns_load_provider "$DNS_PROVIDER"; then + return 1 + fi + + dns_log_info "Removing DNS TXT record for $domain using $DNS_PROVIDER" + + if dns_provider_rm "$DNS_PROVIDER" "$domain" "$txt_value"; then + dns_log_info "DNS record removed successfully" + return 0 + else + dns_log_error "Failed to remove DNS record" + return 1 + fi +} + +dns_cmd_test() { + if [ -z "$DNS_PROVIDER" ]; then + dns_log_error "DNS_PROVIDER not set in configuration" + return 1 + fi + + if ! dns_load_provider "$DNS_PROVIDER"; then + return 1 + fi + + dns_log_info "Testing DNS provider: $DNS_PROVIDER" + + if dns_provider_test "$DNS_PROVIDER"; then + dns_log_info "DNS provider test successful" + return 0 + else + dns_log_error "DNS provider test failed" + return 1 + fi +} + +dns_cmd_info() { + provider="${1:-$DNS_PROVIDER}" + + if [ -z "$provider" ]; then + dns_log_error "No DNS provider specified" + return 1 + fi + + if ! dns_load_provider "$provider"; then + return 1 + fi + + dns_provider_info "$provider" + return 0 +} + +dns_cmd_list() { + echo "Supported DNS Providers:" + echo "========================" + echo "" + + for provider in $SUPPORTED_PROVIDERS; do + echo "- $provider" + if [ -f "$DNSAPIDIR/dns_${provider}.sh" ]; then + echo " Status: Available" + else + echo " Status: Missing provider script" + fi + echo "" + done + + echo "Current Configuration:" + echo "- DNS_PROVIDER: ${DNS_PROVIDER:-not set}" + echo "- DNS_PROPAGATION_WAIT: ${DNS_PROPAGATION_WAIT:-120}s" + echo "- DNS_TIMEOUT: ${DNS_TIMEOUT:-30}s" + echo "- MAX_RETRIES: ${MAX_RETRIES:-3}" +} + +# Main function +main() { + command="$1" + domain="$2" + token="$3" + key_auth="$4" + + # Show usage if no command provided + if [ -z "$command" ]; then + echo "DNS API Framework v$DNS_API_VERSION" + echo "Usage: dns_api.sh [token] [key_auth]" + echo "" + echo "Commands:" + echo " add - Add TXT record for ACME challenge" + echo " rm - Remove TXT record" + echo " test - Test DNS provider connectivity" + echo " info [provider] - Show provider information" + echo " list - List all supported providers" + echo "" + echo "Configuration is loaded from renew.cfg" + echo "Set DNS_PROVIDER to specify which provider to use" + echo "" + echo "ACME Integration:" + echo "This script is called by acme_tiny.py during DNS-01 challenges" + echo "The TXT value is calculated from the key_auth parameter" + return 1 + fi + + # Handle commands + case "$command" in + "add") + if [ -z "$domain" ]; then + dns_log_error "Usage: dns_api.sh add " + return 1 + fi + if [ -z "$TXT_VALUE" ]; then + dns_log_error "Failed to calculate TXT value - key authorization required" + return 1 + fi + dns_cmd_add "$domain" "$TXT_VALUE" + ;; + "rm"|"remove") + if [ -z "$domain" ]; then + dns_log_error "Usage: dns_api.sh rm " + return 1 + fi + # For remove, TXT_VALUE is optional as some providers can remove by domain only + dns_cmd_rm "$domain" "$TXT_VALUE" + ;; + "test") + dns_cmd_test + ;; + "info") + dns_cmd_info "$domain" + ;; + "list") + dns_cmd_list + ;; + *) + dns_log_error "Unknown command: $command" + dns_log_info "Run 'dns_api.sh' without arguments to see usage" + return 1 + ;; + esac +} + +# Initialize ESXi environment +dns_init_esxi_environment + +# Run main function if script is executed directly +if [ "${0##*/}" = "dns_api.sh" ]; then + main "$COMMAND" "$DOMAIN" "$TOKEN" "$KEY_AUTH" +fi diff --git a/dnsapi/dns_cloudflare.sh b/dnsapi/dns_cloudflare.sh new file mode 100644 index 0000000..2146972 --- /dev/null +++ b/dnsapi/dns_cloudflare.sh @@ -0,0 +1,341 @@ +# Cloudflare DNS API Provider +# Requires: CF_API_TOKEN or CF_API_KEY + CF_EMAIL +# + +# Provider information +dns_cloudflare_info() { + echo "Cloudflare DNS API Provider" + echo "Website: https://cloudflare.com" + echo "Documentation: https://developers.cloudflare.com/api/operations/dns-records-for-a-zone-create-dns-record" + echo "" + echo "Required Environment Variables:" + echo " CF_API_TOKEN - Cloudflare API Token (recommended)" + echo " OR" + echo " CF_API_KEY - Cloudflare Global API Key" + echo " CF_EMAIL - Cloudflare account email" + echo "" + echo "Optional Settings:" + echo " CF_TTL - TTL for DNS records (default: 120)" + echo " CF_PROXY - Enable Cloudflare proxy (default: false)" +} + +# Cloudflare API endpoints +CF_API_BASE="https://api.cloudflare.com/client/v4" + +# Default settings +CF_TTL=${CF_TTL:-120} +CF_PROXY=${CF_PROXY:-false} + +# Authentication setup +_cf_setup_auth() { + if [ -n "${CF_API_TOKEN:-}" ]; then + dns_log_debug "Using Cloudflare API Token authentication" + CF_AUTH_HEADER="Authorization: Bearer $CF_API_TOKEN" + return 0 + elif [ -n "${CF_API_KEY:-}" ] && [ -n "${CF_EMAIL:-}" ]; then + dns_log_debug "Using Cloudflare Global API Key authentication" + CF_AUTH_HEADER="X-Auth-Key: $CF_API_KEY" + CF_EMAIL_HEADER="X-Auth-Email: $CF_EMAIL" + return 0 + else + dns_log_error "Cloudflare credentials not found. Please set CF_API_TOKEN or (CF_API_KEY + CF_EMAIL)" + return 1 + fi +} + +# Helper: Extract base domain from FQDN (e.g., sub.domain.example.com -> example.com) +cf_get_base_domain() { + fqdn="$1" + # Use awk to get the last two labels (handles most cases) + echo "$fqdn" | awk -F. '{if (NF>=2) print $(NF-1)"."$NF; else print $0}' +} + +# Get zone ID for domain +_cf_get_zone_id() { + domain="$1" + base_domain="$(cf_get_base_domain "$domain")" + + dns_log_debug "[CF] Looking up zone for base domain: $base_domain" + # Try exact match first + zone_response="" + headers="$CF_AUTH_HEADER" + [ -n "$CF_EMAIL_HEADER" ] && headers="$headers +$CF_EMAIL_HEADER" + headers="$headers +Content-Type: application/json" + zone_response=$(dns_http_get "$CF_API_BASE/zones?name=$base_domain" "$headers") + dns_log_debug "[CF] Zone lookup response: $zone_response" + + zone_id=$(dns_json_get "$zone_response" "result.0.id") + + if [ -n "$zone_id" ] && [ "$zone_id" != "null" ]; then + dns_log_debug "[CF] Found zone_id: $zone_id for $base_domain" + echo "$zone_id" + return 0 + fi + + # Try parent domains (should rarely be needed) + parent_domain="$base_domain" + while [ "$(echo "$parent_domain" | awk -F'.' '{print NF}')" -gt 2 ]; do + parent_domain=$(echo "$parent_domain" | cut -d. -f2-) + dns_log_debug "[CF] Trying parent domain: $parent_domain" + if [ -n "$CF_EMAIL_HEADER" ]; then + headers="$CF_AUTH_HEADER +$CF_EMAIL_HEADER +Content-Type: application/json" + else + headers="$CF_AUTH_HEADER +Content-Type: application/json" + fi + zone_response=$(dns_http_get "$CF_API_BASE/zones?name=$parent_domain" "$headers") + dns_log_debug "[CF] Parent zone lookup response: $zone_response" + zone_id=$(dns_json_get "$zone_response" "result.0.id") + if [ -n "$zone_id" ] && [ "$zone_id" != "null" ]; then + dns_log_debug "[CF] Found parent zone_id: $zone_id for $parent_domain" + echo "$zone_id" + return 0 + fi + done + + dns_log_error "Could not find Cloudflare zone for domain: $base_domain" + return 1 +} + +# Get existing TXT record ID +_cf_get_txt_record_id() { + zone_id="$1" + record_name="$2" + txt_value="$3" + + dns_log_debug "[CF] Looking for TXT record in zone $zone_id with name $record_name and value $txt_value" + records_response="" + headers="$CF_AUTH_HEADER" + [ -n "$CF_EMAIL_HEADER" ] && headers="$headers +$CF_EMAIL_HEADER" + headers="$headers +Content-Type: application/json" + records_response=$(dns_http_get "$CF_API_BASE/zones/$zone_id/dns_records?type=TXT&name=$record_name" "$headers") + dns_log_debug "[CF] TXT record lookup response: $records_response" + + i=0 + max_iter=20 + while [ $i -lt $max_iter ]; do + record_id=$(dns_json_get "$records_response" "result.$i.id") + record_content=$(dns_json_get "$records_response" "result.$i.content") + + if [ -z "$record_id" ] || [ "$record_id" = "null" ]; then + break + fi + + if [ "$record_content" = "$txt_value" ]; then + dns_log_debug "[CF] Found matching TXT record id: $record_id" + echo "$record_id" + return 0 + fi + + i=$((i + 1)) + done + + if [ $i -ge $max_iter ]; then + dns_log_warn "[CF] TXT record search exceeded $max_iter iterations, possible malformed API response." + fi + + return 1 +} + +# Add TXT record +dns_cloudflare_add() { + domain="$1" + txt_value="$2" + + dns_log_debug "[CF] Starting dns_cloudflare_add for $domain" + _cf_setup_auth || return 1 + + record_name="_acme-challenge.$domain" + zone_id="" + + # Always use base domain for zone lookup + base_domain="$(cf_get_base_domain "$domain")" + dns_log_debug "[CF] Using base domain for zone lookup: $base_domain" + zone_id=$(_cf_get_zone_id "$base_domain") + if [ -z "$zone_id" ]; then + dns_log_error "[CF] No zone_id found for $base_domain" + return 1 + fi + + dns_log_debug "[CF] Found Cloudflare zone ID: $zone_id" + + # Check if record already exists + existing_record_id="" + existing_record_id=$(_cf_get_txt_record_id "$zone_id" "$record_name" "$txt_value") + + if [ -n "$existing_record_id" ]; then + dns_log_info "TXT record already exists with ID: $existing_record_id" + echo "$existing_record_id" > "/tmp/acme_cf_record_${domain}.id" + return 0 + fi + + # Create new TXT record + record_data="{\n \"type\": \"TXT\",\n \"name\": \"$record_name\",\n \"content\": \"$txt_value\",\n \"ttl\": $CF_TTL,\n \"proxied\": $CF_PROXY\n }" + + dns_log_debug "[CF] Creating new TXT record: $record_data" + create_response="" + headers="$CF_AUTH_HEADER" + [ -n "$CF_EMAIL_HEADER" ] && headers="$headers +$CF_EMAIL_HEADER" + headers="$headers +Content-Type: application/json" + create_response=$(dns_http_post "$CF_API_BASE/zones/$zone_id/dns_records" "$record_data" "$headers") + dns_log_debug "[CF] TXT record create response: $create_response" + + record_id=$(dns_json_get "$create_response" "result.id") + success=$(dns_json_get "$create_response" "success") + # Normalize success to lowercase for comparison (BusyBox/ESXi compatible) + success_lc=$(echo "$success" | sed 'y/ABCDEFGHIJKLMNOPQRSTUVWXYZ/abcdefghijklmnopqrstuvwxyz/') + + if { [ "$success_lc" = "true" ] || [ "$success" = "1" ]; } && [ -n "$record_id" ] && [ "$record_id" != "null" ]; then + dns_log_info "Created Cloudflare TXT record: $record_id" + echo "$record_id" > "/tmp/acme_cf_record_${domain}.id" + echo "$zone_id" > "/tmp/acme_cf_zone_${domain}.id" + return 0 + else + error_msg=$(dns_json_get "$create_response" "errors.0.message") + # Fallback: if no error message but record_id exists, treat as success + if [ -z "$error_msg" ] && [ -n "$record_id" ] && [ "$record_id" != "null" ]; then + dns_log_info "Created Cloudflare TXT record (fallback): $record_id" + echo "$record_id" > "/tmp/acme_cf_record_${domain}.id" + echo "$zone_id" > "/tmp/acme_cf_zone_${domain}.id" + return 0 + fi + dns_log_error "Failed to create Cloudflare TXT record: $error_msg" + return 1 + fi +} + +# Remove TXT record +dns_cloudflare_rm() { + domain="$1" + txt_value="$2" + + _cf_setup_auth || return 1 + + record_file="/tmp/acme_cf_record_${domain}.id" + zone_file="/tmp/acme_cf_zone_${domain}.id" + + # Try to get record ID from file first + record_id="" + zone_id="" + + if [ -f "$record_file" ]; then + record_id=$(cat "$record_file" 2>/dev/null) + fi + + if [ -f "$zone_file" ]; then + zone_id=$(cat "$zone_file" 2>/dev/null) + fi + + # If we don't have the IDs, try to find them + if [ -z "$zone_id" ]; then + base_domain="$(cf_get_base_domain "$domain")" + zone_id=$(_cf_get_zone_id "$base_domain") + if [ -z "$zone_id" ]; then + dns_log_warn "Could not find zone ID for cleanup" + rm -f "$record_file" "$zone_file" + return 0 + fi + fi + + if [ -z "$record_id" ] && [ -n "$txt_value" ]; then + record_id=$(_cf_get_txt_record_id "$zone_id" "_acme-challenge.$domain" "$txt_value") + fi + + if [ -n "$record_id" ] && [ "$record_id" != "null" ]; then + dns_log_debug "Deleting Cloudflare record ID: $record_id" + + delete_response="" + if [ -n "$CF_EMAIL_HEADER" ]; then + delete_response=$(dns_http_delete "$CF_API_BASE/zones/$zone_id/dns_records/$record_id" "$CF_AUTH_HEADER" "$CF_EMAIL_HEADER" "Content-Type: application/json") + else + delete_response=$(dns_http_delete "$CF_API_BASE/zones/$zone_id/dns_records/$record_id" "$CF_AUTH_HEADER" "Content-Type: application/json") + fi + + success=$(dns_json_get "$delete_response" "success") + + if [ "$success" = "true" ]; then + dns_log_info "Deleted Cloudflare TXT record" + else + error_msg=$(dns_json_get "$delete_response" "errors.0.message") + dns_log_warn "Failed to delete Cloudflare TXT record: $error_msg" + fi + else + dns_log_warn "No record ID found for cleanup (record may have already been deleted)" + fi + + # Clean up temporary files + rm -f "$record_file" "$zone_file" + return 0 +} + +# Check if zone exists (override default implementation) +dns_zone_exists() { + zone="$1" + provider="$2" + + if [ "$provider" != "cloudflare" ]; then + return 1 + fi + + _cf_setup_auth || return 1 + + base_domain="$(cf_get_base_domain "$zone")" + zone_id=$(_cf_get_zone_id "$base_domain") + + [ -n "$zone_id" ] +} + +# Get zone for domain (override default implementation) +dns_cloudflare_get_zone() { + domain="$1" + + _cf_setup_auth || return 1 + + base_domain="$(cf_get_base_domain "$domain")" + # Try exact match first + zone_response="" + # Fix: Pass all headers as a single string, not as multiple arguments + headers="$CF_AUTH_HEADER" + [ -n "$CF_EMAIL_HEADER" ] && headers="$headers +$CF_EMAIL_HEADER" + headers="$headers +Content-Type: application/json" + zone_response=$(dns_http_get "$CF_API_BASE/zones?name=$base_domain" "$headers") + + zone_name=$(dns_json_get "$zone_response" "result.0.name") + + if [ -n "$zone_name" ] && [ "$zone_name" != "null" ]; then + echo "$zone_name" + return 0 + fi + + # Try parent domains (should rarely be needed) + parent_domain="$base_domain" + while [ "$(echo "$parent_domain" | awk -F'.' '{print NF}')" -gt 2 ]; do + parent_domain=$(echo "$parent_domain" | cut -d. -f2-) + if [ -n "$CF_EMAIL_HEADER" ]; then + headers="$CF_AUTH_HEADER +$CF_EMAIL_HEADER +Content-Type: application/json" + else + headers="$CF_AUTH_HEADER +Content-Type: application/json" + fi + zone_response=$(dns_http_get "$CF_API_BASE/zones?name=$parent_domain" "$headers") + zone_name=$(dns_json_get "$zone_response" "result.0.name") + if [ -n "$zone_name" ] && [ "$zone_name" != "null" ]; then + echo "$zone_name" + return 0 + fi + done + + return 1 +} diff --git a/dnsapi/dns_manual.sh b/dnsapi/dns_manual.sh new file mode 100644 index 0000000..17ee7a8 --- /dev/null +++ b/dnsapi/dns_manual.sh @@ -0,0 +1,121 @@ +# Manual DNS API Provider +# For testing or when automatic DNS management is not available +# + +# Provider information +dns_manual_info() { + echo "Manual DNS API Provider" + echo "Description: Interactive manual DNS record management" + echo "" + echo "WARNING: This provider requires manual interaction and is NOT suitable" + echo " for automated certificate renewals (cron jobs, etc.)" + echo "" + echo "This provider requires manual intervention to:" + echo "1. Create TXT records in your DNS provider's control panel" + echo "2. Verify DNS propagation" + echo "3. Clean up records after certificate issuance" + echo "" + echo "Use Cases:" + echo "- Testing certificate issuance process" + echo "- One-time certificate generation" + echo "- DNS providers not yet supported by automated providers" + echo "- Learning how DNS-01 challenges work" + echo "" + echo "For automated renewals, use providers like:" + echo "- cloudflare (Cloudflare DNS)" + echo "- route53 (AWS Route 53)" + echo "- gcloud (Google Cloud DNS)" + echo "- azure (Azure DNS)" + echo "- And others listed in: ./dnsapi/dns_api.sh list" + echo "" + echo "Optional Settings:" + echo " MANUAL_AUTO_CONTINUE - Skip manual prompts (default: false)" + echo " MANUAL_TTL - Recommended TTL value (default: 120)" +} + +# Default settings +MANUAL_TTL=${MANUAL_TTL:-120} +MANUAL_AUTO_CONTINUE=${MANUAL_AUTO_CONTINUE:-false} + +# Add TXT record +# Usage: dns_manual_add +dns_manual_add() { + domain="$1" + txt_value="$2" + record_name="_acme-challenge.$domain" + + dns_log_info "Manual DNS Challenge Setup Required for $domain" + echo "============================================" + echo "Manual DNS Challenge Setup Required" + echo "============================================" + echo "Domain: $domain" + echo "Record Type: TXT" + echo "Record Name: $record_name" + echo "Record Value: $txt_value" + echo "TTL: $MANUAL_TTL (seconds)" + echo "" + echo "Please create the above TXT record in your DNS provider's control panel." + echo "" + echo "Steps:" + echo "1. Log into your DNS provider's management interface" + echo "2. Navigate to DNS settings for your domain" + echo "3. Add a new TXT record with the details above" + echo "4. Save the changes" + echo "5. Wait for DNS propagation (usually 1-5 minutes)" + echo "" + + if [ "$MANUAL_AUTO_CONTINUE" = "true" ]; then + dns_log_info "Auto-continue mode enabled, proceeding without confirmation" + return 0 + fi + + echo "Press Enter when the record is created and has propagated..." + read dummy + + dns_log_info "Verifying DNS propagation..." + if dns_check_propagation "$domain" "$txt_value" 180 15; then + dns_log_info "DNS propagation verified successfully" + return 0 + else + dns_log_warn "DNS propagation verification failed, but continuing anyway" + echo "" + echo "The verification failed, but this might be due to:" + echo "- DNS propagation delays" + echo "- Firewall blocking DNS queries" + echo "- Different DNS resolvers" + echo "" + echo "Press Enter to continue anyway, or Ctrl+C to abort..." + read dummy + return 0 + fi +} + +# Remove TXT record +# Usage: dns_manual_rm +dns_manual_rm() { + domain="$1" + txt_value="$2" + record_name="_acme-challenge.$domain" + + dns_log_info "Manual DNS Challenge Cleanup for $domain" + echo "============================================" + echo "Manual DNS Challenge Cleanup" + echo "============================================" + echo "Domain: $domain" + echo "Record Type: TXT" + echo "Record Name: $record_name" + if [ -n "$txt_value" ]; then + echo "Record Value: $txt_value" + fi + echo "" + echo "You can now remove the TXT record from your DNS provider." + echo "This is optional as the record is no longer needed." + echo "" + if [ "$MANUAL_AUTO_CONTINUE" = "true" ]; then + dns_log_info "Auto-continue mode enabled, cleanup message displayed" + return 0 + fi + echo "Press Enter to continue..." + read dummy + return 0 +} diff --git a/renew.cfg.example b/renew.cfg.example new file mode 100644 index 0000000..aa244c0 --- /dev/null +++ b/renew.cfg.example @@ -0,0 +1,76 @@ +# Let's Encrypt ESXi Configuration Template +# +# USAGE: +# 1. Copy this file: cp renew.cfg.example renew.cfg +# 2. Edit renew.cfg and uncomment/configure settings as needed +# 3. For HTTP-01 (default): No configuration required - works out of the box +# 4. For DNS-01: Uncomment CHALLENGE_TYPE, DNS_PROVIDER, and provider credentials +# +# This file is safe to use as-is with default HTTP-01 behavior. +# All non-default settings are commented out to prevent configuration errors. + +# ============================================================================= +# LET'S ENCRYPT SETTINGS +# ============================================================================= +# Let's Encrypt server URL (default: production) +# Uncomment below line for staging/testing (issues test certificates) +#DIRECTORY_URL="https://acme-staging-v02.api.letsencrypt.org/directory" + +# Certificate renewal interval in days (default: 30) +# Certificates are renewed this many days before expiration +#RENEW_DAYS=14 + +# Domain name for certificate (default: uses ESXi hostname) +# Override only if hostname doesn't match desired certificate domain +#DOMAIN=$(hostname -f) + +# Challenge type: "http-01" or "dns-01" (default: "http-01") +#CHALLENGE_TYPE="dns-01" + +# ============================================================================= +# DNS PROVIDER CONFIGURATION (Required for DNS-01) +# ============================================================================= +# Primary DNS provider - choose one: +# Supported: cloudflare, manual +#DNS_PROVIDER="cloudflare" + +# WARNING: The "manual" provider requires user interaction and will NOT work +# with automated renewals (cron jobs). Use only for testing or one-time +# certificate generation. For production ESXi deployments, use an automated +# provider like cloudflare, route53, gcloud, azure, etc. + +# DNS challenge settings +#DNS_PROPAGATION_WAIT=120 # Seconds to wait for DNS propagation +#DNS_PROPAGATION_CHECK=1 # Enable active DNS propagation checking (1) or use fixed wait (0) +#DNS_TIMEOUT=30 # API request timeout in seconds +#MAX_RETRIES=3 # Maximum retry attempts for failed API calls +#RETRY_DELAY=5 # Base delay between retries (exponential backoff) +#DEBUG=0 # Enable debug logging (0=off, 1=on) +#DNS_CACHE_TTL=120 # Cache TTL in seconds (2 minutes for ESXi) + +# DNS provider-specific settings +# Uncomment and configure the provider you want to use + +# Cloudflare +# Create an API token at https://dash.cloudflare.com/profile/api-tokens with Zone:Edit permissions +#CF_API_TOKEN="your-cloudflare-api-token" + +# OR use Global API Key (legacy method) +#CF_API_KEY="your-cloudflare-api-key" +#CF_EMAIL="your-cloudflare-account-email" + +# Cloudflare-specific settings +#CF_TTL=120 # TTL for DNS records (seconds) +#CF_PROXY=false # Enable Cloudflare proxy for records (true/false) + +# ============================================================================= +# ADVANCED SETTINGS (rarely need to change) +# ============================================================================= +# File paths for certificates and keys +# Override only if you need custom file locations +#ACCOUNTKEY="esxi_account.key" +#KEY="esxi.key" +#CSR="esxi.csr" +#CRT="esxi.crt" +#VMWARE_CRT="/etc/vmware/ssl/rui.crt" +#VMWARE_KEY="/etc/vmware/ssl/rui.key" diff --git a/renew.sh b/renew.sh index 14f4c29..9af69a6 100644 --- a/renew.sh +++ b/renew.sh @@ -19,10 +19,22 @@ CRT="esxi.crt" VMWARE_CRT="/etc/vmware/ssl/rui.crt" VMWARE_KEY="/etc/vmware/ssl/rui.key" +# Default to HTTP-01 challenge +CHALLENGE_TYPE="http-01" +DNS_PROVIDER="" +DNS_PROPAGATION_WAIT=30 + +# Load user configuration (if available) if [ -r "$LOCALDIR/renew.cfg" ]; then . "$LOCALDIR/renew.cfg" fi +# Export configuration variables +export CHALLENGE_TYPE DNS_PROVIDER DNS_PROPAGATION_WAIT \ + CF_API_TOKEN CF_API_KEY CF_EMAIL \ + DIRECTORY_URL CONTACT_EMAIL \ + ACCOUNTKEY KEY CSR CRT VMWARE_CRT VMWARE_KEY SSL_CERT_FILE + log() { echo "$@" logger -p daemon.info -t "$0" "$@" @@ -65,31 +77,126 @@ if [ -e "$VMWARE_CRT" ]; then fi cd "$LOCALDIR" || exit -mkdir -p "$ACMEDIR" -# Route /.well-known/acme-challenge to port 8120 -if ! grep -q "acme-challenge" /etc/vmware/rhttpproxy/endpoints.conf; then - echo "/.well-known/acme-challenge local 8120 redirect allow" >> /etc/vmware/rhttpproxy/endpoints.conf - /etc/init.d/rhttpproxy restart -fi +# Detect if we're running in an automated context (cron, etc.) +is_automated_run() { + if [ ! -t 0 ]; then return 0; fi + if [ -z "$TERM" ] || [ "$TERM" = "dumb" ]; then return 0; fi + if [ -n "$CRON" ]; then return 0; fi + if ps -o comm= -p $PPID 2>/dev/null | grep -q "crond"; then return 0; fi + return 1 +} + +# Cleanup function to restore firewall rules +cleanup_firewall() { + if [ "$CHALLENGE_TYPE" = "http-01" ]; then + # Kill HTTP server if still running + if [ -n "$HTTP_SERVER_PID" ]; then + kill -9 "$HTTP_SERVER_PID" 2>/dev/null || true + fi + # Restore original firewall states + if [ -n "$ORIGINAL_WEBACCESS_STATE" ] && [ "$ORIGINAL_WEBACCESS_STATE" = "false" ]; then + esxcli network firewall ruleset set -e false -r webAccess 2>/dev/null || true + log "Restored webAccess firewall rule to disabled" + fi + if [ -n "$ORIGINAL_VSPHERE_STATE" ] && [ "$ORIGINAL_VSPHERE_STATE" = "false" ]; then + esxcli network firewall ruleset set -e false -r vSphereClient 2>/dev/null || true + log "Restored vSphereClient firewall rule to disabled" + fi + elif [ "$CHALLENGE_TYPE" = "dns-01" ]; then + # Restore original httpClient state + if [ -n "$ORIGINAL_HTTPCLIENT_STATE" ] && [ "$ORIGINAL_HTTPCLIENT_STATE" = "false" ]; then + esxcli network firewall ruleset set -e false -r httpClient 2>/dev/null || true + log "Restored httpClient firewall rule to disabled" + fi + fi +} + +trap cleanup_firewall EXIT INT TERM -# Cert Request -[ ! -r "$ACCOUNTKEY" ] && openssl genrsa 4096 > "$ACCOUNTKEY" +# Helper to enable outbound ACME firewall rule and store state +enable_httpclient_firewall() { + httpclient_enabled=$(esxcli network firewall ruleset list | grep "httpClient" | awk '{print $NF}') + ORIGINAL_HTTPCLIENT_STATE="$httpclient_enabled" + if [ "$httpclient_enabled" = "false" ]; then + esxcli network firewall ruleset set -e true -r httpClient + log "Enabled httpClient firewall rule for ACME communication" + fi +} -openssl genrsa -out "$KEY" 4096 -openssl req -new -sha256 -key "$KEY" -subj "/CN=$DOMAIN" -config "./openssl.cnf" > "$CSR" -chmod 0400 "$ACCOUNTKEY" "$KEY" +# Setup based on challenge type +if [ "$CHALLENGE_TYPE" = "http-01" ]; then + mkdir -p "$ACMEDIR" -# Start HTTP server on port 8120 for HTTP validation -esxcli network firewall ruleset set -e true -r httpClient -python -m "http.server" 8120 & -HTTP_SERVER_PID=$! + # Route /.well-known/acme-challenge to port 8120 + if ! grep -q "acme-challenge" /etc/vmware/rhttpproxy/endpoints.conf; then + echo "/.well-known/acme-challenge local 8120 redirect allow" >> /etc/vmware/rhttpproxy/endpoints.conf + /etc/init.d/rhttpproxy restart + fi -# Retrieve the certificate -export SSL_CERT_FILE -CERT=$(python ./acme_tiny.py --account-key "$ACCOUNTKEY" --csr "$CSR" --acme-dir "$ACMEDIR" --directory-url "$DIRECTORY_URL") + # Firewall management for HTTP-01 (needs inbound access on port 80/443) + log "Configuring firewall for HTTP-01 challenge..." + firewall_enabled=$(esxcli network firewall get | grep "Enabled:" | awk '{print $NF}') + webaccess_enabled=$(esxcli network firewall ruleset list | grep "webAccess" | awk '{print $NF}') + vsphere_enabled=$(esxcli network firewall ruleset list | grep "vSphereClient" | awk '{print $NF}') + # Store original states for restoration + ORIGINAL_FIREWALL_STATE="$firewall_enabled" + ORIGINAL_WEBACCESS_STATE="$webaccess_enabled" + ORIGINAL_VSPHERE_STATE="$vsphere_enabled" + # Enable required rulesets for HTTP-01 + if [ "$webaccess_enabled" = "false" ]; then + esxcli network firewall ruleset set -e true -r webAccess + log "Enabled webAccess firewall rule for HTTP-01" + fi + if [ "$vsphere_enabled" = "false" ]; then + esxcli network firewall ruleset set -e true -r vSphereClient + log "Enabled vSphereClient firewall rule for HTTP-01" + fi + # Enable outbound HTTP client for ACME communication (consolidated) + enable_httpclient_firewall + # Start HTTP server on port 8120 for HTTP validation + python3 -m "http.server" 8120 & + HTTP_SERVER_PID=$! + +elif [ "$CHALLENGE_TYPE" = "dns-01" ]; then + # Validate DNS provider configuration for DNS-01 challenge + if [ -z "$DNS_PROVIDER" ]; then + log "Error: DNS_PROVIDER must be set for dns-01 challenge" + exit 1 + fi + # Prevent automated renewal with manual DNS provider + if [ "$DNS_PROVIDER" = "manual" ]; then + if is_automated_run; then + log "Manual DNS provider detected in automated context (likely cron job)." + log "Skipping renewal to prevent user interaction requirements." + log "Manual DNS certificates should be renewed manually by running:" + log " $LOCALDIR/$LOCALSCRIPT" + log "Or change DNS_PROVIDER to an automated provider in renew.cfg" + exit 0 + else + log "Manual DNS provider detected. This will require interactive input." + log "Press Ctrl+C now if you want to cancel and switch to an automated provider." + sleep 3 + fi + fi + # Ensure DNS API script is present and executable + if [ ! -x "$LOCALDIR/dnsapi/dns_api.sh" ]; then + log "Error: DNS API script not found or not executable: $LOCALDIR/dnsapi/dns_api.sh" + exit 1 + fi + # Only outbound firewall access is needed for DNS-01 + log "Configuring firewall for DNS-01 challenge..." + enable_httpclient_firewall + log "Using DNS provider: $DNS_PROVIDER" +fi -kill -9 "$HTTP_SERVER_PID" +# Cert Request (consolidated) +acme_args="--account-key \"$ACCOUNTKEY\" --csr \"$CSR\" --directory-url \"$DIRECTORY_URL\" --challenge-type \"$CHALLENGE_TYPE\"" +if [ "$CHALLENGE_TYPE" = "http-01" ]; then + acme_args="$acme_args --acme-dir \"$ACMEDIR\"" +fi +CERT=$(eval python ./acme_tiny.py $acme_args 2>acme_error.log) +[ "$CHALLENGE_TYPE" = "http-01" ] && [ -n "$HTTP_SERVER_PID" ] && kill -9 "$HTTP_SERVER_PID" # If an error occurred during certificate issuance, $CERT will be empty if [ -n "$CERT" ] ; then