Use aix extended authentication APIs.

[sswamy48]

In AIX KRB5files or LDAP user authentication may fail, and they see a message as.

[files]: Your encrypted password is invalid.
Only the system administrator can change this password.
sudo: a password is required

In case of KRB5files or LDAP authentication, the presence of the user name in /etc/passwd (which is a necessity for a KRB5files user) causes /etc/security/passwd to be checked.
If there is no password entry for that username, then login will fail.
This shouldn't be something that is even checked since KRB5files authentication goes through a Kerberos server and has no dependence on /etc/security/passwd.

The failure is seen since sudo uses passwdexpired() rather than authenticatex() and passwdexpiredx().
Because of that, it can't pass the state that is determined from the authentication routine on to passwd expiration routine, and it ends up erroneously checking 'file' method rather than KRB5files or LDAP.

Here is link to the APIs document: https://www.ibm.com/docs/en/aix/7.2.0?topic=authenticatex-subroutine

We need to use

loginrestrictionsx
authenticatex
passwdexpiredx

in that order.

For example, with a krb5 user.

lsuser -R KRB5files sudoaixkrb5

sudoaixkrb5 id=209 pgrp=staff groups=staff home=/home/sudoaixkrb5 shell=/usr/bin/ksh gecos=yes login=true su=true rlogin=true daemon=true admin=false sugroups=ALL admgroups= tpath=nosak ttys=ALL expires=0 auth1=SYSTEM auth2=NONE umask=22 registry=KRB5files SYSTEM=KRB5files logintimes= loginretries=0 pwdwarntime=5 account_locked=false minage=4 maxage=0 maxexpired=4 minalpha=2 minloweralpha=1 minupperalpha=1 minother=0 mindigit=1 minspecialchar=1 mindiff=0 maxrepeats=8 minlen=15 histexpire=52 histsize=4 pwdchecks= dictionlist= default_roles= datecreated=0 fsize=2097151 cpu=-1 data=262144 stack=65536 core=2097151 rss=65536 nofiles=2000 time_last_login=1763116089 time_last_unsuccessful_login=1763117957 tty_last_login=ssh tty_last_unsuccessful_login=ssh host_last_login=******* host_last_unsuccessful_login=****** unsuccessful_login_count=7 roles= krb5_principal=sudoaixkrb5@******* krb5_principal_name=sudoaixkrb5@****** krb5_realm=****** krb5_last_pwd_change=1763115490 flags= krb5_attributes=requires_preauth krb5_mod_name=admin/admin@******* krb5_mod_date=1763118356 krb5_kvno=6 krb5_mkvno=1 krb5_max_renewable_life=604800 krb5_names=sudoaixkrb5:*********

We can see a error as below.

su - sudoaixkrb5

$ sudo -k
$ sudo ksh
Password:
[files]: 3004-330 Your encrypted password is invalid.
3004-320 Only the system administrator can change this password.
sudo: a password is required
$

With the proposed changes we can see sudo works as expected.

su - sudoaixkrb5

$ sudo -k
$ sudo ksh
Password:

All the default testcases are passing with the new changes.

fuzz_sudo_conf: verifying corpus
base64_test: 12 tests run, 0 errors, 100% success rate
closefrom_test: 5 tests run, 0 errors, 100% success rate
digest_test: 259 tests run, 0 errors, 100% success rate
fnmatch: 6 tests run, 0 errors, 100% success rate
strsig_test: 90 tests run, 0 errors, 100% success rate
getgrouplist_test: 7 tests run, 0 errors, 100% success rate
hexchar_test: 515 tests run, 0 errors, 100% success rate
hltq_test: 19 tests run, 0 errors, 100% success rate
json_test: 15 tests run, 0 errors, 100% success rate
open_parent_dir_test: 9 tests run, 0 errors, 100% success rate
parse_gids_test: 6 tests run, 0 errors, 100% success rate
regex_test: 38 tests run, 0 errors, 100% success rate
strsplit_test: 29 tests run, 0 errors, 100% success rate
strtobool_test: 14 tests run, 0 errors, 100% success rate
strtoid_test: 9 tests run, 0 errors, 100% success rate
strtomode_test: 4 tests run, 0 errors, 100% success rate
strtonum_test: 25 tests run, 0 errors, 100% success rate
uuid_test: 16 tests run, 0 errors, 100% success rate
sudo_conf: 14 tests run, 0 errors, 100% success rate
sudo_parseln: 12 tests run, 0 errors, 100% success rate

check_parse_json: 3 tests run, 0 errors, 100% success rate
store_json_test: 4 tests run, 0 errors, 100% success rate
store_sudo_test: 4 tests run, 0 errors, 100% success rate

fuzz_iolog_legacy: verifying corpus
fuzz_iolog_json: verifying corpus
fuzz_iolog_timing: verifying corpus
iolog_filter: 3 tests run, 0 errors, 100% success rate
iolog_path: 8 tests run, 0 errors, 100% success rate
iolog_mkpath: 3 tests run, 0 errors, 100% success rate
iolog_timing: 14 tests run, 0 errors, 100% success rate
host_port_test: 16 tests run, 0 errors, 100% success rate

fuzz_logsrvd_conf: verifying corpus
dotdot_test: 15 tests run, 0 errors, 100% success rate
logsrvd_conf_test: 2 tests run, 0 errors, 100% success rate

fuzz_sudoers: verifying corpus
fuzz_sudoers-ldif: verifying corpus
fuzz_policy: verifying corpus
check_addr: 9 tests run, 0 errors, 100% success rate
check_editor: 16 tests run, 0 errors, 100% success rate
check_env_pattern: 22 tests run, 0 errors, 100% success rate
check_exptilde: 6 tests run, 0 errors, 100% success rate
check_fill: 18 tests run, 0 errors, 100% success rate
check_gentime: 17 tests run, 0 errors, 100% success rate
check_iolog_plugin: 8 tests run, 0 errors, 100% success rate
check_serialize_list: 2 tests run, 0 errors, 100% success rate
check_starttime: 3 tests run, 0 errors, 100% success rate
check_unesc: 19 tests run, 0 errors, 100% success rate
check_symbols: 8 tests run, 0 errors, 100% success rate
sudoers: 174 tests run, 0 errors, 100% success rate
testsudoers: 64 tests run, 0 errors, 100% success rate
visudo: 20 tests run, 0 errors, 100% success rate
cvtsudoers: 82 tests run, 0 errors, 100% success rate

check_net_ifs: 1 tests run, 0 errors, 100% success rate
check_noexec: 3 tests run, 0 errors, 100% success rate
check_ttyname: 2 tests run, 0 errors, 100% success rate

Please review and let me know your comments.

[millert]

Thanks for your contribution, I just had a few minor comments.

[millert]

I'm not sure this is needed. The message should be displayed immediately after the loop exits when result is 0.

[sswamy48]

I didn't find condition where all three will be satisfied.
Removed this part of the code.