Ansible Remote Execution with sudo password broken after 2.1/3.16 to 2.2/3.17 upgrade

Matthew_Hyclak · December 9, 2020, 6:50pm

Problem:
Remote Execution job to run Ansible Roles failing after upgrading from Foreman/Katello 2.1/3.16 to 2.2/3.17 when using sudo escalation WITH password. Job ends with:

fatal: [target.host]: FAILED! => { “msg”: “Missing sudo password” }

This worked prior to the upgrade by setting remote_execution_sudo_password. It appears that the db:migrate for remote_execution_sudo_password -> remote_execution_effective_user_password ran successfully, but I went ahead and re-set it anyway to no avail. It just appears that the runner isn’t supplying the password for sudo at all.

Expected outcome:

Job executes successfully by providing SSH and sudo password when running

Foreman and Proxy versions:

foreman and foreman-proxy 2.2.1 running on CentOS 7

Foreman and Proxy plugin versions:

ansible-runner-1.4.6-1.el7.noarch
tfm-rubygem-foreman_ansible-6.0.0-1.fm2_2.el7.noarch
tfm-rubygem-foreman_ansible_core-3.0.4-1.fm2_2.el7.noarch
tfm-rubygem-foreman_remote_execution-4.1.0-1.fm2_2.el7.noarch
tfm-rubygem-foreman_remote_execution_core-1.3.1-1.el7.noarch

Distribution and version:

CentOS Linux release 7.9.2009 (Core)

Other relevant data:

ansible-playbook 2.9.15
config file = /usr/share/foreman-proxy/.ansible.cfg
configured module search path = [u’/usr/share/foreman-proxy/.ansible/plugins/modules’, u’/usr/share/ansible/plugins/modules’]
ansible python module location = /usr/lib/python2.7/site-packages/ansible
executable location = /usr/bin/ansible-playbook
python version = 2.7.5 (default, Nov 16 2020, 22:23:17) [GCC 4.8.5 20150623 (Red Hat 4.8.5-44)]
Using /usr/share/foreman-proxy/.ansible.cfg as config file
host_list declined parsing /tmp/d20201209-12177-1waaldu/inventory/hosts as it did not pass its verify_file() method
Parsed /tmp/d20201209-12177-1waaldu/inventory/hosts inventory source with script plugin

PLAYBOOK: playbook.yml *********************************************************
1 plays in playbook.yml

PLAY [all] *********************************************************************

TASK [Gathering Facts] *********************************************************
task path: /tmp/d20201209-12177-1waaldu/project/playbook.yml:2
<remote.server> ESTABLISH SSH CONNECTION FOR USER: svc-remoteexec
<remote.server> SSH: EXEC sshpass -d8 ssh -o ProxyCommand=none -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o Port=22 -o 'IdentityFile="/usr/share/foreman-proxy/.ssh/id_rsa_foreman_proxy"' -o 'User="svc-remoteexec"' -o ConnectTimeout=10 -o ControlPath=/var/lib/foreman-proxy/ansible/cp/35fac8a072 remote.server '/bin/sh -c '"'"'echo ~svc-remoteexec && sleep 0'"'"''
<remote.server> (0, '/home/svc-remoteexec\n', '')
<remote.server> ESTABLISH SSH CONNECTION FOR USER: svc-remoteexec
<remote.server> SSH: EXEC sshpass -d8 ssh -o ProxyCommand=none -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o Port=22 -o 'IdentityFile="/usr/share/foreman-proxy/.ssh/id_rsa_foreman_proxy"' -o 'User="svc-remoteexec"' -o ConnectTimeout=10 -o ControlPath=/var/lib/foreman-proxy/ansible/cp/35fac8a072 remote.server '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo /home/svc-remoteexec/.ansible/tmp `"&& mkdir "` echo /home/svc-remoteexec/.ansible/tmp/ansible-tmp-1607538373.43-7696-22467728853433 `" && echo ansible-tmp-1607538373.43-7696-22467728853433="` echo /home/svc-remoteexec/.ansible/tmp/ansible-tmp-1607538373.43-7696-22467728853433 `" ) && sleep 0'"'"''
<remote.server> (0, 'ansible-tmp-1607538373.43-7696-22467728853433=/home/svc-remoteexec/.ansible/tmp/ansible-tmp-1607538373.43-7696-22467728853433\n', '')
<remote.server> Attempting python interpreter discovery
<remote.server> ESTABLISH SSH CONNECTION FOR USER: svc-remoteexec
<remote.server> SSH: EXEC sshpass -d8 ssh -o ProxyCommand=none -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o Port=22 -o 'IdentityFile="/usr/share/foreman-proxy/.ssh/id_rsa_foreman_proxy"' -o 'User="svc-remoteexec"' -o ConnectTimeout=10 -o ControlPath=/var/lib/foreman-proxy/ansible/cp/35fac8a072 remote.server '/bin/sh -c '"'"'echo PLATFORM; uname; echo FOUND; command -v '"'"'"'"'"'"'"'"'/usr/bin/python'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'python3.7'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'python3.6'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'python3.5'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'python2.7'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'python2.6'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'/usr/libexec/platform-python'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'/usr/bin/python3'"'"'"'"'"'"'"'"'; command -v '"'"'"'"'"'"'"'"'python'"'"'"'"'"'"'"'"'; echo ENDFOUND && sleep 0'"'"''
<remote.server> (0, 'PLATFORM\nLinux\nFOUND\n/usr/bin/python\n/usr/bin/python2.7\n/usr/libexec/platform-python\n/usr/bin/python\nENDFOUND\n', '')
<remote.server> ESTABLISH SSH CONNECTION FOR USER: svc-remoteexec
<remote.server> SSH: EXEC sshpass -d8 ssh -o ProxyCommand=none -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o Port=22 -o 'IdentityFile="/usr/share/foreman-proxy/.ssh/id_rsa_foreman_proxy"' -o 'User="svc-remoteexec"' -o ConnectTimeout=10 -o ControlPath=/var/lib/foreman-proxy/ansible/cp/35fac8a072 remote.server '/bin/sh -c '"'"'/usr/bin/python && sleep 0'"'"''
<remote.server> (0, '{"osrelease_content": "NAME=\\"CentOS Linux\\"\\nVERSION=\\"7 (Core)\\"\\nID=\\"centos\\"\\nID_LIKE=\\"rhel fedora\\"\\nVERSION_ID=\\"7\\"\\nPRETTY_NAME=\\"CentOS Linux 7 (Core)\\"\\nANSI_COLOR=\\"0;31\\"\\nCPE_NAME=\\"cpe:/o:centos:centos:7\\"\\nHOME_URL=\\"https://www.centos.org/\\"\\nBUG_REPORT_URL=\\"https://bugs.centos.org/\\"\\n\\nCENTOS_MANTISBT_PROJECT=\\"CentOS-7\\"\\nCENTOS_MANTISBT_PROJECT_VERSION=\\"7\\"\\nREDHAT_SUPPORT_PRODUCT=\\"centos\\"\\nREDHAT_SUPPORT_PRODUCT_VERSION=\\"7\\"\\n\\n", "platform_dist_result": ["centos", "7.9.2009", "Core"]}\n', '')
Using module file /usr/lib/python2.7/site-packages/ansible/modules/system/setup.py
<remote.server> PUT /tmp/ansible-local-7685B2SZDN/tmpdlWegS TO /home/svc-remoteexec/.ansible/tmp/ansible-tmp-1607538373.43-7696-22467728853433/AnsiballZ_setup.py
<remote.server> SSH: EXEC sshpass -d8 sftp -o BatchMode=no -b - -o ProxyCommand=none -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o Port=22 -o 'IdentityFile="/usr/share/foreman-proxy/.ssh/id_rsa_foreman_proxy"' -o 'User="svc-remoteexec"' -o ConnectTimeout=10 -o ControlPath=/var/lib/foreman-proxy/ansible/cp/35fac8a072 '[remote.server]'
<remote.server> (0, 'sftp> put /tmp/ansible-local-7685B2SZDN/tmpdlWegS /home/svc-remoteexec/.ansible/tmp/ansible-tmp-1607538373.43-7696-22467728853433/AnsiballZ_setup.py\n', '')
<remote.server> ESTABLISH SSH CONNECTION FOR USER: svc-remoteexec
<remote.server> SSH: EXEC sshpass -d8 ssh -o ProxyCommand=none -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o Port=22 -o 'IdentityFile="/usr/share/foreman-proxy/.ssh/id_rsa_foreman_proxy"' -o 'User="svc-remoteexec"' -o ConnectTimeout=10 -o ControlPath=/var/lib/foreman-proxy/ansible/cp/35fac8a072 remote.server '/bin/sh -c '"'"'chmod u+x /home/svc-remoteexec/.ansible/tmp/ansible-tmp-1607538373.43-7696-22467728853433/ /home/svc-remoteexec/.ansible/tmp/ansible-tmp-1607538373.43-7696-22467728853433/AnsiballZ_setup.py && sleep 0'"'"''
<remote.server> (0, '', '')
<remote.server> ESTABLISH SSH CONNECTION FOR USER: svc-remoteexec
<remote.server> SSH: EXEC sshpass -d8 ssh -o ProxyCommand=none -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o Port=22 -o 'IdentityFile="/usr/share/foreman-proxy/.ssh/id_rsa_foreman_proxy"' -o 'User="svc-remoteexec"' -o ConnectTimeout=10 -o ControlPath=/var/lib/foreman-proxy/ansible/cp/35fac8a072 -tt remote.server '/bin/sh -c '"'"'sudo -H -S -n  -u root /bin/sh -c '"'"'"'"'"'"'"'"'echo BECOME-SUCCESS-jigrjdoviuzdpisumtyuidilgfzyuyiw ; /usr/bin/python /home/svc-remoteexec/.ansible/tmp/ansible-tmp-1607538373.43-7696-22467728853433/AnsiballZ_setup.py'"'"'"'"'"'"'"'"' && sleep 0'"'"''
Escalation requires password
<remote.server> ESTABLISH SSH CONNECTION FOR USER: svc-remoteexec
<remote.server> SSH: EXEC sshpass -d8 ssh -o ProxyCommand=none -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o Port=22 -o 'IdentityFile="/usr/share/foreman-proxy/.ssh/id_rsa_foreman_proxy"' -o 'User="svc-remoteexec"' -o ConnectTimeout=10 -o ControlPath=/var/lib/foreman-proxy/ansible/cp/35fac8a072 remote.server '/bin/sh -c '"'"'rm -f -r /home/svc-remoteexec/.ansible/tmp/ansible-tmp-1607538373.43-7696-22467728853433/ > /dev/null 2>&1 && sleep 0'"'"''
<remote.server> (0, '', '')
fatal: [remote.server]: FAILED! => {
    "msg": "Missing sudo password"
}
PLAY RECAP *********************************************************************
remote.server : ok=0    changed=0    unreachable=0    failed=1    skipped=0    rescued=0    ignored=0

Marek_Hulan · December 14, 2020, 9:28am

This could have been caused by https://github.com/theforeman/foreman_ansible/commit/8570ca3a543826ed29541d2508a974cfffa2ca63

aruzicka · December 14, 2020, 9:48am

We missed the core part when doing a release, it should be fixed in foreman_ansible_core-4.0.0

Matthew_Hyclak · December 14, 2020, 2:55pm

Glad it’s something simple. Is there a plan for releasing foreman_ansible_core-4.0.0 for Foreman 2.2? I tried grabbing the plugin rpms for ansible, ansible_core, remote_execution and remote_execution_core, but they didn’t play nice with the rest of the system - so I presume they’re built against 2.3.

Thanks!

Matthew_Hyclak · January 13, 2021, 8:03pm

@aruzicka Will the updated foreman_ansible_core be released with 2.2.3? Or is this going to remain broken for all of 2.2.x? I didn’t see anything mentioned in the meeting notes for the 2.2.3 release.

Thanks,
Matt

aruzicka · January 15, 2021, 10:46am

Updated foreman_ansible_core will be eventually released for 2.2.z, but I first have to check if we can just take the entire release or if I’ll need to cherry-pick only selected bits.

Matthew_Hyclak · January 20, 2021, 4:28pm

Thanks for the update. I’m sure you’re aware, but it looks like this is also still an issue in the 2.3 branch. I just updated to 2.3.2/3.18 this morning. Looks like we have foreman_ansible_core 3.0.4-1 so still not 4.0. I tried rebuilding the newer foreman_ansible_core package on 2.2 and it started down a path of dependencies that didn’t look like it would end well for me, so I abandoned that (and why I hoped it would be fixed in 2.3). I have a workaround that I’m comfortable with, but the rest of my team wouldn’t be, so I’m hoping to get this resolved sooner than later.

Thanks,
Matt

aruzicka · January 21, 2021, 8:22am

Packaging PRs were merged ~12 hours ago and the packages should now be available in the repos.

Matthew_Hyclak · January 21, 2021, 2:25pm

Yep, see it available and installed. Ran a foreman-maintain service restart but I’m still running into issues.

It looks like it’s connecting and logging in ok, but not passing the sudo password. I’m setting remote_execution_ssh_password and remote_execution_effective_user_password (tried both in the default system settings as well as at the Host Group level). Here’s the foreman output:

1: ansible-playbook 2.9.15
2: config file = /usr/share/foreman-proxy/.ansible.cfg
3: configured module search path = [u'/usr/share/foreman-proxy/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
4: ansible python module location = /usr/lib/python2.7/site-packages/ansible
5: executable location = /usr/bin/ansible-playbook
6: python version = 2.7.5 (default, Nov 16 2020, 22:23:17) [GCC 4.8.5 20150623 (Red Hat 4.8.5-44)]
7: Using /usr/share/foreman-proxy/.ansible.cfg as config file
8: host_list declined parsing /tmp/d20210121-11569-wqy5z5/inventory/hosts as it did not pass its verify_file() method
9: Parsed /tmp/d20210121-11569-wqy5z5/inventory/hosts inventory source with script plugin
10:
11: PLAYBOOK: playbook.yml *********************************************************
12: 1 plays in playbook.yml
13:
14: PLAY [all] *********************************************************************
15:
16: TASK [Gathering Facts] *********************************************************
17: task path: /tmp/d20210121-11569-wqy5z5/project/playbook.yml:2
18: <mrhdevlnx01.domain.com> ESTABLISH SSH CONNECTION FOR USER: svc-remoteexec
19: <mrhdevlnx01.domain.com> SSH: EXEC ssh -o ProxyCommand=none -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o Port=22 -o 'IdentityFile="/usr/share/foreman-proxy/.ssh/id_rsa_foreman_proxy"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="svc-remoteexec"' -o ConnectTimeout=10 -o ControlPath=/var/lib/foreman-proxy/ansible/cp/35fac8a072 mrhdevlnx01.domain.com '/bin/sh -c '"'"'echo ~svc-remoteexec && sleep 0'"'"''
20: <mrhdevlnx01.domain.com> (255, '', 'Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).\r\n')
21: fatal: [mrhdevlnx01.domain.com]: UNREACHABLE! => {
22:  "changed": false, 
23: "msg": "Failed to connect to the host via ssh: Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).", 
24:  "unreachable": true
25: }
26: PLAY RECAP *********************************************************************
27: mrhdevlnx01.domain.com : ok=0    changed=0    unreachable=1    failed=0    skipped=0    rescued=0    ignored=0
28: Exit status: 1

And here’s the target system output (/var/log/secure):

Jan 21 08:46:12 mrhdevlnx01 sshd[24930]: pam_sss(sshd:auth): authentication success; logname= uid=0 euid=0 tty=ssh ruser= rhost=sat01.domain.com user=svc-remoteexec
Jan 21 08:46:12 mrhdevlnx01 sshd[24930]: Accepted password for svc-remoteexec from xxx.yyy.12.181 port 59044 ssh2
Jan 21 08:46:12 mrhdevlnx01 sshd[24930]: pam_unix(sshd:session): session opened for user svc-remoteexec by (uid=0)
Jan 21 08:46:13 mrhdevlnx01 sudo: svc-remoteexec : a password is required ; TTY=pts/0 ; PWD=/home/svc-remoteexec ; USER=root ; COMMAND=/bin/sh -c echo BECOME-SUCCESS-saxptemlbdxwtpaetverfrauksrwqddk ; /usr/bin/python /home/svc-remoteexec/.ansible/tmp/ansible-tmp-1611236772.95-10006-35933303970005/AnsiballZ_setup.py
Jan 21 08:47:12 mrhdevlnx01 sshd[24933]: Received disconnect from xxx.yyy.12.181 port 59044:11: disconnected by user
Jan 21 08:47:12 mrhdevlnx01 sshd[24933]: Disconnected from xxx.yyy.12.181 port 59044

Am I just missing something? Anything else I can do to troubleshoot this?

Thanks!
Matt

aruzicka · January 22, 2021, 9:12am

Could you bump ansible verbosity to maximum and try running it again? I just tried on a machine I have and it seemed to work just fine with similar settings to what you described. Also to narrow the possibilities down a bit, could you try running something using regular ssh-based rex?

Matthew_Hyclak · January 22, 2021, 5:48pm

Regular SSH appears to work. I have a daily check for yum updates that is successful:
Evaluated at: 2021-01-22 12:06:43 -0500

1: rex login:
2: Loaded plugins: enabled_repos_upload, fastestmirror, package_upload, product-id,
3:           : search-disabled-repos, subscription-manager
4: Loading mirror speeds from cached hostfile
5: Uploading Enabled Repositories Report
6: Loaded plugins: fastestmirror, product-id, subscription-manager
7: Exit status: 0

And /var/log/secure on the target system shows:

Jan 22 12:06:44 mrhdevlnx01 sshd[13392]: pam_sss(sshd:auth): authentication success; logname= uid=0 euid=0 tty=ssh ruser= rhost=odeesat01.it.ohio-state.edu user=svc-remoteexec
Jan 22 12:06:44 mrhdevlnx01 sshd[13392]: Accepted password for svc-remoteexec from xxx.yyy.12.181 port 38188 ssh2
Jan 22 12:06:44 mrhdevlnx01 sshd[13392]: pam_unix(sshd:session): session opened for user svc-remoteexec by (uid=0)
Jan 22 12:06:46 mrhdevlnx01 sudo: pam_sss(sudo:auth): authentication success; logname=svc-remoteexec uid=1049208040 euid=0 tty=/dev/pts/1 ruser=svc-remoteexec rhost= user=svc-remoteexec
Jan 22 12:06:46 mrhdevlnx01 sudo: svc-remoteexec : TTY=pts/1 ; PWD=/home/svc-remoteexec ; USER=root ; COMMAND=/var/tmp/foreman-ssh-cmd-e55b7a34-33fd-423a-9aa8-976c7de29075/script
Jan 22 12:06:46 mrhdevlnx01 sudo: pam_unix(sudo:session): session opened for user root by svc-remoteexec(uid=0)
Jan 22 12:06:49 mrhdevlnx01 sudo: pam_unix(sudo:session): session closed for user root
Jan 22 12:06:49 mrhdevlnx01 sshd[13392]: pam_unix(sshd:session): session closed for user svc-remoteexec

I bumped up the ansible logging from -vvv to -vvvv on the failing job, and reran:
Ansible Roles - Ansible Default
Evaluated at: 2021-01-22 11:56:02 -0500

1: ansible-playbook 2.9.15
2:   config file = /usr/share/foreman-proxy/.ansible.cfg
3:   configured module search path = [u'/usr/share/foreman-proxy/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
4:   ansible python module location = /usr/lib/python2.7/site-packages/ansible
5:   executable location = /usr/bin/ansible-playbook
6:   python version = 2.7.5 (default, Nov 16 2020, 22:23:17) [GCC 4.8.5 20150623 (Red Hat 4.8.5-44)]
7: Using /usr/share/foreman-proxy/.ansible.cfg as config file
8: setting up inventory plugins
9: host_list declined parsing /tmp/d20210122-11569-rbma1h/inventory/hosts as it did not pass its verify_file() method
10: Parsed /tmp/d20210122-11569-rbma1h/inventory/hosts inventory source with script plugin
11: Loading callback plugin awx_display of type stdout, v2.0 from /usr/lib/python2.7/site-packages/ansible_runner/callbacks/awx_display.pyc
12: Loading callback plugin foreman of type notification, v2.0 from /usr/lib/python2.7/site-packages/ansible/plugins/callback/foreman.pyc
13:
14: PLAYBOOK: playbook.yml *********************************************************
15: Positional arguments: playbook.yml
16: become_method: sudo
17: inventory: (u'/tmp/d20210122-11569-rbma1h/inventory',)
18: forks: 5
19: tags: (u'all',)
20: verbosity: 4
21: connection: smart
22: timeout: 10
23: 1 plays in playbook.yml
24:
25: PLAY [all] *********************************************************************
26:
27: TASK [Gathering Facts] *********************************************************
28: task path: /tmp/d20210122-11569-rbma1h/project/playbook.yml:2
29: <mrhdevlnx01.domain.com> ESTABLISH SSH CONNECTION FOR USER: svc-remoteexec
30: <mrhdevlnx01.domain.com> SSH: EXEC ssh -vvv -o ProxyCommand=none -C -o ControlMaster=auto -o ControlPersist=60s -o StrictHostKeyChecking=no -o Port=22 -o 'IdentityFile="/usr/share/foreman-proxy/.ssh/id_rsa_foreman_proxy"' -o KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi-keyex,hostbased,publickey -o PasswordAuthentication=no -o 'User="svc-remoteexec"' -o ConnectTimeout=10 -o ControlPath=/var/lib/foreman-proxy/ansible/cp/35fac8a072 mrhdevlnx01.domain.com '/bin/sh -c '"'"'echo ~svc-remoteexec && sleep 0'"'"''
31: <mrhdevlnx01.domain.com> (255, '', 'OpenSSH_7.4p1, OpenSSL 1.0.2k-fips  26 Jan 2017\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 58: Applying options for *\r\ndebug1: auto-mux: Trying existing master\r\ndebug1: Control socket "/var/lib/foreman-proxy/ansible/cp/35fac8a072" does not exist\r\ndebug2: resolving "mrhdevlnx01.domain.com" port 22\r\ndebug2: ssh_connect_direct: needpriv 0\r\ndebug1: Connecting to mrhdevlnx01.domain.com [xxx.yyy.12.70] port 22.\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug1: fd 3 clearing O_NONBLOCK\r\ndebug1: Connection established.\r\ndebug3: timeout: 9999 ms remain after connect\r\ndebug1: identity file /usr/share/foreman-proxy/.ssh/id_rsa_foreman_proxy type 1\r\ndebug1: key_load_public: No such file or directory\r\ndebug1: identity file /usr/share/foreman-proxy/.ssh/id_rsa_foreman_proxy-cert type -1\r\ndebug1: Enabling compatibility mode for protocol 2.0\r\ndebug1: Local version string SSH-2.0-OpenSSH_7.4\r\ndebug1: Remote protocol version 2.0, remote software version OpenSSH_7.4\r\ndebug1: match: OpenSSH_7.4 pat OpenSSH* compat 0x04000000\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug1: Authenticating to mrhdevlnx01.domain.com:22 as \'svc-remoteexec\'\r\ndebug3: hostkeys_foreach: reading file "/usr/share/foreman-proxy/.ssh/known_hosts"\r\ndebug3: record_hostkey: found key type ECDSA in file /usr/share/foreman-proxy/.ssh/known_hosts:4\r\ndebug3: load_hostkeys: loaded 1 keys from mrhdevlnx01.domain.com\r\ndebug3: order_hostkeyalgs: prefer hostkeyalgs: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521\r\ndebug3: send packet: type 20\r\ndebug1: SSH2_MSG_KEXINIT sent\r\ndebug3: receive packet: type 20\r\ndebug1: SSH2_MSG_KEXINIT received\r\ndebug2: local client KEXINIT proposal\r\ndebug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1,ext-info-c\r\ndebug2: host key algorithms: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,ssh-dss-cert-v01@openssh.com,ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa,ssh-dss\r\ndebug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-cbc,aes192-cbc,aes256-cbc\r\ndebug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-cbc,aes192-cbc,aes256-cbc\r\ndebug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: compression ctos: zlib@openssh.com,zlib,none\r\ndebug2: compression stoc: zlib@openssh.com,zlib,none\r\ndebug2: languages ctos: \r\ndebug2: languages stoc: \r\ndebug2: first_kex_follows 0 \r\ndebug2: reserved 0 \r\ndebug2: peer server KEXINIT proposal\r\ndebug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1\r\ndebug2: host key algorithms: ssh-rsa,rsa-sha2-512,rsa-sha2-256,ecdsa-sha2-nistp256,ssh-ed25519\r\ndebug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-cbc,aes192-cbc,aes256-cbc,blowfish-cbc,cast128-cbc,3des-cbc\r\ndebug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-cbc,aes192-cbc,aes256-cbc,blowfish-cbc,cast128-cbc,3des-cbc\r\ndebug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: compression ctos: none,zlib@openssh.com\r\ndebug2: compression stoc: none,zlib@openssh.com\r\ndebug2: languages ctos: \r\ndebug2: languages stoc: \r\ndebug2: first_kex_follows 0 \r\ndebug2: reserved 0 \r\ndebug1: kex: algorithm: curve25519-sha256\r\ndebug1: kex: host key algorithm: ecdsa-sha2-nistp256\r\ndebug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: zlib@openssh.com\r\ndebug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: zlib@openssh.com\r\ndebug1: kex: curve25519-sha256 need=64 dh_need=64\r\ndebug1: kex: curve25519-sha256 need=64 dh_need=64\r\ndebug3: send packet: type 30\r\ndebug1: expecting SSH2_MSG_KEX_ECDH_REPLY\r\ndebug3: receive packet: type 31\r\ndebug1: Server host key: ecdsa-sha2-nistp256 SHA256:G5IPeVMu6ejtwofct3yE2tTqvDJ68A21MtsjQ4IgdOA\r\ndebug3: hostkeys_foreach: reading file "/usr/share/foreman-proxy/.ssh/known_hosts"\r\ndebug3: record_hostkey: found key type ECDSA in file /usr/share/foreman-proxy/.ssh/known_hosts:4\r\ndebug3: load_hostkeys: loaded 1 keys from mrhdevlnx01.domain.com\r\ndebug3: hostkeys_foreach: reading file "/usr/share/foreman-proxy/.ssh/known_hosts"\r\ndebug3: record_hostkey: found key type ECDSA in file /usr/share/foreman-proxy/.ssh/known_hosts:4\r\ndebug3: load_hostkeys: loaded 1 keys from xxx.yyy.12.70\r\ndebug1: Host \'mrhdevlnx01.domain.com\' is known and matches the ECDSA host key.\r\ndebug1: Found key in /usr/share/foreman-proxy/.ssh/known_hosts:4\r\ndebug3: send packet: type 21\r\ndebug2: set_newkeys: mode 1\r\ndebug1: rekey after 134217728 blocks\r\ndebug1: SSH2_MSG_NEWKEYS sent\r\ndebug1: expecting SSH2_MSG_NEWKEYS\r\ndebug3: receive packet: type 21\r\ndebug1: SSH2_MSG_NEWKEYS received\r\ndebug2: set_newkeys: mode 0\r\ndebug1: rekey after 134217728 blocks\r\ndebug2: key: /usr/share/foreman-proxy/.ssh/id_rsa_foreman_proxy (0x55dc975e9a80), explicit\r\ndebug3: send packet: type 5\r\ndebug3: receive packet: type 7\r\ndebug1: SSH2_MSG_EXT_INFO received\r\ndebug1: kex_input_ext_info: server-sig-algs=<rsa-sha2-256,rsa-sha2-512>\r\ndebug3: receive packet: type 6\r\ndebug2: service_accept: ssh-userauth\r\ndebug1: SSH2_MSG_SERVICE_ACCEPT received\r\ndebug3: send packet: type 50\r\ndebug3: receive packet: type 51\r\ndebug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password\r\ndebug3: start over, passed a different list publickey,gssapi-keyex,gssapi-with-mic,password\r\ndebug3: preferred gssapi-with-mic,gssapi-keyex,hostbased,publickey\r\ndebug3: authmethod_lookup gssapi-with-mic\r\ndebug3: remaining preferred: gssapi-keyex,hostbased,publickey\r\ndebug3: authmethod_is_enabled gssapi-with-mic\r\ndebug1: Next authentication method: gssapi-with-mic\r\ndebug1: Unspecified GSS failure.  Minor code may provide more information\nNo Kerberos credentials available (default cache: KEYRING:persistent:990)\n\r\ndebug1: Unspecified GSS failure.  Minor code may provide more information\nNo Kerberos credentials available (default cache: KEYRING:persistent:990)\n\r\ndebug2: we did not send a packet, disable method\r\ndebug3: authmethod_lookup gssapi-keyex\r\ndebug3: remaining preferred: hostbased,publickey\r\ndebug3: authmethod_is_enabled gssapi-keyex\r\ndebug1: Next authentication method: gssapi-keyex\r\ndebug1: No valid Key exchange context\r\ndebug2: we did not send a packet, disable method\r\ndebug3: authmethod_lookup publickey\r\ndebug3: remaining preferred: ,publickey\r\ndebug3: authmethod_is_enabled publickey\r\ndebug1: Next authentication method: publickey\r\ndebug1: Offering RSA public key: /usr/share/foreman-proxy/.ssh/id_rsa_foreman_proxy\r\ndebug3: send_pubkey_test\r\ndebug3: send packet: type 50\r\ndebug2: we sent a publickey packet, wait for reply\r\ndebug3: receive packet: type 51\r\ndebug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password\r\ndebug2: we did not send a packet, disable method\r\ndebug1: No more authentication methods to try.\r\nPermission denied (publickey,gssapi-keyex,gssapi-with-mic,password).\r\n')
32: fatal: [mrhdevlnx01.domain.com]: UNREACHABLE! => {
33:     "changed": false, 
34:     "msg": "Failed to connect to the host via ssh: OpenSSH_7.4p1, OpenSSL 1.0.2k-fips  26 Jan 2017\r\ndebug1: Reading configuration data /etc/ssh/ssh_config\r\ndebug1: /etc/ssh/ssh_config line 58: Applying options for *\r\ndebug1: auto-mux: Trying existing master\r\ndebug1: Control socket \"/var/lib/foreman-proxy/ansible/cp/35fac8a072\" does not exist\r\ndebug2: resolving \"mrhdevlnx01.domain.com\" port 22\r\ndebug2: ssh_connect_direct: needpriv 0\r\ndebug1: Connecting to mrhdevlnx01.domain.com [xxx.yyy.12.70] port 22.\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug1: fd 3 clearing O_NONBLOCK\r\ndebug1: Connection established.\r\ndebug3: timeout: 9999 ms remain after connect\r\ndebug1: identity file /usr/share/foreman-proxy/.ssh/id_rsa_foreman_proxy type 1\r\ndebug1: key_load_public: No such file or directory\r\ndebug1: identity file /usr/share/foreman-proxy/.ssh/id_rsa_foreman_proxy-cert type -1\r\ndebug1: Enabling compatibility mode for protocol 2.0\r\ndebug1: Local version string SSH-2.0-OpenSSH_7.4\r\ndebug1: Remote protocol version 2.0, remote software version OpenSSH_7.4\r\ndebug1: match: OpenSSH_7.4 pat OpenSSH* compat 0x04000000\r\ndebug2: fd 3 setting O_NONBLOCK\r\ndebug1: Authenticating to mrhdevlnx01.domain.com:22 as 'svc-remoteexec'\r\ndebug3: hostkeys_foreach: reading file \"/usr/share/foreman-proxy/.ssh/known_hosts\"\r\ndebug3: record_hostkey: found key type ECDSA in file /usr/share/foreman-proxy/.ssh/known_hosts:4\r\ndebug3: load_hostkeys: loaded 1 keys from mrhdevlnx01.domain.com\r\ndebug3: order_hostkeyalgs: prefer hostkeyalgs: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521\r\ndebug3: send packet: type 20\r\ndebug1: SSH2_MSG_KEXINIT sent\r\ndebug3: receive packet: type 20\r\ndebug1: SSH2_MSG_KEXINIT received\r\ndebug2: local client KEXINIT proposal\r\ndebug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1,ext-info-c\r\ndebug2: host key algorithms: ecdsa-sha2-nistp256-cert-v01@openssh.com,ecdsa-sha2-nistp384-cert-v01@openssh.com,ecdsa-sha2-nistp521-cert-v01@openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519-cert-v01@openssh.com,ssh-rsa-cert-v01@openssh.com,ssh-dss-cert-v01@openssh.com,ssh-ed25519,rsa-sha2-512,rsa-sha2-256,ssh-rsa,ssh-dss\r\ndebug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-cbc,aes192-cbc,aes256-cbc\r\ndebug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-cbc,aes192-cbc,aes256-cbc\r\ndebug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: compression ctos: zlib@openssh.com,zlib,none\r\ndebug2: compression stoc: zlib@openssh.com,zlib,none\r\ndebug2: languages ctos: \r\ndebug2: languages stoc: \r\ndebug2: first_kex_follows 0 \r\ndebug2: reserved 0 \r\ndebug2: peer server KEXINIT proposal\r\ndebug2: KEX algorithms: curve25519-sha256,curve25519-sha256@libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1\r\ndebug2: host key algorithms: ssh-rsa,rsa-sha2-512,rsa-sha2-256,ecdsa-sha2-nistp256,ssh-ed25519\r\ndebug2: ciphers ctos: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-cbc,aes192-cbc,aes256-cbc,blowfish-cbc,cast128-cbc,3des-cbc\r\ndebug2: ciphers stoc: chacha20-poly1305@openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm@openssh.com,aes256-gcm@openssh.com,aes128-cbc,aes192-cbc,aes256-cbc,blowfish-cbc,cast128-cbc,3des-cbc\r\ndebug2: MACs ctos: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: MACs stoc: umac-64-etm@openssh.com,umac-128-etm@openssh.com,hmac-sha2-256-etm@openssh.com,hmac-sha2-512-etm@openssh.com,hmac-sha1-etm@openssh.com,umac-64@openssh.com,umac-128@openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1\r\ndebug2: compression ctos: none,zlib@openssh.com\r\ndebug2: compression stoc: none,zlib@openssh.com\r\ndebug2: languages ctos: \r\ndebug2: languages stoc: \r\ndebug2: first_kex_follows 0 \r\ndebug2: reserved 0 \r\ndebug1: kex: algorithm: curve25519-sha256\r\ndebug1: kex: host key algorithm: ecdsa-sha2-nistp256\r\ndebug1: kex: server->client cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: zlib@openssh.com\r\ndebug1: kex: client->server cipher: chacha20-poly1305@openssh.com MAC: <implicit> compression: zlib@openssh.com\r\ndebug1: kex: curve25519-sha256 need=64 dh_need=64\r\ndebug1: kex: curve25519-sha256 need=64 dh_need=64\r\ndebug3: send packet: type 30\r\ndebug1: expecting SSH2_MSG_KEX_ECDH_REPLY\r\ndebug3: receive packet: type 31\r\ndebug1: Server host key: ecdsa-sha2-nistp256 SHA256:G5IPeVMu6ejtwofcf3yE2tTqvDJ68A21MtsjQ4IgdOA\r\ndebug3: hostkeys_foreach: reading file \"/usr/share/foreman-proxy/.ssh/known_hosts\"\r\ndebug3: record_hostkey: found key type ECDSA in file /usr/share/foreman-proxy/.ssh/known_hosts:4\r\ndebug3: load_hostkeys: loaded 1 keys from mrhdevlnx01.domain.com\r\ndebug3: hostkeys_foreach: reading file \"/usr/share/foreman-proxy/.ssh/known_hosts\"\r\ndebug3: record_hostkey: found key type ECDSA in file /usr/share/foreman-proxy/.ssh/known_hosts:4\r\ndebug3: load_hostkeys: loaded 1 keys from xxx.yyy.12.70\r\ndebug1: Host 'mrhdevlnx01.domain.com' is known and matches the ECDSA host key.\r\ndebug1: Found key in /usr/share/foreman-proxy/.ssh/known_hosts:4\r\ndebug3: send packet: type 21\r\ndebug2: set_newkeys: mode 1\r\ndebug1: rekey after 134217728 blocks\r\ndebug1: SSH2_MSG_NEWKEYS sent\r\ndebug1: expecting SSH2_MSG_NEWKEYS\r\ndebug3: receive packet: type 21\r\ndebug1: SSH2_MSG_NEWKEYS received\r\ndebug2: set_newkeys: mode 0\r\ndebug1: rekey after 134217728 blocks\r\ndebug2: key: /usr/share/foreman-proxy/.ssh/id_rsa_foreman_proxy (0x55dc975e9a80), explicit\r\ndebug3: send packet: type 5\r\ndebug3: receive packet: type 7\r\ndebug1: SSH2_MSG_EXT_INFO received\r\ndebug1: kex_input_ext_info: server-sig-algs=<rsa-sha2-256,rsa-sha2-512>\r\ndebug3: receive packet: type 6\r\ndebug2: service_accept: ssh-userauth\r\ndebug1: SSH2_MSG_SERVICE_ACCEPT received\r\ndebug3: send packet: type 50\r\ndebug3: receive packet: type 51\r\ndebug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password\r\ndebug3: start over, passed a different list publickey,gssapi-keyex,gssapi-with-mic,password\r\ndebug3: preferred gssapi-with-mic,gssapi-keyex,hostbased,publickey\r\ndebug3: authmethod_lookup gssapi-with-mic\r\ndebug3: remaining preferred: gssapi-keyex,hostbased,publickey\r\ndebug3: authmethod_is_enabled gssapi-with-mic\r\ndebug1: Next authentication method: gssapi-with-mic\r\ndebug1: Unspecified GSS failure.  Minor code may provide more information\nNo Kerberos credentials available (default cache: KEYRING:persistent:990)\n\r\ndebug1: Unspecified GSS failure.  Minor code may provide more information\nNo Kerberos credentials available (default cache: KEYRING:persistent:990)\n\r\ndebug2: we did not send a packet, disable method\r\ndebug3: authmethod_lookup gssapi-keyex\r\ndebug3: remaining preferred: hostbased,publickey\r\ndebug3: authmethod_is_enabled gssapi-keyex\r\ndebug1: Next authentication method: gssapi-keyex\r\ndebug1: No valid Key exchange context\r\ndebug2: we did not send a packet, disable method\r\ndebug3: authmethod_lookup publickey\r\ndebug3: remaining preferred: ,publickey\r\ndebug3: authmethod_is_enabled publickey\r\ndebug1: Next authentication method: publickey\r\ndebug1: Offering RSA public key: /usr/share/foreman-proxy/.ssh/id_rsa_foreman_proxy\r\ndebug3: send_pubkey_test\r\ndebug3: send packet: type 50\r\ndebug2: we sent a publickey packet, wait for reply\r\ndebug3: receive packet: type 51\r\ndebug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password\r\ndebug2: we did not send a packet, disable method\r\ndebug1: No more authentication methods to try.\r\nPermission denied (publickey,gssapi-keyex,gssapi-with-mic,password).", 
35:    "unreachable": true
36: }
37: PLAY RECAP *********************************************************************
38: mrhdevlnx01.domain.com : ok=0    changed=0    unreachable=1    failed=0    skipped=0    rescued=0    ignored=0
39: Exit status: 1

/var/log/secure on the target system seems even less verbose now, only showing:

Jan 22 11:56:04 mrhdevlnx01 sshd[12869]: Connection closed by xxx.yyy.12.181 port 37474 [preauth]

Open to any/all suggestions!

Thanks,
Matt

aruzicka · January 22, 2021, 8:31pm

Ahh, I see. While foreman_ansible-6.0.1 and foreman_ansible_core-3.0.4 didn’t pass the sudo password properly, foreman_ansible-6.0.1 and foreman_ansible_core-4.0.0 pass the sudo password, but don’t pass the connection password properly. Is using a key instead of a password an option for you for the time being? I assume the fix will roll out earlier next week.

Matthew_Hyclak · January 22, 2021, 9:29pm

Ah, that makes sense. I can workaround with a key if needed (not preferred because of security), but I’ll keep an eye out for the updates either way.

Thanks for the help debugging, and if it’s helpful to confirm fixes just let me know if there’s specific versions to look out for.

Matthew_Hyclak · January 26, 2021, 3:04pm

Just wanted to confirm that things appear to be working correctly after the updates today:

tfm-rubygem-foreman_ansible-6.1.1-1.fm2_3.el7.noarch Tue 26 Jan 2021 09:33:43 AM EST
tfm-rubygem-foreman_remote_execution-4.2.2-1.fm2_3.el7.noarch Tue 26 Jan 2021 09:33:00 AM EST
tfm-rubygem-foreman_ansible_core-4.0.0-1.fm2_3.el7.noarch Thu 21 Jan 2021 08:36:32 AM EST

Thanks for all the help!