Diagnosing SSH Authentication Failures in Ansible

The Problem: 'Permission Denied (publickey)'

When Ansible fails to connect to a remote host, it often returns a generic Authentication failed or Permission denied (publickey) error. This prevents any tasks from executing because the control node cannot establish the initial secure shell (SSH) tunnel required to push modules to the target.

The takeaway: SSH failures are rarely Ansible bugs; they are almost always mismatches between the control node's identity (private key) and the remote host's expectations (authorized keys and sshd configuration).

Quick Diagnostic Matrix

Symptom	Likely Cause	Primary Check
Immediate 'Permission denied'	Missing/Wrong Private Key	ansible.cfg or --private-key flag
Prompt for password despite keys	Public key not in authorized_keys	Remote ~/.ssh/authorized_keys file
Key exists but is ignored	SSH Agent not loaded	ssh-add -l output
Connection refused for root	SSHD config restriction	PermitRootLogin in sshd_config

Step-by-Step Troubleshooting Sequence

1. Isolate Ansible from the SSH Handshake

Before adjusting Ansible configurations, determine if the underlying SSH connection works. Run this command from the control node terminal:

ssh -v <remote_user>@<remote_host>

What to look for: Search the verbose output for Next authentication method: publickey. If the server rejects every key offered by the client, the issue is on the remote host. If the client doesn't offer a key, the issue is on the control node.

2. Verify Identity Availability

Ansible defaults to the user's default SSH key (usually ~/.ssh/id_rsa). If you use a custom key path, Ansible may not find it.

• Check the SSH Agent: Run ssh-add -l. If it returns The agent has no identities, your key isn't loaded into memory.
• Fix: Load the key manually:

  ssh-add ~/.ssh/my_custom_key

3. Explicitly Define the Key in Ansible

If you cannot use an SSH agent, tell Ansible exactly which file to use. You can do this via the command line for a single run:

ansible-playbook site.yml --private-key=~/.ssh/my_custom_key

Or, for a permanent per-host configuration, add the variable to your inventory file (INI format):

[webservers]
web01 ansible_host=10.0.0.1 ansible_ssh_private_key_file=~/.ssh/my_custom_key

4. Audit the Remote Target's Permissions

If the key is being sent but rejected, the remote host likely has a permission or configuration mismatch. Log in via a console or alternative method and check the following:

• Authorized Keys: Ensure the public key from the control node is present in /home/<user>/.ssh/authorized_keys.
• File Permissions: SSH will ignore keys if permissions are too open. Ensure these settings:
  • chmod 700 ~/.ssh
  • chmod 600 ~/.ssh/authorized_keys
• SSHD Config: Check /etc/ssh/sshd_config. If you are connecting as root, ensure PermitRootLogin yes or PermitRootLogin prohibit-password is set.

Verification and Testing

Once the fix is applied, use the ping module. This is not an ICMP ping, but a test of Ansible's ability to authenticate and execute a Python snippet on the target.

ansible all -m ping

Expected Result: "ping": "pong". If this succeeds, your authentication layer is functional.

Limitations and Safety Warnings

• StrictHostKeyChecking: You may see suggestions to set host_key_checking = False in ansible.cfg. Avoid this in production; it disables the verification of the remote host's identity, making you vulnerable to Man-in-the-Middle (MITM) attacks.
• Privilege Escalation: Do not attempt to fix authentication by using --become (sudo) until the basic SSH connection is established. become happens after the SSH tunnel is open.
• Credential Storage: Never hardcode passwords in playbooks. Use ansible-vault encrypt vars.yml to secure sensitive credentials.

Rollback Procedure

If you modified the remote sshd_config and lost access, you must use a cloud console or physical terminal to revert the changes. To undo a key addition, simply remove the specific line from the remote ~/.ssh/authorized_keys file.