PAM – Pluggable Authentication Modules for Linux and how to edit the defaults

Most of us have been using PAM when authenticating without really thinking about it, but for the few of us that have actually tried to make sense of it, PAM is the partner that always says “no”, unless otherwise stated. It’s the bane of any sysadmin’s existence when it comes to making system x secure, and it becomes a major pain point on and off when I forget about the normal rules of engagement.

Rules of Engagement

Session windows

To engage with PAM in any combative situation, please ensure belts and braces are on, and keep your arms and legs inside the vehicle at all times. Backup the /etc/pam.d/ directory, and make sure that you have one or two non-terminating sessions open on your system – ideally a console, and an ssh session.

The unexpected overwrite

In RHEL/Fedora like systems, PAM is configured to have two main files which are included by the rest of the PAM configuration: /etc/pam.d/system-auth-ac and /etc/pam.d/password-auth-ac.  Normally, system-auth and password-auth in the same /etc/pam.d directory are links to the above files. authconfig tools will overwrite the configuration in the files with a suffix of -ac. This means that if the changes need to be persistent and not overwritten, the symlinks can be set to the new location As follows:

ls -l /etc/pam.d/*-auth

lrwxrwxrwx. 1 root root 19 Feb 19 12:57 /etc/pam.d/fingerprint-auth -> fingerprint-auth-ac
lrwxrwxrwx. 1 root root 16 Feb 19 12:57 /etc/pam.d/password-auth -> password-auth-ac
lrwxrwxrwx. 1 root root 17 Feb 19 12:57 /etc/pam.d/smartcard-auth -> smartcard-auth-ac
lrwxrwxrwx. 1 root root 14 Feb 19 12:57 /etc/pam.d/system-auth -> system-auth-ac
[root@node1 pam.d]# rm password-auth
rm: remove symbolic link ‘password-auth’? y
[root@node1 pam.d]# ln -s password-auth-local password-auth

The Language

When the usual PAM translator (authconfig) is not enough to achieve the right system authentication, one has to start thinking about communicating directly with it. PAM has four different keywords for controlling authentication with the system:

awk '!/^[#\-]/{print $1}' /etc/pam.d/*  | sort | uniq


auth is used for providing some kind of challenge/response (depending on the module)  – usually username/password.

account is used to time or otherwise restrict the user account -i.e. user must use faillock, load all the sssd account requirements etc.

password is used to update the authtoken associated with the user account. This is mainly used to change passwords and it can be where the rules around local password strength can be formulated.

session is used to determine what the user needs before they are allowed a session: working home directory, to which system limits apply (open filehandles, number of terminals, etc.), and user keyring.

Each of these keywords has  these common modes:

required - if this fails return failure but continue executing anyway
[success=ok new_authtok_reqd=ok ignore=ignore default=bad]

requisite - if this fails return failure and die (don't even attempt to preauth)
[success=ok new_authtok_reqd=ok ignore=ignore default=die]

sufficient - this is enough for success and exit if nothing previously has failed
[success=done new_authtok_reqd=done default=ignore]

optional - we don't care unless this is the only module in the stack associated with this type
[success=ok new_authtok_reqd=ok default=ignore]

PAM execution stack

PAM executes everything sequentially unless told otherwise. The following snippet of password-auth-ac will:

# set environment variables
auth required
# delay (ms) by this amount if last time this user failed
# otherwise if absent check login.defs for specified delay
auth required delay=2000000
# check user authentication from the system 
# try_first_pass - use the password that has already been entered if any
# nullok allow - allow blank password
auth sufficient nullok try_first_pass
# succeed if uid is greater than 1000 and don't log success to the system log
auth requisite uid >= 1000 quiet_success
# deny everything else
auth required

If anything is sufficient and it succeeds then the execution stack exits for the component – i.e. auth successful when local user signs in with username/password.

PAM if statements

PAM doesn’t have very obvious if statements, but given the right parameters it allows jumps of execution. Below is a way of incorporating an SSSD back-end with PAM to allow users with IdM logins access to the system:

# check if the user is allowed to log in with preauthorisation (i.e. has faillock entries)
auth        required preauth silent audit deny=5 unlock_time=900
# skip two rules if successful 
# NOTE: ​default ignore  means sufficient
# and check if it's a unix user - use the password provided by the auth stack
auth        [success=2 new_authtok_reqd=done default=ignore] try_first_pass
# if it's not a unix user, then use sssd backend for logging in
auth        sufficient forward_pass
# otherwise fail 
auth        [default=die] authfail audit deny=5 unlock_time=900
# this is the skip step from pam_unix module
# it allows for resetting the faillock when necessary
auth        sufficient authsucc audit deny=5 unlock_time=900

The comments in line explain what each module is doing. The execution sequence reminds me a bit of jump statements in assembly language and it helps me to think about them in that manner.

Putting them all together gives us this auth section:

auth        required
auth        required delay=2000000
auth        required preauth silent audit deny=5 unlock_time=900
auth        [success=2 new_authtok_reqd=done default=ignore] try_first_pass 
auth        sufficient forward_pass
auth        [default=die] authfail audit deny=5 unlock_time=900 
auth        sufficient authsucc audit deny=5 unlock_time=900
auth        required

If we were to make slight adjustments to the above snippet, it may have the frightening effect of allowing users to log in without having the correct password:

auth        required
auth        required delay=2000000
auth        required preauth silent audit deny=5 unlock_time=900
# reducing this number from 2 to 1 (success=1)
auth        [success=1 new_authtok_reqd=done default=ignore] try_first_pass
auth        sufficient forward_pass
# swapping these two lines
auth        sufficient authsucc audit deny=5 unlock_time=900
auth        [default=die] authfail audit deny=5 unlock_time=900
auth        required


PAM is a very powerful, yet quite obscure tool. It can be configured to allow people in without even a valid password, or it can deny everyone access apart from every alternate Tuesday between 19:00 and 20:00 (in combination with other tools). Whenever I have configured it, I have found it useful to test for access allowed, access denied, and access locked in order to ensure predictable operation.

NOTE: To get more information around PAM, visit the man pages: (5) pam.conf , (8) pam ,  (5) password-auth

Identity Management (Continued) – The AD integration

Thoughts about AD integration with IPA

Have you ever spent days wondering what would happen if the Windows guys talked to the Linux guys and visa versa? It’s odd; organizations decide to manage identity with Active Directory and thus the Linux server estates come in as an afterthought, an add-on that needs to be integrated – and maintained – alongside the Windows boxes.

Centralised authentication is one of those integration points that someone like me has to address. A preexisting Active Directory is usually (and regrettably) the cornerstone for this concept, as it provides authentication to the desktop estate where end users usually remain, for better or for worse. Introducing Linux central authentication to servers and to users is something wonderful. Something to be celebrated by a party! No longer would you manage individual users and passwords, no longer would you have to go through individual servers deleting previous sysadmins that have pained and upset you by leaving you in this mess and no longer would you have the arduous task of clearing up after them. You can remove them from your centrally managed Linux authentication once and for all.

In fact, you can do one better than that. If your company is managing users via AD, and you are integrating AD with IPA you have the possibility to offload this task to the Windows team. They can go and disable the user from AD. All you have to do is sit back and relax.

The Integration Choice

I know my thoughts wandered at this point. All the different possibilities. How best to integrate, what are the benefits of the different options, why would people care. Decision time!

Assuming that you already decided on using IPA and have one up and running and ready to go, you will want to consider the following two possibilities:

  • the user sync from AD to IPA
  • the cross realm trust

User synchronizing from AD with IPA may leave you with user name collision, if you have similarly named users. It also doesn’t allow SSO and unique identities but it does give you the option of having two different passwords in the windows and Linux realms. Some may consider that a plus. On the down side, user management on putting users into groups is also necessary because all the users are synchronized but not the AD groups.

I prefer the cross realm trust route. It means that user management falls completely under the S.E.P field. All I need to care about is which groups of users does the company trust enough not to harm themselves and others when using Linux. Each realm would be responsible in authenticating its own users and the groups the groups that the users are in would allow them to be authorized for certain actions.

NOTE: If you don’t have an IPA server ready to go, you have very limited time, you’re not allowed to run 2 different realms, you don’t care about policy enforcement, consider integrating directly with AD. SSSD and kerberos should help you manage different users from AD and your Linux boxes talk directly to the AD forest.

The rocky road

I already have an installation that I can use for my integration. The previous article showed how to disable most IPA features. This part is showing how to live with your choices with IPA 4.1 and AD 2012R2.
IPA should have a netbios name as well as samba and winbind services configured:

ipa-adtrust-install --netbios-name=MGMT

Enable the following ports:

 firewall-cmd --permanent --add-port={138/tcp,139/tcp,445/tcp,138/udp,139/udp,389/udp,445/udp}
 firewall-cmd --reload

Add the service records to DNS. This is one of the things that allows AD to talk to IPA on a native level:

_ldap._tcp.dc._msdcs IN SRV 0 100 389 idm
_kerberos._tcp.dc._msdcs IN SRV 0 100 88 idm
_kerberos._udp.dc._msdcs IN SRV 0 100 88 idm
_ldap._tcp.Default-First-Site-Name._sites.dc._msdcs IN SRV 0 100 389 idm
_kerberos._tcp.Default-First-Site-Name._sites.dc._msdcs IN SRV 0 100 88 idm
_kerberos._udp.Default-First-Site-Name._sites.dc._msdcs IN SRV 0 100 88 idm

And create the trust (Yes, in IPA 4.1 you do have to stop the firewall and then add the trust, there are ports that aren’t mentioned in the docs):

systemctl stop firewalld
ipa trust-add --type=ad "windows.local" --admin Administrator --password
systemctl start firewalld
ipactl status

Check that the trust exists by finding the domains

[root@idm ~]# ipa trust-find windows.local
1 trust matched
  Realm name: windows.local
  Domain NetBIOS name: WINDOWS
  Domain Security Identifier: S-1-5-21-4218785893-350090421-2374357632
  Trust type: Active Directory domain
Number of entries returned 1

OK, now one kerberos realm trusts the other and this is how it should look:

Cross Realm Trust
Cross Realm Trust

The group mapping

To enable your AD users to log on to clients of IPA, you need to tell IPA how to find them. To do that you will need to add the AD group where your user is as an external group in IPA. Let’s assume you want to allow the AD Domain Admins group to ssh on to servers.

 ipa group-add --desc='AD Domain admins' domain_admins_external  --external
 ipa group-add-member domain_admins_external --external 'WINDOWS\Domain Admins

These groups won’t have any POSIX attributes as they are external groups. Every user that connects to a Linux computer requires a Linux UID and GID. This means that you will need to create another group to contain the mapped external one:

ipa group-add --desc='AD Posix Domain Admins' domain_admins
ipa group-add-member domain_admins --groups domain_admins_external

Great! You can now start configuring your sudo, host based access control and Identity Views!

Identity Management (IPA) – The ‘on the side’ installation

Thoughts about IPA installation

IPA or IdM in its Red Hat productised form is a very neat product. It allows centralised authentication and policy management while providing that over secure channels (kerberos and TLS). IdM provides quite a few features and you may decide that you’re better off without some (saving the extra calories/effort for later) as your infrastructure may already provide those on the side.

This example installation is without DNS, without a CA, and without NTP (VM installations shouldn’t really be running NTP anyway).

Once you’re past the stage of convincing management that it’ll be good for you (and good for them) to allow this installation to happen, this is what you need to think about and discuss with the team managing Certificate Authorities, NTP servers, and DNS:

  • DNS – DNS zones need to be configured in such a way that IPA acts as a KDC to its own group of servers if there are existing KDC in a different realm in the environment, they will need to be in a different subdomain/domain. The SRV records will only return the IPA servers when queried about kerberos in this subdomain.
  • Certificates – IPA uses SSL for ldap and http. IPA could be acting as a Certificate Authority but not in this instance. Active Directory (or something else) may already be configured as a Certificate Authority which could allow you to present your windows team with a certificate request from IPA to sign in order to obtain a valid web certificate.
  • Time – a uniform time source across the estate IPA servers and clients. Think about business meetings, SSL, sex, humour, and trains – all require good timing.

NOTE: IPA/IdM used to have to provide certificates by default to its clients on installation. As this is no longer the case, IPA can be installed without a CA in an easier fashion than you’re used to. Give it a try.

Prerequisite Checking for IPA installation

NOTE: The following installation is for IPA version 4.1 and AD version 2012R2.

Check that you have:

  • Access to the right software packages via yum (normal RHEL/CentOS base repo should do)
  • Forward and reverse resolvable hostname
  • An entry in the /etc/hosts with the ip address and hostname
  • nscd off
  • An up-to-date OS installation
  • Still got your sanity (test to be performed by an external third party)

The certificate creation

Create your secret private key for your server. Here, we are using openssl to generate a private key:

mkdir /root/certs 
openssl genrsa -out /root/certs/http.$(hostname).key 2048

And then, the certificate request below:

And then the certificate request below:
[root@idm certs]# openssl req  \
-key /root/certs/http.idm.mgmt.linux.local.key \
-out /root/certs/$(hostname -f).csr -new

You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
Country Name (2 letter code) [XX]:GB
State or Province Name (full name) []:Norfolk
Locality Name (eg, city) [Default City]:Norwich
Organization Name (eg, company) [Default Company Ltd]:MGMT.LINUX.LOCAL
Organizational Unit Name (eg, section) []:She ITs And Giggles
Common Name (eg, your name or your server's hostname) []:idm.mgmt.linux.local  
Email Address []:root@localhost 

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:SheITsAndGiggles
An optional company name []:

Your AD certificate Authority should be able to sign this csr and then retrieve the CA chain with the cert for IPA installation. An example on how to sign certificates in a windows CA can be found here: Generate a Digital Certificate from CSR.

To examine the Certificates, open them as follows:

openssl x509 -in <certificate> -text

The Installation

The following step just downloads the software on the server. It doesn’t start any services:

yum -y install ipa-server ipa-server-trust-ad

Now for the fun part:

ipa-server-install --http-cert-file /root/certs/http.idm.mgmt.linux.local.crt \
--http-cert-file /root/certs/http.idm.mgmt.linux.local.key \
--http-pin SheITsAndGiggles \
--dirsrv-cert-file /root/certs/http.idm.mgmt.linux.local.crt \
--dirsrv-cert-file /root/certs/http.idm.mgmt.linux.local.key \
--dirsrv-pin SheITsAndGiggles --ca-cert-file ca-chain.p7b \
 -n mgmt.linux.local -r LINUX.LOCAL --mkhomedir

NOTE: I have used the same certificate and key for the http and directory servers. The p7b file that has been downloaded from the CA is the chain.

After the installation, you will need to open all the ports for the services that we are running and add some DNS entries to advertise those services:

systemctl enable firewalld
firewall-cmd --permanent --zone=public \
firewall-cmd --reload

To tell your IPA clients what you are serving you need to advertise the services via DNS. Find an example below:

_ldap._tcp IN SRV 0 100 389 idm
_ldap._udp IN SRV 0 100 389 idm
_kerberos._tcp IN SRV 0 100 88 idm
_kerberos._udp IN SRV 0 100 88 idm
_kerberos-master._tcp IN SRV 0 100 88 idm
_kerberos-master._udp IN SRV 0 100 88 idm
_kpasswd._tcp IN SRV 0 100 464 idm
_kpasswd._udp IN SRV 0 100 464 idm

Check that the installation is running as it should by getting kerberos credentials for your admin user and using admin to ssh on ipa:

[root@idm ~]# kinit admin
[root@idm ~]# ssh  admin@$(hostname -f)
Creating home directory for admin.