NFS and the 16 groups

 

So, here’s one that someone else solved before me, but it’s really hard to find. Consider this a boost: https://www.xkyle.com/solving-the-nfs-16-group-limit-problem/
The story is: NFS mounts, group permissions and the group limit.
Say you’ve got a filesystem mounted from a server over NFS. You’ve got a multi-user environment and a stack of groups to manage permissions, those groups are somehow kept in line by something like LDAP or active directory.
You go to cd to a directory (maybe even the mount point) and get permission denied.
Something like this:
me@box~$ ls /data/somedir
ls: cannot open directory /data/somedir: Permission denied
(Yes, it’s an unorthodox mount point…)
me@box~$ ls /data/somedir -lhd
drwxrws--x. 21 notme group5 4.0K May  2 14:32 /data/somedir
But you’re definitely in (for sake of argument) group5:
me@box:~$ groups
group1 group2 group3 group4 group5 group6 group7 group8 group9 group10 group11 group12 group13 group14 group15 group16 group17 group18 group19 group20
What’s going on? What to do about it?
NFS requests pass the group memberships of a user from the client to the server when you try to access something. But requests only have room for 16 group ids. If you are in more than 16 groups then not all ids are included in the list (and which ones are can vary, this is why group5 might get blocked but group17 be okay).
There are various elaborate ways to address this, including Kerberos, but it’s not an NFS3 vs NFS4 thing, ACLs won’t save you, and there are a lot of other dead ends. However there is one very simple solution:
--manage-gids
This is an option to rpc.mountd and exportd (part of the server infrastructure that NFS uses) to tell it to manage group ids. With this option your group list gets looked up at the server side and that (not the limited list passed over NFS) is used to control access. No more 16 group limit!
It’s simple enough to run:
me@srvbox:~# rpc.mountd --manage-gids
But this won’t stick on reboot. On RedHat 8 and similar systems you want this in the mountd and exportd sections of the NFS server's /etc/nfs.conf:
[exportd]
…
manage-gids=y
…
[mountd]
…
manage-gids=y
Then restart NFS however your system does it (possibly systemctl restart nfs-server).
Why both [mountd] and [exportd]? exportd is not widely documented; if you read the nfs.conf manual you’ll see this section controls exports, but you probably don’t have a manpage for exportd on your system. It is an NFS4-only alternative to rpc.mountd, sometimes named as nfsv4.exportd, but I’ve not found it on an RPM-based system yet. Most likely you’re using rpc.mountd and the exportd entry is unnecessary, but it should be harmless to add as well.
There are still things to deal with. You need the group list to be available for the user on the server side, but if you’re in this situation then likely you have something like LDAP or Active Directory which the machines are already talking to. There is also still a limit to the number of groups, which is now how many ids can fit into the buffer for each user. It’s not clear exactly how many it is, but since the buffer is apparently 32kB in size (https://access.redhat.com/solutions/2570781) the answer is a lot more than 16.