One thing I’ve come to realize from being in IT so long is that you should be constantly learning new things. If you aren’t, it’s not because you’re smart or because you know everything; it’s because you’re stagnating.

So, I was not surprised when I heard it was possible to apply NFSv4.x ACLs to files and folders and then mount them via NFSv3 and have the ACLs still work! I already knew that you could do audit ACEs from NFSv4.x for NFSv3 (covered in TR-4067), but had no idea this could extend into the permissions realm. If so, this solves a pretty big problem with NFSv3 in general, where your normal permissions are limited only to owner, group and then everyone else. That makes it hard to do any sort of granular access control for NFSv3 mounts, presents problems for some environments.

It also allows you to keep using NFSv3 for your workloads, whether for legacy application or general performance concerns. NFSv4.x has a lot of advantages over NFSv3, but if you don’t need stateful operations or the NFSv4.x features, or integrated locking, then you are safe to stay with NFSv3.

So, is it possible to use NFSv4.x ACLs with NFSv3 objects?

You betcha!

The method for doing this is pretty straightforward.

Configure and enable NFSv4.x in ONTAP and on your client Enable NFSv4.x ACL support in ONTAP Mount the export via NFSv4.x Apply the NFSv4.x ACLs Unmount and then remount the export using NFSv3 and test it out!

Configuring NFSv4.x

When you’re setting up NFSv4.x in an environment, there are a few things to keep in mind:

Client and NFS server support for NFSv4.x

NFS utilities installed on clients (for NFSv4.x functionality)

NFSv4.x configured on the client in idmapd.conf

NFSv4.x configured on the server in ONTAP (ACLS allowed)

Export policies and rules configured in ONTAP

Ideally, a name service server (like LDAP) to negotiate the server/client conversation of user identities

One of the reasons NFS4.x is more secure than NFSv3 is the use of user ID strings (such as user@domain.com) to help limit cases of user spoofing in NFS conversations. This ID string is required to be case-sensitive. If the string doesn’t match on both client and server, then the NFSv4.x mounts will get squashed to the defined “nobody” user in the NFSv4.x client. One of the more common issues seen with NFSv4.x mounts is the “nobody:nobody” user and group on files and folders. One of the most common causes of this is when a domain string is mismatched on the client and server.

In a client that domain string is defined in the idmapd.conf file. Sometimes, it will default to the DNS domain. In ONTAP, the v4-id-domain string should be configured to the same value on the client to provide proper NFSv4.x authentication.

Other measures, such as Kerberos encryption, can help lock the NFS conversations down further. NFSv4.x ACLs are a way to ensure that files and folders are only seen by those entities that have been granted access and is considered to be authorization, or, what you are allowed to do once you authenticate. For more complete steps on setting up NFSv4.x, see TR-4067 and TR-4073.

However, we’re only setting up NFSv4.x to allow us to configure the ACLs…

What are NFSv4.x ACLs?

NFSv4.x ACLs are a way to apply granular permissions to files and folders in NFS outside of the normal “read/write/execute” of NFSv3, and across more objects than simple “owner/group/everyone.” NFSv4.x ACLs allow administrators to set permissions for multiple users and groups on the same file or folder and treat NFS ACLs more like Windows ACLs. For more information on NFSv4.x ACLs, see:

http://wiki.linux-nfs.org/wiki/index.php/ACLs

https://linux.die.net/man/5/nfs4_acl

http://www.netapp.com/us/media/tr-4067.pdf

NFSv3 doesn’t have this capability by default. The only way to get more granular ACLs in NFSv3 natively is to use POSIX ACLs, which ONTAP doesn’t support.

Once you’ve enabled ACLs in ONTAP (v4.0-acl and/or v4.1-acl options), you can mount an NFS export via NFSv4.x and start applying NFSv4.x ACLs.

In my environment, I mounted a homedir volume and then set up an ACL on a file owned by root for a user called “prof1” using nfs4_setfacl -e (which allows you to edit a file rather than have to type in a long command).

[root@centos7 /]# mount demo:/home /mnt [root@centos7 /]# mount | grep mnt demo:/home on /mnt type nfs4 (rw,relatime,vers=4.0,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,port=0,timeo=600,retrans=2,sec=sys,clientaddr=10.193.67.225,local_lock=none,addr=10.193.67.237)

The file lives in the root user’s homedir. The root homedir is set to 755, which means anyone can read them, but no one but the owner (root) can write to them.

drwxr-xr-x 2 root root 4096 Jul 13 10:42 root

That is, unless, I set NFSv4.x ACLs to allow a user full control:

[root@centos7 mnt]# nfs4_getfacl /mnt/root/file A::prof1@ntap.local:rwaxtTnNcCy A::OWNER@:rwaxtTnNcCy A:g:GROUP@:rxtncy A::EVERYONE@:rxtncy

I can also see those permissions from the ONTAP CLI:

ontap9-tme-8040::*> vserver security file-directory show -vserver DEMO -path /home/root/file Vserver: DEMO File Path: /home/root/file File Inode Number: 8644 Security Style: unix Effective Style: unix DOS Attributes: 20 DOS Attributes in Text: ---A---- Expanded Dos Attributes: - UNIX User Id: 0 UNIX Group Id: 1 UNIX Mode Bits: 755 UNIX Mode Bits in Text: rwxr-xr-x ACLs: NFSV4 Security Descriptor Control:0x8014 DACL - ACEs ALLOW-user-prof1-0x1601bf ALLOW-OWNER@-0x1601bf ALLOW-GROUP@-0x1200a9-IG ALLOW-EVERYONE@-0x1200a9

I can also expand the mask to translate the hex:

ontap9-tme-8040::*> vserver security file-directory show -vserver DEMO -path /home/root/file -expand-mask true Vserver: DEMO File Path: /home/root/file File Inode Number: 8644 Security Style: unix Effective Style: unix DOS Attributes: 20 DOS Attributes in Text: ---A---- Expanded Dos Attributes: 0x20 ...0 .... .... .... = Offline .... ..0. .... .... = Sparse .... .... 0... .... = Normal .... .... ..1. .... = Archive .... .... ...0 .... = Directory .... .... .... .0.. = System .... .... .... ..0. = Hidden .... .... .... ...0 = Read Only UNIX User Id: 0 UNIX Group Id: 1 UNIX Mode Bits: 755 UNIX Mode Bits in Text: rwxr-xr-x ACLs: NFSV4 Security Descriptor Control:0x8014 1... .... .... .... = Self Relative .0.. .... .... .... = RM Control Valid ..0. .... .... .... = SACL Protected ...0 .... .... .... = DACL Protected .... 0... .... .... = SACL Inherited .... .0.. .... .... = DACL Inherited .... ..0. .... .... = SACL Inherit Required .... ...0 .... .... = DACL Inherit Required .... .... ..0. .... = SACL Defaulted .... .... ...1 .... = SACL Present .... .... .... 0... = DACL Defaulted .... .... .... .1.. = DACL Present .... .... .... ..0. = Group Defaulted .... .... .... ...0 = Owner Defaulted DACL - ACEs ALLOW-user-prof1-0x1601bf 0... .... .... .... .... .... .... .... = Generic Read .0.. .... .... .... .... .... .... .... = Generic Write ..0. .... .... .... .... .... .... .... = Generic Execute ...0 .... .... .... .... .... .... .... = Generic All .... ...0 .... .... .... .... .... .... = System Security .... .... ...1 .... .... .... .... .... = Synchronize .... .... .... 0... .... .... .... .... = Write Owner .... .... .... .1.. .... .... .... .... = Write DAC .... .... .... ..1. .... .... .... .... = Read Control .... .... .... ...0 .... .... .... .... = Delete .... .... .... .... .... ...1 .... .... = Write Attributes .... .... .... .... .... .... 1... .... = Read Attributes .... .... .... .... .... .... .0.. .... = Delete Child .... .... .... .... .... .... ..1. .... = Execute .... .... .... .... .... .... ...1 .... = Write EA .... .... .... .... .... .... .... 1... = Read EA .... .... .... .... .... .... .... .1.. = Append .... .... .... .... .... .... .... ..1. = Write .... .... .... .... .... .... .... ...1 = Read ALLOW-OWNER@-0x1601bf 0... .... .... .... .... .... .... .... = Generic Read .0.. .... .... .... .... .... .... .... = Generic Write ..0. .... .... .... .... .... .... .... = Generic Execute ...0 .... .... .... .... .... .... .... = Generic All .... ...0 .... .... .... .... .... .... = System Security .... .... ...1 .... .... .... .... .... = Synchronize .... .... .... 0... .... .... .... .... = Write Owner .... .... .... .1.. .... .... .... .... = Write DAC .... .... .... ..1. .... .... .... .... = Read Control .... .... .... ...0 .... .... .... .... = Delete .... .... .... .... .... ...1 .... .... = Write Attributes .... .... .... .... .... .... 1... .... = Read Attributes .... .... .... .... .... .... .0.. .... = Delete Child .... .... .... .... .... .... ..1. .... = Execute .... .... .... .... .... .... ...1 .... = Write EA .... .... .... .... .... .... .... 1... = Read EA .... .... .... .... .... .... .... .1.. = Append .... .... .... .... .... .... .... ..1. = Write .... .... .... .... .... .... .... ...1 = Read ALLOW-GROUP@-0x1200a9-IG 0... .... .... .... .... .... .... .... = Generic Read .0.. .... .... .... .... .... .... .... = Generic Write ..0. .... .... .... .... .... .... .... = Generic Execute ...0 .... .... .... .... .... .... .... = Generic All .... ...0 .... .... .... .... .... .... = System Security .... .... ...1 .... .... .... .... .... = Synchronize .... .... .... 0... .... .... .... .... = Write Owner .... .... .... .0.. .... .... .... .... = Write DAC .... .... .... ..1. .... .... .... .... = Read Control .... .... .... ...0 .... .... .... .... = Delete .... .... .... .... .... ...0 .... .... = Write Attributes .... .... .... .... .... .... 1... .... = Read Attributes .... .... .... .... .... .... .0.. .... = Delete Child .... .... .... .... .... .... ..1. .... = Execute .... .... .... .... .... .... ...0 .... = Write EA .... .... .... .... .... .... .... 1... = Read EA .... .... .... .... .... .... .... .0.. = Append .... .... .... .... .... .... .... ..0. = Write .... .... .... .... .... .... .... ...1 = Read ALLOW-EVERYONE@-0x1200a9 0... .... .... .... .... .... .... .... = Generic Read .0.. .... .... .... .... .... .... .... = Generic Write ..0. .... .... .... .... .... .... .... = Generic Execute ...0 .... .... .... .... .... .... .... = Generic All .... ...0 .... .... .... .... .... .... = System Security .... .... ...1 .... .... .... .... .... = Synchronize .... .... .... 0... .... .... .... .... = Write Owner .... .... .... .0.. .... .... .... .... = Write DAC .... .... .... ..1. .... .... .... .... = Read Control .... .... .... ...0 .... .... .... .... = Delete .... .... .... .... .... ...0 .... .... = Write Attributes .... .... .... .... .... .... 1... .... = Read Attributes .... .... .... .... .... .... .0.. .... = Delete Child .... .... .... .... .... .... ..1. .... = Execute .... .... .... .... .... .... ...0 .... = Write EA .... .... .... .... .... .... .... 1... = Read EA .... .... .... .... .... .... .... .0.. = Append .... .... .... .... .... .... .... ..0. = Write .... .... .... .... .... .... .... ...1 = Read

In the above, I gave prof1 full control over the file. Then, I mounted via NFSv3:

[root@centos7 /]# mount -o nfsvers=3 demo:/home /mnt [root@centos7 /]# mount | grep mnt demo:/home on /mnt type nfs (rw,relatime,vers=3,rsize=1048576,wsize=1048576,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,mountaddr=10.193.67.219,mountvers=3,mountport=635,mountproto=udp,local_lock=none,addr=10.193.67.219)

When I become a user that isn’t on the NFSv4.x ACL, I can’t write to the file:

[root@centos7 /]# su student1 sh-4.2$ cd /mnt/root sh-4.2$ ls -la total 8 drwxr-xr-x 2 root root 4096 Jul 13 10:42 . drwxrwxrwx 11 root root 4096 Jul 10 10:04 .. -rwxr-xr-x 1 root bin 0 Jul 13 10:23 file -rwxr-xr-x 1 root root 0 Mar 29 11:37 test.txt sh-4.2$ touch file touch: cannot touch ‘file’: Permission denied sh-4.2$ rm file rm: remove write-protected regular empty file ‘file’? y rm: cannot remove ‘file’: Permission denied

When I change to the prof1 user, I have access to do whatever I want, even though the mode bit permissions in v3 say I can’t:

[root@centos7 /]# su prof1 sh-4.2$ cd /mnt/root sh-4.2$ ls -la total 8 drwxr-xr-x 2 root root 4096 Jul 13 10:42 . drwxrwxrwx 11 root root 4096 Jul 10 10:04 .. -rwxr-xr-x 1 root bin 0 Jul 13 10:23 file -rwxr-xr-x 1 root root 0 Mar 29 11:37 test.txt sh-4.2$ vi file sh-4.2$ cat file NFSv4ACLS!

When I do a chmod, however, nothing seems to change from the NFSv4 ACL for the user. I set 700 on the file, which shows up in NFSv3 mode bits:

sh-4.2$ chmod 700 file sh-4.2$ ls -la total 8 drwxr-xr-x 2 root root 4096 Jul 13 10:42 . drwxrwxrwx 11 root root 4096 Jul 10 10:04 .. -rwx------ 1 root bin 11 Aug 11 09:58 file -rwxr-xr-x 1 root root 0 Mar 29 11:37 test.txt

But notice how the prof1 user still has full control:

ontap9-tme-8040::*> vserver security file-directory show -vserver DEMO -path /home/root/file Vserver: DEMO File Path: /home/root/file File Inode Number: 8644 Security Style: unix Effective Style: unix DOS Attributes: 20 DOS Attributes in Text: ---A---- Expanded Dos Attributes: - UNIX User Id: 0 UNIX Group Id: 1 UNIX Mode Bits: 700 UNIX Mode Bits in Text: rwx------ ACLs: NFSV4 Security Descriptor Control:0x8014 DACL - ACEs ALLOW-user-prof1-0x1601bf ALLOW-OWNER@-0x1601bf ALLOW-GROUP@-0x120088-IG ALLOW-EVERYONE@-0x120088

This is because of an ONTAP option known as “ACL Preservation.”

ontap9-tme-8040::*> nfs show -vserver DEMO -fields v4-acl-preserve vserver v4-acl-preserve ------- --------------- DEMO enabled

When I set the option to enabled, the NFSv4.x ACLs will survive mode bit changes. If I disable the option, the ACLs get blown away when a chmod is done:

ontap9-tme-8040::*> nfs modify -vserver DEMO -v4-acl-preserve disabled ontap9-tme-8040::*> nfs show -vserver DEMO -fields v4-acl-preserve vserver v4-acl-preserve ------- --------------- DEMO disabled [root@centos7 root]# chmod 755 file

And the ACLs are wiped out:

ontap9-tme-8040::*> vserver security file-directory show -vserver DEMO -path /home/root/file Vserver: DEMO File Path: /home/root/file File Inode Number: 8644 Security Style: unix Effective Style: unix DOS Attributes: 20 DOS Attributes in Text: ---A---- Expanded Dos Attributes: - UNIX User Id: 0 UNIX Group Id: 1 UNIX Mode Bits: 755 UNIX Mode Bits in Text: rwxr-xr-x ACLs: -

I’d personally recommend setting that option to “enabled” if you want to do v3 mounts with v4.x ACLs.

So, there you have it… a new way to secure your NFSv3 mounts!