I was tasked recently to find which of our processes was accessing a NFS share. During this process, I found that some tools are better adapted than others for the task.In this article, I want to share with you my findings. The whole process was fun and gave me ideas on how to use these tools to tackle similar problems in the future.According to Wikipedia:Network File System (NFS) is a distributed file system protocol originally developed by Sun Microsystems (Sun) in 1984, allowing a user on a client computer to access files over a computer network much like local storage is accessed. NFS, like many other protocols, builds on the Open Network Computing Remote Procedure Call (ONC RPC) system. NFS is an open IETF standard defined in a Request for Comments (RFC), allowing anyone to implement the protocol.Please note that this is not a full tutorial on NFS. For that, please take a look at the following tutorial.In this article, we will focus only on detecting access to a shared drive using several techiques as well setting up two servers and one client.Also, I do use a different OS to set up both the server and the client, so instructions on how to do the task change a little bit.My lab setup has one Nfs Server and two clients:On my setup, I will have three computers talking to each other. One of them will be the NFS server and the other two will be a client.I will prepare my OrangePI machine to be the NFS server. Do so, I will enter the following commands:Next step is to tell the server we want to share.For that, we will edit the /etc/exports file (sudo vi /etc/exports history):Please check the man page to understand what these options mean.In a nutshell, export /data:Now it is time to activate our shared directories:I did something similar to the other host, raspberrypi:First thing is to confirm we can indeed see the shared mount points from our server:Data is shared with two machines – just what we expected.Now, there are several ways to mount this drive. One of them is manually, another one is at startup, and the last one, my preferred one, is on demand.First we set the service:Then we set this up, so we end mounting remote /data into local /misc/data. For that, sdd the following line to your /etc/auto.master:Restart the service one more time:And the smoke test:Now we are ready to play with our service.For our example, we want to determine if a Python application is reading data from this directory. This Script has two features:Here is how our test script looks like in action:The code, written in Python, is pretty simple:Now, let's go over how we can see if our script is indeed accessing an NFS partition.First we need to learn where to look for. So on the machine, check for NFS in /etc/fstab (for mount points that are available since the machine was rebooted):Then on the AutoMount files:The regular expressions are not exact science, but you get the idea what to look for next.We need to confirm if there was access to any of the following partitions mounted over NFS:Next, I will show you a set of tools that will make the task easier, each one of them with their own strength and limitations.Starting with lsof and ripgrep combined.I passed the -b option to lsof to avoid it from getting stuck, in case the NFS handle is stale.A few things about lsof:Still, lsof is a great Tool to investigate.Next strategy involves monitoring from the beginning, to catch the elusive short-read. We will use straceThe openat(AT_FDCWD) entries give away the two files our script is reading from NFS. But as you can tell this approach has some caveats:Is there a different way? Time to move on to the next tool, tshark and see how to use a network capture to confirm access to the share.We can also capture the network traffic and filter out only NFS. It is not perfect, but it may be sufficient.First, find out which network interface is used to communicate with the NFS server. In my case it is easy – they all connected using a wired private network:For this example it is eno1 with IP address '192.168.68.70'. Then capture the traffic, and with some luck we will get the file path:This is great, there is activity against two NFS servers, 192.168.68.59 and 192.168.68.60. But, is there a way to see the name of files?tshark has a way to spit information by field. The problem is that NFS has lots of them:So, let's capture them into a variable (also need to enable some options):I managed to get the filename only once, then after interrupting and restarting the program I got no luck.And yet no sign of the file name. The file handle was in the contents but this is not very useful if you want a quick way to see what was accessed.Is there an easier way to do this? Sysdig may offer some answers.While trying to find the elusive mount points, I stumbled into Sysdig:Sysdig instruments your physical and virtual machines at the OS level by installing into the Linux kernel and capturing system calls and other OS events. Sysdig uses DTrace to get access to the system kernel.Sysdig also makes it possible to create trace files for system activity, similarly to what you can do for networks with tools like tcpdump and Wireshark.I decided to use the latest version (0.33.1) for Fedora 37 where my script is running):How easy is to probe out the script so it is indeed accessing the NFS mounted directories? Let's print three fields of interest and the name of the accesed file:What if you want to capture all the data, and filter later? One way to do it is capturing to a file:And then replay the contents, with filtering (replay doesn't need elevated privileges):Sysdig supports scripting, using the LUA language. For example, it has a very convenient version of lsof:So let's use it:What I liked about this tool:Before finishing up let's look at one more tool, BPF.Originally Berkeley Packet Filter, is a kernel and user-space observability scheme for Linux.The BPF is a very powerful tool, and this short article won't even scratch the surface.Yes, this is huge. I'm learning this myself.I found that the bcc repository has lots of ready to use scripts that we could use to track our NFS access, and even check for performance (you can find more examples here, and on the BPF Performance Book repository).But it is more interesting to write tools yourself that monitor pretty much anything you want. For this tutorial, I will use some ready to use programs that use the traces to capture useful information.As a first step, we will need to install a high level interpreter for our scripts. Again, on my Fedora Linux machine:On a separate terminal run again the NFS test script:You can trace all the files opened by a program, like top:But it doesn't print the full path. It's more useful to ask a NFS snoop and see if one of our files shows up:This is much better. Also, we can see than the latency is almost two milliseconds.We can also monitor mount/ umount operations:This is good as well, we can see the activity over NFS we wanted to confirm.You learned several tools and as you may have guessed, you can use them to snoop on more than just opened files on NFS.It is always useful to know more than one tool. Sysdig has a special mention for being very versatile, powerful and yet easy to use. Also, it can be extended with scripts written in the LUA language.BPF is another alternative and will give you incredible access to the kernel calls. Be prepared to spend time reading and learning how to use the tools.The code for the scripts used on this tutorial can be obtained from my GitHub repository: SpyOnNfs.🇻🇪 🇺🇸, proud dad and husband, software developer and sysadmin, recreational runner and geek. If this article was helpful, tweet it. Learn to code for free. freeCodeCamp's open source curriculum has helped more than 40,000 people get jobs as developers. Get started freeCodeCamp is a donor-supported tax-exempt 501(c)(3) charity organization (United States Federal Tax Identification Number: 82-0779546)Our mission: to help people learn to code for free. We accomplish this by creating thousands of videos, articles, and interactive coding lessons - all freely available to the public. We also have thousands of freeCodeCamp study groups around the world.Donations to freeCodeCamp go toward our education initiatives, and help pay for servers, services, and staff. You can make a tax-deductible donation here.