[ale] Stale NFS mounts - Switch to iSCSI?

[Raj Wurttemberg via Ale]

Many of our client systems share data via NFS (v3) and everything works fine
except when the server with the NFS export is rebooted, which happens
regularly due to OS patching (not my decision). The systems that have the
NFS export mounted can no longer do even a simple 'ls', they just hang.
Unmounting and remounting the NFS export or rebooting the client system does
resolve the issue, but I was curious if I switched to iSCSI if I would
experience the same issue? I know the easy answer is to mount the NFS
directly off the SAN (EMC) but I was told that the storage team would not
create NFS exports for me. I was also looking at using autofs for the NFS
mounts instead of putting them in the fstab file.



Just curious if anyone else had any advice.



Thanks,

/Raj

[Lightner, Jeffrey via Ale]

I'd think you'd have a worse issue and more complexity with iSCSI.

I highly recommend autofs for filesystems that aren't in use a high percentage of the time. If the exporting server goes down you see impact only on the servers where the mount is currently active.

If you're doing planned maintenance you can run a script from a central server to go to each of your other servers, unmount the share if mounted, then disable autofs. After maintenance run another script that goes and re-enables autofs. (No need to remount as autofs will do that when someone tries to access the shared filesystem.

For unplanned events you can resolve the hung NFS mount effect by running the "lazy" unmount (umount -l <filesystem>) on the affected host. That works where "force (umount -f) doesn't.

Of course there are cluster shares like gfs available but that also has some complexity.


From: Ale [mailto:ale-***@ale.org] On Behalf Of Raj Wurttemberg via Ale
Sent: Tuesday, March 27, 2018 9:12 AM
To: 'Atlanta Linux Enthusiasts'
Subject: [ale] Stale NFS mounts - Switch to iSCSI?

Many of our client systems share data via NFS (v3) and everything works fine except when the server with the NFS export is rebooted, which happens regularly due to OS patching (not my decision). The systems that have the NFS export mounted can no longer do even a simple 'ls', they just hang. Unmounting and remounting the NFS export or rebooting the client system does resolve the issue, but I was curious if I switched to iSCSI if I would experience the same issue? I know the easy answer is to mount the NFS directly off the SAN (EMC) but I was told that the storage team would not create NFS exports for me. I was also looking at using autofs for the NFS mounts instead of putting them in the fstab file.

Just curious if anyone else had any advice.

Thanks,
/Raj

[Scott Plante via Ale]

Are you using the "soft" parameter on your NFS mount?


https://www.centos.org/docs/5/html/Deployment_Guide-en-US/s1-nfs-client-config-options.html



hard or soft — Specifies whether the program using a file via an NFS connection should stop and wait (hard) for the server to come back online, if the host serving the exported file system is unavailable, or if it should report an error (soft).


If soft is specified, the user can set an additional timeo=<value> option, where <value> specifies the number of seconds to pass before the error is reported.
--
Scott Plante
404-873-0058 x104


----- Original Message -----

From: "Raj Wurttemberg via Ale" <***@ale.org>
To: "Atlanta Linux Enthusiasts" <***@ale.org>
Sent: Tuesday, March 27, 2018 9:12:16 AM
Subject: [ale] Stale NFS mounts - Switch to iSCSI?



Many of our client systems share data via NFS (v3) and everything works fine except when the server with the NFS export is rebooted, which happens regularly due to OS patching (not my decision). The systems that have the NFS export mounted can no longer do even a simple ‘ls’, they just hang. Unmounting and remounting the NFS export or rebooting the client system does resolve the issue, but I was curious if I switched to iSCSI if I would experience the same issue? I know the easy answer is to mount the NFS directly off the SAN (EMC) but I was told that the storage team would not create NFS exports for me. I was also looking at using autofs for the NFS mounts instead of putting them in the fstab file.

Just curious if anyone else had any advice.

Thanks,
/Raj

_______________________________________________
Ale mailing list
***@ale.org
https://mail.ale.org/mailman/listinfo/ale
See JOBS, ANNOUNCE and SCHOOLS lists at
http://mail.ale.org/mailman/listinfo

[Ed Cashin via Ale]

But you might *want* an important write to succeed after the rebooting NFS
server comes back up.

Has anybody mentioned the fsid mount option yet? If so, sorry. If not,
you can consider using hard but helping the file handles to keep from
becoming stale.

https://lxadm.com/Stale_NFS_handle

Post by Scott Plante via Ale
Are you using the "soft" parameter on your NFS mount?
https://www.centos.org/docs/5/html/Deployment_Guide-en-US/
s1-nfs-client-config-options.html
hard or soft — Specifies whether the program using a file via an NFS
connection should stop and wait (hard) for the server to come back online,
if the host serving the exported file system is unavailable, or if it
should report an error (soft).
If soft is specified, the user can set an additional timeo=<value> option,
where <value> specifies the number of seconds to pass before the error is
reported.
--
Scott Plante
404-873-0058 x104 <(404)%20873-0058>
------------------------------
*Sent: *Tuesday, March 27, 2018 9:12:16 AM
*Subject: *[ale] Stale NFS mounts - Switch to iSCSI?
Many of our client systems share data via NFS (v3) and everything works
fine except when the server with the NFS export is rebooted, which happens
regularly due to OS patching (not my decision). The systems that have the
NFS export mounted can no longer do even a simple ‘ls’, they just hang.
Unmounting and remounting the NFS export or rebooting the client system
does resolve the issue, but I was curious if I switched to iSCSI if I would
experience the same issue? I know the easy answer is to mount the NFS
directly off the SAN (EMC) but I was told that the storage team would not
create NFS exports for me. I was also looking at using autofs for the NFS
mounts instead of putting them in the fstab file.
Just curious if anyone else had any advice.
Thanks,
/Raj
_______________________________________________
Ale mailing list
https://mail.ale.org/mailman/listinfo/ale
See JOBS, ANNOUNCE and SCHOOLS lists at
http://mail.ale.org/mailman/listinfo
_______________________________________________
Ale mailing list
https://mail.ale.org/mailman/listinfo/ale
See JOBS, ANNOUNCE and SCHOOLS lists at
http://mail.ale.org/mailman/listinfo

[Dow Hurst via Ale]

It is a problem is the NFS server is patched often. I use:

rw,bg,intr,hard,proto=tcp,noatime

as my options on mounting a NFS server. We rarely have a reboot or downtime
so stale file handles are rare.

Sincerely,
Dow
⚛Dow Hurst, Research Scientist
340 Sullivan Science Bldg.
Dept. of Chem. and Biochem.
University of North Carolina at Greensboro
PO Box 26170 Greensboro, NC 27402-6170

Post by Raj Wurttemberg via Ale
Many of our client systems share data via NFS (v3) and everything works
fine except when the server with the NFS export is rebooted, which happens
regularly due to OS patching (not my decision). The systems that have the
NFS export mounted can no longer do even a simple ‘ls’, they just hang.
Unmounting and remounting the NFS export or rebooting the client system
does resolve the issue, but I was curious if I switched to iSCSI if I would
experience the same issue? I know the easy answer is to mount the NFS
directly off the SAN (EMC) but I was told that the storage team would not
create NFS exports for me. I was also looking at using autofs for the NFS
mounts instead of putting them in the fstab file.
Just curious if anyone else had any advice.
Thanks,
/Raj
_______________________________________________
Ale mailing list
https://mail.ale.org/mailman/listinfo/ale
See JOBS, ANNOUNCE and SCHOOLS lists at
http://mail.ale.org/mailman/listinfo