Over the past month the scratch file system on Helium, which was based on Lustre, was replaced with one based on NFS, accessed at /nfsscratch. By most accounts this has worked out pretty well and allowed us to evaluate options for what to do with the hardware that Lustre was running on. After an evaluation we have come to a couple of decisions.
- Since the NFS scratch system (/nfsscratch) has worked out much better for some workloads we will be keeping this file system available on Helium permanently.
- The hardware that was running a Lustre file system is now deployed with a Gluster file system. Gluster is a distributed file system that is in some ways similar to Lustre but yet different. This scratch file system is accessible at /glusterscratch.
There is more information available on the Wiki page at https://icts.uiowa.edu/confluence/display/ICTSit/scratch+file+system but essentially the two different scratch file systems provide options for handling different IO loads. It is difficult to say which will be better for any particular IO load so a little benchmarking may be in order. A side benefit of this is that we will have more scratch storage available, although split between two systems. There is no default scratch file system so you must specify either /nfsscratch/Users/HawkID or /glusterscratch/Users/HawkID for your jobs.
We will also be re-instituting the scratch cleanup policy, which was on hold during the transition period. To minimize the impact of this, all files on /nfsscratch have had their access times updated to July 22, 2012 and therefore no files will be cleaned until 30 days from that date. However, please be aware that files extracted from archive files (tar, zip, etc.) after that date will have the access time of the files in the archive.