Photo by John @ ThinkHole.com

On Tuesday night I gave a presentation on ZFS to the Central PA Linux User Group. Since the audience was a Linux user group, I wasn’t expecting too many in the crowd to be familiar with ZFS, but I was pleasantly surprised that about 40% of the ~ 20 people in attendance had used ZFS in some capacity. If you are already a seasoned ZFS user, I would highly recommend Richard Elling’s ZFS presentation which he uses in his day-long tutorials.

On Monday I attended a tutorial by Richard Elling on ZFS: A Filesystem for Modern Hardware.

On Tuesday I attended two tutorials.  The first was Jacob Farmer’s Disk-to-Disk Backup and Eliminating Backup System Bottlenecks.  The second was Tom Limoncelli’s Design Patterns for System Administrators.

Unfortunately on both Monday and Tuesday I had to spend a significant amount of time on conference calls helping to troubleshoot some work related issues, but the time I spent in all 3 sessions and viewing their materials was helpful.  I would definitely recommend attending tutorials by any of the 3 people above if they are teaching a topic of interest to you.

On Tuesday night I attended some (Open)Solaris birds-of-a-feather sessions.  There were a few times that people in the crowd were being belligerent towards a speaker (mostly complaining about the difficulty of finding information of various types), even though the speaker certainly had no sway over what the person in the crowd was upset about.  I don’t care how much money your company spends with a vendor, there is never a reason to be rude.   David Miner gave a talk about whats coming in Solaris.next and Ben Rockwood gave an entertaining and informative presentation on ZFS in the Trenches.

I was lucky enough to get a chance to talk with David Miner over a quick lunch later in the week and talk about the new opportunities and challenges with the OpenSolaris installation technologies.

On Wednesday through Friday I attended a mix of presentations, met with a bunch of vendors, and also sat in some of the ‘Guru is in’ sessions and talked with a number of conference attendees.  The highlights for me were:

• Werner Vogel (CTO of Amazon) gave a fascinating talk on the history of Amazon’s IT philosophy and infrastructure and how they evolved from a humble internal IT shop to adding a business which is the dominant  cloud computing provider.
• Elizabeth Zwicky’s talk on “Searching for Truth, or at Least Data: How to Be an Empiricist Skeptic”
• Bryan Cantrill’s talk on “Visualizing DTrace: Sun Storage 7000 Analytics”
  
• Talking with the folks from Splunk (awesome log searching analysis tool)

My environment consists of a Sun x4150 (32G RAM and dual quad-core Xeons @ 2.93 Ghz) running Solaris 10 05/2008 with (8) 73GB 10k RPM SAS drives and an LSI SAS RAID controller with 256M of memory.

For this test I created the following 6 disk zpool:

Two options I decided to test for ZFS using BlogBench were atime and compression.  For those unfamiliar with it, the atime of a file is changed when a file is accesed.  All UNIX file systems I’m aware of have atime updating enabled by default. As a sysadmin, it can be very handy to have the atime available to see when a file was last accessed, but  atime updating can add a significant amount of overhead in some access patterns, so a lot of sites disable it on mounts that need performance. For ZFS to disable atime you use:

zfs set atime=off $datasetname

for most UNIX file system types there is noatime or similar mount option that can be used.

I ran 5 iterations of BlogBench (using ./blogbench –directory=/data/blogbench ) for each permutation of atime and compression settings. I had the script sleep for 60 seconds between runs to make sure any background activity for memory or ZFS housekeeping had finished before the next run started.  Averaging the 5 runs together for each permutation gave me the following results:

When atime was on (which is the default), there was very little difference in the non-compressed versus compressed results.  With atime disabled there was a 30-50% increase in read transactions performed and about a 250% increase in write operations.  Note that the data size of the benchmark ( ~ 3.6G) was significantly smaller than the memory on the machine (32G) , so all reads were satisfied out of file system cache.

These results are only applicable to this specific test and software/hardware, so your environment may vary significantly but I would like people to be aware of the atime setting so they can be aware of another potential knob to turn in their environment.