How to optimize NexentaStor performance?
The first thing to remember is that if you do nothing but run NexentaStor with your data and SSDs, it will get faster automatically over time. There are algorithms within ZFS that pull frequently requested data onto the SSDs. More specifically, ZFS caching strategy is designed around Adaptive Replacement Cache (ARC)- see http://en.wikipedia.org/wiki/Adaptive_Replacement_Cache. For this reason, it is also worth noting that in many situations, benchmarks may produce results that are lower than actual performance received in production; which is why it is recommended to take benchmarks with a grain of salt, run more than one kind, and try to mimic as closely as possible your actual data usage with a real data set.
At runtime the ZFS ARC is getting populated with pools of free buffers of different size, just in case the file system will ever need those buffers. The memory allocation algorithm is highly sensitive to both the workload and the memory pressure from other subsystems; ZFS constantly tries to perform a balancing act of allocating maximum for itself while at the same time not starving the applications. Assuming the only applications are those that use the filesystem, ZFS (more exactly, the ARC) will by default try to use 7/8 of the RAM in the system.
There is an easy way to restrict it, and secondly, again, ZFS will relinquish memory under external pressure. But in general, it'll allocate almost all (usually leaving at least 1 GB free), and effectively take full advantage of it. This caching mechanism is designed from ground up to take full advantage of Solid State Drives - please see:
Can I use Solid State Drives (SSD) to improve performance?
Starting from version 1.1.8, NexentaStor provides an additional built-in module called NexentaStor Analytics. This module implements a number of easy-to-use easy-to-choose graphical statistics providing a window into all aspects of the internal operation of the appliance.
For more information on NexentaStor Analytics, please see:
How to use DTrace to investigate I/O performance
User Guide, Section "DTrace and DTrace based Graphical Statistics" (this document is also available online)
In addition, the appliance's management software includes a variety of tools to help (1) analyze and (2) optimize performance. The appliance's management console provides 'show performance' command. In particular:
nmc$ show performance arc
Will show realtime and summary ZFS Adaptive Replacement Cache (ARC) statistics. (Aside: Adaptive Replacement Cache was originally designed at IBM. Implemented and improved for ZFS, ARC has proved to be efficient and effective across variety of all possible I/O workloads).
In addition, the management console (NMC) includes:
nmc$ show performance iostat
to report I/O statistics, and
nmc$ show performance dtrace
The latter delivers a powerful DTrace toolkit, also accessible via the newly added top level NMC command:
nmc$ dtrace
DTrace can be used to generate performance profiles and analyze performance bottlenecks. DTrace can help to troubleshoot problems, by providing detailed views of the system internals. NexentaStor makes it easy to use this power: examples, man pages, short descriptions of all DTrace tools, and certainly the tools themselves are available at your fingertips. For in-depth guide on DTrace language and details, please visit DTrace at OpenSolaris.org.
To get a comprehensive view of system performance and examine all areas of the system in one shot, run:
nmc$ dtrace report-all
NexentaStor GUI and CLI based management clients provide interfaces to tune up system settings. As always when modifying the system defaults, a special attention need to be paid as to how the changes will affect your particular deployment environment.
In addition, there is a collection of guides and blogs on the subject of I/O performance, including: http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide