{"id":4531,"date":"2023-02-16T15:21:52","date_gmt":"2023-02-16T21:21:52","guid":{"rendered":"https:\/\/www.vanderbilt.edu\/csb\/?page_id=4531"},"modified":"2023-02-17T17:30:55","modified_gmt":"2023-02-17T23:30:55","slug":"zfs-howto","status":"publish","type":"page","link":"https:\/\/www.vanderbilt.edu\/csb\/zfs-howto\/","title":{"rendered":"ZFS HOWTO"},"content":{"rendered":"<p><strong>Introduction<\/strong><\/p>\n<p>The Zed File System (ZFS) is an advanced filesystem that merges a file system, logical volume manager, and software RAID.\u00a0 It consists of the following components:<\/p>\n<ul>\n<li>A ZFS virtual device (vdev) is logical device in a zpool, which can be a physical device, a file, or a collection of devices.<\/li>\n<li>A ZFS zpool is a logical group of devices describing the layout and physical characteristics of the available storage. Disk space for datasets is allocated from a zpool.\u00a0 RAID is implemented in a zpool and can be striped, mirrored, or raid-z.<\/li>\n<li>A ZFS dataset behaves like other file systems and is mounted within the standard system namespace.<\/li>\n<li>A ZFS snapshot is a read-only copy of a dataset at a given point in time.<\/li>\n<\/ul>\n<p><strong>Basic Commands<\/strong><\/p>\n<p>List zpools<\/p>\n<pre># zpool list<\/pre>\n<p>Check zpool status<\/p>\n<pre># zpool status<\/pre>\n<p>Create a zpool<\/p>\n<pre># zpool create <em>zpool <\/em>-o ashift=12<\/pre>\n<p>Delete a zpool<\/p>\n<pre># zpool destroy <em>zpool<\/em><\/pre>\n<p>List datasets<\/p>\n<pre># zfs list<\/pre>\n<p>Create a dataset<\/p>\n<pre># zfs create <em>dataset<\/em> -o atime=off -o mountpoint=<em>mountpoint<\/em><\/pre>\n<p>Delete a dataset<\/p>\n<pre># zfs destroy <em>dataset<\/em><\/pre>\n<p>List snapshots<\/p>\n<pre># zfs list -t snapshot<\/pre>\n<p>Check to see how much space snapshots use<\/p>\n<p>Note: Be very careful with this command!\u00a0 Snapshots may be deleted without the &#8216;-n&#8217; (dry-run) argument.\u00a0 Snapshots that are deleted before they are replicated to a backup server cannot be recovered.<\/p>\n<p>For example, the see how much space can be recovered by deleting snapshots that are older than one second (all of them):<\/p>\n<pre class=\"p1\"><span class=\"s1\">$ \/sb\/admin\/scripts\/zfs-prune-snapshots -v -n 1s -p tank\/hd1<\/span>\r\n<span class=\"s1\">removing tank\/hd1@zfs-auto-snap_monthly-2023-02-01-00h28: 2 weeks old &lt;dry-run: no action taken&gt;<\/span>\r\n<span class=\"s1\">would reclaim 1.89G<\/span>\r\n<span class=\"s1\">removing tank\/hd1@zfs-auto-snap_daily-2023-02-10-00h07: 6 days old &lt;dry-run: no action taken&gt;<\/span>\r\n<span class=\"s1\">would reclaim 684M<\/span>\r\n:\r\n<span class=\"s1\">removing tank\/hd1@zfs-auto-snap_frequent-2023-02-16-15h45: 1 minute old &lt;dry-run: no action taken&gt;<\/span>\r\n<span class=\"s1\">would reclaim 6.67M<\/span><\/pre>\n<p>View Snapshots<\/p>\n<p>ZFS creates a special .zfs\/snapshot directory which shows the snapshots as directories of the dataset as it existed at a particular time.<\/p>\n<pre class=\"p1\"><span class=\"s1\">rinerwc@halibut \/home\/rinerwc% cd .zfs\/snapshot<\/span>\r\n<span class=\"s1\">rinerwc@halibut \/home\/rinerwc\/.zfs\/snapshot% ls<\/span>\r\n<span class=\"s1\">zfs-auto-snap_daily-2023-02-10-00h07\/ <span class=\"Apple-converted-space\">\u00a0 <\/span>zfs-auto-snap_hourly-2023-02-16-00h00\/<\/span>\r\n<span class=\"s1\">zfs-auto-snap_daily-2023-02-11-00h07\/ <span class=\"Apple-converted-space\">\u00a0 <\/span>zfs-auto-snap_hourly-2023-02-16-01h00\/<\/span>\r\n<span class=\"s1\">zfs-auto-snap_daily-2023-02-12-00h07\/ <span class=\"Apple-converted-space\">\u00a0 <\/span>zfs-auto-snap_hourly-2023-02-16-02h00\/<\/span>\r\n<span class=\"s1\">zfs-auto-snap_daily-2023-02-13-00h07\/ <span class=\"Apple-converted-space\">\u00a0 <\/span>zfs-auto-snap_hourly-2023-02-16-03h00\/<\/span>\r\n<span class=\"s1\">zfs-auto-snap_daily-2023-02-14-00h07\/ <span class=\"Apple-converted-space\">\u00a0 <\/span>zfs-auto-snap_hourly-2023-02-16-04h00\/<\/span>\r\n<span class=\"s1\">zfs-auto-snap_daily-2023-02-15-00h07\/ <span class=\"Apple-converted-space\">\u00a0 <\/span>zfs-auto-snap_hourly-2023-02-16-05h00\/<\/span>\r\n<span class=\"s1\">zfs-auto-snap_daily-2023-02-16-00h07\/ <span class=\"Apple-converted-space\">\u00a0 <\/span>zfs-auto-snap_hourly-2023-02-16-06h00\/<\/span>\r\n<span class=\"s1\">zfs-uto-snap_frequent-2023-02-16-14h45\/<span class=\"Apple-converted-space\">\u00a0 <\/span>zfs-auto-snap_hourly-2023-02-16-07h00\/<\/span>\r\n<span class=\"s1\">zfs-auto-snap_frequent-2023-02-16-15h15\/<span class=\"Apple-converted-space\">\u00a0 <\/span>zfs-auto-snap_hourly-2023-02-16-08h00\/<\/span>\r\n<span class=\"s1\">zfs-auto-snap_frequent-2023-02-16-15h30\/<span class=\"Apple-converted-space\">\u00a0 <\/span>zfs-auto-snap_hourly-2023-02-16-09h00\/<\/span>\r\n<span class=\"s1\">zfs-auto-snap_frequent-2023-02-16-15h45\/<span class=\"Apple-converted-space\">\u00a0 <\/span>zfs-auto-snap_hourly-2023-02-16-10h00\/<\/span>\r\n<span class=\"s1\">zfs-auto-snap_hourly-2023-02-15-16h00\/ <span class=\"Apple-converted-space\">\u00a0 <\/span>zfs-auto-snap_hourly-2023-02-16-11h00\/<\/span>\r\n<span class=\"s1\">zfs-auto-snap_hourly-2023-02-15-17h00\/ <span class=\"Apple-converted-space\">\u00a0 <\/span>zfs-auto-snap_hourly-2023-02-16-12h00\/<\/span>\r\n<span class=\"s1\">zfs-auto-snap_hourly-2023-02-15-18h00\/ <span class=\"Apple-converted-space\">\u00a0 <\/span>zfs-auto-snap_hourly-2023-02-16-13h00\/<\/span>\r\n<span class=\"s1\">zfs-auto-snap_hourly-2023-02-15-19h00\/ <span class=\"Apple-converted-space\">\u00a0 <\/span>zfs-auto-snap_hourly-2023-02-16-14h00\/<\/span>\r\n<span class=\"s1\">zfs-auto-snap_hourly-2023-02-15-20h00\/ <span class=\"Apple-converted-space\">\u00a0 <\/span>zfs-auto-snap_hourly-2023-02-16-15h00\/<\/span>\r\n<span class=\"s1\">zfs-auto-snap_hourly-2023-02-15-21h00\/ <span class=\"Apple-converted-space\">\u00a0 <\/span>zfs-auto-snap_monthly-2023-02-01-00h28\/<\/span>\r\n<span class=\"s1\">zfs-auto-snap_hourly-2023-02-15-22h00\/ <span class=\"Apple-converted-space\">\u00a0 <\/span>zfs-auto-snap_weekly-2023-02-12-00h14\/<\/span>\r\n<span class=\"s1\">zfs-auto-snap_hourly-2023-02-15-23h00\/<\/span><\/pre>\n<p><strong>Backups<\/strong><\/p>\n<p>ZFS uses snapshot replication to backup datasets.<\/p>\n<p>Snapshots operate on a block level.\u00a0 Once a snapshot is created, dataset blocks which change are copied to the snapshot (copy on write or COW).\u00a0 The original block is not changed.\u00a0 Snapshot initially are zero size and grow as the associated dataset changes.<\/p>\n<p>Root cron jobs create snapshots frequently (every 15 minutes), hourly, daily, weekly, and monthly.\u00a0 Only a certain number of snapshots are kept: 4 frequent, 24 hourly, 7 daily, 1 weekly, and 1 monthly.<\/p>\n<pre class=\"p1\"><span class=\"s1\">[root@trigger ~]# crontab -l<\/span>\r\n<span class=\"s1\">#Ansible: Frequent snapshots<\/span>\r\n<span class=\"s1\">15,30,45 * * * * \/bin\/bash -l -c '\/usr\/local\/bin\/zfs-auto-snapshot frequent 4'<\/span>\r\n<span class=\"s1\">#Ansible: Hourly snapshots<\/span>\r\n<span class=\"s1\">0 * * * * \/bin\/bash -l -c '\/usr\/local\/bin\/zfs-auto-snapshot hourly 24'<\/span>\r\n<span class=\"s1\">#Ansible: Daily snapshots<\/span>\r\n<span class=\"s1\">7 0 * * * \/bin\/bash -l -c '\/usr\/local\/bin\/zfs-auto-snapshot daily 7'<\/span>\r\n<span class=\"s1\">#Ansible: Weekly snapshots<\/span>\r\n<span class=\"s1\">14 0 * * 7 \/bin\/bash -l -c '\/usr\/local\/bin\/zfs-auto-snapshot weekly 1'<\/span>\r\n<span class=\"s1\">#Ansible: Monthly snapshots<\/span>\r\n<span class=\"s1\">28 0 1 * * \/bin\/bash -l -c '\/usr\/local\/bin\/zfs-auto-snapshot monthly 1'<\/span><\/pre>\n<p>The daily, weekly, and monthly snapshots are replicated (copied) to a backup server each night and are kept for a maximum of 30 days.\u00a0 Using snapshots, only the changed blocks are replicated. This make ZFS replication much more efficient than rsync.\u00a0 Rsync must find the files that have changed (which can take a significant amount of time) and then copy each modified file to the backup server.<\/p>\n<p><strong>Best Practices<\/strong><\/p>\n<p>Keep ZFS files systems below 80% of capacity.\u00a0 Performance degrades quickly as the usage increases beyond 80%.<\/p>\n<p><strong>Miscellaneous<\/strong><\/p>\n<p>Using the &#8216;df&#8217; command with ZFS datasets is confusing.\u00a0 It is not aware of ZFS snapshots, so as the amount of space used by snapshots increases, it sees the disk size decrease.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Introduction The Zed File System (ZFS) is an advanced filesystem that merges a file system, logical volume manager, and software RAID.\u00a0 It consists of the following components: A ZFS virtual device (vdev) is logical device in a zpool, which can be a physical device, a file, or a collection of devices. A ZFS zpool is&#8230;<\/p>\n","protected":false},"author":1556,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"spay_email":""},"tags":[],"acf":[],"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.vanderbilt.edu\/csb\/wp-json\/wp\/v2\/pages\/4531"}],"collection":[{"href":"https:\/\/www.vanderbilt.edu\/csb\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/www.vanderbilt.edu\/csb\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/www.vanderbilt.edu\/csb\/wp-json\/wp\/v2\/users\/1556"}],"replies":[{"embeddable":true,"href":"https:\/\/www.vanderbilt.edu\/csb\/wp-json\/wp\/v2\/comments?post=4531"}],"version-history":[{"count":11,"href":"https:\/\/www.vanderbilt.edu\/csb\/wp-json\/wp\/v2\/pages\/4531\/revisions"}],"predecessor-version":[{"id":4554,"href":"https:\/\/www.vanderbilt.edu\/csb\/wp-json\/wp\/v2\/pages\/4531\/revisions\/4554"}],"wp:attachment":[{"href":"https:\/\/www.vanderbilt.edu\/csb\/wp-json\/wp\/v2\/media?parent=4531"}],"wp:term":[{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.vanderbilt.edu\/csb\/wp-json\/wp\/v2\/tags?post=4531"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}