• Netapp Deduplication – quick setup guide

    Deduplication refers to the elimination of redundant data in the storage. In the deduplication process, duplicate data is deleted, leaving only one copy of the data to be stored. However, indexing of all data is still retained should that data ever be required. De-duplication is able to reduce the required storage capacity since only the unique data is stored. 
    Netapp supports deduplication where only unique blocks in the flex volume is stored and it creates a small amount of additional metadata in the dedup process. The NetApp deduplication technology allows duplicate 4KB blocks anywhere in the flexible volume to be deleted and stores a unique one.
    The core enabling technology of deduplication is fingerprints. These are unique digital signatures for every 4KB data block in the flexible volume.
    When deduplication runs for the first time on a flexible volume with existing data, it scans the blocks in the flexible volume and creates a fingerprint database, which contains a sorted list of all fingerprints for used blocks in the flexible volume. After the fingerprint file is created, fingerprints are checked for duplicates and if found, first a byte-by-byte comparison of the blocks is done to make sure that the blocks are indeed identical. If they are found to be identical, the block’s pointer is updated to the already existing data block and the duplicate data block is released and inode is updated.
     Netapp Deduplication commands:

    1. Enable dedup (asis) license.

      fractal-design> sis on /vol/demovol

    2. If you have a new flex volume which was just created, follow this step to enable ASIS deduplication

      fractal-design> sis on /vol/demovol
      Deduplication for “/vol/demovol” is enabled.
      Already existing data could be processed by running “sis start -s /vol/demovol”

    3. If you have already existing flex volume with data in it, follow this step.

      fractal-design> sis start -s /vol/demovol

    4. Checking the status of deduplication.
      fractal-design> vol status demovol
      Volume          State   Status          Options
      VolArchive      online  raid_dp, flex   nosnap=on
      Containing aggregate: ‘aggr0’

      fractal-design> sis status /vol/demovol
      Path            State   Status      Progress
      /vol/demovol    Enabled Idle        Idle for 00:02:12
    5. Check the storage space saved due to deduplication
      fractal-design> df -s /vol/demovol
      Filesystem      used    saved   %saved
      /vol/demovol/   9316052 0       0%

    6. If you have to run deduplication at a later point of time on this volume, just do a “sis start /vol/demovol”.
    7. The sis can be scheduled using “sis config” command.
    8. Done.

    More netapp blog posts at :

  • OCFS2 cluster – quick setup guide

    OCFS2 is a POSIX-compliant shared-disk cluster file system for Linux capable of providing both high performance and high availability.  Cluster-aware applications can make use of parallel I/O for higher performance. OCFS2 is mostly used to host Oracle Real application clusters (RAC) database on Linux clusters. 
    The below steps shows how to create ocfs2 filesystem on top a multipath’d SAN lun and mount it on Linux clusters.
    1. Identify the nodes that will be part of your cluster.
    2. Export/Zone the LUNs on the SAN end and check whether they are accessible on all the hosts of the cluster. (fdisk -l or multipath -ll)
    3. If you need multipathing, configure multipath and the multipathing policy based on your requirement. For Linux multipath setup, refer Redhat’s multipath guide.
    4. Create OCFS2 configuration file (/etc/ocfs2/cluster.conf) on all the cluster nodes.
    5. The example presents you a sample cluster.conf for a 3 node pool. If you have heartbeat IP configured on these cluster nodes, use the heartbeat IP for ocfs2 cluster communication and specify the hostname (without FQDN). Copy the same file to all the hosts in the cluster.

      [[email protected] ~]# cat /etc/ocfs2/cluster.conf
              ip_port = 7777
              ip_address =
              number = 0
              name = oracle-cluster-1
              cluster = ocfs2

              ip_port = 7777
              ip_address =
              number = 1
              name = oracle-cluster-2
              cluster = ocfs2

              ip_port = 7777
              ip_address =
              number = 2
              name = oracle-cluster-3
              cluster = ocfs2

              node_count = 3
              name = ocfs2

      [[email protected] ~]#

    6. On each node check the status of OCFS2 cluster service and stop “o2cb” if the service is already running.

      # service o2cb status
      # service o2cb stop

    7. On each node, load the OCFS2 module.

      # service o2cb load

    8. Make the OCFS2 service online on all the nodes.

      # service o2cb online
    9. Now your OCFS2 cluster is ready.
    10. Format the SAN lun device from any one of the cluster node.

      # mkfs.ocfs2 -b 4k -C 32k -L oraclerac /dev/mapper/mpath0

      -b : Block size (values are 512, 1K, 2K and 4K bytes per block)
      -C : Cluster size (values are 4K, 8K, 16K, 32K, 64K, 128K, 256K, 512K and 1M)
      -L : Label

      Note : Replace /dev/mapper/mpath0 with your device name.
    11. Update /etc/fstab on all the nodes in the cluster with the mount point.

      Like : /dev/mapper/mpath0 /u01 ocfs2 _netdev 0 0

    12. Mount the /u01 volume using mount command

      # mount /u01

    13. Enable ocfs and o2b service at runlevel 3.

      # chkconfig –level 345 o2cb on ; chkconfig –level 345 ocfs2 on

    14. The /u01 repository setup on a SAN Lun is done.
    15. You can now configure Oracle RAC database on this filesystem.
  • Netapp Snapmirror Setup Guide

    Snapmirror is an licensed utility in Netapp to do data transfer across filers. Snapmirror works at Volume level or Qtree level. Snapmirror is mainly used for disaster recovery and replication.

    Snapmirrror needs a source and destination filer. (When source and destination are the same filer, the snapmirror happens on local filer itself.  This is when you have to replicate volumes inside a filer. If you need DR capabilities of a volume inside a filer, you have to try syncmirror ).

    Synchronous SnapMirror is a SnapMirror feature in which the data on one system is replicated on another system at, or near, the same time it is written to the first system. Synchronous SnapMirror synchronously replicates data between single or clustered storage systems situated at remote sites using either an IP or a Fibre Channel connection. Before Data ONTAP saves data to disk, it collects written data in NVRAM. Then, at a point in time called a consistency point, it sends the data to disk.

    When the Synchronous SnapMirror feature is enabled, the source system forwards data to the destination system as it is written in NVRAM. Then, at the consistency point, the source system sends its data to disk and tells the destination system to also send its data to disk.

    This guides you quickly through the Snapmirror setup and commands.

    1) Enable Snapmirror on source and destination filer

    source-filer> options snapmirror.enable
    snapmirror.enable            on
    source-filer> options snapmirror.access
    snapmirror.access            legacy

    2) Snapmirror Access

    Make sure destination filer has snapmirror access to the source filer. The snapmirror filer’s name or IP address should be in /etc/snapmirror.allow. Use wrfile to add entries to /etc/snapmirror.allow.

    source-filer> rdfile /etc/snapmirror.allow

    3) Initializing a Snapmirror relation

    Volume snapmirror : Create a destination volume on destination netapp filer, of same size as source volume or greater size. For volume snapmirror, the destination volume should be in restricted mode. For example, let us consider we are snapmirroring a 100G volume – we create the destination volume and make it restricted.

    destination-filer> vol create demo_destination aggr01 100G
    destination-filer> vol restrict demo_destination

    Volume SnapMirror creates a Snapshot copy before performing the initial transfer. This copy is referred to as the baseline Snapshot copy. After performing an initial transfer of all data in the volume, VSM (Volume SnapMirror) sends to the destination only the blocks that have changed since the last successful replication. When SnapMirror performs an update transfer, it creates another new Snapshot copy and compares the changed blocks. These changed blocks are sent as part of the update transfer.

    Snapmirror is always destination filer driven. So the snapmirror initialize has to be done on destination filer. The below command starts the baseline transfer.

    destination-filer> snapmirror initialize -S source-filer:demo_source destination-filer:demo_destination
    Transfer started.
    Monitor progress with ‘snapmirror status’ or the snapmirror log.

    Qtree Snapmirror : For qtree snapmirror, you should not create the destination qtree. The snapmirror command automatically creates the destination qtree. So just volume creation of required size is good enough.

    Qtree SnapMirror determines changed data by first looking through the inode file for inodes that have changed and changed inodes of the interesting qtree for changed data blocks. The SnapMirror software then transfers only the new or changed data blocks from this Snapshot copy that is associated with the designated qtree. On the destination volume, a new Snapshot copy is then created that contains a complete point-in-time copy of the entire destination volume, but that is associated specifically with the particular qtree that has been replicated.

    destination-filer> snapmirror initialize -S source-filer:/vol/demo1/qtree destination-filer:/vol/demo1/qtree
    Transfer started.
    Monitor progress with ‘snapmirror status’ or the snapmirror log.

    4) Monitoring the status : Snapmirror data transfer status can be monitored either from source or destination filer. Use “snapmirror status” to check the status.

    destination-filer> snapmirror status
    Snapmirror is on.
    Source                          Destination                          State          Lag Status
    source-filer:demo_source        destination-filer:demo_destination   Uninitialized  –   Transferring (1690 MB done)
    source-filer:/vol/demo1/qtree   destination-filer:/vol/demo1/qtree   Uninitialized  –   Transferring (32 MB done)

    5) Snapmirror schedule : This is the schedule used by the destination filer for updating the mirror. It informs the SnapMirror scheduler when transfers will be initiated. The schedule field can either contain the word sync to specify synchronous mirroring or a cron-style specification of when to update the mirror. The cronstyle schedule contains four space-separated fields.

    If you want to sync the data on a scheduled frequency, you can set that in destination filer’s /etc/snapmirror.conf . The time settings are similar to Unix cron. You can set a synchronous snapmirror schedule in /etc/snapmirror.conf by adding “sync” instead of the cron style frequency.

    destination-filer> rdfile /etc/snapmirror.conf
    source-filer:demo_source        destination-filer:demo_destination – 0 * * *  # This syncs every hour
    source-filer:/vol/demo1/qtree   destination-filer:/vol/demo1/qtree – 0 21 * * # This syncs every 9:00 pm

    6) Other Snapmirror commands

    • To break snapmirror relation – do snapmirror quiesce and snapmirror break.
    • To update snapmirror data  – do snapmirror update
    • To resync a broken relation – do snapmirror resync.
    • To abort a relation – do snapmirror abort

    Snapmirror do provide multipath support. More than one physical path between a source and a destination system might be desired for a mirror relationship. Multipath support allows SnapMirror traffic to be load balanced between these paths and provides for failover in the event of a network outage.

    To read how to tune the performance & speed of the netapp snapmirror or snapvault replication transfers and adjust the transfer bandwidth , go to Tuning Snapmirror & Snapvault replication data transfer speed

  • ZFS : Basic administration guide

    ZFS is a combined file system and logical volume manager designed by Oracle/Sun Microsystems. The features of ZFS include support for high storage capacities, integration of the concepts of filesystem and volume management, snapshots and copy-on-write clones, continuous integrity checking and automatic repair, RAID-Z and native NFSv4 ACLs.

    This ZFS guide provides an overview of ZFS and its administration commands that will be helpful for beginners.

    ZFS Pool:ZFS organizes physical devices into logical pools called storage pools. Both individual disks and array logical unit numbers (LUNs) that are visible to the operating system can be included in a ZFS pools. These pools can be created as disks striped together with no redundancy (RAID 0), mirrored disks (RAID 1), striped mirror sets (RAID 1 + 0), or striped with parity (RAID Z). Additional disks can be added to pools at any time but they must be added with the same RAID level.


    ZFS Filesystem : ZFS offers a POSIX-compliant file system interface to the Solaris/OpenSolaris operating system. ZFS file systems must be built in one and only one storage pool, but a storage pool may have more than one defined file system. ZFS file systems are managed & mounted through /etc/vfstab file. The common way to mount a ZFS file system is to simply define it against a pool. All defined ZFS file systems automatically mount at boot time unless otherwise configured.Here are the basic commands for getting started with ZFS.

    Creating Storage pool using “zpool create” :

    bash-3.00# zpool create demovol raidz c2t1d0 c2t2d0 
    bash-3.00# zpool status 
      pool: demovol 
    state: ONLINE 
    scrub: none requested 
            NAME         STATE     READ WRITE CKSUM 
            demovol      ONLINE       0     0     0 
              raidz1     ONLINE       0     0     0 
                c2t1d0   ONLINE       0     0     0 
                c2t2d0   ONLINE       0     0     0
    errors: No known data errors 

    “zfs list” will give the details of the pool and other zfs filesytems.

    bash-3.00# zfs list 
    NAME                   USED  AVAIL  REFER  MOUNTPOINT 
    demovol               1.00G  900G  38.1K  /demovol 

    Creating File Systems : “zfs create” is used to create zfs filesytem.

    bash-3.00# zfs create demovol/testing 
    bash-3.00# zfs list 
    NAME                   USED  AVAIL  REFER  MOUNTPOINT 
    demovol               1.00G  900G  38.1K  /demovol 
    demovol/testing       32.6K  900G  32.6K  /demovol/testing 
    bash-3.00# ls /dev/zvol/dsk/demovol — This should show you the disk file.

    Setting Quota for the filesytem : Until Quota is set, the filesytem shows the total available space of the containter zfs pool.

    bash-3.00# zfs set quota=10G emspool3/testing 
    bash-3.00# zfs list 
    NAME                     USED  AVAIL  REFER  MOUNTPOINT 
    demovol                 1.00G  900G   39.9K  /demovol 
    demovol/testing         32.6K  10.0G  32.6K  /demovol/testing

    Creating a snapshot :

    bash-3.00# zfs snapshot demovol/[email protected] 
    bash-3.00# zfs list 
    NAME                     USED  AVAIL  REFER  MOUNTPOINT 
    demovol                 1.00G  900G   39.9K  /demovol 
    demovol/testing         32.6K  10.0G  32.6K  /demovol/testing 
    demovol/[email protected]      0      –  32.6K  - 

    Get all properties of a ZFS filesytem :

    bash-3.00# zfs get all demovol/testing 
    NAME             PROPERTY         VALUE                  SOURCE 
    demovol/testing  type             filesystem             - 
    demovol/testing  creation         Mon Feb  9  9:05 2009  - 
    demovol/testing  used             32.6K                  - 
    demovol/testing  available        10.0G                  - 
    demovol/testing  referenced       32.6K                  - 
    demovol/testing  compressratio    1.00x                  - 
    demovol/testing  mounted          yes                    - 
    demovol/testing  quota            10G                    local 
    demovol/testing  reservation      none                   default 
    demovol/testing  recordsize       128K                   default 
    demovol/testing  mountpoint       /demovol/testing       default 

    Cloning a ZFS filesystem from a snapshot :

    bash-3.00# zfs clone demovol/[email protected] demovol/clone22 
    bash-3.00# zfs list 
    NAME                     USED  AVAIL  REFER  MOUNTPOINT 
    demovol                 1.00G  900G   39.9K  /demovol 
    demovol/clone22             0  900G   32.6K  /demovol/clone22 
    demovol/testing         32.6K  10.0G  32.6K  /demovol/testing 
    demovol/[email protected]      0      –  32.6K  - 

    Performance IO Monitoring the ZFS storage pool:

    bash-3.00# zpool  iostat 1 
                   capacity     operations    bandwidth 
    pool         used  avail   read  write   read  write 
    ———-  —–  —–  —–  —–  —–  —– 
    demovol     4.95M  900G      0      0      0     35 
    demovol     4.95M  900G      0      0      0      0 
    demovol     4.95M  900G      0      0      0      0 
    demovol     4.95M  900G      0      0      0      0
Posts navigation