Author Archives: erailine

Tech Smorgasbord #3

My on-going reference series for interesting technology or projects which deserve further investigation, or for technical documentation (of one media format or another) that looks to be especially good reference material.

 

OpenSOC – The Open Security Operations Center

The OpenSOC project is a new initiative announced and driven by Cisco to provide an open source, extensible, and scalable advanced security analytics tool. This tool would allow any organization to create incident detection tools customized for their specific infrastructure components and their own security processes. This is a Big Data service leveraging multiple existing open source components like Hadoop, Hbase, Elastic Search, and MySQL.

Let’s Encrypt 

Let’s Encrypt is a service announced by the Internet Security Research Group (ISRG) to provide a free certificate authority for public use, and planned to officially launch in Q2 of 2015. The ultimate goal is well beyond “just” a free CA, however: they want to provide a certificate management framework that allows an administrator to run a single command to automatically generate and install a validated SSL certificate. This certificate management framework will include client software to run on any host (the current preview is written in Python), server software for the actual CA (a demo library written in node.js is currently available), and a new protocol used between client and server.

This protocol is called the Automated Certificate Management Environment (ACME), and a draft spec is available for review.  The protocol is expected to be submitted to the IETF as an official, open standard.

The ISRG is a California-based public benefit corporation sponsored by organizations like Cisco, the Electronic Frontier Foundation (EFF), Mozilla, Akamai, and IdenTrust. The Board of Directors for the ISRG include members from each of the sponsors (except for IdenTrust), as well as the University of Michigan, Stanford Law School, and CoreOS.

Schprokits

Schprokits is a new automation framework like Puppet, Chef, Ansible, Salt and others, but specifically targeted for networking. Their tag line is “Inspired By DevOps. Built For NetOps”. The company was founded by Jeremy Schulman, formerly a Global Solutions Architect at Juniper and the person responsible for the Puppet agent implementation for Juniper switches.

The product is still currently in stealth mode, but some information is getting out.  Ivan Pepelnjak had Jeremy on his podcast to discuss Schprokits, other configuration management systems, devops, and more. In addition, several posts by alpha/beta participants (posted with the apparent consent of Schprokits) are coming out. John Herbert has a couple posts, and Jason Edelman has a couple even longer and more detailed ones.

 

Tours of the Black Prompt: Clustered NetApp Data ONTAP – Part 5

The Tours of the Black Prompt series so far:

As promised, in this post we’re going to cover some more real-world examples of using Data ONTAP’s cluster shell.

Common Configuration

The following examples were actually used in our just-released deployment guide NVA-1110-FP-DEPLOY. This is the Deployment Guide for NVA-1110-FP: FlexPod Datacenter with NetApp All-Flash FAS and VMware Horizon (with View) for which we released the Design Guide back in October.

Note: in our official documentation examples we will normally use the most complete or verbose version of a command, rather than an abbreviated one, for clarity and easier reading. For even greater efficiency, an administrator can absolutely use some of the abbreviations or shortcuts discussed in Part 1.

One of the areas where you may need to perform a lot of repetitive operational tasks is networking. In an ONTAP cluster, you will typically have multiple physical interfaces per node, and then multiple nodes within the cluster. Modifying a setting on each of these individually is the definition of tedious as well as consuming significant time.

Specifying the correct flow control settings on your interfaces is one of these operations. In the NVA-1110-FP environment, we needed to modify eight interfaces each on two nodes (16 total interfaces). Without using queries and operators, this would have required 16 commands

cluster01::> network port modify -node  -port e0a -flowcontrol-admin none
Warning: Changing the network port settings will cause a several second interruption in carrier.
Do you want to continue? {y|n}: y
1 entry was modified.

cluster01::>  network port modify -node  -port e0b -flowcontrol-admin none
Warning: Changing the network port settings will cause a several second interruption in carrier.
Do you want to continue? {y|n}: y

1 entry was modified. 
cluster01::>  network port modify -node  -port e0c -flowcontrol-admin none
Warning: Changing the network port settings will cause a several second interruption in carrier.
Do you want to continue? {y|n}: y

1 entry was modified. 
cluster01::> network port modify -node  -port e0d -flowcontrol-admin none
Warning: Changing the network port settings will cause a several second interruption in carrier.
Do you want to continue? {y|n}: y

1 entry was modified. 
cluster01::>  network port modify -node  -port e0e -flowcontrol-admin none
Warning: Changing the network port settings will cause a several second interruption in carrier.
Do you want to continue? {y|n}: y

1 entry was modified. 
cluster01::>  network port modify -node  -port e0f -flowcontrol-admin none
Warning: Changing the network port settings will cause a several second interruption in carrier.
Do you want to continue? {y|n}: y

1 entry was modified.

cluster01::>  network port modify -node  -port e0g -flowcontrol-admin none
Warning: Changing the network port settings will cause a several second interruption in carrier.
Do you want to continue? {y|n}: y

1 entry was modified. 
cluster01::>  network port modify -node < -port e0h -flowcontrol-admin none
Warning: Changing the network port settings will cause a several second interruption in carrier.
Do you want to continue? {y|n}: y

1 entry was modified.

cluster01::> network port modify -node  -port e0a -flowcontrol-admin none
Warning: Changing the network port settings will cause a several second interruption in carrier.
Do you want to continue? {y|n}: y

1 entry was modified. 
cluster01::>  network port modify -node  -port e0b -flowcontrol-admin none
Warning: Changing the network port settings will cause a several second interruption in carrier.
Do you want to continue? {y|n}: y

1 entry was modified. 
cluster01::>  network port modify -node  -port e0c -flowcontrol-admin none
Warning: Changing the network port settings will cause a several second interruption in carrier.
Do you want to continue? {y|n}: y

1 entry was modified. 
cluster01::>  network port modify -node  -port e0d -flowcontrol-admin none
Warning: Changing the network port settings will cause a several second interruption in carrier.
Do you want to continue? {y|n}: y

1 entry was modified. 
cluster01::>  network port modify -node  -port e0e -flowcontrol-admin none
Warning: Changing the network port settings will cause a several second interruption in carrier.
Do you want to continue? {y|n}: y

1 entry was modified. 
cluster01::>  network port modify -node  -port e0f -flowcontrol-admin none
Warning: Changing the network port settings will cause a several second interruption in carrier.
Do you want to continue? {y|n}: y

1 entry was modified.
cluster01::>  network port modify -node  -port e0g -flowcontrol-admin none
Warning: Changing the network port settings will cause a several second interruption in carrier.
Do you want to continue? {y|n}: y

1 entry was modified. 
cluster01::>  network port modify -node  -port e0h -flowcontrol-admin none
Warning: Changing the network port settings will cause a several second interruption in carrier.
Do you want to continue? {y|n}: y

1 entry was modified.

But that’s crazy when we can do it with one command instead:

cluster01::> network port modify -node * -port e0a..e0h -flowcontrol-admin none
Warning: Changing the network port settings will cause a several second interruption in carrier.
Do you want to continue? {y|n}: y

16 entries were modified.

This will modify the range of ports (using the RANGE operator of “..”) starting with e0a and ending with e0h on every node (using the ALL operator of “*”) in the cluster. That’s 15 commands (and quite a bit of time/frustration) eliminated in this environment, but the savings are even more significant as you scale up: no changes to that command are necessary whether we have one node or 24 nodes. Think of that for a moment: 24 nodes with eight interfaces each would be 192 commands – and we’ve reduced that down to a single command. That’s the equivalent of a 99.5% dedupe ratio!

In addition, we are able to apply the same efficiencies to setting the correct MTU on the physical ports:

cluster01::> network port modify -node * -port a0a -mtu 9000
WARNING: Changing the network port settings will cause a serveral second interruption in carrier.
Do you want to continue? {y|n}: y
2 entries were modified.


cluster01::> network port modify -node * -port e0e..e0h -mtu 9000
WARNING: Changing the network port settings will cause a serveral second interruption in carrier.
Do you want to continue? {y|n}: y
16 entries were modified.

In this case, we needed to modify an existing interface group (a0a) on both controllers, and then also modify each of the physical ports on each node.

There were several other areas of the configuration where we were able save steps using the ALL operator.

We needed to enable Cisco Discovery Protocol (CDP) on all nodes in the cluster.

cluster01::> system node run -node * options cdpd.enable on
2 entries were acted on.

If we wanted to completely abbreviate this, it could have been as short as

cluster01::> ru * options cdpd.enable on
2 entries were acted on.

We also needed to modify AutoSupport settings on each node:

cluster01::> system node autosupport modify -node * -state enable -mail-hosts [SMTPserver] -transport https -support enable -to [admin_email]

2 entries were modified.

Finally, we modified some web services for all Storage Virtual Machines in the cluster:

cluster01::> vserver services web modify -name spi|ontapi|compat -vserver * -enabled true
4 entries were modified.

In this case, we used the OR operator of “|” (the pipe symbol) for the “-name” parameter and the ALL operator for the “-vserver” parameter.

Here are a few other examples where I’ve been able to save significant amounts of effort and time thanks to these built-in efficiencies of the cluster shell. You’ll notice that I also indulged in various shortcuts and abbreviations in these examples, but the cluster shell outputs the unabbreviated command to the console for clarity anyway (the line in parentheses just beneath each example command).

Other Networking Configuration

At one point I needed to fix some Logical Interface (LIF) names that I had mis-typed (mis-specified, really) on 16 different SVMs. One command fixed it:

cluster01::> net int rename -vse test_nas* -lif mgmt1 -newname nfs1
  (network interface rename)
16 entries were modified.

In another case, I needed to modify the home-ports for the management interfaces (35 in this instance) on all of the SVMs in a cluster. A single command again sufficed:

cluster01::> net int modify mgmt1 -home-port e0d -vse *
  (network interface modify)
35 entries were modified.

A special note for this example: specifying the home node was not required – the configuration for each LIF stayed on its current home-node and merely the home-port changed. When initially creating these LIFs, you do need to specify both home-node and home-port.

Reconfiguring the physical networking (VLAN interfaces, interface groups, etc.) happens fairly often as well. Here I needed to tear down an interface group, with a corresponding child VLAN interface, on each node in the cluster:

cluster01::> net port vlan delete -node * -vlan-name a0a-100
  (network port vlan delete)
4 entries were deleted.

cluster01::> net port ifgrp delete -node * -ifgrp a0a
  (network port ifgrp delete)
4 entries were deleted.

SAN Access Cleanup

Sometimes I just need to be able to clean up someLUNs, volumes, and SAN configuration while validating a solution:

cluster01::> lun offline -vserver svm_vmware -lun *
4 entries were acted on.

cluster01::> lun delete -vserver svm_vmware -lun *
4 entries were acted on.

cluster01::> igroup delete -vserver svm_vmware -igroup * 
1 entry was acted on.

cluster01::> volume offline -vserver svm_vmware -volume *

Error: This command does not support queries.

I kept the volume command in there as a good example of where, for safety’s sake, operators are not allowed. If a LUN is deleted in error, you will be able to easily recover using a volume Snapshot. It is significantly harder to recover a volume if it has been accidentally deleted.

SnapMirror Operations

Initializing multiple SnapMirrors at once:
cluster01::> snapmirror initialize {-destination-path tours*}
Operation is queued: snapmirror initialize of destination "tours:tours_sm1".                                                                                  
Operation is queued: snapmirror initialize of destination "tours:tours_sm2".                                                                                  
2 entries were acted on.
Modifying all SnapMirror relationships that currently are on a daily schedule to now have an 8hour schedule:
cluster01::> snapmirror modify {-schedule daily} -schedule 8hour 
Operation succeeded: snapmirror modify for the relationship with destination "tours:tours_sm1".                                                               
Operation succeeded: snapmirror modify for the relationship with destination "tours:tours_sm2".                                                               
2 entries were acted on.

The examples above are just a few of the ways that the built-in efficiencies of the cluster shell can dramatically save an administrator’s time, and turn a tedious experience into a fun one. (At least for some of us!)

Tours of the Black Prompt: Clustered NetApp Data ONTAP – Part 4

The Tours of the Black Prompt series so far:

Today we’re continuing our look at more advanced CLI operations for clustered Data ONTAP with the use of extended queries.

Extended Queries

You specify an extended query by enclosing it within curly braces: “{}”. Extended queries are similar to standard queries and use the same operators, but there are some distinct differences in how and when you use each type of query. Show commands can be run with standard queries, while modify or delete commands can be run with either standard or extended queries (but not both).  Extended queries can be used to filter on parameters that a modify command is capable of changing, whereas standard queries can only be used for parameters that do not get changed with a modify command (most commonly, these are name parameters associated with vservers, volumes, aggregates, etc.). Neither type of query can be used with create commands.

Let’s look at some examples of how you can use modify commands with standard and extended queries.

With standard queries you can use multiple instances within the same command. In this case, we’ll thin-provision all volumes on all SVMs except for vol0:

cdot_mba1::> volume modify -vserver * -volume !vol0 -space-guarantee none  

Volume modify successful on volume: black_root

Volume modify successful on volume: avol1

Volume modify successful on volume: citrixvol1

Volume modify successful on volume: dept1_share

Volume modify successful on volume: dept2_share

Volume modify successful on volume: testvol1

Volume modify successful on volume: tours_root

Volume modify successful on volume: user_share

Volume modify successful on volume: vmwarevol1
9 entries were modified.

We can accomplish something similar using a single extended query instead:

cdot_mba1::> volume modify {-volume !vol0} -space-guarantee none

Volume modify successful on volume: black_root

Volume modify successful on volume: avol1

Volume modify successful on volume: citrixvol1

Volume modify successful on volume: dept1_share

Volume modify successful on volume: dept2_share

Volume modify successful on volume: testvol1

Volume modify successful on volume: tours_root

Volume modify successful on volume: user_share

Volume modify successful on volume: vmwarevol1
9 entries were modified.

You can’t use both a standard query and an extended query within the same command, nor can you use more than one extended query in a single command:

cdot_mba1::> volume modify {-volume !vol0} -vserver * -space-guarantee none

Error: invalid argument "-vserver"

cdot_mba1::> volume modify {-volume !vol0} {-vserver *} -space-guarantee none

Error: "-vserver *" was not expected. Please specify -fieldname first.

As seen in our first extended query example above, an extended query must always be the first argument after the command before any other parameters are specified.

cdot_mba1::> volume modify -vserver tours {-volume !vol0} -space-guarantee none

Error: Volume name: The first character must be a letter or underscore.

An extended query can also not be used with subsequent filtering parameters (such as volume names, SVM names, etc.) even when specifying their values rather than using standard queries:

cdot_mba1::> volume modify {-volume !vol0} -vserver tours -space-guarantee none

Error: invalid argument "-vserver"

The inverse is also true: with extended queries, you don’t need to specify additional parameters that you may need to include when using standard queries:

cdot_mba1::> volume modify -volume !vol0  -space-guarantee none              

Error: Either specify all keys, or set at least one key to "*".

cdot_mba1::> volume modify -volume !vol0 -vserver tours -space-guarantee none

Volume modify successful on volume: avol1

Volume modify successful on volume: citrixvol1

Volume modify successful on volume: dept1_share

Volume modify successful on volume: dept2_share

Volume modify successful on volume: testvol1

Volume modify successful on volume: tours_root

Volume modify successful on volume: user_share

Volume modify successful on volume: vmwarevol1
8 entries were modified.

cdot_mba1::> volume modify {-volume !vol0}  -space-guarantee volume

Volume modify successful on volume: black_root

Volume modify successful on volume: avol1

Volume modify successful on volume: citrixvol1

Volume modify successful on volume: dept1_share

Volume modify successful on volume: dept2_share

Volume modify successful on volume: testvol1

Volume modify successful on volume: tours_root

Volume modify successful on volume: user_share

Volume modify successful on volume: vmwarevol1
9 entries were modified.

You can always tell if what you’re trying to do is going to work or not simply by trying to tab-complete. If the syntax is correct, you will be able to tab-complete commands within the extended query (between the curly braces) and after it; if your syntax is incorrect, you will not.

Putting It All Together

The combination of standard and extended queries provide a lot of power and efficiency for modifying storage configurations. Here’s just a few simple examples using the volume and lun commands:

Change the size of volumes matching a certain naming pattern

cdot_mba1::> volume modify {*share} -size +20M  

Volume modify successful on volume: dept1_share

Volume modify successful on volume: dept2_share

Volume modify successful on volume: user_share
3 entries were modified.

Change the size of volumes matching a certain current size

cdot_mba1::> volume modify {-size 40M} -size 20M

Volume modify successful on volume: dept1_share

Volume modify successful on volume: dept2_share

Volume modify successful on volume: user_share
3 entries were modified.

As noted in the System Administration Guide, you do need to be careful when modifying the same parameter that you are using for the queries as there can be unintended consequences as the command processes.

Modify the space guarantee settings for a volume based on its policy type

For example, you may want to do this to ensure that all of your virtualization volumes are thin provisioned.

cdot_mba1::> volume modify {-policy esx_exports} -space-guarantee none

Volume modify successful on volume: vmwarevol1
1 entry was modified.

Map a LUN to an Initiator Group based on its OS type

cdot_mba1::> lun map {-ostype vmware} -igroup esx
2 entries were acted on.

The examples we’ve been using in this series so far have all been fairly simple with the actual CLI output run and validated on a simulator on my laptop. In a forthcoming post I’ll go through some more sophisticated examples from real  projects and technical documentation that I’ve been worked on over the last year.

Stay tuned.

Mini-Review: Pebble Steel

I’m planning a much longer article (or possibly series of articles) on the tools I use for both my work and personal lives, and why it’s so important to be willing to invest in your tools. That may be obvious to a lot of people, but it hasn’t always been for me: I grew up being a “just make do” kind of person, and I was happy to invest an extra amount of time in something if it enabled me to save money out of pocket. My views on that have changed fairly substantially in recent years, and I’ve begun investing much more heavily in tools that enable me to be more efficient and to save myself time.

I’ve been looking at smart watches, and the Pebble in particular, for a couple of years now. I didn’t have a specific need, but I could see some interesting use cases for me personally and – well, I’m a geek and I like my toys.  But going back to paragraph #1 – I just kept evaluating the various options and didn’t buy one.

That changed recently after reading yet-another-smart-watch-article that rated the Pebble Steel as the best of the current crop of smart watches. It wasn’t an especially effusive article, which made it all the more persuasive to me: it came across as a realistic evaluation of the normal uses where the Pebble did well, uses that I was specifically interested in, while acknowledging some of its limitations.  Just a few days later, a coworker offered up his Pebble Steel on an internal distribution list for 50% off the normal price (and with both the leather & metal watchbands). The coincidence was too much for me to ignore, so I’m now the happy owner of a smart watch.

I had three main reasons to get the Pebble Steel:

  1. Notifications:
    • Number one by a mile, and this was the specific use mentioned in the article as one at which Pebble did well. I wear my phone in a belt case on my hip, and I’ve missed many IMs, text messages, or phone calls because I didn’t hear/feel the phone when it alerted me. Yes, I could turn up the ringtone higher, but I’ve never found the right middle ground between the I-almost-never-hear-it and oh-my-god-why-is-it-so-loud volume settings. Since getting the Pebble Steel, I don’t think I’ve missed a single important notification: job well done so far.
    • I’m constantly checking my phone for new information. If I see a new notification on my phone for one app (doesn’t matter which one), I’ll invariably check everything else.  This isn’t the fault of my phone or applications, of course, it’s mine. But I wanted to find a way to reduce the number of times I looked at my phone, which would then reduce the amount of time I spent looking at all of my other sources of distraction. By being able to see any important notifications (personal IM, text, phone, work email, and one personal email account) on my watch, I’ve reduced the number of times I’m opening my phone fairly significantly.
    • A side benefit also showed itself: in order to keep the amount of alerts being sent to my watch from being unbearable, I was forced to go through & re-organize my inbound email so that more of it is being categorized automatically, and I can only alert on things more likely to actually require my attention. In this case, the Pebble has improved my productivity both directly and indirectly – a very good thing indeed.
    • I actually seem to get the alerts faster on my watch than my computer (which is not surprising) or my watch (which is). It may not be every time, but there have been multiple occasions so far where my Pebble has alerted me before even the normal apps on my phone have displayed the new message.
  2. I haven’t worn a watch in years and I have come to find it annoying to constantly use my phone as a watch. I’d like a more convenient way to tell time instead – like I used to have when I had a ….watch. It’s also bad for me to use my phone as my watch because it’s another opportunity for me to start looking at all my sources of information and get distracted
  3. There are a few other bits of information that would be helpful to have readily available, like the weather, and the Pebble has quite a nice variety of applications to play with. I’ve only played with a few so far, but I think I’ll be able to find some good ones to further improve my productivity.

And the last, unnumbered reason: because it looks good, I’m a geek and I like my toys.

Tours of the Black Prompt: Clustered NetApp Data ONTAP – Part 3

The Tours of the Black Prompt series so far:

Today we’re getting into more advanced CLI operations for clustered Data ONTAP with field options, queries, and operators.

Field Options

In Part 2, we looked at enabling a session-level setting that would force the display of all possible fields for a given command’s tabular output, but there’s a lot more flexibility around which fields can be displayed than just a defaults-or-everything approach.

Every command provides a “-fields” parameter which allows you to granularly specify which fields will be displayed as part of the output.

cdot_mba1::> vol show
  (volume show)
Vserver   Volume       Aggregate    State      Type       Size  Available Used%
--------- ------------ ------------ ---------- ---- ---------- ---------- -----
cdot_mba1-01 
          vol0         aggr0        online     RW      851.5MB    223.0MB   73%
tours     avol1        aggr0        online     RW         20MB     2.52MB   87%
tours     citrixvol1   aggr1        online     RW         20MB    18.89MB    5%
tours     dept1_share  aggr1        online     RW         20MB    18.89MB    5%
tours     dept2_share  aggr1        online     RW         20MB    18.89MB    5%
tours     testvol1     aggr0        online     RW         20MB    18.89MB    5%
tours     tours_root   aggr0        online     RW         20MB    18.89MB    5%
tours     user_share   aggr1        online     RW         20MB    18.89MB    5%
tours     vmwarevol1   aggr1        online     RW         20MB    18.89MB    5%
9 entries were displayed.
cdot_mba1::> vol show -fields ?
  (volume show)
  vserver                                          Vserver Name
  volume                                           Volume Name
  aggregate                                        Aggregate Name
  size                                             Volume Size
  dsid                                             Volume Data Set ID
  msid                                             Volume Master Data Set ID
  state                                            Volume State
  type                                             Volume Type
  volume-style                                     Volume Style
  is-cluster-volume                                Is Cluster-Mode Volume
  is-constituent                                   Is Constituent Volume
  policy                                           Export Policy
  user                                             User ID
  group                                            Group ID
  security-style                                   Security Style
  unix-permissions                                 UNIX Permissions
  junction-path                                    Junction Path
  junction-path-source                             Junction Path Source
  junction-active                                  Junction Active
  junction-parent                                  Junction Parent Volume
  comment                                          Comment
  available                                        Available Size
  filesystem-size                                  Filesystem Size
  total                                            Total User-Visible Size
  used                                             Used Size
  percent-used                                     Used Percentage
  space-nearly-full-threshold-percent              Volume Nearly Full Threshold Percent
  space-full-threshold-percent                     Volume Full Threshold Percent
  max-autosize                                     Maximum Autosize (for flexvols only)
  autosize-increment                               Autosize Increment (for flexvols only)
  min-autosize                                     Minimum Autosize
  autosize-grow-threshold-percent                  Autosize Grow Threshold Percentage
  autosize-shrink-threshold-percent                Autosize Shrink Threshold Percentage
  autosize-mode                                    Autosize Mode
  autosize                                         Autosize Enabled (for flexvols only)
  files                                            Total Files (for user-visible data)
  files-used                                       Files Used (for user-visible data)
  space-guarantee                                  Space Guarantee Style
  space-guarantee-enabled                          Space Guarantee in Effect
  snapdir-access                                   Snapshot Directory Access Enabled
  percent-snapshot-space                           Space Reserved for Snapshots
  snapshot-space-used                              Snapshot Reserve Used
  snapshot-policy                                  Snapshot Policy
  create-time                                      Creation Time
  language                                         Language
  clone-volume                                     Clone Volume
  node                                             Node name
  nvfail                                           NVFAIL Option
  filesys-size-fixed                               Is File System Size Fixed
  extent-enabled                                   Extent Option
  overwrite-reserve                                Reserved Space for Overwrites
  fractional-reserve                               Fractional Reserve
  snapshot-clone-dependency                        Snapshot Cloning Dependency 
  space-mgmt-try-first                             Primary Space Management Strategy
  read-realloc                                     Read Reallocation Option
  is-inconsistent                                  Inconsistency in the File System
  is-quiesced-on-disk                              Is Volume Quiesced (On-Disk)
  is-quiesced-in-memory                            Is Volume Quiesced (In-Memory)
  is-sis-volume                                    Volume Contains Shared or Compressed Data
  sis-space-saved                                  Space Saved by Storage Efficiency
  sis-space-saved-percent                          Percentage Saved by Storage Efficiency
  dedupe-space-saved                               Space Saved by Deduplication
  dedupe-space-saved-percent                       Percentage Saved by Deduplication
  dedupe-space-shared                              Space Shared by Deduplication
  compression-space-saved                          Space Saved by Compression
  compression-space-saved-percent                  Percentage Space Saved by Compression
  block-type                                       Block Type
  flexcache-connection-status                      FlexCache Connection Status
  is-moving                                        Is Volume Moving
  hybrid-cache-eligibility                         Flash Pool Caching Eligibility
  hybrid-cache-write-caching-ineligibility-reason  Flash Pool Write Caching Ineligibility Reason
  is-managed-by-service                            Managed By Storage Service
  enable-snapdiff                                  Create Namespace Mirror Constituents For SnapDiff Use
  constituent-role                                 Constituent Volume Role
  qos-policy-group                                 QoS Policy Group Name
  is-volume-in-cutover                             Is Volume Move in Cutover Phase
  snapshot-count                                   Number of Snapshot Copies in the Volume

There are obviously a lot of different fields that we could display depending on the use case.

Perhaps we actually want to display fewer fields, not more:

cdot_mba1::> vol show -fields vserver,volume,aggregate,size,percent-used
  (volume show)
vserver      volume aggregate size    percent-used 
------------ ------ --------- ------- ------------ 
cdot_mba1-01 vol0   aggr0     851.5MB 73%          
tours        avol1  aggr0     20MB    87%          
tours        citrixvol1 
                    aggr1     20MB    5%           
tours        dept1_share 
                    aggr1     20MB    5%           
tours        dept2_share 
                    aggr1     20MB    5%           
tours        testvol1 
                    aggr0     20MB    5%           
tours        tours_root 
                    aggr0     20MB    5%           
tours        user_share 
                    aggr1     20MB    5%           
tours        vmwarevol1 
                    aggr1     20MB    5%           
9 entries were displayed.

Perhaps we want to look at information more relevant to a NAS configuration:

cdot_mba1::> vol show -fields vserver,volume,security-style,junction-path,language,user,group,policy,unix-permissions 
  (volume show)
vserver      volume policy user group security-style unix-permissions junction-path language 
------------ ------ ------ ---- ----- -------------- ---------------- ------------- -------- 
cdot_mba1-01 vol0   -      -    -     -              ------------     -             -        
tours        avol1  default 
                           0    0     unix           ---rwxr-xr-x     -             C.UTF-8  
tours        citrixvol1 
                    default 
                           0    0     unix           ---rwxr-xr-x     -             C.UTF-8  
tours        dept1_share 
                    default 
                           0    0     unix           ---rwxr-xr-x     -             C.UTF-8  
tours        dept2_share 
                    default 
                           0    0     unix           ---rwxr-xr-x     -             C.UTF-8  
tours        testvol1 
                    default 
                           0    0     unix           ---rwxr-xr-x     -             C.UTF-8  
tours        tours_root 
                    default 
                           0    0     unix           ---rwxr-xr-x     /             C.UTF-8  
tours        user_share 
                    default 
                           0    0     unix           ---rwxr-xr-x     -             C.UTF-8  
tours        vmwarevol1 
                    default 
                           0    0     unix           ---rwxr-xr-x     -             C.UTF-8  
9 entries were displayed.

One subtle thing to notice: fields are not displayed in the order in which you list them. The display order is not configurable by the user.

Field Filtering

The tab completion functionality of the cluster shell lends itself to filtering based on field just by its operation. As you tab through the parameters of a command you are continuously filtering out possible options through your choices. While this is explicit in a create or modify command, it’s equally true for displaying information.

cdot_mba1::> vol show -vserver 
    cdot_mba1    cdot_mba1-01 tours        

cdot_mba1::> vol show -vserver tours
  (volume show)
Vserver   Volume       Aggregate    State      Type       Size  Available Used%
--------- ------------ ------------ ---------- ---- ---------- ---------- -----
tours     avol1        aggr0        online     RW         20MB     2.52MB   87%
tours     citrixvol1   aggr1        online     RW         20MB    18.89MB    5%
tours     dept1_share  aggr1        online     RW         20MB    18.89MB    5%
tours     dept2_share  aggr1        online     RW         20MB    18.89MB    5%
tours     testvol1     aggr0        online     RW         20MB    18.89MB    5%
tours     tours_root   aggr0        online     RW         20MB    18.89MB    5%
tours     user_share   aggr1        online     RW         20MB    18.89MB    5%
tours     vmwarevol1   aggr1        online     RW         20MB    18.89MB    5%
8 entries were displayed.

Here, I’ve used [TAB] to first show me the possible values for that field, and then to filter the results based on my choice of “tours”.

I can also filter just by specifying which values I’m interested in. Perhaps I only want to see volumes owned by “tours” that are on aggr0:

cdot_mba1::> vol show -vserver tours -aggregate aggr0
  (volume show)
Vserver   Volume       Aggregate    State      Type       Size  Available Used%
--------- ------------ ------------ ---------- ---- ---------- ---------- -----
tours     avol1        aggr0        online     RW         20MB     2.50MB   87%
tours     testvol1     aggr0        online     RW         20MB    18.88MB    5%
tours     tours_root   aggr0        online     RW         20MB    18.88MB    5%
3 entries were displayed.

I can filter on this further by specifying fields that aren’t even displayed in the output, such as snapshot-count:

cdot_mba1::> vol show -vserver tours -aggregate aggr0 -snapshot-count 3
  (volume show)
Vserver   Volume       Aggregate    State      Type       Size  Available Used%
--------- ------------ ------------ ---------- ---- ---------- ---------- -----
tours     avol1        aggr0        online     RW         20MB     2.50MB   87%
tours     testvol1     aggr0        online     RW         20MB    18.88MB    5%
2 entries were displayed.

I can then take this and limit which fields are displayed so that I get exactly the data I’m looking for and nothing else:

cdot_mba1::> vol show -vserver tours -aggregate aggr0 -snapshot-count 3 -fields vserver,volume 
  (volume show)
vserver volume 
------- ------ 
tours   avol1  
tours   testvol1 
2 entries were displayed.

Queries and Operators

Since Data ONTAP is a Unix-like operating system, it’s been a common request that the CLI should support the use of typical Unix wildcard characters and patterns to provide more flexibility for the administrator – and with clustered Data ONTAP the cluster shell does. These patterns are considered to be “queries” and each wildcard or pattern is considered an “operator.” Using queries, administrators are able to more precisely specify the contents of the output for which they’re looking.

The following table is adapted from the one included in the System Administration Guide for clustered Data ONTAP. You’ll notice that most of the listed operators are the standard wildcards used in many systems.

Operator Description
* ALL operator: match all values
! NOT operator: do not match the specified value
| (pipe) OR operator: the pipe symbol acts as a separator between multiple patterns
.. RANGE operator: match any value within the range
< LESS THAN operator: match any value less than the specified one
> GREATER THAN operator: match any value more than the specified one
<= LESS-THAN-OR-EQUAL-TO operator: match any value equal to or less than the specified one
>= GREATER-THAN-OR-EQUAL-TO operator: match any value equal to or more than the specified one

Let’s provide some examples of each one:

ALL

cdot_mba1::> volume show -volume *
Vserver   Volume       Aggregate    State      Type       Size  Available Used%
--------- ------------ ------------ ---------- ---- ---------- ---------- -----
cdot_mba1-01 
          vol0         aggr0        online     RW      851.5MB    243.5MB   71%
tours     avol1        aggr0        online     RW         20MB     2.66MB   86%
tours     citrixvol1   aggr1        online     RW         20MB    18.89MB    5%
tours     dept1_share  aggr1        online     RW         20MB    18.89MB    5%
tours     dept2_share  aggr1        online     RW         20MB    18.89MB    5%
tours     testvol1     aggr0        online     RW         20MB    18.89MB    5%
tours     tours_root   aggr0        online     RW         20MB    18.89MB    5%
tours     user_share   aggr1        online     RW         20MB    18.89MB    5%
tours     vmwarevol1   aggr1        online     RW         20MB    18.89MB    5%
9 entries were displayed.

The ALL operator can be used with partial values regardless of placement within that value:

cdot_mba1::> volume show -volume *vol1
Vserver   Volume       Aggregate    State      Type       Size  Available Used%
--------- ------------ ------------ ---------- ---- ---------- ---------- -----
tours     avol1        aggr0        online     RW         20MB     2.66MB   86%
tours     citrixvol1   aggr1        online     RW         20MB    18.89MB    5%
tours     testvol1     aggr0        online     RW         20MB    18.89MB    5%
tours     vmwarevol1   aggr1        online     RW         20MB    18.89MB    5%
4 entries were displayed.

cdot_mba1::> vol show -volume dept*_share
  (volume show)
Vserver   Volume       Aggregate    State      Type       Size  Available Used%
--------- ------------ ------------ ---------- ---- ---------- ---------- -----
tours     dept1_share  aggr1        online     RW         20MB    18.89MB    5%
tours     dept2_share  aggr1        online     RW         20MB    18.89MB    5%
2 entries were displayed.

With this flexibility, you can filter and match on more than one value in a specified field, and not just literally all values.

NOT

cdot_mba1::> vol show -volume !vol0
  (volume show)
Vserver   Volume       Aggregate    State      Type       Size  Available Used%
--------- ------------ ------------ ---------- ---- ---------- ---------- -----
tours     avol1        aggr0        online     RW         20MB     2.66MB   86%
tours     citrixvol1   aggr1        online     RW         20MB    18.89MB    5%
tours     dept1_share  aggr1        online     RW         20MB    18.89MB    5%
tours     dept2_share  aggr1        online     RW         20MB    18.89MB    5%
tours     testvol1     aggr0        online     RW         20MB    18.89MB    5%
tours     tours_root   aggr0        online     RW         20MB    18.89MB    5%
tours     user_share   aggr1        online     RW         20MB    18.89MB    5%
tours     vmwarevol1   aggr1        online     RW         20MB    18.89MB    5%
8 entries were displayed.

In this example, there was a single volume we wanted to exclude from the output, but what if we wanted to exclude more than one? Just combine operators:

cdot_mba1::> volume show -volume !*vol1
Vserver   Volume       Aggregate    State      Type       Size  Available Used%
--------- ------------ ------------ ---------- ---- ---------- ---------- -----
cdot_mba1-01 
          vol0         aggr0        online     RW      851.5MB    237.0MB   72%
tours     dept1_share  aggr1        online     RW         20MB    18.89MB    5%
tours     dept2_share  aggr1        online     RW         20MB    18.89MB    5%
tours     tours_root   aggr0        online     RW         20MB    18.89MB    5%
tours     user_share   aggr1        online     RW         20MB    18.89MB    5%
5 entries were displayed.

OR

cdot_mba1::> vol show -volume citrix* | vmware*
  (volume show)
Vserver   Volume       Aggregate    State      Type       Size  Available Used%
--------- ------------ ------------ ---------- ---- ---------- ---------- -----
tours     citrixvol1   aggr1        online     RW         20MB    18.89MB    5%
tours     vmwarevol1   aggr1        online     RW         20MB    18.89MB    5%
2 entries were displayed.

RANGE

Unsurprisingly, we can’t use operators like RANGE, LESS THAN, GREATER THAN, etc. on free-text fields like volume names. Continuing with our examples, let’s start filtering based on the “-percent-used” field and find our most under-utilized volumes:

cdot_mba1::> vol show -percent-used 1..10
  (volume show)
Vserver   Volume       Aggregate    State      Type       Size  Available Used%
--------- ------------ ------------ ---------- ---- ---------- ---------- -----
tours     citrixvol1   aggr1        online     RW         20MB    18.88MB    5%
tours     dept1_share  aggr1        online     RW         20MB    18.88MB    5%
tours     dept2_share  aggr1        online     RW         20MB    18.88MB    5%
tours     testvol1     aggr0        online     RW         20MB    18.88MB    5%
tours     tours_root   aggr0        online     RW         20MB    18.88MB    5%
tours     user_share   aggr1        online     RW         20MB    18.88MB    5%
tours     vmwarevol1   aggr1        online     RW         20MB    18.88MB    5%
7 entries were displayed.

We can do a similar search to find volumes of a certain size range:

cdot_mba1::> vol show -size 500M..1000M
  (volume show)
Vserver   Volume       Aggregate    State      Type       Size  Available Used%
--------- ------------ ------------ ---------- ---- ---------- ---------- -----
cdot_mba1-01 
          vol0         aggr0        online     RW      851.5MB    227.1MB   73%

Note the “M” in that last example: when using fields that display different units of measurement, you need to specify that unit in your RANGE operation.

cdot_mba1::> vol show -size 500..1000
  (volume show)
There are no entries matching your query.

cdot_mba1::> vol show -size 1024k..30000k
  (volume show)
Vserver   Volume       Aggregate    State      Type       Size  Available Used%
--------- ------------ ------------ ---------- ---- ---------- ---------- -----
tours     avol1        aggr0        online     RW         20MB     2.56MB   87%
tours     citrixvol1   aggr1        online     RW         20MB    18.88MB    5%
tours     dept1_share  aggr1        online     RW         20MB    18.88MB    5%
tours     dept2_share  aggr1        online     RW         20MB    18.88MB    5%
tours     testvol1     aggr0        online     RW         20MB    18.88MB    5%
tours     tours_root   aggr0        online     RW         20MB    18.88MB    5%
tours     user_share   aggr1        online     RW         20MB    18.88MB    5%
tours     vmwarevol1   aggr1        online     RW         20MB    18.88MB    5%
8 entries were displayed.

Let’s try to get the same information as above, but using LESS THAN or MORE THAN instead of a RANGE operation.

LESS THAN

cdot_mba1::> vol show -percent-used <10
  (volume show)
Vserver   Volume       Aggregate    State      Type       Size  Available Used%
--------- ------------ ------------ ---------- ---- ---------- ---------- -----
tours     citrixvol1   aggr1        online     RW         20MB    18.89MB    5%
tours     dept1_share  aggr1        online     RW         20MB    18.89MB    5%
tours     dept2_share  aggr1        online     RW         20MB    18.89MB    5%
tours     testvol1     aggr0        online     RW         20MB    18.89MB    5%
tours     tours_root   aggr0        online     RW         20MB    18.88MB    5%
tours     user_share   aggr1        online     RW         20MB    18.89MB    5%
tours     vmwarevol1   aggr1        online     RW         20MB    18.89MB    5%
7 entries were displayed.

GREATER THAN

cdot_mba1::> vol show -size >500M
  (volume show)
Vserver   Volume       Aggregate    State      Type       Size  Available Used%
--------- ------------ ------------ ---------- ---- ---------- ---------- -----
cdot_mba1-01 
          vol0         aggr0        online     RW      851.5MB    225.6MB   73%

LESS-THAN-OR-EQUAL-TO

cdot_mba1::> vol show -size <=20M 
  (volume show)
Vserver   Volume       Aggregate    State      Type       Size  Available Used%
--------- ------------ ------------ ---------- ---- ---------- ---------- -----
tours     avol1        aggr0        online     RW         20MB     2.46MB   87%
tours     citrixvol1   aggr1        online     RW         20MB    18.89MB    5%
tours     dept1_share  aggr1        online     RW         20MB    18.89MB    5%
tours     dept2_share  aggr1        online     RW         20MB    18.89MB    5%
tours     testvol1     aggr0        online     RW         20MB    18.89MB    5%
tours     tours_root   aggr0        online     RW         20MB    18.89MB    5%
tours     user_share   aggr1        online     RW         20MB    18.89MB    5%
tours     vmwarevol1   aggr1        online     RW         20MB    18.89MB    5%
8 entries were displayed.

To demonstrate that this is really being evaluated differently than just LESS THAN:

cdot_mba1::> vol show -size <20M 
  (volume show)
There are no entries matching your query.

GREATER-THAN-OR-EQUAL-TO

cdot_mba1::> vol show -size >=20M
  (volume show)
Vserver   Volume       Aggregate    State      Type       Size  Available Used%
--------- ------------ ------------ ---------- ---- ---------- ---------- -----
cdot_mba1-01
          vol0         aggr0        online     RW      851.5MB    224.3MB   73%
tours     avol1        aggr0        online     RW         20MB     2.52MB   87%
tours     citrixvol1   aggr1        online     RW         20MB    18.89MB    5%
tours     dept1_share  aggr1        online     RW         20MB    18.89MB    5%
tours     dept2_share  aggr1        online     RW         20MB    18.89MB    5%
tours     testvol1     aggr0        online     RW         20MB    18.89MB    5%
tours     tours_root   aggr0        online     RW         20MB    18.89MB    5%
tours     user_share   aggr1        online     RW         20MB    18.89MB    5%
tours     vmwarevol1   aggr1        online     RW         20MB    18.89MB    5%
9 entries were displayed.

Putting It All Together

Even though these examples are rather trivial, the filtering and querying capabilities in the CLI can be very powerful when they are combined.

For example, what if I want to find all volumes which are more than 50% utilized with a certain minimum number of snapshots – except for those belonging to a particular SVM or on a particular aggregate?

cdot_mba1::> vol show -volume * -vserver !cdot_mba1-01  -aggregate !aggr1 -percent-used >50 -snapshot-count >=3
  (volume show)
Vserver   Volume       Aggregate    State      Type       Size  Available Used%
--------- ------------ ------------ ---------- ---- ---------- ---------- -----
tours     avol1        aggr0        online     RW         20MB     2.45MB   87%

As you can see, you can run very detailed queries that are still simple to construct and easy to understand.

In even moderately sized environments, effective use of the capabilities within the clustered Data ONTAP shell can provide significant administrator efficiencies.

Next up: Extended queries.