Tours of the Black Prompt: Clustered NetApp Data ONTAP – Part 3

The Tours of the Black Prompt series so far:

Today we’re getting into more advanced CLI operations for clustered Data ONTAP with field options, queries, and operators.

Field Options

In Part 2, we looked at enabling a session-level setting that would force the display of all possible fields for a given command’s tabular output, but there’s a lot more flexibility around which fields can be displayed than just a defaults-or-everything approach.

Every command provides a “-fields” parameter which allows you to granularly specify which fields will be displayed as part of the output.

cdot_mba1::> vol show
  (volume show)
Vserver   Volume       Aggregate    State      Type       Size  Available Used%
--------- ------------ ------------ ---------- ---- ---------- ---------- -----
cdot_mba1-01 
          vol0         aggr0        online     RW      851.5MB    223.0MB   73%
tours     avol1        aggr0        online     RW         20MB     2.52MB   87%
tours     citrixvol1   aggr1        online     RW         20MB    18.89MB    5%
tours     dept1_share  aggr1        online     RW         20MB    18.89MB    5%
tours     dept2_share  aggr1        online     RW         20MB    18.89MB    5%
tours     testvol1     aggr0        online     RW         20MB    18.89MB    5%
tours     tours_root   aggr0        online     RW         20MB    18.89MB    5%
tours     user_share   aggr1        online     RW         20MB    18.89MB    5%
tours     vmwarevol1   aggr1        online     RW         20MB    18.89MB    5%
9 entries were displayed.
cdot_mba1::> vol show -fields ?
  (volume show)
  vserver                                          Vserver Name
  volume                                           Volume Name
  aggregate                                        Aggregate Name
  size                                             Volume Size
  dsid                                             Volume Data Set ID
  msid                                             Volume Master Data Set ID
  state                                            Volume State
  type                                             Volume Type
  volume-style                                     Volume Style
  is-cluster-volume                                Is Cluster-Mode Volume
  is-constituent                                   Is Constituent Volume
  policy                                           Export Policy
  user                                             User ID
  group                                            Group ID
  security-style                                   Security Style
  unix-permissions                                 UNIX Permissions
  junction-path                                    Junction Path
  junction-path-source                             Junction Path Source
  junction-active                                  Junction Active
  junction-parent                                  Junction Parent Volume
  comment                                          Comment
  available                                        Available Size
  filesystem-size                                  Filesystem Size
  total                                            Total User-Visible Size
  used                                             Used Size
  percent-used                                     Used Percentage
  space-nearly-full-threshold-percent              Volume Nearly Full Threshold Percent
  space-full-threshold-percent                     Volume Full Threshold Percent
  max-autosize                                     Maximum Autosize (for flexvols only)
  autosize-increment                               Autosize Increment (for flexvols only)
  min-autosize                                     Minimum Autosize
  autosize-grow-threshold-percent                  Autosize Grow Threshold Percentage
  autosize-shrink-threshold-percent                Autosize Shrink Threshold Percentage
  autosize-mode                                    Autosize Mode
  autosize                                         Autosize Enabled (for flexvols only)
  files                                            Total Files (for user-visible data)
  files-used                                       Files Used (for user-visible data)
  space-guarantee                                  Space Guarantee Style
  space-guarantee-enabled                          Space Guarantee in Effect
  snapdir-access                                   Snapshot Directory Access Enabled
  percent-snapshot-space                           Space Reserved for Snapshots
  snapshot-space-used                              Snapshot Reserve Used
  snapshot-policy                                  Snapshot Policy
  create-time                                      Creation Time
  language                                         Language
  clone-volume                                     Clone Volume
  node                                             Node name
  nvfail                                           NVFAIL Option
  filesys-size-fixed                               Is File System Size Fixed
  extent-enabled                                   Extent Option
  overwrite-reserve                                Reserved Space for Overwrites
  fractional-reserve                               Fractional Reserve
  snapshot-clone-dependency                        Snapshot Cloning Dependency 
  space-mgmt-try-first                             Primary Space Management Strategy
  read-realloc                                     Read Reallocation Option
  is-inconsistent                                  Inconsistency in the File System
  is-quiesced-on-disk                              Is Volume Quiesced (On-Disk)
  is-quiesced-in-memory                            Is Volume Quiesced (In-Memory)
  is-sis-volume                                    Volume Contains Shared or Compressed Data
  sis-space-saved                                  Space Saved by Storage Efficiency
  sis-space-saved-percent                          Percentage Saved by Storage Efficiency
  dedupe-space-saved                               Space Saved by Deduplication
  dedupe-space-saved-percent                       Percentage Saved by Deduplication
  dedupe-space-shared                              Space Shared by Deduplication
  compression-space-saved                          Space Saved by Compression
  compression-space-saved-percent                  Percentage Space Saved by Compression
  block-type                                       Block Type
  flexcache-connection-status                      FlexCache Connection Status
  is-moving                                        Is Volume Moving
  hybrid-cache-eligibility                         Flash Pool Caching Eligibility
  hybrid-cache-write-caching-ineligibility-reason  Flash Pool Write Caching Ineligibility Reason
  is-managed-by-service                            Managed By Storage Service
  enable-snapdiff                                  Create Namespace Mirror Constituents For SnapDiff Use
  constituent-role                                 Constituent Volume Role
  qos-policy-group                                 QoS Policy Group Name
  is-volume-in-cutover                             Is Volume Move in Cutover Phase
  snapshot-count                                   Number of Snapshot Copies in the Volume

There are obviously a lot of different fields that we could display depending on the use case.

Perhaps we actually want to display fewer fields, not more:

cdot_mba1::> vol show -fields vserver,volume,aggregate,size,percent-used
  (volume show)
vserver      volume aggregate size    percent-used 
------------ ------ --------- ------- ------------ 
cdot_mba1-01 vol0   aggr0     851.5MB 73%          
tours        avol1  aggr0     20MB    87%          
tours        citrixvol1 
                    aggr1     20MB    5%           
tours        dept1_share 
                    aggr1     20MB    5%           
tours        dept2_share 
                    aggr1     20MB    5%           
tours        testvol1 
                    aggr0     20MB    5%           
tours        tours_root 
                    aggr0     20MB    5%           
tours        user_share 
                    aggr1     20MB    5%           
tours        vmwarevol1 
                    aggr1     20MB    5%           
9 entries were displayed.

Perhaps we want to look at information more relevant to a NAS configuration:

cdot_mba1::> vol show -fields vserver,volume,security-style,junction-path,language,user,group,policy,unix-permissions 
  (volume show)
vserver      volume policy user group security-style unix-permissions junction-path language 
------------ ------ ------ ---- ----- -------------- ---------------- ------------- -------- 
cdot_mba1-01 vol0   -      -    -     -              ------------     -             -        
tours        avol1  default 
                           0    0     unix           ---rwxr-xr-x     -             C.UTF-8  
tours        citrixvol1 
                    default 
                           0    0     unix           ---rwxr-xr-x     -             C.UTF-8  
tours        dept1_share 
                    default 
                           0    0     unix           ---rwxr-xr-x     -             C.UTF-8  
tours        dept2_share 
                    default 
                           0    0     unix           ---rwxr-xr-x     -             C.UTF-8  
tours        testvol1 
                    default 
                           0    0     unix           ---rwxr-xr-x     -             C.UTF-8  
tours        tours_root 
                    default 
                           0    0     unix           ---rwxr-xr-x     /             C.UTF-8  
tours        user_share 
                    default 
                           0    0     unix           ---rwxr-xr-x     -             C.UTF-8  
tours        vmwarevol1 
                    default 
                           0    0     unix           ---rwxr-xr-x     -             C.UTF-8  
9 entries were displayed.

One subtle thing to notice: fields are not displayed in the order in which you list them. The display order is not configurable by the user.

Field Filtering

The tab completion functionality of the cluster shell lends itself to filtering based on field just by its operation. As you tab through the parameters of a command you are continuously filtering out possible options through your choices. While this is explicit in a create or modify command, it’s equally true for displaying information.

cdot_mba1::> vol show -vserver 
    cdot_mba1    cdot_mba1-01 tours        

cdot_mba1::> vol show -vserver tours
  (volume show)
Vserver   Volume       Aggregate    State      Type       Size  Available Used%
--------- ------------ ------------ ---------- ---- ---------- ---------- -----
tours     avol1        aggr0        online     RW         20MB     2.52MB   87%
tours     citrixvol1   aggr1        online     RW         20MB    18.89MB    5%
tours     dept1_share  aggr1        online     RW         20MB    18.89MB    5%
tours     dept2_share  aggr1        online     RW         20MB    18.89MB    5%
tours     testvol1     aggr0        online     RW         20MB    18.89MB    5%
tours     tours_root   aggr0        online     RW         20MB    18.89MB    5%
tours     user_share   aggr1        online     RW         20MB    18.89MB    5%
tours     vmwarevol1   aggr1        online     RW         20MB    18.89MB    5%
8 entries were displayed.

Here, I’ve used [TAB] to first show me the possible values for that field, and then to filter the results based on my choice of “tours”.

I can also filter just by specifying which values I’m interested in. Perhaps I only want to see volumes owned by “tours” that are on aggr0:

cdot_mba1::> vol show -vserver tours -aggregate aggr0
  (volume show)
Vserver   Volume       Aggregate    State      Type       Size  Available Used%
--------- ------------ ------------ ---------- ---- ---------- ---------- -----
tours     avol1        aggr0        online     RW         20MB     2.50MB   87%
tours     testvol1     aggr0        online     RW         20MB    18.88MB    5%
tours     tours_root   aggr0        online     RW         20MB    18.88MB    5%
3 entries were displayed.

I can filter on this further by specifying fields that aren’t even displayed in the output, such as snapshot-count:

cdot_mba1::> vol show -vserver tours -aggregate aggr0 -snapshot-count 3
  (volume show)
Vserver   Volume       Aggregate    State      Type       Size  Available Used%
--------- ------------ ------------ ---------- ---- ---------- ---------- -----
tours     avol1        aggr0        online     RW         20MB     2.50MB   87%
tours     testvol1     aggr0        online     RW         20MB    18.88MB    5%
2 entries were displayed.

I can then take this and limit which fields are displayed so that I get exactly the data I’m looking for and nothing else:

cdot_mba1::> vol show -vserver tours -aggregate aggr0 -snapshot-count 3 -fields vserver,volume 
  (volume show)
vserver volume 
------- ------ 
tours   avol1  
tours   testvol1 
2 entries were displayed.

Queries and Operators

Since Data ONTAP is a Unix-like operating system, it’s been a common request that the CLI should support the use of typical Unix wildcard characters and patterns to provide more flexibility for the administrator – and with clustered Data ONTAP the cluster shell does. These patterns are considered to be “queries” and each wildcard or pattern is considered an “operator.” Using queries, administrators are able to more precisely specify the contents of the output for which they’re looking.

The following table is adapted from the one included in the System Administration Guide for clustered Data ONTAP. You’ll notice that most of the listed operators are the standard wildcards used in many systems.

Operator Description
* ALL operator: match all values
! NOT operator: do not match the specified value
| (pipe) OR operator: the pipe symbol acts as a separator between multiple patterns
.. RANGE operator: match any value within the range
< LESS THAN operator: match any value less than the specified one
> GREATER THAN operator: match any value more than the specified one
<= LESS-THAN-OR-EQUAL-TO operator: match any value equal to or less than the specified one
>= GREATER-THAN-OR-EQUAL-TO operator: match any value equal to or more than the specified one

Let’s provide some examples of each one:

ALL

cdot_mba1::> volume show -volume *
Vserver   Volume       Aggregate    State      Type       Size  Available Used%
--------- ------------ ------------ ---------- ---- ---------- ---------- -----
cdot_mba1-01 
          vol0         aggr0        online     RW      851.5MB    243.5MB   71%
tours     avol1        aggr0        online     RW         20MB     2.66MB   86%
tours     citrixvol1   aggr1        online     RW         20MB    18.89MB    5%
tours     dept1_share  aggr1        online     RW         20MB    18.89MB    5%
tours     dept2_share  aggr1        online     RW         20MB    18.89MB    5%
tours     testvol1     aggr0        online     RW         20MB    18.89MB    5%
tours     tours_root   aggr0        online     RW         20MB    18.89MB    5%
tours     user_share   aggr1        online     RW         20MB    18.89MB    5%
tours     vmwarevol1   aggr1        online     RW         20MB    18.89MB    5%
9 entries were displayed.

The ALL operator can be used with partial values regardless of placement within that value:

cdot_mba1::> volume show -volume *vol1
Vserver   Volume       Aggregate    State      Type       Size  Available Used%
--------- ------------ ------------ ---------- ---- ---------- ---------- -----
tours     avol1        aggr0        online     RW         20MB     2.66MB   86%
tours     citrixvol1   aggr1        online     RW         20MB    18.89MB    5%
tours     testvol1     aggr0        online     RW         20MB    18.89MB    5%
tours     vmwarevol1   aggr1        online     RW         20MB    18.89MB    5%
4 entries were displayed.

cdot_mba1::> vol show -volume dept*_share
  (volume show)
Vserver   Volume       Aggregate    State      Type       Size  Available Used%
--------- ------------ ------------ ---------- ---- ---------- ---------- -----
tours     dept1_share  aggr1        online     RW         20MB    18.89MB    5%
tours     dept2_share  aggr1        online     RW         20MB    18.89MB    5%
2 entries were displayed.

With this flexibility, you can filter and match on more than one value in a specified field, and not just literally all values.

NOT

cdot_mba1::> vol show -volume !vol0
  (volume show)
Vserver   Volume       Aggregate    State      Type       Size  Available Used%
--------- ------------ ------------ ---------- ---- ---------- ---------- -----
tours     avol1        aggr0        online     RW         20MB     2.66MB   86%
tours     citrixvol1   aggr1        online     RW         20MB    18.89MB    5%
tours     dept1_share  aggr1        online     RW         20MB    18.89MB    5%
tours     dept2_share  aggr1        online     RW         20MB    18.89MB    5%
tours     testvol1     aggr0        online     RW         20MB    18.89MB    5%
tours     tours_root   aggr0        online     RW         20MB    18.89MB    5%
tours     user_share   aggr1        online     RW         20MB    18.89MB    5%
tours     vmwarevol1   aggr1        online     RW         20MB    18.89MB    5%
8 entries were displayed.

In this example, there was a single volume we wanted to exclude from the output, but what if we wanted to exclude more than one? Just combine operators:

cdot_mba1::> volume show -volume !*vol1
Vserver   Volume       Aggregate    State      Type       Size  Available Used%
--------- ------------ ------------ ---------- ---- ---------- ---------- -----
cdot_mba1-01 
          vol0         aggr0        online     RW      851.5MB    237.0MB   72%
tours     dept1_share  aggr1        online     RW         20MB    18.89MB    5%
tours     dept2_share  aggr1        online     RW         20MB    18.89MB    5%
tours     tours_root   aggr0        online     RW         20MB    18.89MB    5%
tours     user_share   aggr1        online     RW         20MB    18.89MB    5%
5 entries were displayed.

OR

cdot_mba1::> vol show -volume citrix* | vmware*
  (volume show)
Vserver   Volume       Aggregate    State      Type       Size  Available Used%
--------- ------------ ------------ ---------- ---- ---------- ---------- -----
tours     citrixvol1   aggr1        online     RW         20MB    18.89MB    5%
tours     vmwarevol1   aggr1        online     RW         20MB    18.89MB    5%
2 entries were displayed.

RANGE

Unsurprisingly, we can’t use operators like RANGE, LESS THAN, GREATER THAN, etc. on free-text fields like volume names. Continuing with our examples, let’s start filtering based on the “-percent-used” field and find our most under-utilized volumes:

cdot_mba1::> vol show -percent-used 1..10
  (volume show)
Vserver   Volume       Aggregate    State      Type       Size  Available Used%
--------- ------------ ------------ ---------- ---- ---------- ---------- -----
tours     citrixvol1   aggr1        online     RW         20MB    18.88MB    5%
tours     dept1_share  aggr1        online     RW         20MB    18.88MB    5%
tours     dept2_share  aggr1        online     RW         20MB    18.88MB    5%
tours     testvol1     aggr0        online     RW         20MB    18.88MB    5%
tours     tours_root   aggr0        online     RW         20MB    18.88MB    5%
tours     user_share   aggr1        online     RW         20MB    18.88MB    5%
tours     vmwarevol1   aggr1        online     RW         20MB    18.88MB    5%
7 entries were displayed.

We can do a similar search to find volumes of a certain size range:

cdot_mba1::> vol show -size 500M..1000M
  (volume show)
Vserver   Volume       Aggregate    State      Type       Size  Available Used%
--------- ------------ ------------ ---------- ---- ---------- ---------- -----
cdot_mba1-01 
          vol0         aggr0        online     RW      851.5MB    227.1MB   73%

Note the “M” in that last example: when using fields that display different units of measurement, you need to specify that unit in your RANGE operation.

cdot_mba1::> vol show -size 500..1000
  (volume show)
There are no entries matching your query.

cdot_mba1::> vol show -size 1024k..30000k
  (volume show)
Vserver   Volume       Aggregate    State      Type       Size  Available Used%
--------- ------------ ------------ ---------- ---- ---------- ---------- -----
tours     avol1        aggr0        online     RW         20MB     2.56MB   87%
tours     citrixvol1   aggr1        online     RW         20MB    18.88MB    5%
tours     dept1_share  aggr1        online     RW         20MB    18.88MB    5%
tours     dept2_share  aggr1        online     RW         20MB    18.88MB    5%
tours     testvol1     aggr0        online     RW         20MB    18.88MB    5%
tours     tours_root   aggr0        online     RW         20MB    18.88MB    5%
tours     user_share   aggr1        online     RW         20MB    18.88MB    5%
tours     vmwarevol1   aggr1        online     RW         20MB    18.88MB    5%
8 entries were displayed.

Let’s try to get the same information as above, but using LESS THAN or MORE THAN instead of a RANGE operation.

LESS THAN

cdot_mba1::> vol show -percent-used <10
  (volume show)
Vserver   Volume       Aggregate    State      Type       Size  Available Used%
--------- ------------ ------------ ---------- ---- ---------- ---------- -----
tours     citrixvol1   aggr1        online     RW         20MB    18.89MB    5%
tours     dept1_share  aggr1        online     RW         20MB    18.89MB    5%
tours     dept2_share  aggr1        online     RW         20MB    18.89MB    5%
tours     testvol1     aggr0        online     RW         20MB    18.89MB    5%
tours     tours_root   aggr0        online     RW         20MB    18.88MB    5%
tours     user_share   aggr1        online     RW         20MB    18.89MB    5%
tours     vmwarevol1   aggr1        online     RW         20MB    18.89MB    5%
7 entries were displayed.

GREATER THAN

cdot_mba1::> vol show -size >500M
  (volume show)
Vserver   Volume       Aggregate    State      Type       Size  Available Used%
--------- ------------ ------------ ---------- ---- ---------- ---------- -----
cdot_mba1-01 
          vol0         aggr0        online     RW      851.5MB    225.6MB   73%

LESS-THAN-OR-EQUAL-TO

cdot_mba1::> vol show -size <=20M 
  (volume show)
Vserver   Volume       Aggregate    State      Type       Size  Available Used%
--------- ------------ ------------ ---------- ---- ---------- ---------- -----
tours     avol1        aggr0        online     RW         20MB     2.46MB   87%
tours     citrixvol1   aggr1        online     RW         20MB    18.89MB    5%
tours     dept1_share  aggr1        online     RW         20MB    18.89MB    5%
tours     dept2_share  aggr1        online     RW         20MB    18.89MB    5%
tours     testvol1     aggr0        online     RW         20MB    18.89MB    5%
tours     tours_root   aggr0        online     RW         20MB    18.89MB    5%
tours     user_share   aggr1        online     RW         20MB    18.89MB    5%
tours     vmwarevol1   aggr1        online     RW         20MB    18.89MB    5%
8 entries were displayed.

To demonstrate that this is really being evaluated differently than just LESS THAN:

cdot_mba1::> vol show -size <20M 
  (volume show)
There are no entries matching your query.

GREATER-THAN-OR-EQUAL-TO

cdot_mba1::> vol show -size >=20M
  (volume show)
Vserver   Volume       Aggregate    State      Type       Size  Available Used%
--------- ------------ ------------ ---------- ---- ---------- ---------- -----
cdot_mba1-01
          vol0         aggr0        online     RW      851.5MB    224.3MB   73%
tours     avol1        aggr0        online     RW         20MB     2.52MB   87%
tours     citrixvol1   aggr1        online     RW         20MB    18.89MB    5%
tours     dept1_share  aggr1        online     RW         20MB    18.89MB    5%
tours     dept2_share  aggr1        online     RW         20MB    18.89MB    5%
tours     testvol1     aggr0        online     RW         20MB    18.89MB    5%
tours     tours_root   aggr0        online     RW         20MB    18.89MB    5%
tours     user_share   aggr1        online     RW         20MB    18.89MB    5%
tours     vmwarevol1   aggr1        online     RW         20MB    18.89MB    5%
9 entries were displayed.

Putting It All Together

Even though these examples are rather trivial, the filtering and querying capabilities in the CLI can be very powerful when they are combined.

For example, what if I want to find all volumes which are more than 50% utilized with a certain minimum number of snapshots – except for those belonging to a particular SVM or on a particular aggregate?

cdot_mba1::> vol show -volume * -vserver !cdot_mba1-01  -aggregate !aggr1 -percent-used >50 -snapshot-count >=3
  (volume show)
Vserver   Volume       Aggregate    State      Type       Size  Available Used%
--------- ------------ ------------ ---------- ---- ---------- ---------- -----
tours     avol1        aggr0        online     RW         20MB     2.45MB   87%

As you can see, you can run very detailed queries that are still simple to construct and easy to understand.

In even moderately sized environments, effective use of the capabilities within the clustered Data ONTAP shell can provide significant administrator efficiencies.

Next up: Extended queries.

Advertisements

4 thoughts on “Tours of the Black Prompt: Clustered NetApp Data ONTAP – Part 3

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s