Tours of the Black Prompt: Clustered NetApp Data ONTAP – Part 1

In the first article in our series, we took a tour of how to navigate and use the CLI with Data ONTAP 7-Mode. Today we’re going to dive into the CLI for clustered Data ONTAP where it gets a lot more fun!

Commands, Navigation, and Privilege Levels

Logging in (via SSH or console – telnet is disabled by default and strongly discouraged) you are provided a simple administrative-level prompt:

cdot_mba1::>

The prompt contains the name of the cluster (“cdot_mba1”) followed by a “::>” which indicates that you are in the cluster shell, at the root of the command tree (more on that in a moment) and at the normal administrative privilege level.

From here, you can see the available command structure by simply typing “?”:

cdot_mba1::> ?
  up                          Go up one directory
  cluster>                    Manage clusters
  dashboard>                  Display dashboards
  event>                      Manage system events
  exit                        Quit the CLI session
  history                     Show the history of commands for this CLI session
  job>                        Manage jobs and job schedules
  lun>                        Manage LUNs
  man                         Display the on-line manual pages
  network>                    Manage physical and virtual network connections
  qos>                        QoS settings
  redo                        Execute a previous command
  rows                        Show/Set the rows for this CLI session
  run                         Run interactive or non-interactive commands in the node shell
  security>                   The security directory
  set                         Display/Set CLI session settings
  sis                         Manage volume efficiency
  snapmirror>                 Manage SnapMirror
  statistics>                 Display operational statistics
  storage>                    Manage physical storage, including disks, aggregates, and failover
  system>                     The system directory
  top                         Go to the top-level directory
  volume>                     Manage virtual storage, including volumes, snapshots, and mirrors
  vserver>                    Manage Vservers

You can already see that this is quite a bit different from 7-mode. The commands are organized into a tree which you can navigate similarly to a directory structure. The ">" at the end indicates this is a directory, or context, into which you can further explore. At each level, you can use "?" to display the commands or sub-contexts available.


cdot_mba1::> storage

cdot_mba1::storage> ?
  aggregate>                  Manage storage aggregates
  array>                      The array directory
  disk>                       Manage physical disks
  errors>                     The errors directory
  failover>                   Manage storage failover
  library>                    The library directory
  load>                       The load directory
  path>                       The path directory
  raid-options>               The raid-options directory

cdot_mba1::storage> 

You can also simply tab to display a more concise view of your options. And yes, that means that tab completion is included in the shell!


cdot_mba1::storage>
    aggregate    array        disk         errors       failover
    library      load         path         raid-options

cdot_mba1::storage> aggregate
    add-disks         create            delete            modify
    offline           online            raidtype          relocation
    rename            restrict          scrub             show
    show-scrub-status show-space       

cdot_mba1::storage> aggregate show -aggregate aggr0 

We'll circle back to tab completion later, but did you notice how the prompt has now changed? The current command sub-directory is now displayed within the prompt following the "::". This provides you with visual breadcrumbs for what context you're operating in and where in the command tree you are currently.


cdot_mba1::storage> aggregate

cdot_mba1::storage aggregate> ?
  add-disks                   Add disks to an aggregate
  create                      Create an aggregate
  delete                      Delete an aggregate
  modify                      Modify aggregate attributes
  offline                     Offline an aggregate
  online                      Online an aggregate
  raidtype>                   The raidtype directory
  relocation>                 Manage aggregate relocation
  rename                      Rename an aggregate
  restrict                    Restrict an aggregate
  scrub                       Aggregate parity scrubbing
  show                        Display a list of aggregates
  show-scrub-status           Display aggregate scrubbing status
  show-space                  Display details of space utilization within an aggregate.

cdot_mba1::storage aggregate> raidtype

cdot_mba1::storage aggregate raidtype>

Ok, going down through the directory is easy, but what about going back up? There's actually several different options. You can simply say that you want to go up a level:

cdot_mba1::storage aggregate raidtype> up

cdot_mba1::storage aggregate>

You can say that you want to go all the way back to the top no matter how far down you are:

cdot_mba1::storage aggregate> top

cdot_mba1::>

Or you can do a standard "change directory" or cd:

cdot_mba1::storage> cd ..

cdot_mba1::>

You can also jump around within the tree either using cd or simply entering the context to which you want to change.

cdot_mba1::> volume

cdot_mba1::volume> cd aggregate

cdot_mba1::storage aggregate> volume

cdot_mba1::volume>

Just like 7-mode Data ONTAP, there are two additional privilege levels available: advanced and diag. You can change to these levels using the set -privilege command.

cdot_mba1::> set -privilege advanced

Warning: These advanced commands are potentially dangerous; use them only when directed to do so by NetApp personnel.
Do you want to continue? {y|n}: y

cdot_mba1::*> set -privilege diagnostic

Warning: These diagnostic commands are for use by NetApp personnel only.
Do you want to continue? {y|n}: y

cdot_mba1::*> 

The asterisk after the "::" indicates that you are in either advanced or diag privilege mode, and there is unfortunately no visual distinction between the two modes.

More commands are available within the higher privilege levels than in the normal admin level, though because of the organization of the command tree they do not stand out as much as they do in 7-mode.

Advanced:

  up                          Go up one directory
  cluster>                    Manage clusters
  dashboard>                  Display dashboards
  event>                      Manage system events
  exit                        Quit the CLI session
  history                     Show the history of commands for this CLI session
  job>                        Manage jobs and job schedules
  lun>                        Manage LUNs
  man                         Display the on-line manual pages
  network>                    Manage physical and virtual network connections
  qos>                        QoS settings
  redo                        Execute a previous command
  rows                        Show/Set the rows for this CLI session
  run                         Run interactive or non-interactive commands in the node shell
  security>                   The security directory
  set                         Display/Set CLI session settings
  sis                         Manage volume efficiency
  snapmirror>                 Manage SnapMirror
  statistics>                 Display operational statistics
  storage>                    Manage physical storage, including disks, aggregates, and failover
  system>                     The system directory
  top                         Go to the top-level directory
  volume>                     Manage virtual storage, including volumes, snapshots, and mirrors
  vserver>                    Manage Vservers

Diag:

  up                          Go up one directory
  cluster>                    Manage clusters
  dashboard>                  Display dashboards
  diag>                       *The diag directory
  event>                      Manage system events
  exit                        Quit the CLI session
  history                     Show the history of commands for this CLI session
  job>                        Manage jobs and job schedules
  logger>                     *Management logger
  lun>                        Manage LUNs
  man                         Display the on-line manual pages
  network>                    Manage physical and virtual network connections
  qos>                        QoS settings
  redo                        Execute a previous command
  repository>                 *Manage a content repository
  rows                        Show/Set the rows for this CLI session
  run                         Run interactive or non-interactive commands in the node shell
  security>                   The security directory
  set                         Display/Set CLI session settings
  sis                         Manage volume efficiency
  snapmirror>                 Manage SnapMirror
  spm>                        *Service Process Manager
  statistics>                 Display operational statistics
  storage>                    Manage physical storage, including disks, aggregates, and failover
  system>                     The system directory
  top                         Go to the top-level directory
  volume>                     Manage virtual storage, including volumes, snapshots, and mirrors
  vserver>                    Manage Vservers
 

Changing between privilege levels is always done the same way, even when going back down from a higher level to the admin level:

cdot_mba1::*> set -privilege admin

cdot_mba1::>

Command Syntax and Help

You can see the syntax for a command by passing it the "-?" flag, but only at the appropriate level.

cdot_mba1::> storage -?         

Error: "-" is not a recognized command

cdot_mba1::> storage aggregate -?

Error: "-" is not a recognized command

cdot_mba1::> storage aggregate show -?
  [ -checksum | -disk | -instance | -raid-info | -fields , ... ]
  [[-aggregate] ]                           Aggregate
  [ -chksumstyle  ]                         Checksum Style
  [ -diskcount  ]                                  Number Of Disks
  [ -nodes , ... ]                                Nodes
  [ -disklist|-d , ... ]                    Disks
  [ -free-space-realloc {on|off|no_redirect} ]              Free Space Reallocation
  [ -ha-policy {sfo|cfo} ]                                  HA Policy
  [ -percent-snapshot-space  ]                     Space Reserved for Snapshot Copies
  [ -hybrid-enabled {true|false} ]                          Hybrid Enabled
  [ -availsize {[KB|MB|GB|TB|PB]} ]                Available Size
  [ -chksumenabled {true|false} ]                           Checksum Enabled
  [ -chksumstatus  ]                                  Checksum Status
  [ -has-mroot {true|false} ]                               Has Mroot Volume
  [ -has-partner-mroot {true|false} ]                       Has Partner Node Mroot Volume
  [ -home-id  ]                                    Home ID
  [ -home-name  ]                                     Home Name
  [ -hybrid-cache-size-total {[KB|MB|GB|TB|PB]} ]  Total Hybrid Cache Size
  [ -hybrid {true|false} ]                                  Hybrid
  [ -inconsistent {true|false} ]                            Inconsistent
  [ -is-home {true|false} ]                                 Is Aggregate Home
  [ -maxraidsize|-s  ]                             Max RAID Size
  [ -cache-raid-group-size  ]                      Flash Pool SSD Tier Maximum RAID Group Size
  [ -owner-id  ]                                   Owner ID
  [ -owner-name  ]                                    Owner Name
  [ -percent-used  ]                               Used Percentage
  [ -plexes , ... ]                                   Plexes
  [ -raidgroups , ... ]                               RAID Groups
  [ -raidstatus  ]                                    RAID Status
  [ -raidtype|-t {raid_dp|raid4} ]                          RAID Type
  [ -root {true|false} ]                                    Is Root
  [ -sis-metadata-space-used {[KB|MB|GB|TB|PB]} ]  Space Used by Metadata for Volume Efficiency
  [ -size {[KB|MB|GB|TB|PB]} ]                     Size
  [ -state  ]                              State
  [ -usedsize {[KB|MB|GB|TB|PB]} ]                 Used Size
  [ -volcount  ]                                   Number Of Volumes
  [ -volume-style  ]                                  Volume Style

cdot_mba1::> storage aggregate show -

In this example, it's not until you get to the third word of the command that you are able to use the "-?" to display the parameters. This is because the first two words were actually directories or contexts (you may recall that both displayed the trailing ">" when we were navigating above) rather than actionable commands. It's that action that is key: you can't issue a command without specifying what it is you want done. Nouns alone won't cut it - you need to use a verb. So a command in the cluster shell will generally always end in a verb (show, create, delete, modify, etc.) and it is then that you can specify the appropriate parameters.

Where it gets even better are the manual (man) pages for clustered Data ONTAP. Similar to 7-mode, administrative-level commands have man page available explaining the operation of the command in detail:

cdot_mba1::> man system


system                          Data ONTAP 8.2                          system

NAME
     system -- The system directory

DESCRIPTION
     The system commands enable you to monitor and control cluster nodes.

SUBDIRECTORIES
     health> - System Health Management and Diagnosis commands

     license> - Manage licenses

     node> - The system node directory

     script> - Capture CLI session to a file for later upload. Analogous to the unix 'script' command

     services> - Manage system services

     smtape> - Manage SMTape operations

     snmp> - The snmp directory

     timeout> - Manage the timeout value for CLI sessions


Data ONTAP 8.2                       -o-                        Data ONTAP 8.2

A key difference here is that you won't have to look at a single man page covering the entire command - or command directory/context - which may run on for many pages of output (some of the man pages might be more accurately described as man books). Instead, you can specify exactly at which level of the command you are looking for more information.

cdot_mba1::> man system health

system health                   Data ONTAP 8.2                   system health

NAME
     system health -- System Health Management and Diagnosis commands

DESCRIPTION
     System Health Management and Diagnosis commands

SUBDIRECTORIES
     alert> - The alert directory

     autosupport> - The autosupport directory

     chassis> - Chassis health monitor directory

     cluster-switch> - The cluster-switch directory

     config> - The config directory

     controller> - Controller health monitor directory

     node-connectivity> - Node storage connectivity health monitor directory

     policy> - The policy directory

     status> - The status directory

     subsystem> - The subsystem directory

     system-connectivity> - System storage connectivity health monitor directory


Data ONTAP 8.2                       -o-                        Data ONTAP 8.2

cdot_mba1::> man system health status show

system health status show       Data ONTAP 8.2       system health status show

NAME
     system health status show -- Display system health monitoring status

AVAILABILITY
     This command is available to cluster administrators at the admin privilege level.

DESCRIPTION
     The system health status show command displays the health monitor status. The possible states are:

     o  ok

     o  ok-with-suppressed

     o  degraded

     o  unreachable

PARAMETERS
     None


EXAMPLES
      This example displays information about health monitoring status:

                                                               +---------------------------------------------------------------------------------+
                                                               |                                                                                 |
                                                               |                                                                                 |
                                                               |  cluster1::> system health status show                                          |
                                                               |       Status                                                                    |
                                                               |       ---------------                                                           |
                                                               |       degraded                                                                  |
                                                               |                                                                                 |
                                                               +---------------------------------------------------------------------------------+


Data ONTAP 8.2                       -o-                        Data ONTAP 8.2

 

Command Completion

As noted previously, tab completion is built into the cluster shell so rather than having to fully type out storage aggregate you can type st[TAB]ag[TAB] instead to have the shell auto-complete the commands for you.

You can also abbreviate the commands rather than tab completing them:


cdot_mba1::> st ag sh
Error: Ambiguous command.  Possible matches include:
       storage aggregate show
       storage aggregate show-scrub-status
       storage aggregate show-space
     
cdot_mba1::> st ag show
  (storage aggregate show)
Aggregate     Size Available Used% State   #Vols  Nodes            RAID Status
--------- -------- --------- ----- ------- ------ ---------------- ------------
aggr0        900MB   43.54MB   95% online       1 cdot_mba1-01     raid_dp,
                                                                   normal

cdot_mba1::>  

As you can see, abbreviation will only work if the abbreviation is unique, but even when it isn't, the shell will show you the possible options in order to make it easier for you to complete the desired command. The uniqueness of the abbreviation is key and is similar to one of the navigation tricks above: you don't necessarily need to provide the entire path of the command as long as the abbreviated command is unique:

cdot_mba1::> ag show
  (aggr)
Aggregate     Size Available Used% State   #Vols  Nodes            RAID Status
--------- -------- --------- ----- ------- ------ ---------------- ------------
aggr0        900MB   43.54MB   95% online       1 cdot_mba1-01     raid_dp,
                                                                   normal

cdot_mba1::> 

You can also easily find where sub-commands or actions are available throughout the command tree simply by typing the word and hitting [Enter]:

cdot_mba1::> status
Error: Ambiguous command.  Possible matches include:
       system services manager status
       system health status
       vserver nfs status
       vserver services ndmp status
       system services ndmp status
       system license status
       volume clone split status
       event status
       volume file clone status
       system smtape status
       system services ndmp node-scope-mode status
       system node coredump status
       system node coredump segment status

cdot_mba1::> health
Error: Ambiguous command.  Possible matches include:
       system health
       vserver dashboard health
       dashboard health
       cluster peer health

This can be a great way to explore the various nooks & crannies of the CLI - there's a lot there to look at!

Navigation and Editing

Command-line editing and navigation utilizes the standard keystrokes and combination previously discussed in CLI Efficiency: Common Basics

You can navigate through your previously-entered commands using the up and down arrows, or Ctrl+n and Ctrl+p, or you can now use the history command to simply display the contents of that history.

cdot_mba1::> history
    1  stor aggr show
    2  man syste
    3  man systm health status
    4  history
    5  vol show

You can enter multiple commands on the same command line by separating each command with a semi-colon. The commands will then be executed in order of entry.

cdot_mba1::> stor aggr show; vol show
  (storage aggregate show)
Aggregate     Size Available Used% State   #Vols  Nodes            RAID Status
--------- -------- --------- ----- ------- ------ ---------------- ------------
aggr0        900MB   43.54MB   95% online       1 cdot_mba1-01     raid_dp,
                                                                   normal

  (volume show)
Vserver   Volume       Aggregate    State      Type       Size  Available Used%
--------- ------------ ------------ ---------- ---- ---------- ---------- -----
cdot_mba1-01
          vol0         aggr0        online     RW      851.5MB    303.2MB   64%

cdot_mba1::> 

After all of the above, we've still only begun to scratch the surface of the features available in the clustered Data ONTAP CLI: there's a lot more yet to come!

DataCenter of the Dead

Sometimes you gotta have fun, and sometimes you just gotta keep the lights on. When you’re responsible for an enterprise datacenter environment, the company is relying on you to ensure that the infrastructure not only performs well and has sufficient capacity for compute, storage, and networking, but also that it’s extremely reliable and resilient. You need to take into account hardware failures, software bugs, power outages, extreme weather conditions, and much, much more.

Such as zombies. No, really – you never know what can happen:

If you attended NetApp Insight in Las Vegas you may have already seen the above video – it was playing on a continuous loop at the MetroCluster booth. The film was actually shot on NetApp’s RTP campus and the actors are almost entirely NetApp employees. Some of the featured actors include Mr. Pete Flecha (@vPedroArrow), co-host of the NetApp Communities Podcast, Mr. Troy Mangum (@CloudReveler), and Mr. Rhett Bigler (@CloudUnbound). DataCenter of the Dead is the brainchild of the FlexPod team’s resident creative geek, Mr. Dave Eckert (@TheGreeches) who wrote, directed, and filmed the entire thing.

Expect more video shenanigans in the future!


Dave is also the creative force behind another multimedia launch at Insight: the NetApp 3D FlexPod app! The app provides an interactive 3D view of a FlexPod Datacenter rack, including detailed information on the various components and even the actual cabling between the components. The app is now officially available for iOS devices (iPhone5+ or iPad3+), and will be coming soon to Android.

If you were at Insight, you may have seen this odd-looking obelisk on the solutions floor:

iPads at the booth were used to interact with that monolith and enable customers and partners to virtually view a FlexPod at life-size! Even with an actual FlexPod Datacenter standing next to it, the app & the monolith both got a lot of positive attention. Now you can play with it yourself by printing out a miniature version and using the iOS app on your own device. Yes – fold your own FlexPod! If that’s too much work, just use the app by itself and you’ll still get a cool (and informative) experience, but without quite as much geeky fun.

CLI File Editing with NetApp Data ONTAP 7-Mode

Let me preface this by saying that it’s really not an expectation that you should, or even that you’re going to be, reading or modifying files from the storage array’s CLI. Files are meant to be accessed using a (surprise, surprise) file access protocol, such as NFS or SMB/CIFS, directly from a client. Having said that, it can be useful to use the CLI to modify system files such as /etc/rc or /etc/hosts, particularly during initial setup or maintenance activities, and I’ve done it myself hundreds if not thousands of times.

The biggest hurdle to overcome in CLI file editing is that there isn’t a real editor included in the shell. Nope – no vi or Vim, no emacs, nano, or any other editor you may expect or like. There isn’t even ed! Instead, file access and editing is done using rdfile to “read” or display the contents of a file, and wrfile to “write” the contents of a file. Rdfile is pretty straightforward – it’s essentially just a cat equivalent that takes a file and sends it to standard output (your screen).

mba-7m-1> rdfile /etc/hosts
#Auto-generated by setup Mon Nov 3 21:54:55 GMT 2014
127.0.0.1 localhost localhost-stack
127.0.10.1 localhost-10 localhost-bsd
127.0.20.1 localhost-20 localhost-sk
172.16.213.18 mba-7m-1 mba-7m-1-e0a

And no, before you ask: there is no included equivalent of more or less (for paging of output), nor anything like tail or head. More’s the pity.

Wrfile, on the other hand, is a little different and less friendly. It does the exact opposite of rdfile – it takes standard input (the keystrokes/characters you type into the console/shell) and sends (writes) it to a file. This makes it very easy to create a file – as long as you don’t make any typos! There is no interactive line editing with wrfile – if you make a mistake you’re essentially starting over. Your best course of action is to create your text locally on your workstation with your tool (Vim, notepad++, etc.) of choice, and then paste it into your terminal window.

After you’ve finished entering the desired content, make sure to hit Enter to be on a new line and then simply use Ctrl+c to end the input and terminate wrfile. If you aren’t on a new line, any text already entered on that line will not be saved. Don’t mind the “error reading standard input” – that’s normal.

mba-7m-1> wrfile /etc/thisisonlyatest
blah blah blah
line 2 blah blah blah
read: error reading standard input: Interrupted system call
mba-7m-1> rdfile /etc/thisisonlyatest
blah blah blah
line 2 blah blah blah

You need to be especially careful using wrfile to edit an existing file: it will overwrite the contents of the file you specify as soon as you hit Enter. Even if you don’t type anything and immediately Ctrl+c out, the damage is done and your file is now empty. For this reason, always be sure to rdfile the file prior to making any modifications to it so that you have a copy in your cache. In most cases, you’ll want to copy the contents printed out via rdfile to a local texteditor for modification; you’ll then copy the modified content and paste into the console window after starting the wrfile operation.

mba-7m-1> wrfile /etc/thisisonlyatest
read: error reading standard input: Interrupted system call
mba-7m-1> rdfile /etc/thisisonlyatest
mba-7m-1>

If all you need to do is to add a line (or lines) to a file, it’s best to use the “-a” parameter for wrfile in order to append to the existing file rather than overwriting it. This also works for creating a new file.

mba-7m-1> rdfile /etc/thisisonlyatest
/etc/thisisonlyatest: No such file or directory
mba-7m-1> wrfile -a /etc/thisisonlyatest Line1 foo
mba-7m-1> wrfile -a /etc/thisisonlyatest Line2 bar
mba-7m-1> rdfile /etc/thisisonlyatest
Line1 foo
Line2 bar

File editing via the Data ONTAP CLI is not a normal operation, it’s not very elegant (ok, it’s crude), and it’s really only intended for atypical administrative use-cases – but it gets the job done when you need it. Otherwise, follow the recommended method for modifying files by using a licensed file access protocol (NFS or SMB/CIFS) and edit from a remote client instead.

Tours of the Black Prompt: NetApp Data ONTAP 7-Mode

NetApp’s FAS series of storage arrays run a single operating system called Data ONTAP across all platforms – from the low end to the high end – though this operating system can run in one of two different modes. There’s the older “classic” mode, which the majority of FAS administrators are used to and which is now referred to as Data ONTAP 7-Mode, and there’s the new, scale-out mode that is simply called clustered Data ONTAP. While the two modes share similar administrative concepts and some similar commands, the two CLI operating environments are quite different in behaviors, structure, and feature sets. We’ll start by talking about 7-Mode.

Commands and Privilege Levels

Logging in (via SSH, telnet, or console) you are provided a simple administrative-level prompt:

 mba-7m-1>

The prompt contains the hostname of the controller (“mba-7m-1”) followed by a “>” which indicates that you are in the normal administrative privilege level.
From here, you can see the available commands by simply typing “?”:

?                   flexcache           options             smtape             
acpadmin            fpolicy             orouted             snap               
aggr                fsecurity           partner             snaplock           
arp                 ftp                 passwd              snapmirror         
autosupport         halt                ping                snapvault          
backup              help                ping6               snmp               
bmc                 hostname            pktt                software           
cdpd                httpstat            portset             source             
cf                  ic                  priority            sp                 
charmap             ifconfig            priv                stats              
cifs                ifgrp               qtree               storage            
clone               ifstat              quota               sysconfig          
cna_flash           igroup              radius              sysstat            
config              ipspace             rdate               system             
coredump            iscsi               rdfile              timezone           
date                key_manager         reallocate          traceroute         
dcb                 keymgr              reboot              traceroute6        
df                  license             restore             ucadmin            
disk                lock                restore_backup      ups                
disk_fw_update      logger              revert_to           uptime             
dns                 logout              rlm                 useradmin          
download            lun                 route               version            
du                  man                 routed              vfiler             
dump                maxfiles            rshstat             vlan               
echo                mt                  sasadmin            vmservices         
ems                 nbtstat             sasstat             vol                
environment         ndmpcopy            savecore            vscan              
exportfs            ndmpd               sectrace            wcc                
fcadmin             ndp                 secureadmin         wrfile             
fcnic               netdiag             setup               ypcat              
fcp                 netstat             sftp                ypgroup            
fcstat              nfs                 shelfchk            ypmatch            
fcvi                nfsstat             sis                 ypwhich            
file                nis                

This shows you all of the commands available for this administrator privilege level.

There are two additional privilege levels available: advanced and diag. You can change to these levels using the “priv set” command.

 mba-7m-1> priv set advanced
 Warning: These advanced commands are potentially dangerous; use
 them only when directed to do so by NetApp
 personnel.
 mba-7m-1\*>
 mba-7m-1\*> priv set diag
 Warning: These diagnostic commands are for use by NetApp
 personnel only.
 mba-7m-1\*>

The asterisk after the hostname indicates that you are in either advanced or diag privilege mode. There’s no visual distinction between the two modes, but you can find out which one you are in by typing simply “priv”:

 mba-7m-1\*> priv
 diag
 mba-7m-1\*>

More commands are available within the higher privilege levels than in the normal admin level.

Advanced:
/etc/rmt            ftp                 nfsstat             sldiag             
?                   getXXbyYY           nis                 sm_mon             
acorn               halt                nv8                 sm_mon_old         
acpadmin            hammer              ontapi              sm_not             
aggr                help                options             smb_hist           
arp                 hostname            orouted             smtape             
autosupport         httpstat            panic               snap               
availtime           ic                  partner             snaplock           
backup              if_addr_filter_info passwd              snapmirror         
blink_off           ifconfig            perf                snapvault          
blink_on            ifgrp               ping                snmp               
bmc                 ifinfo              ping6               software           
bootfs              ifstat              pktt                source             
bringhome           ifstat_test         portset             sp                 
cdpd                ifswitch            priority            statit             
cf                  igroup              priv                stats              
charmap             inodepath           ps                  storage            
cifs                ipspace             qtree               stsb               
clone               iscsi               quota               stty               
cna_flash           key_manager         radius              sum32              
cna_flash_image_reset keymgr              rdate               sysconfig          
cna_flash_image_set l2ping              rdfile              syslog             
cna_flash_version   led_off             reallocate          sysstat            
com                 led_off_all         reboot              system             
config              led_on              registry            systemshell        
coredump            led_on_all          restore             tape_qual          
cxgbtool            led_on_off          restore_backup      test_lcd           
date                led_reset_all       result              timezone           
dcb                 led_test            revert_to           toe                
dd                  led_test_one        rlm                 traceroute         
df                  license             rm                  traceroute6        
disk                lock                rmt                 ucadmin            
disk_fw_update      log                 route               ups                
disk_list           logger              routed              uptime             
disk_stat           logout              rpc_stats           useradmin          
dns                 ls                  rshkill             version            
download            lun                 rshstat             vfiler             
du                  mailbox             rsm                 vif                
dump                man                 rtag                vlan               
echo                maxfiles            rtfo                vmservices         
ems                 mbstat              sasadmin            vol                
environ             mem_scrub_stats     sasstat             vscan              
environment         memerr              savecore            vstorage           
exit                mt                  scsi                waffinity_stats    
exportfs            mv                  sectrace            wafl               
fcadmin             nbtstat             secureadmin         wafl_backdoor_stats
fcnic               ndmpcopy            setup               wafl_susp          
fcp                 ndmpd               sftp                wcc                
fcstat              ndp                 sh                  wrfile             
fcvi                netdiag             shelfchk            ypcat              
file                netstat             showfh              ypgroup            
flexcache           nfs                 showfh4             ypmatch            
fpolicy             nfs_hist            sis                 ypwhich            
fsecurity
Diag:
/etc/rmt            fcstat              netstat             sm_not             
?                   fcvi                nfs                 smb_hist           
acorn               file                nfs_hist            smf                
acpadmin            filersio            nfsstat             smtape             
aggr                flexcache           nis                 snap               
anvl                fpolicy             nm                  snaplock           
arp                 fru_led             nv                  snapmirror         
autosupport         fsecurity           nv8                 snapvault          
availtime           ftp                 ontapi              snmp               
backup              gdb                 options             software           
blink_off           getXXbyYY           orouted             source             
blink_on            halt                panic               sp                 
bmc                 hammer              parityck            spares_zero        
bootargs            help                partner             spinhi_stats       
bootfs              hostname            passwd              spinnp_replay      
bringhome           httpstat            perf                spinnp_replay_stats
cdpd                ic                  ping                stack              
cf                  icbulk              ping6               statit             
ch                  if_addr_filter_info pktt                stats              
charmap             ifconfig            portset             storage            
cifs                ifconfig_priv       printflag           stsb               
clone               ifgrp               priority            stty               
cna                 ifinfo              priv                sum32              
cna_flash           ifstat              prof                sync               
cna_flash_image_reset ifstat_test         ps                  sysconfig          
cna_flash_image_set ifswitch            qtree               syslog             
cna_flash_version   igroup              quota               sysstat            
com                 inodepath           radius              system             
config              iomem               raid_config         systemshell        
coredump            ipspace             rastrace            tape_qual          
ct_dump_t3          iscsi               rdate               tcp_client         
ct_phy_read         iswt                rdfile              tcp_server         
ct_phy_read_t3      key_manager         reallocate          test_lcd           
ct_phy_write        keymgr              reboot              time               
ct_phy_write_t3     kma_stats           registry            timezone           
ct_reg_read         kt                  restore             toe                
ct_reg_read_t3      l2ping              restore_backup      traceroute         
ct_reg_write        label               result              traceroute6        
ct_reg_write_t3     labelmaint          revert_to           treecompare        
ct_reset_t3         led_off             rlm                 ttcp               
ct_tpi_par          led_off_all         rm                  ucadmin            
ct_tpi_read         led_on              rmt                 udp_client         
ct_tpi_write        led_on_all          route               udp_server         
cxgbtool            led_on_off          route_priv          ups                
date                led_reset_all       routed              uptime             
dbg                 led_test            rpc_stats           useradmin          
dcb                 led_test_one        rshkill             vdom               
dd                  license             rshstat             version            
debug               lmgr_diag           rsm                 vfiler             
df                  lock                rtag                vif                
disk                log                 rtfo                vlan               
disk_fw_update      log_fio             sasadmin            vm_stat            
disk_list           logger              sasstat             vmservices         
disk_stat           logout              savecore            vol                
dns                 ls                  scsi                vol_db             
download            lun                 sectrace            vscan              
du                  mailbox             secureadmin         vstorage           
dump                man                 sesdiag             vtic               
dumpblock           maxfiles            setflag             waffinity_stats    
dumpstack           mbstat              setup               wafl               
echo                mem_scrub_stats     sftp                wafl_backdoor_stats
ems                 mem_stats           sh                  wafl_cmd_restrictions
environ             memerr              shelfchk            wafl_steal_stats   
environment         mkfile              show_faults         wafl_susp          
exit                mt                  showfh              wafltop            
export_stats        mv                  showfh4             wcc                
export_tbl_dump     nbtstat             signal              wrfile             
exportfs            ndmpcopy            sis                 xttcp              
fcadmin             ndmpd               sldiag              ypcat              
fcmon               ndp                 slist               ypgroup            
fcnic               netdiag             sm_mon              ypmatch            
fcp                 netmpstat           sm_mon_old          ypwhich  

Typing either “priv set” without specifying a privilege level, or “priv set admin” will take you back to the default admin privilege level.

mba-7m-1\*> priv set
mba-7m-1>

Command Syntax and Help

You can see the syntax for a command by passing it the “-?” flag.

mba-7m-1> arp -?
usage: arp [-n]
arp [-n] -a
arp -d
arp -s [temp] [pub]
arp -F

Even better, administrative-level commands have a full manual (man) page available explaining the operation of the command in detail:

mba-7m-1> man arp

na_arp(1) na_arp(1)

NAME
na_arp - Address resolution display and control

SYNOPSIS
arp [-n] hostname

arp [-n] -a

arp -d hostname

arp -s hostname ether_address [ temp ] [ pub ]

DESCRIPTION
The arp command displays and modifies the tables that the
address resolution protocol uses to translate between
Internet and Ethernet addresses.

With no flags, arp displays the current ARP entry for
hostname. The host may be specified by name or by number,
using Internet dot notation.

OPTIONS
-a Displays all of the current ARP entries.

-d Deletes an entry for the host called hostname.

-n IP addresses are displayed instead of hostnames.

-s Creates an ARP entry for the host called hostname
with the Ethernet address ether_address. The Eth-
ernet address is given as six hex bytes separated
by colons. The entry not will be permanent if the
words following -s includes the keyword temp. Tem-
porary entries that consist of a complete Internet
address and a matching Ethernet address are flushed
from the arp table if they haven't been referenced
in the past 20 minutes. A permanent entry is not
flushed.

If the words following -s include the keyword pub,
the entry will be "published"; that is, this system
will act as an ARP server, responding to requests
for hostname even though the host address is not
its own.

HA CONSIDERATIONS
In takeover mode, each node in an HA pair maintains its
own ARP table. You can make changes to the ARP table on
the live node, or you can make changes to the ARP table on
the failed node using the arp command in partner mode.
However, the changes you make in partner mode are lost
after a giveback.

VFILER CONSIDERATIONS
When run from a vfiler context, (for example, via the
vfiler run command), arp operates on the concerned vfiler.
As currently all vfilers in an ipspace share an arp table,
arp operates on the arp table of the concerned vfiler's
ipspace.

SEE ALSO
na_ifconfig(1), na_partner(1), na_ipspace(1),
na_vfiler(1), RFC1483.

6 June 1998 na_arp(1)

Command Completion

In Data ONTAP 7-mode, commands cannot be tab-completed in the shell, nor can they be abbreviated as with some other shells. Each command needs to be fully specified in order for it to be recognized.

Navigation and Editing

Command-line editing and navigation utilizes the standard keystrokes and combination previously discussed in CLI Efficiency: Common Basics

While you can navigate through your previously-entered commands using the up and down arrows, or Ctrl+n and Ctrl+p, there is no “history” command to simply display the contents of that history.

You can enter multiple commands on the same command line by separating each command with a semi-colon. The commands will then be executed in order of entry.

mba-7m-1> echo "help"; echo "I'm being held prisoner"; echo "in a terminal emulator"
help
I'm being held prisoner
in a terminal emulator

Updated 20141130: Fixed formatting that had somehow gotten broken since original publication.

Tech Smorgasbord #1

There’s always too much out there to learn and not remotely enough time to do it in. That’s always been the case, of course, but it just keeps getting worse – there’s all of the “old” stuff still to be learned (or re-learned) and then every minute of every day there’s more new, cool stuff that you want to dive into as well. There’s never enough time, so we need to try (somehow) to narrow our focus to the most interesting or most important – and hopefully there’s overlap between the two.

Here’s a few of the things that have popped onto my radar over the last week that I need to find some time to look into.


Vagrant Manager for OS X

A cool little addition to the toolbox for those using Vagrant (and Mac OS X). I’ve only dabbled with Vagrant to date but this looks like a good way to get myself using it even more.


Await – A modern implementation of Expect for Windows

This is a PowerShell implementation of Expect and is part of the PowerShell Resource Gallery – which has a ton of other modules and resources that should be investigated.

If you don’t know what Expect is you can read the Wikipedia page for background, but in a nutshell: it’s a program to provide automation of interactive applications. Expect this, do that. It’s a pretty handy tool, and I’m curious about this new implementation.


Etcd and Fleet

There have been a lot of configuration systems to pop up over the last few years, and Etcd (for /etc distributed) is one of these, brought to us by the folks at CoreOS – which is itself another project with which I need to become more conversant. I looked at CoreOS a few years back, but it’s come quite a long ways since then.


Hanlon (aka Razor 2.0)

The ever-prolific Nick Weaver and Tom McSweeney came out with Razor back in 2012 to provide “cloud provisioning” and it got quite a bit of attention at the time. Subsequently, Nick has moved on to other projects, and Tom renamed & relaunched the project as Hanlon. The project seems to be continuing to gather steam and deserves some more attention.


MidoNet

At the OpenStack Summit this week, Midokura announced that they would be open-sourcing their flagship network virtualization product, MidoNet. Midokura have been doing “software-defined networking” since before we began defining everything as SDx. I think I first heard of Midonet via a very positive Brad Hedlund blog post in 2012. Brad’s pretty well-known in the networking community, having made his name while working at Cisco before leaving for Dell, and now working for VMware’s NSX team.

Midonet was interesting technology even then, and open-sourcing that tech is an interesting move today: by doing so, they’re hoping to gain a leadership position in the OpenStack and open-source networking communities. Of course, there’s another vendor working to guide OpenStack networking (Cisco), and then there’s the little matter of the influence of Nicira in the OpenStack world. As we all know, Nicira is the foundation for what has become NSX, at the same time that VMware is also making forays into their own OpenStack distribution with VMware Integrated OpenStack.


SmartDataCenter and Manta

Joyent has been one of the more unique tech startups of the past few years, not least because they have eschewed building off of the typical Linux/FreeBSD base for their own SmartOS, which is based on the now-dead OpenSolaris. You may also know them as the originators of the popular Node.js platform. Joyent have a lot of frighteningly smart people working for them, including quite a few ex-Sun engineers such as Bryan Cantrill. They’ve built an interesting alternative cloud offering on SmartOS, and now they’ve released two of the other key pieces of that cloud as open-source. SmartDataCenter is effectively an OpenStack competitor providing cloud management and container orchestration, and Manta is an object storage system (built on top of ZFS).