1.  Terminal usage of the shell

1.1.  The basic notion of commands

      A shell in UNIX acts mostly as a medium through which other programs are invoked. While it has a set of builtin functions which it performs directly, most commands cause execution of programs that are, in fact, external to the shell. The shell is thus distinguished from the command interpreters of other systems both by the fact that it is just a user program, and by the fact that it is used almost exclusively as a mechanism for invoking other programs.

      Commands in the UNIX system consist of a list of strings or words interpreted as a command name followed by arguments. Thus the command 

mail bill
consists of two words. The first word mail names the command to be executed, in this case the mail program which sends messages to other users. The shell uses the name of the command in attempting to execute it for you. It will look in a number of directories for a file with the name mail which is expected to contain the mail program.

      The rest of the words of the command are given as arguments to the command itself when it is executed. In this case we specified also the argument bill which is interpreted by the mail program to be the name of a user to whom mail is to be sent. In normal terminal usage we might use the mail command as follows. 

% mail bill
I have a question about the csh documentation.
My document seems to be missing page 5.
Does a page five exist?
	Bill
EOT
%

      Here we typed a message to send to bill and ended this message with a ^D which sent an end-of-file to the mail program. (Here and throughout this document, the notation ``^x'' is to be read ``control-x'' and represents the striking of the x key while the control key is held down.) The mail program then echoed the characters `EOT' and transmitted our message. The characters `% ' were printed before and after the mail command by the shell to indicate that input was needed.

      After typing the `% ' prompt the shell was reading command input from our terminal. We typed a complete command `mail bill'. The shell then executed the mail program with argument bill and went dormant waiting for it to complete. The mail program then read input from our terminal until we signalled an end-of-file via typing a ^D after which the shell noticed that mail had completed and signaled us that it was ready to read from the terminal again by printing another `% ' prompt.

      This is the essential pattern of all interaction with UNIX through the shell. A complete command is typed at the terminal, the shell executes the command and when this execution completes, it prompts for a new command. If you run the editor for an hour, the shell will patiently wait for you to finish editing and obediently prompt you again whenever you finish editing.

      An example of a useful command you can execute now is the tset command, which sets the default erase and kill characters on your terminal - the erase character erases the last character you typed and the kill character erases the entire line you have entered so far. By default, the erase character is the delete key (equivalent to `^?') and the kill character is `^U'. Some people prefer to make the erase character the backspace key (equivalent to `^H'). You can make this be true by typing 

tset -e
which tells the program tset to set the erase character to tset's default setting for this character (a backspace).

1.2.  Flag arguments

      A useful notion in UNIX is that of a flag argument. While many arguments to commands specify file names or user names, some arguments rather specify an optional capability of the command which you wish to invoke. By convention, such arguments begin with the character `-' (hyphen). Thus the command 

ls
will produce a list of the files in the current working directory. The option -s is the size option, and 
ls -s
causes ls to also give, for each file the size of the file in blocks of 512 characters. The manual section for each command in the UNIX reference manual gives the available options for each command. The ls command has a large number of useful and interesting options. Most other commands have either no options or only one or two options. It is hard to remember options of commands which are not used very frequently, so most UNIX utilities perform only one or two functions rather than having a large number of hard to remember options.

1.3.  Output to files

      Commands that normally read input or write output on the terminal can also be executed with this input and/or output done to a file.

      Thus suppose we wish to save the current date in a file called `now'. The command 

date
will print the current date on our terminal. This is because our terminal is the default standard output for the date command and the date command prints the date on its standard output. The shell lets us redirect the standard output of a command through a notation using the metacharacter `>' and the name of the file where output is to be placed. Thus the command 
date > now
runs the date command such that its standard output is the file `now' rather than the terminal. Thus this command places the current date and time into the file `now'. It is important to know that the date command was unaware that its output was going to a file rather than to the terminal. The shell performed this redirection before the command began executing.

      One other thing to note here is that the file `now' need not have existed before the date command was executed; the shell would have created the file if it did not exist. And if the file did exist? If it had existed previously these previous contents would have been discarded! A shell option noclobber exists to prevent this from happening accidentally; it is discussed in section 2.2.

      The system normally keeps files which you create with `>' and all other files. Thus the default is for files to be permanent. If you wish to create a file which will be removed automatically, you can begin its name with a `#' character, this `scratch' character denotes the fact that the file will be a scratch file.* The system will remove such files after a couple of days, or sooner if file space becomes very tight. Thus, in running the date command above, we don't really want to save the output forever, so we would more likely do 

date > #now

1.4.  Metacharacters in the shell

      The shell has a large number of special characters (like `>') which indicate special functions. We say that these notations have syntactic and semantic meaning to the shell. In general, most characters which are neither letters nor digits have special meaning to the shell. We shall shortly learn a means of quotation which allows us to use metacharacters without the shell treating them in any special way.

      Metacharacters normally have effect only when the shell is reading our input. We need not worry about placing shell metacharacters in a letter we are sending via mail, or when we are typing in text or data to some other program. Note that the shell is only reading input when it has prompted with `% ' (although we can type our input even before it prompts).

1.5.  Input from files; pipelines

      We learned above how to redirect the standard output of a command to a file. It is also possible to redirect the standard input of a command from a file. This is not often necessary since most commands will read from a file whose name is given as an argument. We can give the command 

sort < data
to run the sort command with standard input, where the command normally reads its input, from the file `data'. We would more likely say 
sort data
letting the sort command open the file `data' for input itself since this is less to type.

      We should note that if we just typed 

sort
then the sort program would sort lines from its standard input. Since we did not redirect the standard input, it would sort lines as we typed them on the terminal until we typed a ^D to indicate an end-of-file.

      A most useful capability is the ability to combine the standard output of one command with the standard input of another, i.e. to run the commands in a sequence known as a pipeline. For instance the command 

ls -s
normally produces a list of the files in our directory with the size of each in blocks of 512 characters. If we are interested in learning which of our files is largest we may wish to have this sorted by size rather than by name, which is the default way in which ls sorts. We could look at the many options of ls to see if there was an option to do this but would eventually discover that there is not. Instead we can use a couple of simple options of the sort command, combining it with ls to get what we want.

      The -n option of sort specifies a numeric sort rather than an alphabetic sort. Thus 

ls -s | sort -n
specifies that the output of the ls command run with the option -s is to be piped to the command sort run with the numeric sort option. This would give us a sorted list of our files by size, but with the smallest first. We could then use the -r reverse sort option and the head command in combination with the previous command doing 
ls -s | sort -n -r | head -5
Here we have taken a list of our files sorted alphabetically, each with the size in blocks. We have run this to the standard input of the sort command asking it to sort numerically in reverse order (largest first). This output has then been run into the command head which gives us the first few lines. In this case we have asked head for the first 5 lines. Thus this command gives us the names and sizes of our 5 largest files.

      The notation introduced above is called the pipe mechanism. Commands separated by `|' characters are connected together by the shell and the standard output of each is run into the standard input of the next. The leftmost command in a pipeline will normally take its standard input from the terminal and the rightmost will place its standard output on the terminal. Other examples of pipelines will be given later when we discuss the history mechanism; one important use of pipes which is illustrated there is in the routing of information to the line printer.

1.6.  Filenames

      Many commands to be executed will need the names of files as arguments. UNIX pathnames consist of a number of components separated by `/'. Each component except the last names a directory in which the next component resides, in effect specifying the path of directories to follow to reach the file. Thus the pathname 

/etc/motd
specifies a file in the directory `etc' which is a subdirectory of the root directory `/'. Within this directory the file named is `motd' which stands for `message of the day'. A pathname that begins with a slash is said to be an absolute pathname since it is specified from the absolute top of the entire directory hierarchy of the system (the root). Pathnames which do not begin with `/' are interpreted as starting in the current working directory, which is, by default, your home directory and can be changed dynamically by the cd change directory command. Such pathnames are said to be relative to the working directory since they are found by starting in the working directory and descending to lower levels of directories for each component of the pathname. If the pathname contains no slashes at all then the file is contained in the working directory itself and the pathname is merely the name of the file in this directory. Absolute pathnames have no relation to the working directory.

      Most filenames consist of a number of alphanumeric characters and `.'s (periods). In fact, all printing characters except `/' (slash) may appear in filenames. It is inconvenient to have most non-alphabetic characters in filenames because many of these have special meaning to the shell. The character `.' (period) is not a shell-metacharacter and is often used to separate the extension of a file name from the base of the name. Thus 

prog.c prog.o prog.errs prog.output
are four related files. They share a base portion of a name (a base portion being that part of the name that is left when a trailing `.' and following characters which are not `.' are stripped off). The file `prog.c' might be the source for a C program, the file `prog.o' the corresponding object file, the file `prog.errs' the errors resulting from a compilation of the program and the file `prog.output' the output of a run of the program.

      If we wished to refer to all four of these files in a command, we could use the notation 

prog.*
This expression is expanded by the shell, before the command to which it is an argument is executed, into a list of names which begin with `prog.'. The character `*' here matches any sequence (including the empty sequence) of characters in a file name. The names which match are alphabetically sorted and placed in the argument list of the command. Thus the command 
echo prog.*
will echo the names 
prog.c prog.errs prog.o prog.output
Note that the names are in sorted order here, and a different order than we listed them above. The echo command receives four words as arguments, even though we only typed one word as an argument directly. The four words were generated by filename expansion of the one input word.

      Other notations for filename expansion are also available. The character `?' matches any single character in a filename. Thus 

echo ? ?? ???
will echo a line of filenames; first those with one character names, then those with two character names, and finally those with three character names. The names of each length will be independently sorted.

      Another mechanism consists of a sequence of characters between `[' and `]'. This metasequence matches any single character from the enclosed set. Thus 

prog.[co]
will match 
prog.c prog.o
in the example above. We can also place two characters around a `-' in this notation to denote a range. Thus 
chap.[1-5]
might match files 
chap.1 chap.2 chap.3 chap.4 chap.5
if they existed. This is shorthand for 
chap.[12345]
and otherwise equivalent.

      An important point to note is that if a list of argument words to a command (an argument list) contains filename expansion syntax, and if this filename expansion syntax fails to match any existing file names, then the shell considers this to be an error and prints a diagnostic 

No match.
and does not execute the command.

      Another very important point is that files with the character `.' at the beginning are treated specially. Neither `*' or `?' or the `[' `]' mechanism will match it. This prevents accidental matching of the filenames `.' and `..' in the working directory which have special meaning to the system, as well as other files such as .cshrc which are not normally visible. We will discuss the special role of the file .cshrc later.

      Another filename expansion mechanism gives access to the pathname of the home directory of other users. This notation consists of the character `~' (tilde) followed by another user's login name. For instance the word `~bill' would map to the pathname `/usr/bill' if the home directory for `bill' was `/usr/bill'. Since, on large systems, users may have login directories scattered over many different disk volumes with different prefix directory names, this notation provides a convenient way of accessing the files of other users.

      A special case of this notation consists of a `~' alone, e.g. `~/mbox'. This notation is expanded by the shell into the file `mbox' in your home directory, i.e. into `/usr/bill/mbox' for me on Ernie Co-vax, the UCB Computer Science Department VAX machine, where this document was prepared. This can be very useful if you have used cd to change to another directory and have found a file you wish to copy using cp. If I give the command 

cp thatfile ~
the shell will expand this command to 
cp thatfile /usr/bill
since my home directory is /usr/bill.

      There also exists a mechanism using the characters `{' and `}' for abbreviating a set of words which have common parts but cannot be abbreviated by the above mechanisms because they are not files, are the names of files which do not yet exist, are not thus conveniently described. This mechanism will be described much later, in section 4.2, as it is used less frequently.

1.7.  Quotation

      We have already seen a number of metacharacters used by the shell. These metacharacters pose a problem in that we cannot use them directly as parts of words. Thus the command 

echo *
will not echo the character `*'. It will either echo an sorted list of filenames in the current working directory, or print the message `No match' if there are no files in the working directory.

      The recommended mechanism for placing characters which are neither numbers, digits, `/', `.' or `-' in an argument word to a command is to enclose it with single quotation characters `'', i.e. 

echo '*'
There is one special character `!' which is used by the history mechanism of the shell and which cannot be escaped by placing it within `'' characters. It and the character `'' itself can be preceded by a single `\' to prevent their special meaning. Thus 
echo \'\!
prints 
'!
These two mechanisms suffice to place any printing character into a word which is an argument to a shell command. They can be combined, as in 
echo \''*'
which prints 
'*
since the first `\' escaped the first `'' and the `*' was enclosed between `'' characters.

1.8.  Terminating commands

      When you are executing a command and the shell is waiting for it to complete there are several ways to force it to stop. For instance if you type the command 

cat /etc/passwd
the system will print a copy of a list of all users of the system on your terminal. This is likely to continue for several minutes unless you stop it. You can send an INTERRUPT signal to the cat command by typing ^C on your terminal.* Since cat does not take any precautions to avoid or otherwise handle this signal the INTERRUPT will cause it to terminate. The shell notices that cat has terminated and prompts you again with `% '. If you hit INTERRUPT again, the shell will just repeat its prompt since it handles INTERRUPT signals and chooses to continue to execute commands rather than terminating like cat did, which would have the effect of logging you out.

      Another way in which many programs terminate is when they get an end-of-file from their standard input. Thus the mail program in the first example above was terminated when we typed a ^D which generates an end-of-file from the standard input. The shell also terminates when it gets an end-of-file printing `logout'; UNIX then logs you off the system. Since this means that typing too many ^D's can accidentally log us off, the shell has a mechanism for preventing this. This ignoreeof option will be discussed in section 2.2.

      If a command has its standard input redirected from a file, then it will normally terminate when it reaches the end of this file. Thus if we execute 

mail bill < prepared.text
the mail command will terminate without our typing a ^D. This is because it read to the end-of-file of our file `prepared.text' in which we placed a message for `bill' with an editor program. We could also have done 
cat prepared.text | mail bill
since the cat command would then have written the text through the pipe to the standard input of the mail command. When the cat command completed it would have terminated, closing down the pipeline and the mail command would have received an end-of-file from it and terminated. Using a pipe here is more complicated than redirecting input so we would more likely use the first form. These commands could also have been stopped by sending an INTERRUPT.

      Another possibility for stopping a command is to suspend its execution temporarily, with the possibility of continuing execution later. This is done by sending a STOP signal via typing a ^Z. This signal causes all commands running on the terminal (usually one but more if a pipeline is executing) to become suspended. The shell notices that the command(s) have been suspended, types `Stopped' and then prompts for a new command. The previously executing command has been suspended, but otherwise unaffected by the STOP signal. Any other commands can be executed while the original command remains suspended. The suspended command can be continued using the fg command with no arguments. The shell will then retype the command to remind you which command is being continued, and cause the command to resume execution. Unless any input files in use by the suspended command have been changed in the meantime, the suspension has no effect whatsoever on the execution of the command. This feature can be very useful during editing, when you need to look at another file before continuing. An example of command suspension follows. 

% mail harold
Someone just copied a big file into my directory and its name is
^Z
Stopped
% ls
funnyfile
prog.c
prog.o
% jobs
[1]  + Stopped	mail harold
% fg
mail harold
funnyfile. Do you know who did it?
EOT
%
In this example someone was sending a message to Harold and forgot the name of the file he wanted to mention. The mail command was suspended by typing ^Z. When the shell noticed that the mail program was suspended, it typed `Stopped' and prompted for a new command. Then the ls command was typed to find out the name of the file. The jobs command was run to find out which command was suspended. At this time the fg command was typed to continue execution of the mail program. Input to the mail program was then continued and ended with a ^D which indicated the end of the message at which time the mail program typed EOT. The jobs command will show which commands are suspended. The ^Z should only be typed at the beginning of a line since everything typed on the current line is discarded when a signal is sent from the keyboard. This also happens on INTERRUPT, and QUIT signals. More information on suspending jobs and controlling them is given in section 2.6.

      If you write or run programs which are not fully debugged then it may be necessary to stop them somewhat ungracefully. This can be done by sending them a QUIT signal, sent by typing a ^\. This will usually provoke the shell to produce a message like: 

Quit (Core dumped)
indicating that a file `core' has been created containing information about the running program's state when it terminated due to the QUIT signal. You can examine this file yourself, or forward information to the maintainer of the program telling him/her where the core file is.

      If you run background commands (as explained in section 2.6) then these commands will ignore INTERRUPT and QUIT signals at the terminal. To stop them you must use the kill command. See section 2.6 for an example.

      If you want to examine the output of a command without having it move off the screen as the output of the 

cat /etc/passwd
command will, you can use the command 
more /etc/passwd
The more program pauses after each complete screenful and types `--More--' at which point you can hit a space to get another screenful, a return to get another line, a `?' to get some help on other commands, or a `q' to end the more program. You can also use more as a filter, i.e. 
cat /etc/passwd | more
works just like the more simple more command above.

      For stopping output of commands not involving more you can use the ^S key to stop the typeout. The typeout will resume when you hit ^Q or any other key, but ^Q is normally used because it only restarts the output and does not become input to the program which is running. This works well on low-speed terminals, but at 9600 baud it is hard to type ^S and ^Q fast enough to paginate the output nicely, and a program like more is usually used.

      An additional possibility is to use the ^O flush output character; when this character is typed, all output from the current command is thrown away (quickly) until the next input read occurs or until the next shell prompt. This can be used to allow a command to complete without having to suffer through the output on a slow terminal; ^O is a toggle, so flushing can be turned off by typing ^O again while output is being flushed.

1.9.  What now?

      We have so far seen a number of mechanisms of the shell and learned a lot about the way in which it operates. The remaining sections will go yet further into the internals of the shell, but you will surely want to try using the shell before you go any further. To try it you can log in to UNIX and type the following command to the system: 

chsh myname /bin/csh
Here `myname' should be replaced by the name you typed to the system prompt of `login:' to get onto the system. Thus I would use `chsh bill /bin/csh'. You only have to do this once; it takes effect at next login. You are now ready to try using csh.

      Before you do the `chsh' command, the shell you are using when you log into the system is `/bin/sh'. In fact, much of the above discussion is applicable to `/bin/sh'. The next section will introduce many features particular to csh so you should change your shell to csh before you begin reading it.