SHELLdorado Newsletter 1/2003 - July 6th, 2003

================================================================
The "SHELLdorado Newsletter" covers UNIX shell script related
topics. To subscribe to this newsletter, leave your e-mail
address at the SHELLdorado home page:

        http://www.shelldorado.com/

View previous issues at the following location:

        http://www.shelldorado.com/newsletter/

"Heiner's SHELLdorado" is a place for UNIX shell script
programmers providing

     Many shell script examples,
     shell scripting tips & tricks + more...
================================================================

Contents

 o  Shell Tip: Returning more than one value from a function (or AWK)
 o  Shell Script: "tarmail" - send "tar" archive by e-mail
 o  Shell Tip: Print lines of a file in reverse (or random) order
 o  Shell Tip: advisory locking for shell scripts

-----------------------------------------------------------------
>> Shell Tip: Returning more than one value from a function (or AWK)
-----------------------------------------------------------------

    A shell function function can return error codes, strings,
    and even multiple values. How? Read on...

    Returning success or failure values from a shell function is
    straightforward, e.g.

        # validinteger - return success if string consists only of
        # numbers
        validinteger () {
            case "$1" in
                *[!0-9]*)   return 1;;  # string contains invalid chars.
                *)          return 0;;  # only digits from 0-9
            esac
        }

    This function can be used as follows:

        if validinteger "$1"
        then echo "valid number: $1"
        else echo "no valid integer number: $1"
        fi

    Command substitution can be used for a string return value, e.g. the
    function "abspath":

        # abspath - return absolute path name (starting with /)
        abspath () {
            D=`dirname "$1"`
            N=`basename "$1"`
            (cd "$D"; echo "`pwd`/$N")
        }

    The following command will resolve the directory "../bin" to an
    absolute path name (e.g. /home/john/bin), and assign this value to
    the variable "path":

        path=`abspath "../bin"`
    
    But can a function set more than one variable? It can, using
    a little trick: the function does not directly print the
    results to standard output, but emits variable assignments
    instead. The calling shell has to evaluate them using, no
    surprise, "eval":

        # Read /etc/password, and get home directory and shell name
        set_home_and_shell () {
            awk -F: '
                $1 == "john" {
                    print "home=" $6
                    print "shell=" $7
                }
            ' /etc/passwd
        }

    Calling this function could result in an output line like the
    following:

        home=/home/john
        shell=/bin/ksh

    A caller then has to evaluate these assignments, making the
    shell interpret the lines as commands:

        eval "`set_home_and_shell`"
        # $home and $shell are set here


-----------------------------------------------------------------
>> Shell Script: "tarmail" - send "tar" archive by e-mail
-----------------------------------------------------------------

    When sending all the contents of a directory by e-mail,
    nobody has to create "tar" archives manually, tediously
    encode them e.g.  using "uuencode", and invoke "mail"
    manually. The following script does all that, resulting in
    commands like

        tarmail john@home.com /home/john

    to send the whole directory hierarchy /home/john to the
    e-mail address john@home.com.

    :
    ############################################################
    # tarmail - create "tar" archive, send it by e-mail

    if [ $# -lt 2 ]
    then
        echo >&2 "usage: tarmail recipient {file|dir} [...]"
        exit 1
    fi

    recipient=$1; shift
    case "$recipient" in
        *@*) ;; # This looks like an e-mail address
        *)  echo >&2 "tarmail: probably no valid e-mail address: $recipient"
            exit 1;;
    esac

    for path
    do
        [ -r "$path" ] || continue

        tarfile=$path.tar
        {
            echo "
    This is an uuencoded 'tar' archive. Save it to a file
    (e.g. "mail.uue"), and unpack it using

                uudecode mail.uue
                tar xvf $tarfile"
            tar cf - "$path" |
                    uuencode "$tarfile"
        } | mailx -s "$tarfile" "$recipient" || exit 1
    done
    ############################################################

    [ Extended version of this script:
        http://www.shelldorado.com/scripts/cmds/tarmail
    ]

    If your system does not have "mailx", you may need to use
    another e-mail client that's able to set subject lines, e.g.
    "Mail" (with a capital 'M') or just "mail".

    The resulting mails can be read e.g. by MS Outlook, Mozilla,
    and Netscape e-mail clients.

    Well, using uuencode is a spartan way to send mail
    attachments.  A more general way is to create MIME file
    attachments, which contain the type of the data that is
    attached (e.g.  "image/gif". It can be used by the receiving
    e-mail client to automatically start the right program to
    display it (whatever this may be, for "application/tar"). The
    following article goes into the details, and lists some
    helper programs and scripts that are useful to have:

        http://www.shelldorado.com/articles/mailattachments.html


-----------------------------------------------------------------
>> Shell Tip: Print lines of a file in reverse (or random) order
-----------------------------------------------------------------

    For printing lines in reverse order, i.e. the last line of a
    file first, we use a simple trick: each line gets a line
    number on which we sort, and then remove later on:

        nl -ba < "input" | sort -nr | cut -f2-

    "nl" numbers lines, "-ba" even empty ones. "sort -nr" sorts
    all lines numerically on the first column, descending order.
    "cut" finally removes the first sorting column.

    Now the idea of printing lines in a random order is not too
    far away. It's e.g. useful for processing MP3 play lists, or
    creating random signature files.  Instead of printing a
    sequential number, we'll print a random number at the start
    of each line, and sort the lines based on that number:

        awk '{ print rand() "   " $0 }' "input" |
                sort -n | cut -f2-

    Note that this requires a recent version of AWK. Solaris
    users should use "nawk" ("new AWK"), GNU awk (gawk) works
    fine, too.

    A slightly more general version for printing lines in a
    random order is available at the SHELLdorado:

        http://www.shelldorado.com/scripts/cmds/shuffle


-----------------------------------------------------------------
>> Shell Tip: advisory locking for shell scripts
-----------------------------------------------------------------

    Sometimes a script may want to run exclusively. A script
    starting a database server may want to ensure that no other
    instance of the same script is starting the server at the
    same time, or a script creating backups on a tape device
    could ensure that it is only run once at a time.

    This kind of advisory locking for shell scripts can be
    implemented by creating (temporary) directories.  Directory
    creation is guaranteed to be atomic, for each process it
    either succeeds or it fails, but it cannot succeed for more
    than one process at the same time. We take advantage of this
    in the following script for advisory locking:

        :
        # makelock - advisory locking for shell scripts

        if [ $# -ne 1 ]
        then
            echo >&2 "usage: makelock lockname"
            exit 1
        fi

        lockname=$1; shift
        lockpath=/tmp/$lockname

        try=1           # first of 5 tries
        until mkdir "$lockpath" >/dev/null 2>&1
        do
            try=`expr $try + 1`
            if [ $try -gt 5 ]
            then
                echo >&2 "giving up on $lockname"
                exit 1
            fi
            sleep 5     # give process time to release lock
        done

        # We now have the lock!

    When the program no longer needs the lock, it can use
    "rmdir /tmp/$lockname" to free it. A usage example:

        if makelock Restore
        then
            # Free lock when program terminates (exit or signal)
            trap 'rmdir /tmp/Restore' 0
            trap "exit 2" 1 2 3 15

            echo >&2 "restore: starting to restore data..."
            # do some work...
        else
            echo >&2 "restore: cannot acquire lock: Restore"
            exit 1
        fi

    Note that this kind of locking does not handle "stale" locks,
    i.e. locks that have been acquired, but have not been released
    by a program (e.g. because of a crash, or "kill -9").

    The program "mklock" is an extended version of "makelock",
    which also implements "unlocking":

        http://www.shelldorado.com/scripts/quickies/mklock


----------------------------------------------------------------
If you want to comment on this newsletter, have suggestions for
new topics to be covered in one of the next issues, or even want
to submit an article of your own, send an e-mail to

        mailto:heiner.steven@shelldorado.com

================================================================
To unsubscribe, send a mail with the body "unsubscribe" to
newsletter@shelldorado.com
================================================================