Real cases/scenarios are some time hard to come by and definitely you want to be prepared when you are in such a situation. It is good that you can create such scenario yourself. In my Part 2 regarding shell script performance, I asked my audiences to come up with the fastest way to create 1000 files. of course you do not want to wait for minutes or hours for just 1000 files. What if you want to test on 10,000 or even 100,000 files ?

With that many files in your directory, you can practice your shell scripting by changing the prefix, suffix, extension, padding with zeros, ..., the amount of things you can script is endless.

In Part 1, we understand that process creation is a very expensive task and we should avoid running too many commands, especially within a loop. So I am not going to use the traditional approach like this one:

count=1; while [ $count -le 100 ]; do touch file-$count.txt; count=`expr $count + 1`; done

Below shows you various ways to 'skin a cat':

create1.sh - use 'seq' to generate 1 to 1000 number to be used in a for loop and each loop create a new file create2.sh - use Bash shell built-in capability to do brace expansion create3.sh - use 'seq -f' to generate the file name and supply that to touch to create all the files. [Thanks to comment by pjz] create.tcl - a Tcl implementation create.py - a Python implementation

And the run time:

As you can see, the less commands you used the shortest the run time. In this case, pjz won!. In create2.sh, I introduced Bash shell brace expansion which is a very useful feature in doing command expansion. See this blog in how I create 48 'devices' (c0d0t0s0, ...) in a single command line.

create1.sh

$ cat create1.sh #! /bin/sh if [ $# -ne 3 ]; then echo "Usage: $0 <prefix> <start#> <end#>" exit 1 fi prefix=$1 start=$2 end=$3 for i in `seq -w $2 $3` do touch $prefix-$i.txt done $ time ./create1.sh file 1 1000 real 2m36.452s user 0m4.260s sys 0m24.288s $ rm -f file-*

create2.sh

$ cat create2.sh #! /bin/sh if [ $# -ne 1 ]; then echo "Usage: $0 <prefix>" exit 1 fi prefix=$1 n="0,1,2,3,4,5,6,7,8,9" eval touch $prefix-0{$n}{$n}{$n}.txt mv $prefix-0000.txt $prefix-1000.txt $ time ./create2.sh file real 0m2.371s user 0m0.031s sys 0m0.856s $ rm -f file-*

create3.sh

$ cat ./create3.sh #! /bin/sh if [ $# -ne 3 ]; then echo "Usage: $0 <prefix> <start#> <end#>" exit 1 fi prefix=$1 start=$2 end=$3 touch `seq -f "$prefix-%04g.txt" $start $end` $ time ./create3.sh file 1 1000 real 0m2.075s user 0m0.108s sys 0m0.746s $ rm -f file-*

create.tcl

$ cat create.tcl #! /cygdrive/c/Tcl8.4.19/bin/tclsh if { $argc != 3 } { puts stderr "Usage: $argv0 <prefix> <start#> <end#>" exit 1 } set prefix [lindex $argv 0] set start [lindex $argv 1] set end [lindex $argv 2] set pad [string length $end] set format "%s-%0${pad}d.txt" for { set i $start } { $i <= $end } { incr i } { set fname [format $format $prefix $i] set fp [open $fname w] close $fp } $ time ./create.tcl file 1 1000 real 0m7.831s user 0m0.000s sys 0m0.031s $ rm -f file-*

create.py

$ cat create.py #! /usr/bin/python import sys if len(sys.argv) != 4: sys.stderr.write('Usage: %s <prefix> <start#> <end#>' % (sys.argv[0])) exit(1) prefix = sys.argv[1] start = int(sys.argv[2]) end = int(sys.argv[3]) pad = len(sys.argv[3]) format = '%s-%0' + str(pad) + 'd.txt' for i in xrange(start, end+1): fname = format % (prefix, i) fp = open(fname, 'w') fp.close() $ time ./create.py file 1 1000 real 0m2.403s user 0m0.031s sys 0m0.951s $ rm -f file-*

Labels: performance, shell script, Tcl