Perl's Diamond Operator in Python

I'm been a full Python convert for over a year now. Perl is just too cumbersome at times. I think the artist who draws XKCD has made the same choice. The one thing that still takes me back to Perl is the ultra-handy <> "diamond operator". You can't deny Perl's dominance when it comes to string processing. Look at this elegance:

#!/usr/bin/perl

while (<>) {
    print
}

Every time I encounter a string processing problem where I have to work with multiple files, I go back to Perl. I didn't think Python could do this. Python should have the same ability as Perl's diamond:

def diamond():
    """
    diamond()
    Pre: Nothing.
    This is my attempt to recreate the very handy diamond operator found in
    Perl, which is my only reason for still using that language.

    To use the code:
    for line in diamond():
        process(line)
    """
    import sys
    if len(sys.argv[1:]) > 0:
        for file in sys.argv[1:]:
            if file == '-':
                for line in sys.stdin:
                    yield line
            else:
                f = open(file, 'r')
                for line in f:
                    yield line
                f.close()
    else:
        for line in sys.stdin:
            yield line

What does this do? It checks the sys.argv variable for file names on the command line and iterates through each line of each file. If it can't find a file, it defaults to standard input.

Of course, before I start to write code, I should always use Google first. There exist a module called "fileinput" which does the same thing.

D'oh! Hey... it's practice.

Regular Expression Speeds

I read a paper today titled "Regular Expression Matching Can Be Simple And Fast". I thought his results were interesting enough to compare to my own regular expression engine. (Yes, I'm dorky enough to have my own engine.) I call my program "chrep", which is short for "Church's Regular Expression Parser".

Below, I've tested four engines with the string "aaaaaaaaaaaaaaaaaaaa" against the expression "a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?aaaaaaaaaaaaaaaaaaaa" (which should always return a successful match). (Note: Sorry if you aren't able to read the entire command. My CSS-foo sucks and you probably have to read the source code.)

jcchurch@linux:~> time echo "aaaaaaaaaaaaaaaaaaaa" | chrep 'a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?aaaaaaaaaaaaaaaaaaaa'
[aaaaaaaaaaaaaaaaaaaa] in : aaaaaaaaaaaaaaaaaaaa

real    0m5.137s
user    0m2.964s
sys     0m0.012s

5 seconds. Not bad, but not really all that good. The next program tested was egrep:

jcchurch@linux:~> time echo "aaaaaaaaaaaaaaaaaaaa" | egrep 'a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?aaaaaaaaaaaaaaaaaaaa'
aaaaaaaaaaaaaaaaaaaa

real    0m0.055s
user    0m0.008s
sys     0m0.008s

0.055 seconds. Roughly 100 times faster than my parser. The speed benchmark has been set pretty high. The next test is the old, reliable AWK:

jcchurch@linux:~> time echo "aaaaaaaaaaaaaaaaaaaa" | awk '/a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?aaaaaaaaaaaaaaaaaaaa/{print}'
aaaaaaaaaaaaaaaaaaaa

real    0m0.049s
user    0m0.004s
sys     0m0.004s

0.049 seconds. Still making my program look bad. Finally, I wanted to test Perl:

jcchurch@linux:~> time echo "aaaaaaaaaaaaaaaaaaaa" | perl -ne'print if /a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?a?aaaaaaaaaaaaaaaaaaaa/'
aaaaaaaaaaaaaaaaaaaa

real    0m0.393s
user    0m0.328s
sys     0m0.008s

Perl takes almost a half second. That's horribly slow compared to grep and awk, but still 13 times faster than my parser.

My parser ('cause I know you are wondering) is a simple interpretative, recursive decent parser. There is no compiling of the expression, which I know would speed things up. I'm glad to see that I'm not too far behind the competition, and this will encourage me to make those benchmarks.

vi Number Sequence Trick

I'm a vim user. Often times in my editing, I want to number some of the lines in a file from 1 to some ending number. It might be only a few lines, or it might be thousands. To save myself some carpal tunnel, I wrote one line of perl to do the job. This program will pass a block of lines in whatever file you are editing to a one line perl script which will number the lines starting with 1 and drop that text back into the editor:

:x,y!perl -ne'print "$. $_";'

In this example, x is the starting line number and y is the ending line number. That's it!