Comments on: Intermediate Python: Pythonic file searches

By: wcyee

wcyee — Sat, 16 Feb 2008 21:32:18 +0000

Ryan, thanks for answering my question. I had guessed, but didn’t know, that the GC handled closing file handles. Also, the “with” syntax is very nice. I’ll be using that a fair bit now. Thanks!

By: Ryan Ginstrom

Ryan Ginstrom — Sat, 16 Feb 2008 09:27:23 +0000

@wcyee
Good question. When the call to open goes out of scope, the file handle will be closed upon the next garbage collection. Since GC is eager in CPython, that’ll be very soon. If you’re paranoid about these things, you can use the “with” syntax:
with open(filename) as f:
for num, line in enumerate(f):
pass

Here’s a nice link that explains how file handles and scope work in CPython:
http://www.diveintopython.org/object_oriented_framework/instantiating_classes.html#fileinfo.scope

By: wcyee

wcyee — Sat, 16 Feb 2008 09:12:29 +0000

I’m just learning python and I see code like this often:

for line_num, line in enumerate(open(filename)):

My question is: don’t you need the file handle to close the file? A lot of python code out there never takes that into account. Is there something I don’t know? Or am I just unnecessarily paranoid about resource leaks.

By: Ryan Ginstrom

Ryan Ginstrom — Fri, 15 Feb 2008 06:25:56 +0000

@Pete Cable
Quite right, sorry about that Anonymous. Nice trick about not unpacking the tuples returned by enumerate, although indexing on result seems somehow kind of vulgar…

By: Pete Cable

Pete Cable — Fri, 15 Feb 2008 05:17:46 +0000

@ryan
Anonymous’ list comprehension still gives you line numbers… he just returns the tuple from enumerate. However, the line numbers are zero-indexed instead of one-indexed like yours.

By: Ryan Ginstrom

Ryan Ginstrom — Fri, 15 Feb 2008 02:10:16 +0000

@Ian
Yes, if the number of expected results is large. The enumerator is already a generator.

@Anonymous
True, but I’m assuming that the line numbers are actually needed. (The function is adapted from an actual query of comp.lang.python that asked something similar)

@mackstann
You’re right, I had totally forgotten about that. So the increment operation isn’t contributing to the slowdown; just the line count

By: mackstann

mackstann — Fri, 15 Feb 2008 01:33:31 +0000

Python generally caches numeric (and maybe string) constants, so it actually only creates the 1 integer object once.

By: Anonymous

Anonymous — Fri, 15 Feb 2008 00:52:09 +0000

Or, even shorter and easier to read – remove the line_num, line stuff and just focus on the result:

[result for result in enumerate(open(filename)) if to_find in result[1]]

By: ashot

ashot — Thu, 14 Feb 2008 23:00:23 +0000

One line, most pythonic:

[(line_num,line) for line_num,line in enumerate(open(filename)) if to_find in line]

By: Ian

Ian — Thu, 14 Feb 2008 21:52:20 +0000

If the file size is large you might find a generator approach useful.

def search(target, stream):
    for line in stream:
        if target in line:
            yield line

search = lambda target, stream: (line for line in stream if target in line)

for line in search(“foo”, open(“bar”)):
process(line)