Playing with Python Generators
by Rob Connolly on Aug.30, 2010, under Tips
Today I’ve been playing with generators in Python. I have to say they are yet another awesome Python feature! They give you a very cool and efficient way to store some state within a function, this is useful if you’re trying to do something like generating a sequence, which you would otherwise need a class for. I’m going to go over a couple of quick examples here in order to demonstrate what generators are and what you can do with them.
Side note: You need Python 2.4 or above to use generators, you’ll probably have this anyway, but it might pay to check before wondering why this stuff doesn’t work.
A Quick Introduction
At it’s heart a generator is a function which can return a value several times within it’s body. Each time the function is called it resumes from where it terminated in the previous call, with all it’s internal state intact (i.e. variables are held between calls). In Python a generator is defined by creating a function which uses the ‘yield’ keyword. Each time that a yield is hit the function will return the specified value. Let’s start with an example which defines a generator implementation of the built-in ‘range’ function:
def genrange(start, end):
i = start
while i < end:
yield i
i += 1
As you can see this is incredibly simple and behaves in the same way as the standard range function. The only difference is that when the generator method is called it will return a generator object rather than a value (in contrast to range, which will return a list of values in the range). These generator objects are iteratable so calling the ‘next’ method on the object will run your code and return the next yielded value. Because the generator object is iteratable, the way in which you are likely to use it should look familiar:
for i in genrange(0, 10):
print i
If you want to go ahead and create a list from the sequence as you generate it, you can use a list comprehension, like so:
l = [i for i in genrange(0, 10)]
Of course, as hist comprehensions can be much more complicated than this you can use it to achieve more than just creating the equivalent list as calling the built-in range function. For example, you could filter the output elements according to some criteria, or call a method on each element returned.
A Trade Off
Although the above example is incredibly simple, it presents an interesting trade off between memory use and the speed of iteration through the ‘list’. The trade off exists because you could just build the list of elements in a separate function and then iterate through it. This is a perfectly valid approach but if there are a large number of elements you will use a significant quantity of memory to store the list. You are also more likely to double handle the data, unless you are building the list up over a period of time. Using a generator in this case is likely to be more efficient, as you are just creating the elements as you need them, so only storing a minimal amount of data at any one time.
The other side of the trade off is performance. If the process that produces your iterative data is computationally expensive, it makes no sense to calculate more values than you will need. So if you have a data processing loop, that can potentially exit early (i.e. before reaching the end of the ‘list’) a generator will be more efficient. But equally, if you require a tight loop which iterates quickly, you would want to pre-compute the values.
When considering using generators you should weigh up these factors and think which method is going to be most efficient for your situation. I think in a lot of cases, you will end up coming down in favour of the generator solution.
Another Example (Dan Brown is going to love me)
I’m going to leave you with another quick example, namely a very efficient way of generating Fibonacci numbers using a generator. Without further ado:
def fibonacci():
p1 = 0
p2 = 1
while True:
f = p1 + p2
p2 = p1
p1 = f
yield f
There you go, easy. Many other Fibonacci implementations use a recursive method, which is horribly inefficient. Most other implementations will require either a class to store persistent data in, or the use of static variables, both of which are more tricky to handle from a programming point of view.
Well that’s it. I think I’ve demonstrated that generators are pretty cool. The examples I’ve presented are very simplistic, but I’m sure you can think of awesome uses for them. Go forth and code!
Some Tramping Photos
by Rob Connolly on Aug.18, 2010, under Photos
So, I finally worked out that WordPress can do awesome photo galleries. This means I have somewhere to put my photos that isn’t evil (i.e. not Facebook), that actually works and is simple to use. Yay!
Here are a few photos from some tramping trips I organised over the last few weeks, the first was to the Dome Forest Park and the other was to Pohuehue Scenic Park (the last two photos).
- Dark Group Photo – Dome Forest
- Aiden at top of Trig
- Brendan looks down from Trig
- A View – Dome Forest
- Aiden balances on the Horizontal Tree
- Rebecca, Brendan and Claire
- Another Dome Forest View
- Pohuehue Waterfall
- Pohuehue Group Photo
UALUG Fedora Article
by Rob Connolly on Aug.03, 2010, under Uncategorized
Ahoy’hoy!
Anyone that follows my Identi.ca feed will probably be aware that I recently switched away from Ubuntu and Archlinux and over to Fedora. This was mainly due to frustrations with Arch (I need a system which lets me get stuff done without too much overhead in looking after it) and my increasing feeling that Ubuntu isn’t going in the right direction for me as a power user/developer.
I recently wrote an article on Fedora 13 for our University LUG – UALUG and I thought I’d post a link for readers of this blog. The article focuses on Fedora 13 as a platform for developers and basically details my own Fedora 13 setup. It’s written with the aim of advocating Fedora for people new to Linux, but it also serves of my review of the latest Fedora release.
Anyway here’s the link: https://ualug.ece.auckland.ac.nz/archives/246.
Hopefully I should be posting some more interesting content here soon as I’ve been playing around with some interesting stuff. It’s just a matter of me finding time to write it up! Bye for now.
Denting via Android Intents
by Rob Connolly on May.30, 2010, under Tips
This is just a quick post to share something cool that I was playing with the other day. I was wondering if it was possible to send a message from an Android application via the Mustard identi.ca client. It turns out it is, and here’s how you do it:
Intent i = new Intent(Intent.ACTION_SEND);
i.setType("text/plain");
i.putExtra(Intent.EXTRA_TEXT, "Testing intents on android");
startActivity(i);
These four lines of code hook into a key feature of the Android platform: Intents. The intent system is a neat way of communicating via apps and enables different apps to integrate together without knowing about each other specifically.
In this case, we are sending the ACTION_SEND intent, which Mustard is set up to handle. We set the mime-type of the message to ‘text/plain’ and put the contents of our message into the EXTRA_TEXT field. We then start the intent, and we’re done!
When the intent is started Android presents the user with a list of all the possible applications which can handle the ACTION_SEND intent. In my case this is Email, Gmail, Messaging and Mustard, when I select mustard the new message screen pops up and is populated with my message content. Hitting send posts the message and returns control to my app. Neat!
There are probably a whole host of ways you can integrate your apps with other apps on the phone using the intents system, I guess it’s just a matter of knowing what intents are received by each app and what they do.
Thanks to @macno (main developer of Mustard) and @bugabundo on identi.ca for their help working this out!
Anyway, I hope someone finds this useful. This is the first time I’ve posted about Android development, but I’ll hopefully write more about it in the future. Bye for now!
Why This Apple Fanboyism Really Hacks Me Off!
by Rob Connolly on Apr.06, 2010, under Rants
Firstly, I apologise if this article isn’t particularly well thought out or well worded. I just wanted to get this out there. Secondly, in the interests of keeping my blog clean, I’ve slightly censored some of the language I would have used. I’m sure you can use your imagination to fill in the blanks!
Is it just me or has there been an increasing amount of Apple fanboyism around recently? This is really beginning to annoy me. Principally, because I just don’t care. Apple are completely irredeemable in my eyes, they are worse than Microsoft, but no-one seems to see this, whatever they do!
This is really hacking me off in the Open Source media, particularly on podcasts (I’m looking at you Shot of Jaq!). Frankly, I don’t give a <censored> what Apple are doing, or what they might do in the future. I don’t listen to Open Source podcasts for this.
Then there’s Ubuntu, which seems to be moving closer and closer to the Mac in terms of looks (this isn’t necessarily a bad thing, but why can’t we be more inventive?). The Ubuntu community seems to be going
this way too. In a recent episode of the Ubuntu UK Podcast the words ‘the Mac just works’ were uttered. Hello! So does Ubuntu! (of course I can only speak from my experience, but I rarely if ever have problems).
As a solution to this growing epidemic, I propose a simple test which you should use if you are tempted to praise Apple in public, particularly in the Open Source media.
Simply answer the following five questions:
1. Name five pieces of Open Source software that Apple have contributed to the community.
2. Would you buy an iPhone over a Nexus One? Seriously?
3. Is the iPad even an original idea? (clue: No.)
4. If the Mac is so great, why is it so damned hard to use? Why can’t it be configured to work they way I want?
5. Do you like DRM? Apparently, neither does Steve. Slightly hypocritical, don’t you think?
If you can’t answer these simple questions in Apple’s favour, KEEP STUM!









