Manni Wood

Here’s a shout-out to a blog entry that says “Design patterns are common structures that appear in computer programs, but that can’t themselves be abstracted into a reusable component”.

I remember early on, when porting some of my Java code to Python, I wanted to make a singleton in Python. I turned to my copy of the Python Cookbook, and found that singletons are very rarely needed in Python, because Python’s modules exhibit essentially the same behaviour. It was one of those learning moments for me, when I realised that I was trying to write Python like Java, and that I should try to write Python more like Python. (In my own defence, we all do this when learning a new language. :-)

Here’s an observation by the always-fascinating Steve Yegge on design patterns:

About half of all design patterns out there (not just the GoF patterns) appear to be ways take perfectly natural design ideas and twist them to fit into someone’s static type system: recipes for pounding square pegs into round holes.

—Steve Yegge, Is Weak Typing Strong Enough?

Yegge’s quote also rings true to me, because I just don’t find myself doing a lot of design-patterny stuff in Python, presumably because Python’s dynamic typing and powerful built-in data structures (lists and dictionaries) prevent me from having to use a lot of objects and their attendant design patterns in the first place.

This makes me want to really, really learn Common Lisp, or Arc, or Scheme, or some language with full-fledged macros. What does this have to do with design patterns? I can’t really tell you, because I don’t know Common Lisp yet.

But here’s a sneaky suspicion I have.

I suspect that the same way half of the design patterns that Yegge mentions were invented to compensate for languages that lack dynamic typing and higher-level constructs, the other half of design patterns, and even entire frameworks (think: Ruby on Rails, Django) compensate for languages that lack true macros.

Now it could turn out that I’m wrong about this, or that there’s only a kernel of truth in my suspicion, but that the whole truth is more complicated.

But readers of my blog know that I give a lot of credence to Paul Graham’s idea that mainstream programming languages are slowly converging on the feature set of Lisp.

I’ve never heard of a Lisp or Arc framework for building database-backed web sites. Where’s Lisp’s Django, or Arc’s Rails, or Scheme’s Struts? I suspect that these languages don’t need frameworks, because macros are essentially tools for building project-specific frameworks.

Who among us has taken a framework like Django or Rails or Struts, and used the parts that solved our problems, and re-written or ignored the parts that didn’t solve our problems, or that introduced new problems? Now imagine what it would be like using a programming language that was essentially a framework construction kit; that allowed you to write a framework more quickly than learning and adjusting to a pre-existing framework?

The only higher-level abstraction I can think of off the top of my head is a language that enables you to easily construct your own domain-specific languages. (I like regular expressions for text searching and manipulation, and SQL for data searching and manipulation; imagine your own DSL for your exact business problem! That would even beat frameworks or even macros…)

Apparently, Perl 6 and the Parrot VM are going to try to tackle that exact problem. The problem is, Perl 6 hasn’t shipped yet. (I’ve only been waiting 10 years… ;-)

In the meantime, I really need to learn Lisp or Arc once and for all. I really want to learn about macros.

Any readers of my blog will know that my experience with data lies almost exclusively in data that are best modelled in a relational way. So I have a lot of biases that have evolved through playing a lot in that problem space:

• I like SQL a lot, but realise the SQL standard is an imperfect implementation of the relational model.
• (Heck, I know what the relational model is, after reading lots of C. J. Date.)
• I like writing SQL by hand, (roughly analogous to the way designers like to code their HTML/CSS by hand) because I think I get the best results that way.
• I think the object/relational impedance mismatch is best solved by simply avoiding the situation: don’t use ORM.
• If your data set is small but must be accurate (as has been the constraint on most of my projects), use a database that does this for you. (That is, declaring constraints in DML and having your RDBMS do the rest is a beautiful thing.)

On the other hand… :-)

I’ve been interviewing a places that use non-traditional data stores, sometimes on massive data sets, where ACID compliance is not paramount.

The challenges some of these project face sound really cool, and the tools they are using to meet those challenges would be a lot of fun to learn.

So first off, I should make something really clear: I have reasonably passionate views about RDBMSs, especially regarding small, accurate data sets, because I’ve done a lot of work in this area, with real-world consequences if I didn’t get things correct.

However, I also feel that not all data are relational!

Although I agree with C. J. Date that no other data model compares to the relational data model (at least in terms of being able to be specified in a rigorous way), not all data have to be stored in a way that conforms to the relational model.

For instance, I remember being a little bit afraid early in the development of Windows Longhorn (later to become Vista) when it seemed like they were going to make the entire file system a database. Really? I rather like the hierarchical file system! It turns out the Windows design team came to the same conclusion too.

Take server access logs. They are stored as flat files. Would they benefit from being stored in an RDBMS? I don’t know. It would be nice being able to run SQL queries on them. But you could also make the argument that sometimes a flat file is just a flat file.

Take documents. Some people store them in Lotus notes, some people store them as binary blobs in RDBMSs, and most of us still keep documents on whatever hierarchical file system our operating system currently uses.

Although I love the relational data model, as soon as you decide to use it for your data, you are immediately making certain decisions about the nature of your data, either implicitly, or explicitly.

I’ve played in the relational world for quite some time now, and am excited about looking at data through a different lens.

So maybe I’ll work on a project where modelling the data with objects rather than relations is the absolute right thing to do!

Maybe I’ll be on a project where transactions and constraints have to be enforced in the application layer, because the underlying data store cannot enforce these things itself (and that would be because the underlying data store has other strengths).

Maybe I’ll be on a project where the object model can be the definitive data model, because no other applications are ever given direct access to the underlying data store; so the data store will not have to be the final arbiter of what the data model is. Instead, the data will only ever be accessible as a web service, and so the API that we expose to the data will be the only legitimate view of the data.

I’ve played in the world of small, accurate relational data sets for quite some time. It could be fun to work with really large, semi-structured data sets, and see what sorts of solutions they require, and maybe challenge some of my core assumptions about data.

One thing I like about the job interview process (as I am going through it now) is that you learn what your blind spots are as a developer. It’s not very much fun to find out what those blind spots are, but I did get into this profession because I liked learning new things. And what’s more useful than learning about your weaknesses?

I had a great discussion with a coder I used to work with today, and he pointed out something that was staring me right in the face: I may agree with Ted Neward, Jeff Atwood, et. al. about object-relational mapping being an insurmountable problem (the Vietnam of computer science, as Neward called it) but my coder friend said:

“Look at the market place! ORM has won!”

And he’s right! So what I need to be asking myself is, if ORM really is the Vietnam of computer science, why is everybody using it? There’s an opportunity for me to learn something here, and my attitude towards ORM is not reflective of my more nuanced attitude towards a lot of other things.

Case in point: I was recently asked my opinion on stored procedures in the context of building web applications, and I gave my typically nuanced answer about their strengths and weaknesses, and how they had their uses, but did not need to be embraced 100% nor rejected 100%.

And that same truth is staring me in the face with ORM, and I’ve been ignoring it.

I have mostly worked on sites where the data model was already written, was quite messy, and the CRUD (create, read, update, delete) operations were all non-trivial. My blind spot so far has been assuming all situations are like this. The popularity of ORM seems to indicate that this is not always the case.

In fact, in discussions with other developers, I find that most projects take a hybrid approach to ORM: they use it where it speeds up development and does not adversely affect application performance, and they do not use it where hand-coded SQL is the better way.

My programmer friend recommended I take a look at ibator, a code generator for iBATIS that is sort of like a reverse ORM: it introspects your database, and creates all the CRUD code and Java objects for you. I don’t think I’m quite ready to dive into Hibernate yet, but ibator seems like a good place to start. If I get time, I will blog about my experiences with ibator.

I just finished re-reading Paul Graham’s hackers and painters essay (I actually have the book), and my love of the REPL came at me from another angle.

Graham talks about how programming is like painting, and it struck me that I like the REPL because it’s like a sketch pad. If I was painting some large work, and I came to something that I wasn’t good at (let’s say a house on a hill, and I was not very good at houses yet), I’d do a few test houses on some bits of paper or maybe even a small canvas, to sort of explore how to do houses well, and how I wanted my house to look. Then, after I was confident of a good result, I’d go back to my larger work and paint the house in.

(And, as Graham points out, because I’d be working in oils, if I really messed the house up, I could always paint over it. Apparently, X-rays of paintings show that even the great masters did this.)

I admire Graham because he so fully understands what an exploratory process programming is—in the real world, anyway. So anyway, if my main project is a large canvas, the REPL is my sketch pad.