The most common metric is LoC – Lines of Code. It’s a simple metric: How many lines of code did it take to create a particular piece of software? For non-programmers this is a pretty good metric. It gives them something to nod their heads at in meetings. For programmers this isn’t a very good metric at all. As with any prose, lines of code can vary in utility, complexity, and meaning.

Programmers would like to be able to gauge size and complexity in a more meangingful way. Other metrics are offered from time to time. The number of classes. The number of endpoints on a service. The number of possible execution paths. Many possibilities exist. Automated tools start to crop up.

All of these measurements are fine if they give some insight into a project. But there’s this missing piece that never seems to figure in: People.

In particular, how many people have touched the code?

This is an unavoidably fuzzy metric. Each person’s impact on the code is unpredictable on its own. What is undeniable is that a 50k line application with a 3 programmer history is a totally different beast than a 50k line application with a 15 programmer history.

(Tellingly, neither the Wikipedia page for Programming Complexity nor the page for Software Metric contain — as of this writing — the word “person” or “people”.)

Tallied up, perhaps the number of people who worked on a codebase can give us an added dimension to consider.

One lingering regret is that I never learned to ollie. After I got comfortable skating again I set my mind to learning how to ollie. Last summer I made a small bit of progress, and this summer I’m hoping to finally nail the damned thing.

This weekend I shot some footage of myself practicing. Then I wanted to use that footage to make a little movie. For me, on my Mac, that means iMovie.

I’ve had some modest successes with iMovie in the past, but I’ve never really understood it. The interface is confusing. I’m sure they are trying to make things easy, but it’s like there are so many things to make easy that they make it crazy instead.

I had two and a half minutes of footage of me flubbing around with the skateboard. What I wanted was to grab two portions of the video (where I was actually practicing) and join those together with a modest transition. Then I’d add a title screen and upload it to YouTube to embarass myself.

Simple, right?

First I had to figure out how to get my video out of Photos and into iMovie. I imagined that I could just export the file and then import it, but that seems like The Hard Way. Apple ships the computer with both programs. There must be a Better Way. Off to Google I went, which is ok because I’m basically a Professional Google User (uncertified). Much to my consternation, the recommended way to get your movie from Photos to iMovie is… to export it to a folder and then import it. Stone knives and bear skins!

I did the export/import and created a new project in iMovie. That part went fine. I had a project, I imported my footage, and then I started to make a title. That took a while to figure out (despite the fact that I’ve done it before) but soon my title was in place. Ah, but I needed to split up my footage into two parts. How do I do that? More googling. It was easy but not obvious. I finally figured out how to split it up, but then realized that I have the opportunity to have a cold open on my video where I walk up to the skateboard. So this will be great. Cold open to establish I’m doing something with a skateboard, then title, then my two clips. Ah, but what would be really cool is to have the background audio going over the title sequence so there is continuity in my little story. Googling again, I find out how to do this in iMovie 11. (Later I find out I’m on iMovie 10.) I can’t find the option I found with Google, but poking around in the right place helps me figure out that I can detach the audio. Perfect! This is great. I mess around a little more and then I have a cold open, a title, and then I’m on the skateboard. Something is not quite right though. Ah, the title card just pops in. There is a tiny effect but no smooth transitions. I see the option for transitions and pick the simplest and put it in place. Now everything is set up! I sit back and watch. Something is still wrong…. Ah! The audio is out of sync. Probably by the amount that it is offset during the title sequence. Or maybe I just dragged the wrong slider and screwed something up. I really don’t know. It’s all voodoo at this point. I have no idea what is going on and no idea how to fix it.

I gave up. I wasn’t pissed, just disheartened. I could see nothing but frustration in my future, so I stopped.

And this is The Hump.

The Hump is this point on your learning curve, usually after some early success, where you decide that the effort just isn’t worth the reward. This is the point where opportunities are lost, where you talk yourself out of victory, and where dreams die.

Seriously.

Look how close I got to success before I gave up! This is common. And then there was my bad attitude. I felt like I was being held back. I blamed the software. I blamed Apple. It wasn’t until I realized this was just The Hump that my attitude started to change. I stopped thinking about how messed up the situation was. I started to think about how I was going to get my video done. I went from being indignant to resolute. I went from being pissed at tools to being pissed that I was letting little things get in my way.

I tossed out the crappy work I’d done earlier and stepped back. I started to reframe the problem. I realized that since I now knew how to split footage, I could at least start by breaking things up into clips. With that done it was easy to delete the cruft. I was left with a series of scenes that were keepers. That formed the backbone of my movie. Instead of trying to build the movie from the front to the back, I created a scaffold along the critical path of what I was trying to do.

I eventually got my movie done. It took about an hour more than I thought it would, but I shipped. (WARNING: Super-boring single-shot video of 42 year-old legs mostly not ollying. My wife panned it, it’s that boring.) The movie isn’t the exciting part. Beating The Hump is the exciting part.

I worry that The Hump is this quiet killer of potential engineering careers. It’s great to see the energy behind programs to encourage people to learn to code, but it’s not enough. You can show people how to code all day long and all you’ll ever create is a bunch of people who can follow instructions. That isn’t what programming is. Software is a particularly cruel medium. You don’t just get over The Hump when you learn how to code. The act of building software is a non-stop series of Humps. Even worse, software eats problems. That means today’s hard problems are always a little harder than yesterday’s. Meaning the Humps never stop coming.

May you turn your Humps into moguls.

Bored. So bored.

My aunt walks in with a magazine she thinks I might like.

I read Radio-Electronics for years. From grade school, through junior high, and into high school. I kept every copy. I would read and re-read. Trying to understand what I didn’t already. Dreaming about having the tools and parts to build the kinds of projects that I would read about. Very often I’d dream up my own projects, and sometimes cobble them together with the parts I could lay my hands on. Electronics absorbed me. I spent most of my teenage years split between reading science fiction and scheming to build circuits that reached toward making that fiction into fact.

Fast forward to my late teens. College wasn’t in the cards for me right away, despite all of the college prep. Somehow I’d managed to move out on my own into a little apartment I could barely afford on my electronics-assembler wages. It was a miserable existence in many ways, but my one great pleasure was Tower Books, my local bookstore. I would spend hours browsing their cramped stacks. I stayed mostly in the areas I understood – physics and electronics. (There was this baffling section on software, but it wasn’t accessible. Every book seemed to be about things that were totally made up!)

Electronics books always got me excited. With a few exceptions (another build-your-own-robot book?), I was stoked whenever something new would come in. I would linger in the bookstore for as long as I would have spent at the movies, but usually spent less. I often left empty handed, or maybe with a copy of Circuit Cellar magazine. In any case, a cheap evening of entertainment.

One day I saw a book by a familiar author I remembered from Radio-Electronics, Robert Grossblatt. I can’t remember which of his two books I bought first. It doesn’t really matter. There’s a common theme between them. These books are ostensibly about engineering electronics, but are applicable to engineering in general. Because of that, if you can read between the lines, they are some of the best foundation-building books on software development you’ll read.

I’m not sure I can properly articulate the value of these books to me. At that time especially I was broke and couldn’t afford to build the projects described. I read them anyway, preparing my mind for what I didn’t know was coming next: A career as a software developer. Those books gave me a perspective that helped me through struggles and to value and appreciate the field I was entering.

So whatever happened to Bob? I don’t know. I’ve googled a bit and haven’t found anything.

Wherever you are now Bob, thanks.

I’ll close with just a few of Bob’s “Laws of Life & Design”.

These excerpts are from “Bob Grossblatt’s Guide To Creative Circuit Design” and “The 8088 Project Book”, both © Robert Grossblatt, published by TAB Books. (A lot of TAB books were crap but they did publish a few gems and these were two of them.)

Here’s an example from Satchmo, the web shopping cart software that is built on the Django framework. A bug involving MySQL was recently patched in the Satchmo code base. The bug fix itself isn’t very interesting, but the remaining code is. Here’s the post-fix code:

def payments_completed(self):
    q = self.payments.exclude(transaction_id__isnull = False, transaction_id = "PENDING")
    result = [p for p in q if p.amount]
    return result

Naturally, I have questions.

Why does the ‘result’ variable exist?

def payments_completed(self):
    q = self.payments.exclude(transaction_id__isnull = False, transaction_id = "PENDING")
    return [p for p in q if p.amount]

What are ‘p’ and ‘q’?

def payments_completed(self):
    query_set = self.payments.exclude(transaction_id__isnull = False, transaction_id = "PENDING")
    return [payment for payment in query_set if payment.amount]

Would a non-programmer understand this?

def payments_completed(self):
    payments = self.payments.exclude(transaction_id__isnull = False, transaction_id = "PENDING")
    return [payment for payment in payments if payment.amount]

Don’t we already have a ‘payments’ variable?

def payments_completed(self):
	return self.payments.exclude(transaction_id__isnull = False, transaction_id = "PENDING", amount__isnull = false)

I think that’s better. What do you think?

The past few years of wide-ranging language interest have cast doubt on that perspective. Still, one theme that I can’t shake is that you should be able to model new features in an existing language. It might be awkward, but it should still be possible. (Especially if you’re going to implement the language!)

In that spirit, I’ve tried to express one of the core ideas in Wheeler with a library for Python called AssociativeTools. The star of the show is the Workspace class, which represents a data structure I call an Associative Network.

The word “network” is overloaded. Most often it is used to refer to computer networks. Here, I’m referring to a network or web of data. Think of a big pool of data in which nodes can be arranged into graphs of your choosing.

The Workspace lets you relate items (anything that is hashable in Python) to one another. Mostly I’m using strings and numbers. You relate items to build graphs so that later you can come back and fetch those graphs with partial information. It’s a lot like querying a database. It’s more like querying a search engine. Look:


s = Workspace()
s.associate("foo", "bar", "baz")
# Later in the code...
print(s.comprehend("foo"))


Outputs:

set(['bar', 'baz'])

Btw, the naming for all of this (AssociativeNetwork, associate, comprehend) is all flexible. Suggestions are welcome.

We work with a small but quite tangled system. The system itself is only a few years old but contains many technologies and bears the scars of several smart but inexperienced people learning how to design as they went. I won’t articulate the details of the bug we were experiencing but I will say that it is something that recently appeared and was troublesome but non-critical.

The difference in our two points of view became apparently very quickly. My desired approach was to try to pin down the bug, deconstruct it to the point of understanding, and then apply a fix. My coworker’s approach was to make a well-educated guess about the cause of the bug and apply a refactoring to our codebase that would address the anticipated problem.

His reasoning was that our codebase was too complex to spend time trying to understand it in detail, and in the worst-case scenario we would still have completed a useful refactoring that improved the quality of our codebase. I found this a bit hopeless. Is software really so complex that we as programmers now have to act like doctors – diagnosticians – making reasoned guesses about the world around us? Is there no more engineering?

10 or 15 years ago I would have considered his approach ridiculous. Now I’m honestly not sure. Modern software systems contain so much abstraction and layering that it is really hard to judge the level of effort that will be involved in addressing any one problem.

So what option did we choose? We guessed and did the refactoring. I don’t think I would have agreed to that with just any pair partner, but this particular coworker is a very productive programmer. He had a clear path forward and a clear rationale. And one thing you don’t do with a productive programmer is get in their way.

Did it work? No. I almost want to say “OF COURSE NOT!” but that would be unfair. It could very easily have worked. What did eventually work? A pair sitting down and tracing through the application stack with gdb. And even that has only located the approximate location of the error. We still haven’t solved the problem yet. But we’re much closer.

That video is about how cell membranes work. Did you notice how proteins, enzymes, etc. can only pass through the cell membrane if they are the right shape (and flavor)? That’s how values trigger transitions in Wheeler.

Transitions

If categories are the most minimal representation of state, then transitions are the most minimal representation of action. Transitions are how computation happens in Wheeler. After all, organizing data is only half the battle with programming. Once you have the data in the form you want, you have to actually do something.

You can think of transitions as being very much like an event handler. You define a transition to trigger when an interaction between one or more categories happens.

Since Wheeler is still a work in progress, transitions are not yet native. Instead, transitions are created in Python, the language Wheeler is built on.

Remember our Hello example?

Here’s the transition defined for it in Python:

def printer(category):
	for item in category:
		print item.name[1:-1],
	print
	return []

ROOT.create('print').add_handler( STRING, printer )

(I’ve omitted some unimportant details for clarity.)

What you might be able to suss out here is that there is a ROOT category that all categories live within, and I’ve created a “print” category within that. In addition, I’ve added a transition (with add_handler) which knows to watch for interactions with items in the string category. If the transition finds an interaction that matches, it executes the printer() function, which simply calls Python’s ‘print’ function.

Mutual Dispatch

When more than one transition is triggered by an expression, the transitions all execute AND – at least in concept – they all execute *at the same time*.

This is very much like the real world. If two objects collide you don’t say that one or the other of them collided first. The collision is mutual and simultaneous. In that same way, when categories interact, the interaction is mutual and simultaneous.

Here’s the final snippet of Wheeler for this introduction:

In case it’s not clear what is going on here, we have some numbers, which are in the category ‘number’, some text in the category ’string’, and some other categories (’print’, ‘+’) which give it all meaning with transitions. One transition knows to add all of the numbers in the expression when the plus sign is present. Another transition knows to print all of the strings in the expression to stdout when the ‘print’ category is present.

When presented with this jumble of categories, Wheeler figures out what to do based on the criteria of the transitions. This effectively creates a typed-dispatch mechanism. I call this simultaneous, polymorphic behavior “Mutual Dispatch”.

That sums up Wheeler as currently implemented.

There’s still a lot of work to be done. I’ve covered what Wheeler is currently capable of. There are other planned features that I didn’t cover and hope to implement soon. I will provide more info here as new features are added.

I hope you find it as oddly fascinating as I do! Feedback is not just requested, it is begged for. I also strongly encourage you to grab the code, play, and contribute.

(Thanks to Westside Programmers and #pdxfunc for letting me present on Wheeler a few times over the past year. Having the chance to present and receive feedback has firmed up the concepts in my mind and made them easier to distill into this introduction.)

Someone recently asked me if Wheeler had types. My answer was “No. Uhhh… and yes.” What a terrible answer! I’ll try to do better in this post.

Wheeler has no explicit type system. You will never declare a value to be of a certain type. You might, however, associate a value with a particular category through an interaction. In that way you’re establishing a relationship that will allow typing *behavior*.

What is a value?

Values are the imaginary, interstitial things floating between categories. When you establish a relationship between categories in Wheeler, you’re creating values.

What is a type?

I won’t attempt to answer this in general. I am not a type theorist. From the perspective of Wheeler we might say that a type is determined by a graph’s “shape” and “flavor”. Shape is the actual form of the graph we create in memory (tree, ring, object, etc.). Flavor is what specific categories are being connected by that graph.

What the hell is the difference?

Context. It’s all context. A value will trigger typing behavior when something matching that shape and flavor is being watched for. At that point it could be considered a type. The rest of the time, a value.

Layering values into types

Have you ever seen an artist paint a landscape? First they’ll paint the background… the sky, the horizon, the land. Then they start to layer on more and more detail. At what point does their work become a painting? Existential arguments aside, you might say that the work is a painting when it meets the criteria of the artist. Criteria will vary from artist to artist. Everyone has their own objectives, aesthetics, etc. In other words, everyone brings their own point of view or context to a situation. This idea is so fundamental in Wheeler that it is a core value of the language:

Context Drives Behavior

To help understand this, let’s look at a Venn diagram. (Venn diagrams are quite handy for discussing concepts in Wheeler.)

Here you see we’ve got just a category, Cars. You could think of that as a type. Let’s pile on some more specifics:

Now we’ve got all cars that are Honda Elements. That’s more specific, but it’s still more like a type than anything.

Ah, but now that we have a VIN, the unique Vehicle Identification Number. That’s so unique as to specify a particular vehicle. That seems more like a concrete value now. Yay.

Next time we’ll look at Transitions which allow computation in Wheeler. Then we’ll exercise our type and value scheme.

(You can play along at home by grabbing the latest code and running the examples. You’ll need Python 2.5 and patience.)

First, let’s fire up “bigwheel”, the Wheeler interpreter:

Here is “Hello, world” in Wheeler:

Wheeler is flexible though, so you could also say:

More on that later.

Categories

Wheeler is different from many languages you may be familiar with. All languages have abstractions. Some common examples are functions, lists, collections, stacks, pointers, and objects. In Wheeler, state is the fundamental abstraction.

State is represented by an abstraction called a Category. Categories are lightweight, atomic things. In Wheeler, everything is a category. Here are some sample categories:

• Fred
• blue
• 5.3
• “Hello, world!”
• number
• 3/15/10
• send
• 05:10:59
• AM
• PM
• true
• print

Categories express boolean state. You’re either in a category or you’re not. Here’s an example:

Do you see that? No? Good. What you do not see is ‘apple’ being put into the ‘red’ category. That is happening behind the scenes through an interaction between apple and red.

Categories aren’t containers, but from a practical standpoint they act like containers. This allows you to make statements about the world. We can pile some more attributes on “apple”:

Interactions

Interactions are expressions that combine categories to create new states. Our “Hello, world!” was an example of an interaction. Making our apple red was another interaction. Interaction is a mutual, simultaneous event where a connection (graph edge) is created between each item in the expression.

Wheeler has a “dump” command that will let you see (in GraphViz) the relationships you’ve created. Let’s dump “apple” and see the relationships:

And the relationships as diagrammed in GraphViz:

Here you can see that “apple” is related to “red”, “tasty”, and “sweet”. (You might notice that it is also related to “dump”, the category it interacted with to generate our diagram, and “metadata”, a built-in category.)

You can see that not only is ‘apple’ connected to ‘red’, but ‘red’ is also connected to ‘apple’. In Wheeler all relationships are bidirectional. This may seem odd if you’re used to typical hierarchical relationships in programming. Wheeler doesn’t think about the world that way. Wheeler has a few strongly-held beliefs, and one of these is:

Hierarchy Is Bullshit

Next time we’ll talk about composing categories into types.

Note: This is the conclusion of a two-part entry in my ongoing “Road to Wheeler” series.

In my last post I covered some of the popular type schemes used in programming today. I continued a little past where most discussions of type go when I pulled in pattern matching.

Deconstructing Types

Imagine that you are using a language with no data types and you want to implement some of the typing behavior that I showed in my last post. One way you might do it is with a record used to associate type information with a value.

A C-style int might look something like this:

x = (int, 5)

That’s pretty simple. We’re storing our value along with it’s type. Using that same scheme we can also represent a typical object instance:

vendor = (Vendor, FirstName, LastName)

“Vendor” here is the classname, but it could also be an interface name. Or the record could contain a list of interfaces supported. Or, a list of methods implemented.

The pattern matching example from last time is already in record format. We’ll just change the Erlang braces to parentheses for continuity with the other examples:

(customer, FirstName, LastName)

Let’s list all of these records together to compare them:

(int, 5)
(Vendor, FirstName, LastName)
(customer, FirstName, LastName)

See the similarity? The types are from different methodologies, but they boil down to equivalent concepts.

Our Own Imaginary Type Implementation

I could easily define a function in my imaginary typeless language to properly handle any of these records:

def display(record):

	type = record[0] # Key off first item in the record.

	if type == int:
		print_int(record)

	elif type == Vendor:
		print_vendor(record)

	elif type == customer:
		print_customer(record)

Is pattern matching a typing scheme? Do typing schemes devolve to pattern matching?

One thing I left out of my pattern matching example from last time: Patterns can match concrete values, such as specific numbers. In this example I’m expecting to find out just how overdue a customer’s account is:

bill_with_msg({customer, 0}) -> send_bill("Here's your new bill.");
bill_with_msg({customer, DaysOverdue}) -> send_bill("Your bill is ~p days overdue.", DaysOverdue).

The first pattern matches on the atom “customer” and the specific value of 0 for the number of days overdue.

If we rewrote this in our typeless language with records it might look something like this:

def bill_with_msg(record):

	type = record[0]
	days_overdue = record[1]

	if days_overdue == 0:
		send_bill("Here's your new bill")
	else:
		send_bill("You're bill is $days_overdue days overdue.")

Guards Point The Way

Have you seen guards? They’re pretty handy. Here’s an example of guards in Erlang:

bill_with_msg({customer, 0}) -> send_bill("New bill.");
bill_with_msg({customer, DaysOverdue}) when DaysOverdue <= 30 -> send_bill("Your bill is overdue.");
bill_with_msg({customer, DaysOverdue}) when DaysOverdue > 30 -> send_bill("YOUR BILL IS SERIOUSLY OVERDUE. PLEASE PAY NOW.").

Those “when” clauses are guards. And they’re pretty nice. In a fairly concise and clear way we’ve expressed our desire to do several things with relatively little code. In the process we’ve managed to unlock a new dimension to typing. If you accept that typing is essentially pattern matching you can see that the guards are actually adding extra dimension to our types by letting us do very specific pattern matching.

The trouble with guards is that they are bound to a function. What if you want to capture a certain type of customer (one who has spent more than $10k this year) so you can target them directly? You might find yourself with functions like this:

bill_customer(customer, spent) when spent > 10000 -> preferred_billing().

find_shipping_options(customer, spent) when spent > 10000 -> preferred_shipping().

lookup_special_offers(customer, spent) when spent > 10000 -> offer_preferred_promos().

That’s a lot of repetition; a code smell. If you’re an OO type of person you might break that kind of customer out into it’s own class, generated by a factory. Or perhaps you might decorate your customer objects with particular account classes. And of course there might be many other strategies.

This might also be a good place for a macro if you’re a Lisper.

Another way might be to declare a type with the guard built-in. Here’s one way this might work with some made-up syntax:

typedef Customer = Customer when spent < 10000
typedef GoldCustomer = Customer when spent >= 10000 and spend < 10000
typedef PlatinumCustomer = Customer when spent >= 100000

Here we’re creating some concise types on top of an existing type. Instead of subtyping or aggregating classes, we’re taking a class or data structure (Customer) and describing some situations where we’d like it to be treated specially. Now we can have polymorphic behavior on anything – types, values, or ranges of values.

This scheme will stretch a little farther:

typedef Spam = String when contains "viagra"

def proc_mail(Spam msg):
	mark_and_toss()

def proc_mail(String msg):
	deliver(msg)

Of course you can only do this with lazy type evaluation, so there’s some runtime cost. I’m ok with that. Laziness has its virtues.

What I’ve presented so far is interesting but still rather flat. We can go farther. What if we can look at the shape of things too? After all, an interface represents a sort of shape. What if you just want to support objects that have a “Text” property? What if you only want to deal with Customers who have blue items in their RecentPurchases collection? Or more abstractly, what if we only want to deal with objects that expose a hashtable containing other hashtables?

Sound weird? It shouldn’t. It happens all the time, it just depends on the environment you work and code in. See also: jQuery, LINQ, RSpec, Hamcrest, etc., etc..

This Way To The Egress

I haven’t seen a type system that does exactly what I describe. Maybe you have. If so I hope you’ll tell me about it. Haskell’s type system is pretty nice (and I’m still learning), but seems to stop short of the dynamic behavior I’m looking for. Wheeler, the language I’m working on, aims (at least in part) for something very much like this.