Yesterday (essentially), I woke up and decide to stir the pot a little bit. I had come across this post while looking into the company behind MapBox. As I noted, I do this to get a feel for what the company is doing. What technology is important to them? What do they value? Are they funded, and so on, and so on.

That particular job posting rubbed me the wrong way, however, because of it’s emphatic no telecommute policy, with them going so far as to call attention to it with a bold font. To automatically shut the door to any potential employee who might be an amazing fit, just not in their particular office seems extremely one-sided.

I woke up ready to write, and did. Before writing my first book, my post would have been rather mellow. I would have presented the case, but not gone quite so far in making my case. This post, I decided to let it rip. Companies that didn’t allow telecommuting don’t get it, end of discussion.

And boy did it work. Nearly 90 comments—between Hacker News and my blog—later, I’m still amazed at the fervor with which both sides attacked the issue. Almost to a T, no one was in the middle on this. Everyone had an opinion. They loved it, or they hated it. Everything from management types saying how their teams wouldn’t be the same if IRC was their only interaction to people with severe social anxiety talking about how telecommuting affords them the opportunity to deal with that anxiety by focusing on their job, not their proximity to people.

I like to mine the edges of the conversation to get the bigger picture. My post emphasized the importance of taking a stand. Had I not been so opinionated, I doubt much interested would have been given to the post.

Honestly, the post is a little harsher than my personal opinion on the subject. Companies that start the conversation with prospective employees by outlining the things they aren’t going to stand for are starting off on the wrong foot, but beyond that I realize that telecommuting does present challenges in some environments and some companies—at their loss—aren’t willing to try to overcome those challenges.

A few months back a recruiter—the flaky kind it turns out—was talking with me about a gig. I was honest that I wasn’t looking to move, but for the right opportunity every thing was on the table. I told him:

I want to be up front with you. I’m not looking to move, but I’m not ruling it out. All I ask for in return is that you approach this with the same open-mindedness. Let’s continue the process, see if its a good fit, then figure out if we can make the logistics work.

Needless to say, they didn’t. Companies, particularly in the tech space, are asking their potential employees to take a chance on them. Companies are taking a risk too, but when companies expect the employees to be the only ones giving (broadly disregarding monetary compensation from the companies for the moment), they set the wrong tone.

A few years back, I’m not even sure when, I started looking at job postings of companies I found interesting. The point wasn’t to find a new job, but to understand the company a bit more.

You can almost always discover what technology a company is using if they’re hiring. I found out Plurk uses Python through this method, that Twitter hasn’t given up on Ruby, and even when it doesn’t make sense Washington DC shops still use Drupal.

That last company is what gave rise to this tweet:

They’re pretty blunt about telecommuting, actually. Going so far as to say “[i]t is not ok to telecommute.” The emphasis is theirs (as seen in this Skitch). This gives me a couple of possible explanations:

• They’ve been burned in the past. They hired someone on who misrepresented himself, his abilities, his capacity, or all of the above. They feel that they gave it a try and it isn’t for them, so they’re not going down that road again. This is the most generous possibility.
• They don’t have their heads screwed on right. They live in a “constantly evolving and fast-paced” environment (as seen on other job postings) which translates roughly into we can’t control our product team, our CEO, or worst, what our sales team promises, so you’re going to have to sit by and wait for the hour-by-hour priorities. Communicating them by any other means than directly from our mouth to your ass, sitting at a desk waiting to turn our ideas into money via that magical electronic device in front of you is too inefficient.
• A slight variation of the previous option is that they lack the confidence in their abilities. They have the vision, the product specs are nailed down, but they don’t know if they can convey them without “reading” the other person.
• Finally, the last possible option is that they’re so out of touch with reality that they believe the only way for work to be done is with “asses in seats.”

Of these options, shy away—no, run, very quickly, away from the 2nd and 4th. Those companies don’t get it. The other two provide some hope, it’s up to as to whether to stick around (or start) and find out.

Companies that don’t allow telecommuting, especially companies that are up front about it, don’t get it. Telecommuting is no longer a technical challenge. Reliable high-speed Internet is ubiquitous at this point. I can see adding that as a requirement to a job posting, but not where you’re located.

With the technical out of the way, that leaves only the social aspect of the company and the position they’re hiring for. I get the culture that requires someone on site. I also realize I’m not interested in working for a company with that culture.

It’s the culture that specs and requirements are fluid based on the latest hallway session. It says interruptions are a part of the day that you should be used to—we like to chat a lot. It’s the we don’t have any realistic metrics to use to judge your performance, so we can only know if you’re doing work by whether or not you show up. That last one has worked so well in the past.

The world’s a big place. There’s a lot of developers in it. There’s even a few kick ass ones. Most of those don’t live near you—even if “this position is located at our offices in Palo Alto,” which is another euphemism for no telecommute. By being up front about telecommuting not being and option, you’re telling me that company comes first (we go to the mountain). Hiring someone good is important, but hiring the best you can afford isn’t.

A company that doesn’t go out of its way to build the best team possible, regardless of where they’re located, isn’t a company I want to work for. It’s a matter of priorities. I’ve got mine straight, they don’t have theirs.

By now, I hope we’ve all read Is Google Making Us Stupid from The Atlantic. Here’s the quick version, which ironically makes the author’s point:

People are changing the way they interact with the world, which is changing the way their brains model the world, which is changing their brains. We’re not exercising the critical, reflective part of our brains; instead, we’re using almost exclusively the parts that focus on quick, immediate activities. We’re skimming everything rather than thinking more critically, more reflectively.

The reaction to the Shirley Sherrod case is an excellent example of this. Rather than step back, the NAACP and White House jumped on it and stepped in the heaping pile of burning feces that was laid at their doorstep.

So what’s this got to do with thinking and Twitter and such? Well, it’s the reason I’ve started blogging again. I don’t want to lose my ability, what little it sometimes appears to be, of critical thinking and reasoning so I’m starting to blog more.

I noticed my thought process was becoming much more reactionary and shallow. I was allowing the insignificant to bubble up and take over. Rather than ponder something and try to write out, I was firing off a quick tweet and calling it a day. Couldn’t fit it in a tweet? Oh well, it must not be that important.

So that’s why I’m coming back to blogging. The first four years I ran this blog, I had over 100 posts each year—closing in on 200 in 2006. Starting in 2008, however, I let it slide. 66, 22, and then only 13 posts so far this year. I’m working on reversing that trend.

I’ve spent a lot of time working on a lot of different code. I’m fortunate in that I’ve been exposed to all three of the modern scripting languages and their communities. All of them, to varying degrees, bemoan their code. All of them, way too much of, bemoan the code of everyone else.

Ruby is too clever, I can never figure out what’s going on or what to expect.

Python developer’s can’t write a test to save their lives.

PHP developers, well, do I really need to say more?

Everyone’s moaning, from the top to the bottom about something with software. To listen to them, you’d think we had no tools to do anything and that all software sucked.

I’ve got news for you. It all does, so can we please move on now?

This isn’t a new concept. Actually, I’m stealing it mostly from Chris and his keynote as Ruby Midwest, but I’m coming at it from a slightly different point of view.

I had the unfortunate task of talking with a client today about the state of their code and recommending that they put a few small tweaks on hold until a larger revisiting could happen. Ultimately, it’s their decision and I’ll do what they want, but it’s going to cost them more for what can only be a temporary solution now, versus revisiting it in a few months as part of a larger redesign.

Why? Because their current code sucks. Everything is hard coded—nary a loop in sight. There’s HTML mixed up in their business logic and business logic in their HTML. They have a haphazard layout to their project. Form validation? They got it. To the tune of if statements in functions that are hundreds of lines line. Touching one part invariably breaks two other parts because they depended on that particular HTML tag to exist in that particular place, with those types of siblings, etc. etc., etc.

This code is the definition of spaghetti code. Adding features that are less than 30 minutes on any normal project are taking upwards of 5 to 6 hours, most of it in testing and fixes for other things you break along the way.

I’m relatively new to the project and wasn’t brought on in a code review capacity. I was explaining my reasoning and a knowing grin came across their face. “Well, I’ve been told that most developers hate projects that they come into when they’re new to it because they didn’t code it. It’s not how they would have done it.”

Gee, thanks. Every single developer out there who’s complained because someone used a different bracing system or they used a different naming convention than they would have or any one of the hundreds (if not thousands) of other asinine things coders like to complain about make it impossible for me to convey to a new client that his code truly is in need of some serious help because “developers hate projects.”

Can we all please shut up?

My point is this. Code sucks. It’s all a compromise to take an idea that we (or our clients) have formed and turn it into something machines can understand. It’s all trade offs. There’s always another way to approach something, a different language it could be written in, a more scalable data back end that could have been used. Always.

So why spend time talking about it? All we’re doing is making it easier for the people who need to be listening to us to tune us out. They don’t know the difference someone like me complaining about the internal structure of Wordpress (which I think is an abomination) and my explaining that it is the wrong choice for their project (which I’d argue is the case for at least half of business deployments of WP). That’s not their job to known and understand the difference, but it is ours. It’s also our job to be able to convey that information to them.

This reminds me of a few tweets from Ivo (if I recall correctly) during Confoo. One of the speakers had tweeted about the wifi and issues they were having with it. Then a bunch of people started “me too” tweeting about how bad it was and Ivo called the original person out on it. What good did it do for a speaker to complain publicly about a situation that was already known (and being worked on, I might add) other than to encourage everyone else to follow suit?

We’re doing the same thing every time we bitch and moan about relatively insignificant things in code. We’re doing everyone a disservice by increasing the noise-to-signal ratio when we complain for no good reason. So please, let’s all put a sock in it.

I’m lazy when it comes to code. Not in a bad way, but in an efficient way. I want to get to the crux of the matter quickly and move on. Truth be told, that’s why I like TDD—I don’t have to remember anything more than I need to know right now. My tests remember everything else I knew, but I digress.

When I start evaluating a new library to see if its something I want to use, code is what I want to see. Sure, tell me what it does, briefly, then move into the basic use-case. As an example, consider the front page of Ruby-Lang.org.

You’ve got access to all the information you need, but right up front and center (almost literally) is some code showing you how to use Ruby.

If you’re starting a new library, make sure to put the code right up front. I know there are any number of projects I have on GitHub that fail this test, but I’m not out purchasing domain names and designing custom sites for my projects either. If you bill your project as “easy” and “powerful”, show me, don’t tell me.

Otherwise, you’re no better than the phone commercials][] that promise the world will bow before me if only I use their phone, without, you know, showing their phone being used.

Finally got around to converting some old code from SVN to Git and getting it up on GitHub. It’s like looking back through a time-warp actually, as most of the code hasn’t been touched since the summer of 2007.

Nearly all of the code is usable, but it’s all abandoned at this point. If there’s something there that strikes your fancy and you’d be interested in forking it into your own project, feel free.

There are still a few more to go, but you can start checking them out now at the Domain51 Github account. Just search for Domain51_ to filter the listing as they’re all named in the old PEAR-style package naming scheme.

This may be premature, but it looks like I’m live with the new site design and new blog engine. The design is HTML5 (i.e., it looks great to me in Chrome, not sure what it’ll be like elsewhere) and the new engine is jekyll.

What does this mean for you, my loyal reader? Not much, really. I believe my port is transparent.

Actually, the only problem I’m seeing right now is related to disqus—some of my comments that I know are imported are not showing up yet. I just dumped nearly 2,000 comments into their system for this blog, so my guess is that it’s a caching issue and they’ll catch up. The comment count number are correct inside their admin interface, so I know the comments are in their system somewhere. :-)

If anything looks out of whack, please let me know.

Lately I’ve been playing with testing frameworks all over the spectrum of languages. I’ve come to really enjoy using Cucumber for testing web APIs. Since most of my coding lately has been in JavaScript or Python, using Ruby with Cucumber allows me to completely segregate my tests from the system under test (SUT). This separation has worked great until recently when I needed to have the test system running in Python.

I started looking at the Python bridge in Cucumber and that’s when I came across Lettuce. It’s a Cucumber-inspired, Python BDD library. I like the syntax, it had built in Django support, tons of tests (including functional and integration), so I was ready to go.

Then I ran the test suite. And it failed.

A failing test suite is a massive red-flag for me with any project. In a test suite, it’s a nuclear launch siren. I poked around a bit and figured out what was triggering the test, but not why, opened a bug report and found out that the tests were never meant to be run together.

I let it go, but last weekend decided I was going to dig into the framework, figure out what was causing the tests to fail if the functional tests were run before the unit tests, and submit a patch.

I spent the better part of Sunday afternoon cursing at code, trying various paths of exploration, trying to grok the entire framework’s codebase to understand what was happening. I ended up going so far as doing the equivalent of var_dump() debugging (pdb didn’t prove very helpful because of the intense setup required before the tests started running)

Finding the code that was causing the problem was easy - modifications to the lettuce.registry.STEP_REGISTRY were causing the failure. Figuring out how to fix that proved more difficult.

The issue, it turned out, was global state. The unit tests assumed that once they setup, the state wouldn’t change. The functional tests didn’t much care for that and stomped all over the registry of steps. By the time the unit tests rolled around, the steps that were so carefully defined inside the unit test modules were no gone and the test suite was throwing failures.

I finally landed on the solution by redefining all of the steps inside a @with_setup() for the tests that need them in the unit tests. It brings up a couple of interesting learning moments.

First, this shows the need to make no assumptions when writing tests. Need a database connection? Make sure its initialized and ready for each test that uses it. Want to make sure a step is defined for the test your checking in a BDD framework, define it immediately before running the test. It’s a good example of defensive coding.

Second, it shows the mess that global state can create. Each of the test modules was being loaded by nose, then the tests were being executed after a global state (the STEP_REGISTRY object) was defined. When other tests changed that state, things started falsely failing.

The “fix” currently is to reset that state to what you expect every time, but this causes issues with tests running in parallel. What happens when two of the same tests both try to reset the state at the same time? Don’t know, I haven’t tried it yet, but I imagine it’s gonna cause some more failures.

My fix now is short-term (and was included) and it gets the job done. Hopefully, this shows you a bit about what we mean when developers say that global state is a bad thing that leads to tricky bugs that are hard to comprehend. In this case, I literally had to understand the entire step definition system in order to comprehend what was happening here

The test cases at least gave me some guard rails to help guide me toward the solution, but had there not be global state in the first place, these tests would have worked across the board with no problems.

This is it. I’ve had enough! Seriously, people. OAuth is about maintaining control as a user and everyone wants me to give it up! I’m tired of constantly clicking deny.

What am I complaining about? The constant abuse of Twitter OAuth login. Every site that I’ve visited that uses Twitter OAuth requires both read and write access to my account. The latest to do this is Paper.li, a service that looks really cool,
but…

So what’s the fix? Websites should ask for the minimum amount of information needed to get started. In nearly every single case, the sites are using it for login purposes. Instead of a username and password, you talk to Twitter to verify that you have a legitimate user. Those “Tweet This For Me” buttons are optional add-ons that you can do.

You should handle those automatic cases by performing an upgrade when the user decides they want to allow your application to update for them. Unfortunately, Twitter doesn’t allow you to specify which level of access you want when you request a token, you have to do it when you setup your application.

Registering two applications is an easy solution to this problem. You use the read-only application for authentication, then switch to the other app when you’re attempting to write. It requires a little overhead when you store the authentication token, but it’s trivial to store a flag showing which set of credentials to use.

Honestly, I know most applications are completely trustworthy. Especially those I’ve found through recommendations of others, but it’s still unnerving to give 100% access to my account to a new service for the shear pleasure of being able to login and see if I like it. It should be to you too.

The entire subject of two talks and mentioned in several other, MongoDB was definitely a buzz at TekX this year. It’s long been in favor in the tech community in Lawrence and has been used for some data crunching for a few projects at the local paper. Even with all of this exposure, I’ve yet to sit down and actually explore it.

That changed Friday afternoon while I sat at O’Hare waiting on my flight back to Lawrence (which subsequently got canceled). I installed Mongo earlier in the week and opened up a bunch of tabs on the various intros and tutorials available on the Mongo wiki. The rest of this article a mix of stream-of-conscious as I played around with Mongo for the first time and some of my reflections this past week.

Note on typefaces

I use both Mongo and mongo throughout this article. The first, the title-case Mongo refers to the software as a whole. Whenever you see mongo with a lowercase and in monospace, it’s referring to the Mongo client program you run from the command line.

Installation

On a Mac, it’s a breeze. I use Homebrew to manage software on my Mac, so a quick brew install mongodb was all I needed and a minute later I was ready to go.

Starting Up the Server

Mongo is run by the mongod process. I don’t know if it’s pronounced mongo-d or mon-god though. It’s a fun play on words if the latter is the case.

Brew includes a basic configuration to get up and running, so I use that inside a screen instance so I can leave it running in the background while I use the mongo tool to interact with it.

Interacting with Mongo

I started out with the basic tutorial to get going. It looks like that needs some love though. It shows the version in the startup as 0.9.8. Homebrew ships with 1.4.2 and I did find a few things that were out of date. No, I’ haven’t been a good open source community member and submitted fixes yet.

The first thing that’s different than a traditional RMDBS with Mongo is that you don’t have to explicitly create a database. Pretty straight forward: from within mongo, type use <database>. This creates a brand new database for you and you’re off. For the examples below, I’m using use mydb to select mydb as my database.

It’s kind of nice to just be able to connect and go, but it feels odd. Not good or bad, just odd. Sort of like the first time you run git checkout inside a repository to switch branches when you’re used to Subversion.

The shell feels like a Javascript console. I don’t have access to the source code in my off-line mode, so I don’t know but that it is. The syntax seems remarkably similar, so it’s at least Javascript inspired.

Adding Records

Mongo stores documents, not rows of columns. This distinction allows Mongo to ignore schema—continuing the theme of leaving it up to the developer. Those documents can be made up any number key-values that look remarkably like JSON. Need to store a new data point, just add it as a field to a document and you’re set.

Here’s an example inspired by Mongo’s tutorial for adding a few records:

> person = {name: "Travis Swicegood"}
> city = {city: "Lawrence", state: "KS"}
> db.things.save(person)
> db.things.save(city)

Here I created two new objects with various data attached to them, then saved them all inside the things collection. Collections in Mongo are like a table inside the SQL world. You don’t have to create a collection, you just declare it on the db object, and you’re set.

Comparing this to the same code in a database, I’ve got to say I love this. No boilerplate code to get going. I didn’t have to create a database, no tables were created. I just started using them. This appeals to my laziness—err, I mean desire for efficiency, but also looks very promising to teach someone new. Every abstract idea you can remove is one less potential stumbling block for someone starting out.

Back to the data I entered. Notice that neither have the same fields. Collections inside Mongo are made up of a series of keys and values—they can be whatever you want them to be. This is perfect for lazy migrations: migrating the data as its requested instead of doing it all at once. ming, a Python wrapper around Mongo already provides this. This is especially useful for large sites with lots of data that may or may not ever been requested again.

Finding Records

Now that the records are there, finding them. The db.things object comes back now:

> db.things.find()
{ "_id" : ObjectId("4bf9a96b7d04f51b48499011"), "name" : "Travis Swicegood" }
{ "_id" : ObjectId("4bf9a96f7d04f51b48499012"), "city" : "Lawrence", "state" : "KS" }

That gives me everything. The find method takes optional parameters to filter the results. This is actually a good time to bring up the built-in help in mongo. Entering only the value of any function (i.e., without calling it) displays the implementation of the function:

> db.things.find
function (query, fields, limit, skip) {
    return new DBQuery(
        this._mongo, this._db, this, this._fullName,
        this._massageObject(query), fields, limit, skip);
}

Note: I changed the formatting so it’s more easily viewable online.

The parameters are optional (like all Javascript function), so you can pass in as many or as few as you want. Filtering the results is done by providing a hash for the query parameter (the first one). For example, to find my record:

> db.things.find({name: "Travis Swicegood"})
{ "_id" : ObjectId("4bf9a96b7d04f51b48499011"),
  "name" : "Travis Swicegood" }

One thing you can’t do is full-text searching. I can’t ask for all of the records that begin with Travis or have a portion of my name in it. The current recommendation (at least via the wiki) is to build your own list of keywords as an array, then search that array. For example:

> var person2 = {name: "Travis Swicegood",
>                name_field: ["Travis", "Swicegood"]};
> db.things.save(person2)
> db.things.find({name_field: "Travis"})
{ "_id" : ObjectId("4bf9afa17d04f51b48499014"),
  "name" : "Travis Swicegood", 
  "name_field" : [ "Travis", "Swicegood" ] }

For something like a name, this can be useful. For full-text searching of an article, it’s probably best to delegate searching off to something like Solr and let Mongo focus on storage and retrieval.

Querying for sub-objects

Of course, I had to try sub-objects to see if they would work:

> db.things.find({person: person2})
{ "_id" : ObjectId("4bf9b02b7d04f51b48499015"), 
  "person" : { "name" : "Travis Swicegood",
               "name_field" : [ "Travis", "Swicegood" ],
               "_id" : ObjectId("4bf9afa17d04f51b48499014") },
  "city" : { "city" : "Lawrence",
             "state" : "KS",
             "_id" : ObjectId("4bf9a96f7d04f51b48499012") } }

You can also query using the dot-notation to &lquot;reach through&rquot; an object and look at its children. This returns the same result as the previous query:

> db.things.find({"person.name_field": "Travis"})

Limiting returned columns

This ability to dynamically add columns to a record and definitely provides a breading ground for massive documents with lots of keys. Most of the time a small subset of those keys are all that’s needed. The second parameter in find provides us with that functionality:

> db.things.find({person: person2}, {city:1})  
{ "_id" : ObjectId("4bf9b02b7d04f51b48499015"), 
  "city" : { "city" : "Lawrence",
             "state" : "KS",
             "_id" : ObjectId("4bf9a96f7d04f51b48499012") } }

Likewise, you can reach through the object and pull out a subfield:

> db.things.find({person: person2}, {"city.state":1})
{ "_id" : ObjectId("4bf9b02b7d04f51b48499015"), 
  "city" : { "state" : "KS" } }

These examples bring up a syntax thing with Mongo that I’m not crazy about: the use of the number one. It’s the standard C style: 1 is true, 0 is false. I’d love to see the client and the libraries adopt an intent revealing name. Granted, this is a minor niggle, but the little things are what make a good system an amazing one.

Few issues

The docs, being that they are community run and Mongo’s still relatively new, are a little loose. I’ve found a bunch of examples looking through them that don’t work the way they were documented.

Another potential issue (or at least something you need to be aware of) is that Mongo’s geospatial support isn’t 100% year. They only provide 2d and the math they use assumes that 1° of longitude is the same at the poles as it is at the equator. For many applications, this isn’t a huge issue, but if precision is important, Mongo’s not ready for this type of use.

One thing that I’m looking forward to is Mongo’s sharding. That is going to allow Mongo to scale horizontally really well. Some of the initial test results look amazing. What will be really interesting is to see how well is scales down. It’s one thing to have over 300,000 ops/sec on a bigger box, another thing to be able to manage it on something like a 1gb instance on Rackspace Cloudservers.

Two Biggest Issues

First, Mongo’s a master-slave system. It appears really robust, but whenever a box takes on a special role I start to get nervous. One of the promises of &lquot;NoSQL&rquot; is that it provides a tremendous amount of resilience. Any time you start to add special nodes you’re taking away from that.

For example, if you’re running 5 homogeneous servers and one goes down, the other 4 can pick up the slack—assuming you’re not running 5 servers at peak capacity. This makes failure planning easy: figure up the amount of CPU time you need to handle your load, provision that many servers, then add enough servers to be comfortable when they start failing. Need 3 servers, provision 5 and you can have two failures before you peg your machines.

This isn’t to say Mongo can’t handle failures. It’s current model is rebalancing the load when one of the servers goes out. mongos is the tool to read up on for handling this. Unfortunately, I haven’t been able to dive into it yet. The only way to know for sure is to build up a cluster then start killing servers. Of course, this type of testing is preferred for any data storage system.

Second, the license. I’m not anti-AGPL, but there’s some ambiguity. The Mongo team has addressed this both on the wiki and through an in-depth blog post. According to that, I can write up a service such as MongoHQ and as long as I don’t actually change the mongod or mongos code I’m fine.

On the other hand, most of the definitions I’ve read of the AGPL mean that code that talks to it is subject to being hit with the AGPL. I don’t have any doubts with 10gen, but if they don’t always own the copyright…

Of course, those last two paragraphs are with the caveat I am not a lawyer.

I think Mongo is an amazingly compelling piece of software in the non-standard database realm. With the upcoming sharding and what I would have to imagine is an eminent fix to the geospatial queries, Mongo’s definitely worth a look.