Brogrammer Killed The Requirements Engineering Star

Here are a couple, seemingly contradictory facts: we, as an industry, understand much, much more about how to write software securely today than we did ten years ago. And – fact number two: there’s far more, insecure software being written today than there was ten years ago. Why?

There are lots of possible explanations, but I’ll cut to the chase: I think it’s Mark Zuckerberg’s fault. OK – I know. That’s not fair. After all, Facebook’s creator and CEO is only 28 years old and – gifted though he is – wasn’t much of a factor in shaping the software engineering profession before about five years ago. Still, I think we can find a lot in Zuckerberg and The Social Network – the movie that celebrated his meteoric ascent from Harvard University undergrad to billionaire – that helps explain why software quality is nose diving, despite heightened attention to the material and reputational risk that buggy, insecure software can cause.

If you recall, The Social Network delves into the early days of Facebook, which began as a dorm room project by Zuckerberg to create a “Hot Or Not” type site for the Harvard Undergraduate population. Eventually, Zuck tweaked the idea to make a plain social networking site – like MySpace, but more exclusive. The movie, which won three Academy Awards, is peppered with scenes that highlight Zuckerberg’s coding prowess, as well as those of other, technically gifted “coders” who joined the company early-on.

In one scene (http://youtu.be/4c0lk-LtLI0), the “Sean Parker” character visits Zuckerberg in the Bay Area home he’s rented and is using as Facebook’s headquarters. The house is filled with college-aged coders “wired in” – coding furiously away to the exclusion of all else. In another scene, a group of applicants for a summer internship at the nascent company are asked play a drinking game that requires them to do shots while hacking into a “fully patched Python Web Server” (I guess the Zuck wrote that?) “Every 10 lines of code, they have to drink a shot,” the Zuckerberg character explains. (http://youtu.be/uxKmDWDUZ5A)

So what’s the problem? None, really – The Social Network is a great movie. But, let’s face it, the kind of “coding” you’re doing when you’re “wired in” isn’t likely to be very careful or – need we say – secure. By setting that kind of activity as the hallmark of a gifted programmer, Aaron Sorkin’s movie is a paean to what’s been called “brogrammer” culture – a fusion of frat-house style party culture and…well…computer science. Brogramming’s been blamed or has been blamed for creating workplaces that are hostile to women and – generally – dumbing down the profession.

Whatever else it may have done, the focus on flashy, testosterone-fueled “competitive” coding divorces ‘writing software’ – free form, creative, inspirational – from ‘software engineering,’ its older, more thoughtful and reliable cousin.

That’s really too bad, because what we need now, more than anything, is a return to a more sober and methodical approach to software development. That’s the point of a great opinion piece by computer scientist Leslie Lamport in Wired. His piece, “Why We Should Build Software Like We Build Houses,” makes a long overdue argument in favor of slowing things down and turning software development into the methodical, engineering-heavy discipline it has been for much of the last half century.

Lamport, a winner of the IEEE John von Neumann Medal is an expert on distributed systems and a member of the National Academy of Engineering and the National Academy of Sciences. He says that building software should be conducted in the same way that we build houses – with plans and forethought.

“Architects draw detailed plans before a brick is laid or a nail is hammered,” he writes. “Programmers and software engineers don’t. Can this be why houses seldom collapse and programs often crash?”

Lamport takes the super-agile, ‘code first, plan later’ culture head-on. “Most programmers regard anything that doesn’t generate code to be a waste of time,” he says, adding: “Thinking doesn’t generate code, and writing code without thinking is a recipe for bad code.”

Writing functional and technical specifications – even for simple programs – is a vital skill, forcing programmers to think through what it is they want to do before they start doing it. They’re also invaluable for the generation (or two) of programmers who may need to modify or update your code after you’ve moved on. Trying to make even simple changes to a program without introducing new bugs requires a detailed understanding of what the program or function is supposed to do and how it was written. Without proper documentation, that job becomes much, much harder, Lamport says.

Of course, Lamport isn’t voicing some heretical new theory. Requirements engineering has been considered an integral part of the software engineering process for more than thirty years. The idea is that careful requirements planning should be the first stage in sofware development, leading to the creation of detailed software specifications documents that describe the function and structure of the software to be written. Alas, my conversations with other security experts suggests that such activities are more the exception than the rule and that – even in high risk and high stakes fields, like medical devices and critical infrastructure, requirements engineering is becoming less common.

There are arguments against Laport, for sure. The modern market for software is populated with buyers who demand flexibility and speed. The rapid shift to web based computing has made it possible to use agile programming methods and rapid versioning to speed up the evolution of new applications. The days of writing monolithic software applications and leaving updates to annual or semi-annual patches are long gone.

Software development has also moved from being a rarefied profession to a (nearly) ubiquitous skill. Maybe, then, its unrealistic to expect there to be only one kind of software creator – the “software engineer.” “Coders,” “Programmers,” or “Brogrammers” may see their role as quite different from the “software engineer.” They may not consider higher level planning and specification writing to be part of their job description.

But – in the end – is there really any excuse for sloppy, poorly planned work of any kind? It may be true that software doesn’t fail in the highly visible and kinetic way that, say, a suspension bridge does. But that doesn’t mean that the consequences of insecure or poorly crafted software aren’t real and costly for our society.

Ryan Kulla | January 31, 2013 5:13 pm

I have worked seen companies that were operated by “brogrammers” before Facebook and long before that movie.

That said, I think the last paragraph in the article is the most important to consider. For the last few years, at least, people with no other knowledge but some basic PHP or JavaScript have been holding a lot of the “Senior Software Engineer” positions out there in both small and large companies a like. People who don’t even using branching in their Version Control or unit tests or or or or.

Something has got to change. If companies want to hire cowboys, or rock-stars, or bros, and the like — then maybe they should make that their job title. Or, at least refer to everyone as generic “Software Developers” if they’re not going to implement software engineering practices.

There are lots of T.V. shows and movies depicting doctors like this and I hope to death that that never penetrates reality.

Matt Palmer | February 1, 2013 8:32 am

For some reason, people love to compare the process of writing software to the construction of bridges and houses. The big difference is that the end-goal of the software is frequently not understood at the beginning of a software project, in the same way it is for the construction industry.

Not only do the software engineers not understand exactly what they’re building, neither do the people who asked for it, or the people managing the process. This leads to inevitable change in requirements across the development process.

If construction engineers had to build houses, no wait, bridges, no wait, house-bridges, no wait, house-bridge-swimming pools… then they would fail to build anything that worked at all. Note that this is an analogy failure, not a failure of construction or software engineering. They are different domains, with different challenges.

This has led to the adoption of Agile software development methodologies, which seek to embrace change during the development process, rather than try to manage it away at the beginning. They rely on fast-feedback to developers and stakeholders, allowing changes of direction as requirements become clearer to all concerned.

Critically, these methodologies do not seek to get rid of requirements. They rely on well specified requirements (“stories”), with defined acceptance criteria. It’s just that they are typically small, and can be introduced as development proceeds and requirements shift. They are not planned at the beginning before the true shape of the deliverable is understood by anyone.

Does this approach always work? No. Is it always appropriate? No. I wouldn’t want to use a pacemaker developed like this. For safety-critical systems, then more inflexible and better specified methodologies should be used. But it does work, and frequently works better than the traditional top-down planning.

I agree that planning should be done. The question is: “how much and when?”, rather than whether to do it at all.

Robert Davis | February 1, 2013 11:56 am

Other than misogyny, how is this Brogramming portmanteau different from the Hacker culture (in the Jargon File sense of the word) that has pervaded computer programming far longer than its paper-work lesser cousin software engineering?

DaveM | February 1, 2013 12:01 pm

The last paragraph is the money quote. There is NO excuse for sloppy, poorly planned work of any kind.

Agile “stories” are fine for stating requirements, but they do not describe structures and patterns adequately enough to be considered a specification. Imagine wiring up an interface relying only on a story of how it will be used by a person! Every developer will roll their own API, with their own ideas of parameters, scope, timing, messaging, etc. The result of which will be sloppy, poorly planned work.

The point that not all specifications can be written IN ADVANCE is a good one. But that doesn’t let you off the hook of writing them when the time comes!

Daniel Spisak | February 1, 2013 12:09 pm

What the fuck is a scrogrammer?

Marc | February 1, 2013 12:32 pm

I don’t get it? So Programmers can’t be confident/good looking/have social skills because then they are by definition (see picture) BROgrammers?

Sorry, but are you scared some good looking smart programmer is gonna take all the lady attention at your workspace so you have to classify them as idiots.

Grow up.

Shit programmers are shit. It has nothing to do with character.

Karl Mueller | February 1, 2013 3:43 pm

Completely agree with Mr. Palmer. As a matter of fact, I tend to express “Agile” as additional requirements management discipline, not a reduction. In order to accommodate dynamic behavioral and performance requirements from the user community, you need tools to deal with these changes “mid-flight”. That’s where “Agile” comes in. You still need to do all the other things you do in a more traditional waterfall-ish development process (design/spec/build/test/QA/integration/etc), you just do them in smaller pieces spread over a longer time.

Adrian | February 1, 2013 3:57 pm

I would give you more credence, but your not a programmer Paul, your an analyst/pundit/technovangelist. The problem is everything always looks so good on paper but clients don’t care about your carefully laid plans. You can build that house of their dreams, sure no problem, but 3 months into the project they are asking for a functional toilet to be attached to the ceiling of the garage. Your probably familiar with RUP and how it was suppose to better capture client requirements, well it doesn’t eliminate them, it just sanctions them. Developing software to meet a clients requirements will always be this way; An imperfect artist impression of someone else’s dream of perfection.

Ted Marz | February 1, 2013 4:57 pm

Matt’s comment above has also been attributed to the #1 reason why software projects fail – they don’t know what they were intended to do in the first place, so they wander around with highly variable requirements and are never able to accomplish their “goals” (which were never really defined in the first place).

The other problems I’ve encountered with the Use Case / Story approach are:
1) There is a lack of overall context to interpret the stories in.
2) Stories are success oriented, and don’t provide guidance as to how things could go wrong and what to do about it.

Secure systems are about controlling mis-use. Stories are about defining functional aspects. Quality and mis-use cases are typically not addressed.

Mike Little | February 1, 2013 5:01 pm

While such an approach could be argued as needed in the fast pace Internet world we live in, but about in application area such as process control, defense, and space applications do need highly rigous software engineer practices.

Amogh | February 1, 2013 9:15 pm

@Matt Palmer,
I think the author means the same thing. Agile development is still software engineering. He is referring to software development that happens with zero engineering. Just slapping together lines of code that make sense to no one but the original author and at that for the while he has it in his head only.

John | February 1, 2013 9:39 pm

As to Matt’s reply above I agree partially? But the problem he describes is exactly why I think Agile fails. Work should not begin until a solid plan for what the finished project is supposed to accomplish is in place. My experience with Agile is that it just supports this “lack of planning model” that so many customers utilize.

You would never start building a house and then halfway through decide to build a car but often this is metaphorically what happens.

Alan Shutko | February 2, 2013 12:19 am

Great headline, but the problem is far broader. Matt already discussed how changing needs hurt requirements engineering, but short timeframes are equally damaging.

I’m currently at a company which has a process in place with multiple levels of requirements, security reviews, architecture reviews, compliance reviews. And yet, the project has timeframes that are so short that were we to spend enough time to completely and properly do the requirements, we would be in breach of contract. The business need to get functionality out is sufficiently great that we can tolerate issues we will have to address later on.

This tension between correctness and business needs is present throughout the industry. Skyrim needed to be released to make money. The fact that it had large numbers of defects did not kill it. Sony’s Playstation Network experienced a significant security breach, but I can imagine the project meetings before the PS3 launch: “Shortchanging analysis will not kill the company, but failing to launch this console will.” And, in the end, they were right. The PS Network breach was very painful, but not terminal.

Gweihir | February 2, 2013 12:22 am

Well, quite frankly, this “cretinization” of software creation does far more harm than good. Sure, some people will earn entirely undeserved riches, but generally society as a whole suffers, due to reduced stability, increased criminal activity and unmaintainable systems. The only thing “Brogrammers” can create is software destined to be thrown away not too long after creation. That is not innovation, that is holding back innovation by filling a need with a quickly and badly made product, to prevent others with better products to from getting into the market. As such, it is more in the nature of a short-term scam than anything that even remotely resembles engineering.

Valerie | February 2, 2013 10:23 pm

“Not only do the software engineers not understand exactly what they’re building, neither do the people who asked for it, or the people managing the process.”

There are many ways to solve this problem. Lightweight, iterative planning is one way. But, we could get better at planning. We could improve the system at what I assume is the business analyst layer — better requirements gathering, better writing, better communication.

Now I will give the requisite example of building a bridge. If my bridge blueprints weren’t functional, I would get a more capable/thorough/experienced civil engineer, not get rid of the blueprint.

Pieterjan S. | February 4, 2013 8:24 am

Speed is everything nowadays.. Most people just rush stuff out of the door to make more money. It’s simple as that.

Paul | February 4, 2013 11:50 pm

Great comments everyone – thanks! I find lots to agree with in Matt’s comments and Ted’s. I think the gist of all this is: there’s a place for Agile and a place for more structured, methodical, detailed specifications or plans that Prof. Lamport is talking about.

Obviously, the fact that most software development shops get specs wrong or write incomplete specs isn’t a justification for ceasing to use them or try to make them better, right?

Thanks to Ted for noting the connection between security and all this debate over software development methodologies. To quote him “secure systems are about controlling mis-use.” And yet most functional and technical specs still focus only on describing the (proper) function of the software under normal conditions, not its misuse under abnormal conditions.

The bigger story is the way that many wealthy and high profile manufacturers that have operated outside of the tech industry are now becoming, in essence, software publishers – think “smart TV,” “smart auto,” “smart coffee maker,” “medical devices,” etc. In theory, these wealthy manufacturing concerns should extend their quality engineering to the software that runs their hardware. In practice – not so much.

Paul

Twitter revealed that it had been; – Strategic Staff | February 22, 2013 1:23 pm

[...] of trusting third party SOUP (Software of Unknown Pedigree) or the lack of rigor in application design, coding and testing. The news this week of watering hole attacks aimed at developers adds a new [...]

Erik Reppen | June 26, 2013 12:50 am

I love it when people put JavaScript down as a choice of mediocre programmers. I came up writing code for 6-20 platforms that don’t actually agree with each other on what your code actually means. I’d wager the guy putting JS down probably writes code that isn’t even cross-platform compatible even with the aid of a VM. Or he chose Microsoft, because that’s badass. I promise you this: You will never hear a competent JavaScript dev say “There’s only one way to do it right.” And what could possibly be more fratboy or Java or mediocre C# dev than that.

Please Post Your Comments & Reviews

Your email address will not be published. Required fields are marked *

RSS feed for comments on this post