“Software sucks, 80% of projects fail, and most developers are unhappy individuals. Why is this? My answer; complexity. Complexity is the single factor I would attribute poor software to. The more you have to do, the harder you make it on yourself, the way requirements seem to change, the worse the final result is; right? Software needs to be simple“, says Chris Stewart.
Why Software Sucks
About The Author
Eugenia Loli
Ex-programmer, ex-editor in chief at OSAlert.com, now a visual artist/filmmaker.
Follow me on Twitter @EugeniaLoli
Once you present an UI (which generally are quite a bit easier to produce than the model+data access layer) they will expect the application to be done.
So when you at the next iteration show the same UI with one added button, and try to explain that you “uh, had to refractor some parts of the data access layer to support the new functionality” they’d just think your sleeping on the job.
…it’s the programmers that suck.
Of all technical and engineering disciplines, I’ve never come across one that is so rife with unqualified practitioners as software development.
It’s among the poorest defined of technical and engineering disciplines as well as the youngest and most rapidly evolving.
It’s also extremely easy to cheat in university, which is also a problem.
The cheating comment has nothing to do with Computer Science specifically so I’ll just ignore it.
The poorly defined comment has a little more merit, but isn’t it also quite obvious?
I mean, the situation in Mechanical engineering is:
“Here is a set of immutable laws of physics and a set of mathematical models to help you quantify all problems you’ll ever encounter to an exacting standard. Go build a widget that meets this specification.”
The situation in software development is more like:
“Here is a machine that is capable of solving any problem that is finitely describeable. Here is the problem we’d like to solve. We need you to A. decide it is finitely describeable, B. develop a set of rules that constrain inputs and outputs in order to solve the problem, and C. develop a set of mathematical models to quantify any problems you may come across.”
Now, which one of these do you think will have the most well defined set of Academic standards?
It’s not the mathematical part of computer science that is lacking in definition. It’s the methodologies for building complex software.
Building the part is easy.
Building a small program is easy (any freshman in CS who will graduate can write 1K line programs in a weekend).
It’s the big problems that are more difficult. The problems that consist of other problems which consist of yet more problems … ad nauseum.
And even there, there are methods for building the software and making this happen. The lacking part is getting all of those components to work well together with n people working on them.
In most engineering fields I think there are probably standards. Or maybe it just takes that long?
In software you’re expected to finish very complex designs in a short period of time and the standards (that we have) themselves are fairly complicated.
Some of the driving force is probably the market. When you design a car engine you have strong specifications, and your boss wants you to meet them because otherwise it could cost the company a fortune.
In software you have user desired specifications (they want this feature and that) and some design specs. You meet the user requirements and ship. If it fails and deletes their data you don’t care, because they signed a contract saying they’ll deal with it (which isn’t as bad as it sounds, data is cheap and that’s why you copy it several times). Car’s can’t get away with this. The engine can’t break and be cheaply put back together! It needs to work as long as expected, and probably, break as soon as expected.
Writing software is not building cars. It’s not building spaceships, and it’s not building aircraft. If you want to see software done with similar methodologies (although somewhat difference due to the nature of software) then look at air traffic control software or something else where bugs mean dead citizens.
In the end I really think it’s user driven. Most users want something mediocre now and not something finished in ten years. Ask yourself this, would you like to be using the first GUI’s right now (we’ll pretend Win95 was the first GUI) and have them run flawlessly? Or would you rather run Vista and have it hang up and crash every now and then.
Ask yourself this, would you like to be using the first GUI’s right now (we’ll pretend Win95 was the first GUI) and have them run flawlessly? Or would you rather run Vista and have it hang up and crash every now and then.
I’d vote for a flawless running version of the Win95/NT4 desktop.
Anyway the difference between the desktop in Win95/NT4 and Vista is much smaller than between Win95 and WfW 3.11.
Flawless is the best choice for anything professional/serious.
Users just want to app to work – I’ve even worked for a client where they prefered the Unix console based terminals rather than the new Windows based clients – because they were faster and did not crash half way through a long winded sale.
Also, say you have an organisation with 5000 PC’s.
Now I think you’d rather have a flawless Win95 than a regularly crashing Vista.
It’s only the operating system FCC – all it has to do is present the company applications to the user and not get in-the-way/infected.
Once ReactOs is up and running I think you’ll see huge numbers of companies migrating to it.
Agreed. And too many wannabe hacks. Many get an unrealistic view about how good they really are without any formal training on software development, lifecycles, budgeting, business expectations and frankly practical business experience of the roles they’re writing to software for.
I also think the quality of some education is utterly poor. I previously worked with someone who’s belief that he was an “object oriented programmer” was based on the fact that he “dragged and dropped” buttons in VB onto a form. When pressed for the basics such as classes, inheritence or recursion he was left clueness. This was after two years of “training” in “software development”.
Complexity can be managed, understanding is completely different.
Edited 2006-11-03 05:10
Why would you hire someone without at least a bachelors or a lot of experience?
You’ll have a hard time finding good developers who are into “practical business.” Why? Because a lot of your good developers will be somewhat idealist in their thinking, and they won’t want to worry about the money making world.
If you’re a decent manager that won’t matter. As long as they can relate to their customers needs they can get the software done with your help. If you’ve got bright people and all you have to do is abstract out the business overhead for them, then do it!
The guys who know business and software will, likely, generally be much better at the business. Maybe they’ll make good managers?
Budgeting is not your developers job.
***I’m assuming that there are more than 8 people in your company.
It’s not the programmers that suck. It’s the employers. Who would employ someone without any qualifications to do a technical job?
An employ who wants to pay the least wage possible.
I would say that the employers are at fault usually because when I worked for companies not one of them had processes in place to check the quality of the work produced.
I worked for an insurance company where there were about 35 programmers just making things up as they went along – and not a single check made on the code produced.
And as only about 1 in 10 programmers have a proper understanding of the craft the state of the company applications was woeful.
The scope of the document is about software engineering. Companies employs resources: you have to consider an average resource, who has average experience, education level and is able to do the job.
In most cases in industry, lot’s of project will fail to maintain their initial dead line, commitments, etc, it’s an old reality studied extensively by many people, most notably by Frederick P. Brooks http://en.wikipedia.org/wiki/Fred_Brooks.
And, as pointed by many other comments the true essence of software is that it crosses many boundaries and is difficult to define/standardize. Even nowadays, those topics divide a lot of people, but all agree: one of the silver bullet is to refine as much as possible your requirements to the point where you can make it simple (from the model/algorithm point of view).
Just to quote Nicolas Boileau:
“Whatever we well understand we express clearly, and words flow with ease.”
This is something that should be primarily considered when writing a poem, but also when designing a software
Someone beat me to the punch of mentioning Brooks. I highly suggest his, “Mythical Man Month” and the follow up, “There is No Silver Bullet”. If you don’t read either and start making guesses about why software sucks, then you are missing out a man who pretty much nails it on the head. The authors write up smells of Brooks, but I can’t be sure if he actual read it.
Many managers don’t understand that engineering estimates made by in-house programming staff are usually based on actual assessments of the effort involved and aren’t just grabs for more resources, that a certain percentage of the technical folks under their juridiction actually do have a better understanding of the issues than either they, the Gartner Group, or their CTO/CEO/CFO or consultant friends down at the pizza parlor, and that simply throwing more people at a software project will absolutely not help in most cases.
Most of the places I’ve worked had weeded out the chaff before I got there. Most of the poor programmers were long since gone. Management, however, was a mixed bag, with some of them rising from the ranks and having an excellent understanding, and with others coming from other areas of the business and not fully appreciating some of the issues that come with complex software projects.
Oh yeah, and read the Mythical Man Month. Really. I wish more managers would read the One Minute Manager, too. Good book.
(Revised to correct some spelling and case issues and to reword an awkward sentence or two).
Edited 2006-11-03 18:54
“Of all technical and engineering disciplines, I’ve never come across one that is so rife with unqualified practitioners as software development.”
Agreed.
-Chris Stewart
Edited 2006-11-03 02:42
First of all, your hypothesis is completely unoriginal. Secondly, it’s backed up by statistics you made up at random.
“80% of projects fail”.
Right. I’m sure that’s the case. I’d be interested to know where you got such an acurate statistic.
Also consider this:
“B.S., Information Systems, Virginia Commonwealth University, 2005”
Frankly, you don’t have enough experience to make informed comments about “most developers” or “most projects” or levels of complexity and the relationship between client expectations and a delivered product.
I am glad you are taking the time to become involved in your craft, but maybe you should swim in the waters a little longer before you start preaching about how cold it is?
Do you think more than 20% of projects are succeeding?
I guess the question is: What measure are we using?
I know that I am dissatisfied with over 80% of the software I have used.
Have you stopped beating your wife?
Can’t be used here.
The first is an expressed condition, the latter an implied.
A lot of the points in the article seem to imply that the programmer has control of the pacing and influence of the customer and can act as gatekeeper of requirements. That’s usually not true, though. It might be for an indie contractor, but an office rat is going to be stuck with a boss that wants everything done yesterday. Pushing back is often not good for your career, either. The end result is you get a lot of hassled devs who might not live and breathe code like a typical hardcore geek plugging in the first “framework” they see that has something attractive to them listed as a bullet point on the feature list.
There are few developers who actually revel in ‘new’ technology as oppose to getting the job done. The problem is programmers don’t know how to get the job done.
rushed solutions and hacks are all too common. The how is extremely important for any software project. Obviously not a ‘cool’ how. But a practical ‘how’.
Simple is always better, but few people can see the ‘simple’ solution. There is no such thing as common sense. It takes much more talent to see the easy solution than the hard one. I’m still young, but that’s what I’ve learned.
Other than that…exactly drogin. However, cheating is a very very very big issue. It’s more important to Software because it is a relatively new field with lots of demand and a flood of people. A lot of unqualified are in there because of cheating.
“The problem is programmers don’t know how to get the job done.”
It’s not correct to blame this entirely on programmers. Many times the time constraints are so severe and the specifications so bad that getting the job done correctly isn’t an option or even possible. Delaying a project isn’t an option. This is more a problem of bad management and bad managers than bad programmers.
That’s one of the reasons I got out of the programming field.
Blame the programmers? I wish it was the programmer’s fault – I have even seen the code from bad programmers so I know they are out there.
But my personal experience when I made a living as a programmer are that the Bosses above the programmer are the problem.
1) Goals. Give a good programmer the goals of the program, and they can deliver working code. Keep changing the goals or add new ones every time the lastest version is delivered and you never have a finished program.
2) Debugging & Time. No matter what schedule the programmers need the bosses keep trying to cut it down because time = money! Well, bugs = money too! If a programmer tries to sit down and pre-design the logic to avoid bugs the bosses think they are wasting time. If the programmer writes the code quickly then wants to debug the bosses want to start using it right away. I have never seen programmers given the true time needed to deliver good working code.
3) Simple is not easy! It takes a lot of thought and usually a lot of experiments to find the simple solution. And even when you find one there always seems to be someone to complain because it is not the way they want to do it – period!
I blame most of the bad things to stupid decisions. I’m not even talking about the promises made by sales departments, which are a whole other dimension by themselves. I mean the decisions that people who don’t really know what’s behind a framework, technology, architechture, language, plataform… make sacrificing everything else to ‘time to market’.
Sometimes they don’t even spend a tenth of a second thinking about the consecuences of a bad decision. They even say things like “We’ll go with this. I know it’s a bad decision and I know it will cost us a lot later, but right now it’s the fastest route to take”. I’ve been told this many times. I’ve argued and I’ve begged, to no avail.
This is, imho, the single most important reason why software development sucks so bad.
>>I blame most of the bad things to stupid decisions. I’m >>not even talking about the promises made by sales >>departments, which are a whole other dimension by >>themselves.
Agreed.
I was in a meeting once where the client had been promised that their system would be integrated in 3 days. This was between an Insurance company and a catalog company.
I told them it would take me three days just to get through the (fairly large) specs and then I’d give them an estimate of the actual time. This produced an embarrassed silence, uncomfortable shifting in chairs etc. Then a few embaraasing ‘Are you sure?’s and ‘Could we not?’s
Three months of hard work later and I produced a really good solution which had error checking, docs, worked flawlessly and was later extended successfully.
But the damage was done – I was now unpopular with the sales managers. All because THEY tried to estimate a job for which they had no idea what was involved.
I often wonder if sales and management people in other disciplines are as mentally-deficient about making ridiculous claims that no sane person would believe. I can just imagine in my mind the sales people for a construction company telling the government that they could have a stretch of Interstate highway done within three days time.
I often wonder if sales and management people in other disciplines are as mentally-deficient about making ridiculous claims that no sane person would believe. I can just imagine in my mind the sales people for a construction company telling the government that they could have a stretch of Interstate highway done within three days time.
The salesman has to sell the product, right? I’ve been there with ridiculous claims that a software developer can do magic things in 25x shorter time than is actually required. The problem is classical: You have a developer and a salesman/boss. The two are often at odds because they don’t understand eachother. The developer grows weary of the boss, and the boss can’t understand why things can’t be completed by tomorrow.
I think most software developers have been there, and it’s a very uncomfortable way to work. You write crappy software, because it has to be finished yesterday. Tomorrow you’ll be building new software based on the crap you built yesterday. It gets worse, you stress up, you burn out and you leave the business.
This is why I work alone. I’m my own boss. I can decide what to do and when I have a finished product that comes close to perfection, it can be sold. Not sooner.
If I get a project that I’m being asked to do in an unreasonably short time, I simply decline and ask if they can find someone else to do it.
Software development should be much more on the developer's terms than sales or directors, but when money dictates what’s going on, a wise developer would start looking for another job where he/she is allowed to build quality software. I think most if not all developers/engineers want to build quality stuff.
80% of projects fail? It’s probably closer to 95%. Shipping doesn’t necessarily mean success.
Programming is a discipline, if you lack focus and are unable to funnel your energy and manage time properly you will produce nothing but suckware.
Universities cannot make a great programmer, only writing lots and lots of programs can make a programmer great. The amount of code for a comp sci BS is laughable and does not qualify you as one. Professional programmers fill a spec, usually a spec they have no hand in creating, blame marketing and the suits for cutting corners if you’re software sucks. Just exactly what do you think they left out when the time to market for a program slid from two years to two months?
Then I saw the link to “Joel on Software”……
Joel’s great.
Agreed.
Why does software suck?
Its caused by expectations and individual desires.
Marketing has succeeded in “programming” us into accepting limitations for hardware products (but I want my DVD player to be green…yeah…you better get some spray paint) but software products are held to a different standard.
Look at a VCR/DVD/Radio,Car,etc. Single purpose, simple interface. A DVD player has Play, reverse, skip (ok..crazy remotes…but nobody can EVER figure those out).
Software VCR/DVD/etc. But it should have a skip 2 track function, reverse 7 tracks, and forward 1/50th speed. That button color is wrong, i want to skin it..HEY…This software sucks!
People seem to accept simple, limited functionality in hardware devices today, and in fact complain if there are too many options.
Software, well, thats completely different.
So let’s complain that software is buggy, takes too long, programmers are idiots, all because we EXPECT that program to provide a custom solution to address everyones individual desire.
This is why software development will never be even comparable to engineering.
Imagine if your car was designed with the options available to even most simple programs…umm…how much would it cost?
If one looks at a discipline like mechanical engineering, the evolution of that field is made up with disasters going back to the weird looking pyramid in Egypt where they started building at one angle and finished building at another.
Software is a field that is less than 100 years old. We do not know the strength of the materials, the lifespans of the product or even have the requisite tools to perform the analysis.
When I think of coding and think that I have to write an iterator to loop through data or an if statement at such low levels, I know the tools are not there yet. Shouldn’t the tools know I want an iterator when I have a list and write the code so I can just say yes that is it. Shouldn’t the tools detect I am going to branch and write not only the if statements but the result statements.
Software has a long way to go.
Tools. This is how I explain the difference between software engineering and regular engineering to people: “imagine that all your tools, your pencil, your hammer, your lathe etc. were all designed by people who didn’t have a proper understanding of their job, let alone yours. Your pencil only worked after warming up, your hammer couldn’t do angles, and you had a different lathe for each material.”
As programmers, the languages we program in are hackish, full of traps, overly demanding for the task we want to do, don’t encourage proper documentation, and the APIs we use are often a tangled mess.
It’s going to take another 10 years before we have a computing language and API that makes software engineering a mould pressing process, instead of building things with matchsticks all the time.
Software doesn’t suck because it’s complex, and hardware’s not all that simple. It only seems that way to people who don’t have to design it or make it work.
Software sucks for a lot of reasons that basically boil down to “because the customers let it suck.”
When I was young, and enamoured of the writings of Dykstra, I believed his claim that software development should be a mathematical discipline, but, alas, I’m a mathematician and it eventually dawned on me that it is not.
Now I am old, and beset on all sides by those who think that software development is some sort of enginering discipline, but alas, I’ve worked with people in real engineering disciplines, and it’s definitely not one of those, either.
Software development is a business. It has a business model that rewards feature creep and punishes efforts to improve quality. No amount of whining is going to change that.
Are programmers partially responsible? Sure. When they forever follow the latest trend instead of picking a set of skills and polishing them to perfection. When they write bad code because they know it’s “good enough for now”. When they propose the features rather than fix the bugs.
Is whatever the hot-topic “development methodology” of the day is going to fix it? Well, let’s see, I first encountered software development “paradigm shift” 20 years ago, and have been through about 15 of them since with no noticable change in the quality of software. My crystal ball says: don’t bet on it.
The article is cookbook about what software development must be! I think more or less all developers have same thoughts and feelings at desk in hard coding.
One key note:Your client don’t care about your good/bad code, they care about the quality of your final work!
Edited 2006-11-03 08:59
The article struck me as rather naive. Sure, your employer may not care about the quality of your code, but they certainly will care when your “quick to produce but does the job” code is inextensible, meaning that the next time they request a feature it requires a major rewrite rather than simply adding a module or something along those lines.
In my experience “simple” solutions always come back to bite you, because inevitably a new feature will be requested and you’ll end up either having to rewrite the original code to fit in with the new feature, or implementing the new feature as an ugly hack on code that wasn’t designed to support it.
Edited 2006-11-03 09:51
Was it really the most simple solution?
Sometime ago we had to do a software that would send a given messages to a certain connection. My friend designed the software with message processing as its core in mind. I failed to realize that the simple solution was actually to design the software with connection management as its core! Quickly the software was bloated, we eventually rewrote it from scratch. The new design is very easily extensive.
What I am saying is that making a design simple, requires a lot of effort and thinking, and is definitely not the most simple thing :-), but eventually your effort will be rewarded.
No offense, but the example you give is for a rather trivial piece of software. If you’re trying to design, say, an invoicing system for a company, you’ll quickly discover that many programming tasks have no simple solution, and that any attempt to create one will result in inflexible code.
I’m not saying that the simple way is always a bad choice, merely that in many cases it’s just not an option.
I come from a background of developing/delivering software for large scale clients (financial institutions, government departments, etc). As both a business consultant and developer, I think that the problems with *large scale* projects that are perceived to have failed are due to a number of factors:
– Unbound expectations: business people do not always understand what software solutions can and cannot achieve. If underlying business processes, data gathering/storage, reporting lines etc are poor, the software that is integrated into this will not be able to fulfill its purpose. But it is this very visible interface that will be blamed.
Similarly, IT guys can (with the best of ententions) over-engineer a product. I tend to agree that it’s best to keep it as simple as possible: deliver phase 1 well, with the core functionality, and people will come back for more.
– Ill-defined requirements. Too many analysts will interview just one or two users and design a system based on their views. But it is quite possible that their views do not represent the actual business need. Having a more holistic understanding of business needs is crucial. Involve, clarify, confirm, repeat.
– Sloppy coding standards: too many developers want to bash out code without taking the time to document, comment, and apply proper standards. This leads to maintenance hell. No matter how many times I have instructed teams of developers to adhere to certain standards, one or more will inevitably ignore these intructions and do their own thing. Which requires rework. Usually my muggins here :/
– Poor data modelling skills. Get the underlying database model wrong, and it is an absoulte nightmare to change further down the line. I find it astonishing how many times I am called in to try to amend a system where the underlying table structure is so wholly inappropriate to the business needs of the users.
– Politics. be aware of any political undercurrents. One stakeholder’s project may be a threat to another individual. Be prepared.
– Maturity, or lack of it. I’ve been doing this for 16 years now (OMG, where did the time go!). I am prepared to stand up to a client and say no, and to argue my corner. However, it’s much harder to do so when you have not been through the school of hard knocks and backstabs. But saying yes when you know it’s wrong can only backfire on you. You will be blamed when things go wrong, and it will be partly justified.
In a word: communication. Get that wrong, and the whole project is doomed to failure. Get it right and there is no reason why you should not have a success. Avoid the “us and them”, and get people on board. It’s not as hard as it sounds
Oh, and I must be one of the few “happy” programmers alluded to in the intro paragraph. I still enjoy coding, much more so than any of the business-centric skills that I have learnt along the way.
The 80% failure rate is also one I would dispute. In the last 9 years at my current employer, our team has only ever had (I think) a single project where the client was truly unhappy. And this turned out to be because they had built expectations above and beyond their specified requirements. That taught us a lesson in managing expectations throughout the lifecyle of the project! And I hold my hands up – it was one of mine.
Software is unlike anything else humans have done so far.
Building a house or a bridge is simple: you just follow a simple set of physical laws applied on specific equipment, and if the outcome is computed correctly, then construction goes well and the product is fine.
But software has physical limitations: it can not be proven correct! (see the halting problem for proof). Therefore any software is doomed to fail at some point, due to some constraint violation, where the constraint was implicit or introduced as an afterthought.
Hmmm… 80% failed… May be.
100% of projects in my company are finished successfully.
Failed because of complexity? Not true. They are failed because of lack of competency, lack of experience, lack of undestanding and because of “it-so-cool-to-write-software” developers. Software development now is too industrialized,if you know what I mean. There is no “complexity” in our world. All complexity is artificial and created by humans.
Alot of developers and bloody consultants trying to “develop undeveloperable”, “take untakeable”, do more than required.
Another problem – bloody money! Nobody cares about customer, nobody cares about quality of software. Everybody cares only about money.
So boring…
There is no “complexity” in our world. All complexity is artificial and created by humans.
Wow. Care to prove that one, or are proofs a complexity of mankind?
ok, let’s see…
The customer wants an application that performs three tasks: A, B, C.
As you have read this article, you say: “Software must be simple” and implement A, B and C; removing criteria about maintainability, extensibility, modularity, etc… anyway, the customer does not want those features: (s)he just wants A, B and C.
You show the final product to your customer, (s)he sees the result of his/her thoughts turned into software and (s)he says: “Could you please add a button here that performs A1 and A2? I think it should be very easy to you”… But you did not think about extensibility: you just thought about A, B and C and you have to do a lot of work to support A1 and A2… what will happen when your customer will want the features B1, B2 and B3; C, D and E? How will be the final result? Will be easy to achieve the new customer requirements? Will the code be maintainable or will it be just a set of features added with no order or criteria?
Programming has a lot of art and a lot of engineering and complexity and abstraction are part of good and big software… The “simple software” today, will be the “patched software” tomorrow and the “refactored software” after tomorrow…
I’d like to say a few words on the different groups that are blamed to be the reason why software sucks.
First: Machines. Why do we have software? Do we need it? No, we don’t. We could represent algorithms in hardware by hard-wiring the circuits. Hardware-represented OpenGL? Or back to vacuum tubes and patch fields for coding? What an image.
And now: The bosses. In most cases – it’s sad! – they really don’t have a clue about what they’re responsible for. Phrases like “Do it another way!” or “I meant my concept to be different from what you’ve done. Do it again.” can turn every good software into crap. I’ve seen this for myself, and it was a reason for me to work for my own rather than being employed in a company where the bosses are that stupid that they can’t realize the needs why certain things have to be done this and that way, even if they are explained.
And now for the customers, the users of software. In many cases you’ll find (i) they don’t know what they want or (ii) they can’t say (express) what they want. So the programmer has to guess what the customer wants – or needs, that’s a fine but significant difference. Often it’s hard to fulfill both needs.
The author has recognized this fact, but it seems he doesn’t want to see the full truth.
So what can you do? Step one is focusing on your customer. They often don’t know what they want, which is fine. Sit down, listen to them, and I mean listen. […] Next, through hearing their wants and needs, hash out a very general abstract in what you’re going to build.
Not every customer is able to formulate what he wants or needs, that’s a problem. Another problem – on the other hand – is that well formulated wishes or needs are not considered effectively before development starts, and not possible to use when development is in progress. So even good ideas are lost.
I fully agree with his statement:
Don’t add complexity up front, or you’ll fail.
It’s fitting good into his claim “Software needs to be simple”, but complex problems often need a complex solution, even if the problem seems to be easy (and in fact it’s not). The trick is to do the complex solution with simple to use software. Here’s the difference between the easyness of software and its usage.
Many “freshmen” in programming assume that programming means to produce sourcecode. That’s not true, because implementation is one of the smaller parts of programming. Developing concepts, datatypes, (G)UI handling takes more time. Even testing and bugfixing does.
While those who studied CS know about principles and complexitxy, their programming abilities are not very good because programming (planning, coding, testing etc.) are not part of the curriculum, they’re just to illustrate how to do something. During the CS courses, many language concepts and programming languages are touched, no one is “really used”. When the students receive their diploma, they may think, “Wow, I am a programmer!”, but they are not. (I tell from German universities; in other countries it may be different.)
The other programmer group are the ones who learn by practice. Maybe, their scientifical knowledge is not the best, they have gained knowledge about how programming works and which steps are included. These programmers are the real ones – the ones the author seems to speak to.
And then, there are “script kiddies” (I don’t know if there’s an equivalent term in english, sorry). They just go there and clickityclick, thinking about theirselves as professionals. But they have the possibility to develop up to real programmers. Only few of them stay “script kiddies”. I tell you: I’ve seen programs that aren’t worth the occupied storage space.
The final statement from the article:
I think most of us have lost sight of what’s important in software development. […] Lets enjoy delivering real business solutions. […] Sounds easy, eh? Lets make software not suck anymore!
So it’s all a question of money? For a successful business solution, one would need customers to spend money. Are they really willing to? “Why should I have to pay for software when I can download it for free?” would be a typical statement, as well as “Why sould I buy something new while I can use my older software?” I’m not sure.
In the end, I don’t think software sucks in general. There’s sucking software, indeed. I’ve seen one in our local “Arbeitsagentur” (authority for unemployment). Daily updates, daily crashes, daily changes – costs tons of money but doesn’t work properly. This sucks.
An example of how a typical end-user specifies what kind of computer
program they want you to design:
Dear Mr. Architect:
please design and build me a house. I’m not quite sure of what I need or
want in a house, so you should use your discretion.
My new house should have between 2 and 45 bedrooms. Just make sure the
plans are such that bedrooms can be easily added or eliminated. When you
bring the blueprints to me, I’ll make the final decision of what I want.
Also, bring me the cost breakdown for each configuration so that I can
arbtrarily pick one.
Keep in mind that the house I ultimately choose must cost less than the
one in which I currently reside. Make sure, however, that you correct all
the deficiencies that exist in my current home. I’d allow you to view my
current house to help you understand and avoid the problems that I’m refer-
ring to, but I’m afraid that it may interfere with your creative abilities.
Also keep in mind as you design this house that I wish to keep yearly
maintainance costs as low as possible. This should mean the incorporation
of the latest technological advancements in siding & insulation. If you
choose not to specify aluminum siding, be prepared to explain in detail.
Please make sure that modern design practices and the latest materials
are used in constructing the house. The house should be very nice.
However, be alerted that the kitchen should be designed to accommodate my
1952 Gibson refrigerator and any other items with which we don’t wish to
part.
To assure that you are building the correct house for my family, make
sure that you contact each of my children and in-laws. My Mother-in-law
will have very strong feelings about how the house ought to be designed
since she visits us at least once a year. Make sure that you weigh care-
fully all suggestions made by family members and make the right decisions.
I, however, retain the right to override any decisions you make.
Please don’t bother me with details right now. Your job is to develop
the overall plans for this house. Get the big picture. It’s not appro-
priate at this time to be choosing such things as the color of the carpet,
although you should keep in mind that my wife likes green.
Also, do not worry at this time about acquiring the resources needed to
build this house. Your first priority is to develop detailed plans and
specifications. However, once I accept the plans, I’ll expect to have the
house under roof within 48 hours.
While you are designing this house specifically for me, keep in mind
that sooner or later I’ll probably sell the house. It should appeal to
the largest number of potential buyers. Please make sure, before you
finalize the plans, that there is a consensus of the population in my area
as to the desirability of the features included in the house.
You are advised to look at my neighbor’s house, which he constructed
last year. We like it a great deal because it has many features that we
would like to have in our new home, particularly the 75-foot swimming pool.
With careful engineering I believe that you can design this into our new
house without impacting the construction cost.
Please prepare a complete set of blueprints. I’s not necessary at this
time to do the real design since these blueprints will used only for
construction bids. Please be advised, though, that any resulting increase
in the cost caused by future design changes will result in your getting
your hands slapped.
You must be thrilled to be working on such an interesting project such
as this. To be able to use new kinds of construction and to be given such
freedom in design is something that doesn’t happen very often. Contact me
as soon as possible with your design ideas. I’m most enthusiastic in
seeing what you develop.
Sincerely,
Prospective Client
P.S. My wife has just told me that she disagrees with many of the
instructions that I’ve given you in this letter. As the architect, it is
your responsibility to resolve these issues. I’ve tried in the past and
have been unable to accomplish this. If you can’t handle this, I’ll have
to look for a new architect.
P.P.S. Perhaps what I need is not a house at all, but rather, a travel
trailer. Please advise me as early as possible if this is the case.
The problem is much more complex than most people care to understand.
Lets take a simple example, people like car analogies, last year Chevy introduced the new Camaro concept. Now its not going to be released until 2008 – 2009. How long did Chevy have to build the prototype. I’d be willing to bet it has been on the drawing board since around 2002 maybe earlier when they decided to kill off the F-Body platform. So we are talking 6 years to prototype, build, test, work out the bugs, etc… until they release version 1.0 to the public. Each year after that the public will see a new version release with a few minor enhancements.
Software is a completely different story. You don’t get 6-7 years to build your product and then give it to the customer. The customer is sold the product before its ever built. They want it delivered in a year, because after all its only software. You spend millions of dollars engineering a car but no one wants to spend millions of dollars to develop a new software app. The principles to building cars, buildings, bridges, ships, and airplanes have been around for 100+ years and little has changed in their basic design. Software languages, operating systems, standards, are all expected to evolve every six months.
The issue is not the developers or the software its the consumer of the product. They want it and they want it now, and they want it for free. But it’s not that simple it costs. If you try to sell wine before its time you get grape juice.
Depends on the environment and the conditions for success. We’ve had two large projects and a medium sized project in a row come in on time with an acceptable feature set that kept management and marketing happy.
We did, however, have one large nightmare project that ran 1 year over. But for the record I’d like to say that is because management decided to have an outside firm do the work instead of the internal IT developers… after a year of nothing, they called us (IT) in to do the job. We had to more or less start from scratch and had the job done in another year.
And I’d like to say that it’s not that there are a lot of bad developers out there, but often that they simply need more discipline and direction. It seems like everytime management hires a new person, some new technology gets implemented that everyone else has to learn. Not to mention that industry “buzzwords” are added or changed every few months. It’s ridiculous.
I had a meeting with my new boss a couple of weeks ago (new management). His group now consists of a few new people, some consultants and some of the old schoolers (myself included). He started talking about implementing a “service layer” in our new project and I KNEW the other guys were sitting there thinking “what the hell is a “service layer”… so I interrupted and said “you mean a wrapper that simplifies the underlying “shared services.”
I mean, why not just call it what it is? Why do we have to keep inventing new words to describe the same old stuff that has been around since the age of the dinosaur mainframes?
The basic design concepts have not changed in 30 years, yet every year some *new and improved* methodology comes out. It’s all bullshit. (can I say that?)
Any architect will tell you that there’s an infinite variety of ways to construct a building; however, the reason that building codes exist is to ensure conformity to safety standards and prevent loss of life & property. Software has no equivalent of building codes. Yes, there are recommendations and best practices; however, there’s no such thing as a software building inspector who comes around and checks on your work. Hence, divergence becomes the norm.
Most software projects fail because…
1. Participants fail to collect or identify proper customer requirements. ie. Delivering something that nobody requested.
2. Lack of participation by or support from stakeholders.
3. Failure to scope tasks to available time and resources.
4. Political pressure to deliver everything requested — regardless of whether it’s sane — and an unwillingness to compromise.
5. Endless collection of requirements or never-ending (re)design.
6. Faulty core architecture decisions.
7. Unrealistic expectations/unreachable standards of quality (ie. perfection) before setting a release date.
8. Failure to adapt project plans to meet changing circumstances.
9. Failure to keep or maintain a schedule.
10. Lack of understanding about the three-legged stool of software development tradeoffs.
11. Hiring unqualified individuals.
12. Not applying any kind of rational development methodology. ie. Doing everything in an ad hoc fasion.
13. Not trying to reduce risk incrementally.
14. Trying to build something that should probably be bought from a commercial source. ie. “Not-Invented-Here-Syndrome”
15. Etc.
Notice anything about this list? There are actually very few items on this list that are related to poor implementation. Most project failures have more to do with poor project management than poor implementation. Don’t blame individual programmers. Proper oversight (ie. code reviews, design reviews, spec reviews, etc) can identify problems very early in the development cycle. Blaming programmers is nothing more than scapegoating.
Edited 2006-11-03 17:31
“7. Unrealistic expectations/unreachable standards of quality (ie. perfection) before setting a release date.
8. Failure to adapt project plans to meet changing circumstances.
9. Failure to keep or maintain a schedule….”
If you notice Number 8, “Failure to adapt project plans to meet changing circumstances.” basically blows away Numbers 1-9 (and maybe even more). If you adapt to project plans, you cannot maintain a schedule, You cannot set a release date. You end up collecting more requirements (for your adaptation…),etc, etc.
It seems that everything ends up being a “crisis” to the stakeholders. Whether or not its the color of a screen or core functionality…. and if you want the pay-check, you adapt, but in adapting you cannot plan, schedule, etc. reliably. You end up wanting to finish it as quick as possible, before any more “issues” arise.
This is just some XP regurgitation with some seemingly contradictory positions that are vague in nature.
Software is complex, but software should be simple. So we should eschew abstraction but focus on the what and not the how. Or rather we should focus on the customer, and not the what. Or rather we should focus on how to interact with the customer, but not the customer. Or rather we should just be focused on getting the product out the door.
Software isn’t simple and it can never be simple, except when viewed at a sufficient distance. Its complexity is hidden through abstraction. We construct metaphors that enable us to more concisely express our intention, but the machinery underneath is complicated and should we violate that abstraction then its unpleasantness shoots upward like a fountain of disorder. What seems like the right level of abstraction changes with what we think we are doing, and if that changes dramatically all of the time then we have to violate what were previous abstractions and that is a sure fire way to be an unhappy developer, because now you have to go back and redo all sorts of decisions or deal with duct tape that somewhere down the road becomes someone’s sucky software. The reason software developers end up focused on the how, is that they have invested themselves in various abstractions that enable them to express these solutions we’re supposed to care so much about. It is in essence their job to focus on the how, and not on what sort of coffee to bring to meetings with the client, because they are being paid to write software much in the same way as a structural engineer is being paid to work out the details that prevent your building from falling over in a wind storm, and not what kind of biscotti to bring on your weekly chat, or how to make your business money. The only distinction in the mind of the developer between the how and the what, is that the latter defines the nature of the former. The developer is not going to forget the what, because that is why he is wasting 50-60 hours a week sitting on his ass wrecking his wrists typing code in a programming language he probably doesn’t even like, hammering against buggy libraries and working around platform warts while dealing with a real life Dilbert strip.
It’s the management side whose job it is to figure out how to make money, to manage costs, and to keep the musical chairs to a minimum for developers. When they fail to do so, it’s their fault that the software sucks, because they have failed to do their job. It doesn’t help that in a lot of cases the entire project is stupid and pointless from the beginning, and that is what contributes to the spastic development process and useless chum that’s spit out when the money is all gone.
In the end as happy and wonderous as it would be if software just materialized out of thin air through focusing on its existence, it only exists because the how is dealt with. That badass XP book you just got done reading isn’t going to make software materialize out of nowhere, either.
Finally, someone who actually recognizes the problem! Software written for corporate applications is often written by people with no clue what the end user really needs, and approved by some corporate idiot who unqualified to approve anything but a vacation request. The software usually works, more or less, in spite of it’s poor design. But there’s always constant unnecessary compromises being made by the users. I can’t tell you how many programmers don’t even know how to set the tab stop order properly.