History of software engineering


The history of software engineering begins around the 1960s. Writing software has evolved into a profession concerned with how best to maximize the quality of software and of how to create it. Quality can refer to how maintainable software is, to its stability, speed, usability, testability, readability, size, cost, security, and number of flaws or "bugs", as well as to less measurable qualities like elegance, conciseness, and customer satisfaction, among many other attributes. How best to create high quality software is a separate and controversial problem covering software design principles, so-called "best practices" for writing code, as well as broader management issues such as optimal team size, process, how best to deliver software on time and as quickly as possible, work-place "culture", hiring practices, and so forth. All this falls under the broad rubric of software engineering.

Overview

The evolution of software engineering is notable in a number of areas:
  • Emergence as a profession: By the early 1980s software engineering had already emerged as a bona fide profession, to stand beside computer science and traditional engineering.
  • Role of women: Before 1970 men filling the more prestigious and better paying hardware engineering roles often delegated the writing of software to women, and legends such as Grace Hopper or Margaret Hamilton filled many computer programming jobs.
Today, fewer women work in software engineering than in other professions, a situation whose cause is not identified. Many academic and professional organizations consider this situation unbalanced and are trying hard to solve it.
  • Processes: Processes have become a big part of software engineering. They are hailed for their potential to improve software but sharply criticized for their potential to constrain programmers.
  • Cost of hardware: The relative cost of software versus hardware has changed substantially over the last 50 years. When mainframes were expensive and required large support staffs, the few organizations buying them also had the resources to fund large, expensive custom software engineering projects. Computers are now much more numerous and much more powerful, which has several effects on software. The larger market can support large projects to create commercial off the shelf software, as done by companies such as Microsoft. The cheap machines allow each programmer to have a terminal capable of fairly rapid compilation. The programs in question can use techniques such as garbage collection, which make them easier and faster for the programmer to write. On the other hand, many fewer organizations are interested in employing programmers for large custom software projects, instead using commercial off the shelf software as much as possible.

    1945 to 1965: The origins

Early usages for the term software engineering include a 1965 letter from ACM president Anthony Oettinger, lectures by Douglas T. Ross at MIT in the 1950s. Margaret H. Hamilton is the person who came up with the idea of naming the discipline, software engineering, as a way of giving it legitimacy during the development of the Apollo Guidance Computer.
The NATO Science Committee sponsored two conferences on software engineering in 1968 and 1969, which gave the field its initial boost. Many believe these conferences marked the official start of the profession of software engineering.

1965 to 1985: The software crisis and operating systems

Software engineering was spurred by the so-called software crisis of the 1960s, 1970s, and 1980s, which identified many of the problems of software development. Many projects ran over budget and schedule. Some projects caused property damage. Through negligence in publishing software with critical bugs, some lost their lives due to software failures. One of the most striking examples of harm through software bugs was the Therac-25 race condition bug. The bug caused a radiation therapy machine to administer overdoses of radiation in cases where low doses should have been used. The software crisis was originally defined in terms of productivity, but evolved to emphasize quality. Some used the term software crisis to refer to their inability to hire enough qualified programmers. During this time, Silicon Valley cemented itself as the best location for software engineers to work.
  • Cost and Budget Overruns: The OS/360 operating system was a classic example. This decade-long project from the 1960s eventually produced one of the most complex software systems at the time. OS/360 was one of the first large software projects. Fred Brooks claims in The Mythical Man-Month that he made a multimillion-dollar mistake of not developing a coherent architecture before starting development.
  • Property Damage: Software defects can cause property damage. Poor software security allows hackers to steal identities, costing time, money, and reputations.
  • Life and Death: Software defects can kill. Embedded systems used in radiotherapy machines prove the ability to fail so catastrophically that they administered lethal doses of radiation to patients. The most famous of these failures are the Therac-25 incidents.
Peter G. Neumann has kept a contemporary list of software problems and disasters. The software crisis has been fading from view, because it is psychologically extremely difficult to remain in crisis mode for a protracted period. Nevertheless, software – especially real-time embedded software – remains risky and is pervasive, and it is crucial not to give in to complacency. Over the last 10–15 years Michael A. Jackson has written extensively about the nature of software engineering, has identified the main source of its difficulties as lack of specialization, and has suggested that his problem frames provide the basis for a "normal practice" of software engineering, a prerequisite if software engineering is to become an engineering science.
One of the largest projects undertaken by software engineers during this time period was the development of modern operating systems. Starting in Bell Labs and then moving to UC Berkeley, Ken Thompson and Dennis Ritchie, among other software engineers, worked to create Unix V6 in 1975. Unix V6 was a landmark operating system that set standards for future operating systems and is used today to educate students about proper operating system principles. Moreover, future operating systems built on Unix V6's methods, and its descendants can be grouped into five types of operating system paradigms: Grassroots Systems, Large-Scale Systems, Hybrid Systems, Experimental Systems, and Minor Systems. In contrast with Unix, software engineers at MIT in 1983 built GNU as an open source alternative to Unix. As an early open source software, GNU was beloved by a small group of developers and its work grew the open source software development community in the 80s.

1985 to 1989: "''No Silver Bullet''"

For decades, solving the software crisis was paramount to researchers and companies producing software tools.
The cost of owning and maintaining software in the 1980s was twice as expensive as developing the software.
  • During the 1990s, the cost of ownership and maintenance increased by 30% over the 1980s.
  • In 1995, statistics showed that half of surveyed development projects were operational, but were not considered successful.
  • The average software project overshoots its schedule by half.
  • Three-quarters of all large software products delivered to the customer are failures that are either not used at all, or do not meet the customer's requirements.

    Software projects

Seemingly, every new technology and practice from the 1970s through the 1990s was trumpeted as a silver bullet to solve the software crisis. Tools, discipline, formal methods, process, and professionalism were touted as silver bullets:
  • Tools: Especially emphasized were tools: structured programming, object-oriented programming, CASE tools such as ICL's CADES CASE system, Ada, documentation, and standards were touted as silver bullets.
  • Discipline: Some pundits argued that the software crisis was due to the lack of discipline of programmers.
  • Formal methods: Some believed that if formal engineering methodologies would be applied to software development, then production of software would become as predictable an industry as other branches of engineering. They advocated proving all programs correct.
  • Process: Many advocated the use of defined processes and methodologies like the Capability Maturity Model.
  • Professionalism: This led to work on a code of ethics, licenses, and professionalism.
In 1986, Fred Brooks published his No Silver Bullet article, arguing that no individual technology or practice would ever make a 10-fold improvement in productivity within 10 years.
Debate about silver bullets raged over the following decade. Advocates for Ada, components, and processes continued arguing for years that their favorite technology would be a silver bullet. Skeptics disagreed. Eventually, almost everyone accepted that no silver bullet would ever be found. Yet, claims about silver bullets pop up now and again, even today.
Some interpret no silver bullet to mean that software engineering failed. However, with further reading, Brooks goes on to say: "We will surely make substantial progress over the next 40 years; an order of magnitude over 40 years is hardly magical..."
The search for a single key to success never worked. All known technologies and practices have only made incremental improvements to productivity and quality. Yet, there are no silver bullets for any other profession, either. Others interpret no silver bullet as proof that software engineering has finally matured and recognized that projects succeed due to hard work.
However, it could also be said that there are, in fact, a range of silver bullets today, including lightweight methodologies, spreadsheet calculators, customized browsers, in-site search engines, database report generators, integrated design-test coding-editors with memory/differences/undo, and specialty shops that generate niche software, such as information web sites, at a fraction of the cost of totally customized web site development. Nevertheless, the field of software engineering appears too complex and diverse for a single "silver bullet" to improve most issues, and each issue accounts for only a small portion of all software problems.