Software development

The Software Development Process

• acquisition
• implementation
• maintenance
• retirement

It can be divided into three phases:

1. Project definition (what?)
2. Implementation (how?)
3. Support (change)

• Testing
• Technical reviews
• Quality assurance
• Configuration management
• Documentation
• Reuse
• Metrics
• Risk management

Process Maturity

1. Initial
2. Repeatable
3. Defined
4. Managed
5. Optimising

Software Process Models

• Waterfall model (mainly of historical interest)
• Spiral development. Involves cycles of
  • Customer negotiation
  • Design
  • Risk analysis
  • Implementation
  • Measurements
  • Evaluation
  Successive iterations can involve
  • Prototype
  • New product
  • Improved product
  • Maintenance cycles
• Rapid application development (based on business/data/process modelling and 4GL implementation)
• Incremental refinement
• Prototype-based (of e.g. user interface, functionality, or algorithms)

Requirement Types

Product / process

• Product requirements
• process requirements

Product Requirements

• Functional requirements
• Non-functional requirements

Levels of Detail

• User requirements, targeting
  • customer administrative personnel
  • contractor administrative personnel
  • end users
  • customer engineers
  • system architects
• System requirements, targeting
  • end users
  • customer engineers
  • system architects
  • contractor programmers
• Software design specification, targeting
  • customer engineers (possibly)
  • system architects
  • contractor programmers

Requirement Drivers

• High level business goals
• Operating environment
• Management environment

• End-users
• Customers
• Regulatory bodies
• Development engineers

Elicitation Techniques

• Interviews
• Scenarios
• Prototype development
• Structured meetings
• Observation
• Existing internal and external documentation

Non-functional Requirements

Product requirements

• usability
• efficiency
  • space
  • time
• reliability
• portability

Process requirements

• delivery
• implementation
• standards compliance

External requirements

• interoperability
• ethical requirements
• legislative
  • privacy
  • safety

Non-functional Requirement Metrics

• Time
  • Transactions / sec
  • Response time
  • Time to complete an operation
• Space
  • Main memory
  • Auxiliary memory
  • (Cache)
• Usability
  • Training time
  • Number of choices
  • Mouse clicks
• Reliability
  • Mean time to failure
  • Downtime probability
  • Failure rate
  • Availability
• Robustness
  • Time to recovery
  • % of incidents leading to catastrophic failures
  • Data corruption probability after a failure
• Portability
  • % of non-portable code
  • Number of systems where software can run

Requirement Management

• Requirement identification (e.g. numbering)
• Change management
• Traceability
  • Requirement source
  • Requirement dependencies
  • Design mapping
• Support tools

Advantages of Prototype Development

• requirement elicitation
• requirement validation
• bridging communication gap between developers and end-users
• demonstrating a working system
• user training
• testing the final system (parallel testing)

• a more usable final system
• better end-user requirement matching
• higher quality design
• easier maintenance
• lower development cost

Prototype Development Techniques

• High-level languages
• 4GLs
• Component based development
• GUI editors

Implementation Guidelines

• Automate
• Refactor often
• Use existing libraries and components
• Express complex structures using data
  static const int month_days[2][12] = {
    { 31, 28, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31 },
    { 31, 29, 31, 30, 31, 30, 31, 31, 30, 31, 30, 31 }
  };
  
• Do not patch bad code; rewrite it
• Use recursive procedures for recursive structures
• When correcting an error look for similar ones
• Comments should not repeat the code
• Avoid clever tricks
• The code should mainly target people
• Use efficient data structures and algorithms
• Target correctness before efficiency
• Write the comments before the code
• Adopt a structured style even when using an unstructured language
• Use a programming language appropriate for the application you are building
• Employ assertions
• Write self checking code
• Adopt and follow a programming style

Development Tools

• Modelling and design tools
• Editors
• Interpreters and optimising compilers
• Code generators
• Debuggers
• Execution profilers
• Code browsers
• Help systems
• Test frameworks
• Configuration management systems
• Integrated development environments (IDEs)
• Tool workbenches

An application wizard

A debugger environment

The Java class documentation

Domain-specific Languages

• Target a specific problem domain
• Examples
  • lex
  • yacc
  • SQL
  • 4GL
  • PHP
• Efficiently capture the application's logic
• Lower the semantic distance between the problem and the program

Reuse Methods

• Process reuse
• Design patterns
• Component-based programming
• Domain-specific libraries
• Generic libraries
• Application frameworks

Reuse: the Pros and Cons

Pros

• Increased reliability
• Fewer development risks
• Capitalise on expert skills
• Adopt standards
• Decrease development time

Cons

• Application maintenance
• Long term reused code maintenance
• Tool support
• Not invented here syndrome
• Searching, evaluating, and adopting reused code

Optimisation

• The program is really slow
• The optimised program will continue to be correct
• The optimised program will continue to be readable
• The optimised program will continue to be reliable (cf buffer checks)

• Use more efficient data structures and algorithms
• Execution profile; focus on hot spots
• Use a more efficient environment (e.g. move a 4GL into C++)
• Capitalise on the compiler's optimisations
• Optimise code
  • Efficient use of hardware resources (parallelism)
  • Avoid system calls
  • Avoid the use of external data
  • Save temporary results
  • Pre-calculate results

Database Optimisation Techniques

• Use appropriate indexes
• Remove unneeded indexes
• Minimise data transfers from client to server
• Use SQL code instead of program loops
  For example, the code:
          Do While Not tblPersons.EOF
              If tblPersons("Name") = looking Then
                  MsgBox "Your number is " & tblPersons("Number")
              End If
              tblPersons.MoveNext
          Loop
  
  could be replaces with:
          Set qryPersons = dbsDb.OpenQuery("SELECT Number from Persons Where Name ='" & looking & "'")
          Set rstPersons = qryPersons.OpenRecordset
          If Not rstTables.EOF
              MsgBox "Your number is " & rstPersons("Number")
          End If
  
• Use tables with temporary data and triggers
• Use optimisation tools

Portability

• Different processors
• Different operating systems
• Different development tools and libraries

• Using only standard features
• Avoiding exotic and cutting edge features (even if standard)
• Avoiding undocumented features
• Not depending on binary formats and the host's byte order
• Using only standard or portable libraries
• Storing data in plain text files, possibly with XML markup

Internationalisation and Localisation

Internationalisation

Making software usable in different locales

Localisation

Adjusting the software for a given locale

• Local text entry
• Local text processing (e.g. sorting, comparing)
• Local text display (fonts, ligatures)
• Representation of numbers
• Representation of date and time
• Representation of currency values
• Localised messages
• Localised commands

Principles of Testing

• Each test case must address one requirement (or a part of it)
• Preparation of test cases should begin a lot earlier than their execution
• Pareto principle: 80% of the errors will involve 20% of the code
• Testing should start from small units and end with the complete system
• Testing is never complete
• Effective testing can only be performed by a third party
• Testing should involve black-box and black-box techniques
• Ideally, testing should automated, so that it can be executed after every software modification

Maintenance Types and Strategies

• Bug corrections (17%)
• Response to execution environment changes (18%)
• Functionality changes (65%)

Maintenance

Fixes and new functionality

Refactoring

Architectural changes without important functional changes

Re-engineering

Re-implementation often without new architectural or functional changes

Configuration Management

• Multiple programmers working on the same code
• Notifying group members of code changes
• Software element interdependencies
• Management of software releases

• Specifications
• Data dictionary
• Architectural design
• Module design
• Interfaces
• Test plans and procedures
• Test cases
• Source code
• Documentation

Project Management Pitfalls

Software is Different

• The product has no physical body
• There are no common and commonly accepted development processes
• Large projects are bespoke development efforts
• Software is mistakenly considered malleable
• Small software changes can have large impacts
• Component reuse is not commonly practiced

• Personnel shortages
• Non realistic time planning
• Requirements misunderstanding
• Interface usability problems
• Software complexity beyond customer's needs
• Insufficient tracking of changes requirements
• Problems in reused components or external software
• Problems in outsourced elements
• Slow response time
• Project's beyond the current state of the art

Project Management

• Persons
  • Administrative personnel
  • Project management personnel
  • Programmers
  • Customers
  • End users
• Product
• Process
• Project

Project Staffing Case Study

1. Main team; responsible for
   • Project management
   • Software development
   • QA
   • User training
   • Usability
   • User interface
   • Release engineering
2. Support team; responsible for
   • Product management
   • Marketing
   • Support
   • Beta testing management

• Development manager
• QA manager
• User manager
• Usability manager
• Release manager

• Staffing
• Human resource management
• Project planning and implementation
• Team coordination
• Time plan management

• Architectural and technical elements
• Selecting tools, technologies and standards
• Mentoring other tem members
• Following up on project problems
• Writing code

• Agreeing on a common architecture with other managers
• Requirement elicitation and validation
• Designing the specific functionality
• Helping the QA team
• Writing code

Cost Models

• Estimating a project's cost can be very difficult
• Organisational experience plays a major role
• Cost estimates start from code estimates
• Code estimates can be measured in
  • Size (e.g. lines of code)
  • Function points (e.g. queries, screens, tables, ...)
  • Object points
• Constructive estimation techniques (like COCOMO II) exist, but require organisation and project-specific calibration

10 Productivity Aphorisms

• Correcting an error after delivery can be 100 times more expensive than correcting it during the requirement gathering phase
• A time schedule can be compressed by up to 25%, but no more
• One euro in development effort will amount to two euros in maintenance effort
• Development and maintenance effort are mainly dependent on the size of the program in lines of code
• Staff variability is the major cause behind development productivity variations
• The ratio of software to hardware cost has changed from 15/85 in 1955 to 85/15 in 1985 and continues to expand
• The ratio between design, development, and testing is approximately 60-15-25.
• Software systems and products are three times as expensive as a stand-alone software problem
• Testing will uncover 60% of the errors
• Many software development phenomena and practices follow the 80/20 Pareto rule: 80% of the contribution is due to 20% of the contributors

Agile Development

Manifesto for Agile Software Development

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

• Individuals and interactions over processes and tools
• Working software over comprehensive documentation
• Customer collaboration over contract negotiation
• Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

eXtreme Programming

• Communication
• Simplicity
• Feedback
• Courage

• Team-based release plans
• Small rapid releases
• Use of architectural metaphors
• Simple designs
• Complete coverage through automated test cases
• Refactoring
• Pair programming
• Collective ownership
• Continuous integration
• A 40-hour week
• On-site customer
• Adherence to coding standards

Bibliography

• Christopher Alexander, Sara Ishikawa, Murray Silverstein, Max Jacobson, Ingrid Fiksdahl-King, and Shlomo Angel. A Pattern Language. Oxford University Press, 1977.

• Friedrich L. Bauer. Decrypted Secrets: Methods and Maxims of Cryptology. Springer Verlag, 1997.

• Kent Beck and Erich Gamma. Test infected: Programmers love writing tests. Java Report, July 1998.

• Kent Beck. Extreme Programming Explained: Embrace Change. Addison Wesley Longman, 2000.

• Jon Louis Bentley. Writing Efficient Programs. Prentice-Hall, 1982.

• Jon Louis Bentley. Programming Pearls. Addison-Wesley, 1986.

• Jon Louis Bentley. More Programming Pearls: Confessions of a Coder. Addison-Wesley, 1988.

• Jon Louis Bentley. More Programming Pearls: Confessions of a Coder, chapter Little Languages, pages 83-100. Addison-Wesley, 1988.

• Grady Booch, James Rumbaugh, and Ivar Jacobson. The Unified Modeling Language User Guide. Addison-Wesley, 1999.

• Paul Booth. An Introduction to Human-Computer Interaction. Lawrence Erlbaum Associates, Publishers, New Jersey, USA, 1989.

• F. P. Brooks. The Mythical Man Month. Addison-Wesley, 1975.

• William J. Brown, Raphael C. Malveau, Hays W. McCormick III, and Thomas J. Mowbray. AntiPatterns Refactoring Software, Architectures, and Projects in Crisis. Wiley, 1998.

• Mark Burgess. Principles of Network and System Administration. John Wiley and Sons, 2001.

• Stuart K. Card, Thomas P. Moran, and Allen Newell. The Psychology of Human-Computer Interaction. Lawrence Erlbaum Associates, Publishers, New Jersey, USA, 1983.

• James O. Coplien and Douglas C. Schmidt. Pattern Languages of Program Design. Addison-Wesley, 1995.

• Michael A. Cusumano and Richard W. Selby. Microsoft Secrets. The Free Press, 1995.

• Tina Darmohray, editor. Job Descriptions for System Administrators. Short Topics in System Administration. USENIX Association, Berkeley, CA, USA, 1997.

• Alan M. Davis. 201 Principles of Software Development. McGraw-Hill, 1995.

• Tom DeMarco and Timothy R. Lister. Peopleware: Productive Projects and Teams. Dorset House Publishing, 1987.

• Dorothy Elizabeth Robling Denning. Cryptography and Data Security. Addison-Wesley, 1983.

• Institute of Electrical and Electronics Engineers, Inc., New York, NY, USA. Information Technology--Software Life Cycle Processes--Software Development Acquirer-Supplier Agreement, 1995. EIA/IEEE Interim Standard J-Std-016-1995 (Issued for Trial Use).

• Tom Forester and Perry Morrison. Computer Ethics: Cautionary Tales and Ethical Dilemmas in Computing. MIT Press, 1990.

• Martin Fowler. Refactoring: Improving the Design of Existing Code. Addison-Wesley, Boston, MA, 2000. With contributions by Kent Beck, John Brant, William Opdyke, and Don Roberts.

• Erich Gamma, Richard Helm, Ralph Johnson, and John Vlissides. Design Patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley, 1995.

• Daniel M. Hoffman and David M. Weiss, editors. Software Fundamentals: Collected Papers by David L. Parnas. Addison-Wesley, 2001.

• Watts S. Humphrey. Managing the Software Process. Addison-Wesley, 1989.

• Andrew Hunt and David Thomas. The Pragmatic Programmer: From Journeyman to Master. Addison-Wesley, Boston, MA, 2000.

• IEEE standard for software verification and validation plans. Published by the American National Standards Institute, 1430 Broadway, New York, New York 10018, February 1987. ANSI/IEEE Std 1012-1986.

• Institute of Electrical and Electronics Engineers, Inc., New York, NY, USA. IEEE Recommended Practice for Software Design Descriptions, 1998. IEEE Standard 1016-1998.

• Institute of Electrical and Electronics Engineers, Inc., New York, NY, USA. Software Reviews, 1997. IEEE Standard 1028-1997.

• Institute of Electrical and Electronics Engineers, Inc., New York, NY, USA. Software Productivity Metrics, 1992. IEEE Standard 1045-1992.

• Institute of Electrical and Electronics Engineers, Inc., New York, NY, USA. Software Quality Metrics Methodology, 1998. IEEE Standard 1061-1998.

• Institute of Electrical and Electronics Engineers, Inc., New York, NY, USA. IEEE Recommended Practice for Software Acquisition (includes IEEE 1062a), 1998. IEEE Standard 1062-1998.

• Institute of Electrical and Electronics Engineers, Inc., New York, NY, USA. Developing Software Life Cycle Processes, 1997. IEEE Standard 1074-1997.

• Institute of Electrical and Electronics Engineers, Inc., New York, NY, USA. Software Maintenance, 1998. IEEE Standard 1219-1998.

• Institute of Electrical and Electronics Engineers, Inc., New York, NY, USA. IEEE Guide for Developing System Requirements Specifications (including IEEE 1233a), 1998. IEEE Standard 1233-1998.

• Institute of Electrical and Electronics Engineers, Inc., New York, NY, USA. Information Technology - Software Packages - Quality Requirements and Testing, 1998. IEEE Standard 1465-1998 (ISO/IEC 12119:1998).

• Institute of Electrical and Electronics Engineers, Inc., New York, NY, USA. IEEE Recommended Practice for Architectural Description of Software Incentive Systems, 2000. IEEE Standard 1471-2000.

• Institute of Electrical and Electronics Engineers, Inc., New York, NY, USA. Adoption of PMI Standard- A Guide to the Project Management Body of Knowledge, 1998. IEEE Standard 1490-1998.

• Institute of Electrical and Electronics Engineers, Inc., New York, NY, USA. Information Technology - Software Life Cycle Processes - Reuse Processes, 1999. IEEE Standard 1517-1999.

• Institute of Electrical and Electronics Engineers, Inc., New York, NY, USA. Software Life Cycle Processes-Risk Management, 2001. IEEE Standard 1540-2001.

• Institute of Electrical and Electronics Engineers, Inc., New York, NY, USA. Software Quality Assurance Plans, 1998. IEEE Standard 730-1998.

• Institute of Electrical and Electronics Engineers, Inc., New York, NY, USA. Software Configuration Management Plans, 1998. IEEE Standard 828-1998.

• Institute of Electrical and Electronics Engineers, Inc., New York, NY, USA. Software Test Documentation, 1998. IEEE Standard 829-1998.

• Institute of Electrical and Electronics Engineers, Inc., New York, NY, USA. IEEE Recommended Practice for Software Requirements Specifications, 1998. IEEE Standard 830-1998.

• Institute of Electrical and Electronics Engineers, Inc., New York, NY, USA. Guide to the Software Engineering Body of Knowledge, 2001. Available online http://www.swebok.org.

• Cem Kaner, Jack Falk, and Hung Quoc Nguyen. Testing Computer Software. Wiley, 1999.

• Brian W. Kernighan and Rob Pike. The Practice of Programming. Addison-Wesley, 1999.

• Susan Lammers. Programmers at Work. Microsoft Press, Redmond, Washington, USA, 1989.

• Thomas A. Limoncelli and Christine Hogan. The Practice of System and Network Administration. Addison-Wesley, 2001.

• Pete McBreen. Software Craftmanship: The New Imperative. Addison-Wesley, 2002.

• Steve C McConnell. Code Complete : A Practical Handbook of Software Construction. Microsoft Press, Redmond, WA, 1993.

• Donald A. Norman. The Psychology of Everyday Things. BasicBooks, New York, NY, USA, 1988.

• Donald A. Norman. The Invisible Computer. MIT Press, 1998.

• David L. Oppenheimer, David A. Wagner, and Michele D. Crabb. System Security: A Management Perspective. Short Topics in System Administration. USENIX Association, Berkeley, CA, USA, 1997.

• P. J. Plauger. Programming on Purpose II: Essays on Software People. Prentice-Hall, 1993.

• Roger S. Pressman. Software Engineering: A Practitioner's Approach. McGraw-Hill, London, fifth edition, 2000. European Adaptation. Adapted by Darrel Ince.

• J. Christopher Ramming, editor. USENIX Conference on Domain-Specific Languages, Santa Monica, CA, USA, October 1997. Usenix Association.

• Jef Raskin. The Humane Interface: New Directions for Designing Interactive Systems. Addison-Wesley, 2000.

• Eric Steven Raymond. The Art of Unix Programming. Addison-Wesley, 2003.

• Aviel D. Rubin, Daniel Geer, and Marcus J. Ranum. Web Security Sourcebook. John Wiley & Sons, 1997.

• Charles H. Schmauch. ISO 9000 for Software Developers. ASQC Quality Press, Milwaukee, Wisconsin, USA, 1995.

• Bruce Schneier. Applied Cryptography. Wiley, second edition, 1996.

• Bruce Schneier. Secrets & Lies: Digital Security in a Networked World. Wiley Computer Publishing, 2000.

• Ian Sommerville. Software Engineering. Addison-Wesley, sixth edition, 2001.

• Diomidis Spinellis and V. Guruprasad. Lightweight languages as software engineering tools ( http://www.spinellis.gr/pubs/conf/1997-DSL-Lightweight/html/paper.html). In Ramming [Ramming, 1997], pages 67-76.

• Diomidis Spinellis and Konstantinos Raptis. Component mining: A process and its pattern language ( http://www.spinellis.gr/pubs/jrnl/2000-IST-Components/html/comp.html). Information and Software Technology, 42(9):609-617, June 2000.

• Diomidis Spinellis. Explore, excogitate, exploit: Component mining ( http://www.spinellis.gr/pubs/jrnl/1999-Computer-Components/html/comp.html). IEEE Computer, 32(9):114-116, September 1999.

• Diomidis Spinellis. Reliable software implementation using domain specific languages ( http://www.spinellis.gr/pubs/conf/1999-ESREL-SoftRel/html/dsl.html). In G. I. Schuëller and P. Kafka, editors, Proceedings ESREL '99 --- The Tenth European Conference on Safety and Reliability, pages 627-631, Munich-Garching, Germany, September 1999. ESRA, VDI, TUM, A. A. Balkema.

• Diomidis Spinellis. Fear of coding, and how to reduce it ( http://www.spinellis.gr/pubs/jrnl/2001-05-Computer-Fear-of-Coding/html/foc.html). IEEE Computer, 34(8):98-100, August 2001.

• Diomidis Spinellis. Notable design patterns for domain specific languages ( http://www.spinellis.gr/pubs/jrnl/2000-JSS-DSLPatterns/html/dslpat.html). Journal of Systems and Software, 56(1):91-99, February 2001.

• Diomidis Spinellis. Code Reading: The Open Source Perspective. Effective Software Development Series. Addison-Wesley, Boston, MA, 2003.

• John Viega and Gary McGraw. Building Secure Software: How to Avoid Security Problems the Right Way. Addison-Wesley, 2001.

• Damien Watkins, Mark Hammond, and Brad Abrams. Programming in the .NET Environment. Addison-Wesley, Reading, MA, 2002.

• Joseph Weizenbaum. Computer Power and Human Reason. Pelican books, 1984.

• Edward Yourdon. Death March. Prentice-Hall, 1997.

• Contents

 Last change: Monday, December 15, 2003 8:53 pm
 Unless otherwise expressly stated, all original material on this page created by Diomidis Spinellis is licensed under a Creative Commons Attribution-Share Alike 3.0 Greece License.