MRW

Successful Software Projects in C++

It is a common problem in a lot of software projects, that they exceed costs, don't do what they were intended to do, become much too complex. In my life as software engineer, I worked in small size and large scale projects, some of them were successful in quality, time and cost, some too expensive, but with acceptable quality, some aborted. Now I would like to share part of my experience.

Project Classification

In general there are two kind of projects: small size and large scale. By small size project, I mean a project below about five person years, by large scale project, I mean a project of more than ten person years. Projects between must be classified case by case.

Large Scale Projects

It is much easier to successfully complete a small size project. A large scale project can be handled only, if it is divided into several small sized projects:

  1. Start your large scale project with a small release — or even with a prototype.
    • Time frame of first release is a half, up to one year.
    • No more than three (your best) persons should work on it.
    • The same people that defined the overall architecture are involved in the first release.
    • Implement only the basic features, but plan it to be extensible.
    • Write an automated test suite, where you simulate external input.
    • When finished, test it for performance and stability before you proceed.
    • Show your work to your (potential) customers and get feedback.
  2. Divide large projects or releases into naturally given parts.
    Your internal organisation does not define how the project should be splitted, but your architecture.
    • A part can be an independent executable, a library, a module, a sub project, a layer or even an autonomous class.
      (to be preferred more or less in this order)
    • All parts must have clearly defined, simple and minimal intefaces.
    • No more than two persons should work on the same part (better one).
    • All individual parts must implement and pass automated black box tests, where every (kind of) possible input is given, even illegal input, and the output is checked against the specification.

How to Successfully Finish Projects

The steps for a successful completion of small size and large scale projects include:

Rules for the development process

  1. Define your development process.
  2. You must have (and use) a version control system, such as CVS1), Subversion or ClearCase!
  3. Testing takes the same amount of time as coding!
  4. Do reviews, at least for interfaces and other important classes!
    Use reviews to share know-how, but do not revoew anything.
  5. Maintain a change log file after the first delivery!
  6. All project members must be experienced in the language, tooling and system environment!
    It is an illusion to think that every body is replacable, experience is an important good. In a small C++ project, all programmers must have experience in C++ and object oriented programmming, in a large project every fourth or fifth person can be less experienced, but must be assisted and the code must be reviewed by the experienced collegues.
  7. Clearly define the system specification and interfaces, make sure you understand the customer's needs!
  8. Write your code as generic as possible, as long as it is not too expensive. This is not really a contradiction to:
  9. Know what problems you have to solve and solve exactly that problem, not a generic one.
    If you need a more generic solution later, extend your specific solution, not vice versa. Make things only as flexible, generic, extendable as needed. Otherwise you are in danger to enormously loose time and money.
  10. Don't do what you don't have to do, but make use of existing software, code and libraries!
    Don't reinvent the wheel and keep your implementation simple. E.g. if you write a management software for a telecommunication network element, don't manage the floppy on the computer your software is running! There are operating systems and desktops that do this better!
  11. Keep your documentation small! It must be possible to find every important information. Divide your documentation into the following parts:
    • Describe shortly what your system does (use Doxygen's @mainpage for that).
    • List of all customer requirements. If necessary add one or two images or some use cases. The customer must agree with this paper.
    • Small overviews over interfaces and architectural concepts (one or two pages).
    • A detailed interface specification (well sorted, with index).
    • Detailed browsable source code documentation generated with Doxygen. (If you develop a library, use tag @internal for the details and document the interface from the customer's point of view.)
    • A customer documentation. If you have a GUI, write the documentation in HTML (of course, like all other documentation), add screenshots, and implement an online help that jumps to the right anchor in your documentation (this is available at low cost in Qt).
    • Write a small installation guide. Normally the INSTALL file of the GNU Automake / Autoconf tools is enough.
    • It is often a good idea to include all documentation in the Doxygen generation, using @verbinclude, @htmlinclude, @page, and so on. Use @internal to get a customer documentation without unwanted internals. Use Doxygen to create a customer documentation in PDF, if te customer wants a technical documentation.
    • Save time! Don't draw GUIs with a drawing tool, make your draft with QT-Designer and get a screenshot. Reuse the draft as base for the implementation.
    • Don't use an UML development tool, don't try to generate C++ from UML, otherwise you introducde an unneeded complexity!
    • Reduce grafics to the minimum and prefer text where text is simple and short, but use UML to illustrate difficult behaviour, mainly use the state diagram (use GraphViz in Doxygen)!
  12. Define clean and small interfaces, also between sub systems of your own code! Use design patterns: Facade, signal / slot, …
  13. Divide you program into individual parts, that are independent from each other as much as possible! Identify reusable code and place it in a library, publish the library under GNU LGPL license, if you are allowed to, and profit from the communities feedback.

How to evaluate tools

  1. Be careful when you choose your tool set!
    Always remember: «A fool with a tool is still a fool!» A tool is never a replacement for know-how and experiance.
  2. Don't rely on tools that are not yet developped!
    If e.g. your database vendor promises to implement a certain feature in next release, don't rely on it, because it probably won't be finished until you need it, if ever. Base your developement on existing tools.
  3. Prefere tools that produce human readable text output.
    Never use tools that rely on prorietary binary file formats, or can be sure that you will loose yor data when the tool is no more available, and you will be in absolute dependance, you will be the slave of the tool's developer.
  4. Use standards!
    Don't use tools that save your work in proprietary binary file formats. Perfer e.g. HTML before LaTeX before OpenOffice (XML-Format) before MS-Word (proprietary).
  5. Use simple and transparent tools!
    Don't use tool, if you don't understand, what they do. Especially the systembuilding process must be undesrstood. A tool must not hide things from the user; it must be possible to change everything «by hand» if necessary.
  6. Do not use «one for all solution» tools, that means, for every problem, use the best tool and connect them with some glue (normally some scripts). Therefore you must have tools with transparent interfaces, that integrate well with other tools.
  7. Do your development entirely web based! I.e. write all documentation in HTML, (or better use a project Wiki) and make it accessible through a web browser. Use Doxygen. Don't use OpenOffice, MS-Word or similar for your Documentation.

Software Development Steps and Milestones

  1. If you have a customer or an investor, if you have finished the busines oportunity scanning and market analysis, the implementation starts.
  2. First and most important point: Do a good analysis! For this, talk to your customer, learn his needs, present your solution and collect his feedback. Learn or define the folllowing things:
    • What should the final system be able to do, what are the business cases?
    • Why does the customer need your software?
    • What similar software is on the market? Wouldn't a combination of existing software already fit the customer's need?
    • What are all external interfaces and protocols?
    • How will the user interface (GUI?) look like? (Use a GUI designer for the prototype!)
    • What tools and libraries can ease the developement? Test them!
  3. Write down a small system description, including all features and interfaces and discuss (review) this with your collegues and with your customer.
  4. Based on the analysis, write down a rough architectural concept in a small document.
  5. Create some small overview grafics with a simple grafic tool (Dia, Umbrello, …)2), export it to Postscript and PNG (in the makefile), and reference it from the documentation with Doxygen @img tag.
  6. Write your detail design directly into the C++ header files. C++ headers are an interface and therefore not only part of the implementation, but also part of the documentation. First write central headers, interfaces between subsystems. Document the interface in the header, document implementation details in the implementation file (either Doxygen compatible or as normal C++ comments). The code is part of the documentation, so document it!
  7. After writing the header files, write the implementation.
  8. Immediately test the implementation. Tests are done automated by calling make check and must be repeated before every checkin of sources.
  9. Check for memory leaks and access violations!
  10. First write basic classes, test them, then continue with the classes on the next layer.
  11. If your project is not very small, start with a part of the whole project, with a perliminar drop that implements only a reduced set of functionality, assemble it and test your concept and your implementation under all aspects. If you encounter any small problem, work out a correction. If you enconter any big problem, don't hesitate to redesign and reimplement it now! The later you have to redesign your project, the more expensive it will be. That's why you always need a proof of concept for every part where you lack of previous experience.

Tools for C++ Projects

If you evaluate a tool, always evaluate tools that are not proprietary or bound to one platform. Use tools that are available at least on UNIX and Windows. Better if they are available on MacIntosh too. Don't limit your future!

  1. Develop on UNIX. If you must develop for Windows, get Cygwin or Mingw.
  2. Build your makefiles with GNU Automake and GNU Autoconf.
  3. Build your documentation with Doxygen, use @dot for grafics.
  4. Make your illustrations with a simple tool that can export PostScript and PNG. I recommend dia or umbrello. I don't recommend visio, since it is proprietary and saves your data in an undocumented format. Dont't draw too much nor too large images!
  5. Use GNU gcc compiler, also on Windows. It is a good standard conform compiler and available for nearly target.
  6. If you need a GUI, use Qt, also on Windows. It is free for GPL software on Linux and not so expensive otherwise. It is very powerful and comes with a great designer that stores XML user interface files, that can be loaded at runtime!
  7. Use valgrind or mpatrol for finding memory leaks.
  8. Use STL library for all containers. Never use RogueWave!
  9. Use Boost library for what's missing in STL. It will be partly integrated into the next C++ standard release.
  10. Use log4cxx for tracing. Don't trace too much!
  11. Use cppunit for unit tests.
  12. Use gsoap for communication and re-/ storing parameter files.
  13. Use QpThread for threads if Boost is not enough.
  14. You can also use my C++ library, e.g. if you need a stack trace, to execute UNIX system commands or to handle command line arguments.

Read carefully the documentation of the tools you use. Especially Doxygen and GNU Autoconf /Automake are extremely powerful. Make use of the power!

Programming Hints

Case Tools

Follow the simple rule: «Don't use any case tool!» There is only one acceptable way to generate code: If you never have to touch it, if you can delete and regenerate it. This is given with all good GUI designers, especially with Qt. Case tools are bad, becase they complicate, and don't simplify, the way of programming. They are also no replacement for missing C++ experience. You don't need too much UML diagrams and some of them can even be generated by Doxygen. The simplest and most effective way of programming is to do it in a good editor (that does automatical indentation and coloured formatting, such as XEmacs), and to write the documentation directly into the code using Doxygen.

CORBA

The next important rule is: «Don't use CORBA!» The main reason is because the C++ mapping of CORBA is awful, unusable. You must be very experienced if you don't want to have memory leaks, because there is no object oriented memory management. Also the overall handling is slow and complicated. I recommend the use of SOAP as protocol and gsoap as library.

C++

  • Use automatic variables whenever possible, otherwise use auto pointer or smart pointer, but not classical pointer.
  • Assign resources to classes and free them in the destructor.
  • Don't pass pointer in arguments, always pass by value or by reference. If you must pass a pointer, pass an auto pointer or a smart pointer. You don't need pointer for polymorfism, references work too!
  • Don't use C macros. Use templates and inline methods.
  • Write object oriented code. I.e. don't write «If class is X do this, if class is Y do that, …», neither with if nor with switch constructs.
  • Don't use C libraries, use C++ libraries. Only use C libraries where no C++ library exists, or better write your own C++ wrapper and publish it unter GNU LGPL license. Never use char*, malloc, strcmp and the like.
  • For exception handling rules see mrw::exception
1) Version control systems:
CVS
free but outdated
missing the ability to control history of directories
Subversion
free and state of the art
can control directories too, allows moving of files and links
this is definitely my recommendation
ClearCase
powerful but proprietary, expensive and complicated
needs a kernel module which is only available precompiled for certain kernels, so interobability is definitely a great problem
Subversion is better, but a completely different approach; not recommended
2) Don't use Visio because of its proprietary and undocumented file format.