About Software Maintenance

You can develop software at the level of the software-language itself (coding) or at the tool-level (generating). When you use a tool the complexity of the software-development-process is reduced but the complexity of the coding increases.

When you use a tool the software-language is generated but the coding of the generator is not readable. You need the tool to understand what the software is doing. If you use a tool you have to change the software on the tool-level. If you adapt the generated coding the tool is not usable anymore.

The big problem arises when a software-vendor abandons his tools. This happens often. Software-vendors go broke, are sold and merge or they simply don’t want to maintain the software of the tools.

When a software-vendor has stopped to maintain his tools he also stops to adapt his tools to the underlying hardware, the target-environment of the tools. The target-environment changes because hardware-vendors improve their software, go broke, are sold and merge or simply stop the maintenance of their hardware.

It is very easy to develop new software with a new software-tool. The software-development process becomes very complicated when you want to connect the new software to the old software and the old software is generated by obsolete tool-sets generating obsolete coding that is used by obsolete hardware.

In practice every new software-language, every software-tool and every piece of hardware becomes obsolete. This means that you always have to maintain a huge incomprehensible set of software-components.

Many companies wait until the last moment to replace an obsolete software-environment. At that moment the skills to adapt the software-environment are gone and the only thing that is left for them is to cope with the current situation.

What has happened?

The software-industry is still in its infancy. About fifty years ago the industry was born and it took a very long time to understand what was right and wrong. During that time many companies became highly dependent on immature software.

Why do 70% of the IT-projects fail?

The main approach is to define a new system “out of the blue”. When the project has to face the reality of the installed base things go terribly wrong.

How do you Face Reality?

You are able to face reality when you have made an inventory of all the software components, their relationships (the syntax) and the semantics, the meaning, of the components and their relationships. This sounds simple but it is a very complicated and expensive activity to describe the so called “installed base”. When you have never spent enough time to create a reliable inventory you have to do a lot of work. Most companies postpone this activity until eternity.

They know “everything” about their personnel, their assets and their financial situation but they are “in the blind” about the software that is keeping the books about their personnel, assets and finance. If they don’t know what the software-systems are doing how do they know that their databases are reliable? The answer is simple. They don’t.

What is Software?

A Tool, Software, generates Software but Software generates another type of Software called Machine-code. Software generates Software until the software has reached the lowest layer, bits and bytes. At a certain level the generated software is not understandable.

When the lowest level is reached the bits and bytes activate a processor that is controlled by a clock (the CPU). The processor moves the bits and bytes and sends the end-result (other bits and bytes) to the Outside. At the outside the bits and bytes are transformed into human readable symbols that are presented on a user-interface (Screen, Paper).

Why is Software Changed?

Software has to Change because something Happens in the Outside World, an Event. The Event is propagated until the lowest level, bits and bytes.

So Software (N) = Software (N-1) + Event (N-1) and Software (N, Level (L)) = Software (N-1, Level (L-1)) + Event (N-1, Level (L-1)).

If You write out both sequences the end result is a long sequence of Events.

Software = Event (1) + Event (2) + Event (3) + ………………

Problems arise if you let every Event happen at the moment it happens. If a software vendor creates a new version of its software there is no need to implement this version immediately. Sometimes it is much better to wait until the new release is stable. To prevent chaos you have to Manage the Events that are entering your Software-environment (Change & Release-management).

What is an Event?

An Event is Happening when Something is Set into Motion by Pressure (Tension). Most of the time an Event is associated with a Very Short Duration, an Explosion, but An Event can take much longer. A Movie is an Event. The Great Pyramid in Egypt is an Event.

An Event changes its Intensity all the time. A Concert has a Start and an Apotheosis. An Event is a Wave or a Vibration and every Event has its Unique Pattern.

An Event extends and compresses in Space/Time (A Hurricane). An Event is Born, Expands, Compresses, Dies and Disappears. It follows a Life-Cycle.

Software = Wave (1) + Wave (2) + Wave (3) +

If Events are Born, Mature and Die You can wait until the Event has reached the stage of Maturity. Many Events in the Software World never reach maturity. They die before they have reached this stage.

What is the next stage in the Software Wave?

To predict the Software-Cycle we have to define the Level and the Periodicity of the Wave You want to explore. Every Wave moves through five stages. At a certain point the wave moves to another level (Up or Down) and starts to cycle again.

The simplest way to describe the stages is to use the Four Seasons (Winter, Spring, Summer and Fall) in combination with a Fifth Stage called Indian Summer. This is the stage of the Whole or the Centre. It lies between Summer and Fall.

At this moment the Software Wave is in the Stage of the Centre and is preparing to Jump to another Level. In the Fifth stage the Centre, The Infrastructure, The Foundation, itself is on the Move.

What is the new Centre?

It is very clear that we are moving to a world-wide-software infrastructure. The Foundation of the Infrastructure is the Internet.

What to do?

When a new infrastructure is created the old infrastructure will become obsolete. If the old infrastructure becomes obsolete every layer on top of the infrastructure will become obsolete. This means that the best thing you can do at this moment is “wait and see”. Try to maintain the current situation as much as possible and prepare for the next step.

LINKS

About the Foundation of IT

About the Cycle of the Seasons

About Software and the Cycle of the Seasons

Too Many Insights about IT-Architectures and IT-Strategy

I have been responsible for IT-Architectures and IT-Strategy between 1984 and 1997. From 1997 until now I have reviewed many Architectures and Strategies when I was part of Meta Group (now Gartner).

An IT-Architecture is a System that describes the components of a Software System on an Abstract Level.

An IT-Strategy is a Process that contains Stages. In every Stage a new version of the IT-Architecture is implemented.

A Well-Formed IT-Strategy is able to Adapt the Old Version of the Architecture. When you’re Strategy has failed You have to Start All over Again.

There are two types of Systems. The first type contains Systems. The second type I call a Foundation. It is the level where we think “the Real Things are Happening”. The major problem is to define the Level of the Foundation.

If you look at your Own computer the Foundation lies deeper than you think. It depends on “What You Understand About a Computer“.

We could define the Foundation as the Operating System of Your Computer (most likely a Microsoft Operating System) but below this Foundation other Foundations are in Existence.

At a more abstract level you can see the real Problem. The problem is Containing.

If you use the Containing Metaphor you Never Stop but You Have to Stop Somewhere.

The level where you Stop is the level where you give the responsibility to An-Other. This can be an Organization, A Person or when we dig deep enough even Nature.

When you give the responsibility for something to an-other you have to Trust the Other and the Other has to take Care of You.

The reason why Architectures fail is that they are based on a Foundation that is not stable on the long term.

Suddenly somebody starts to tinker with the Foundation and suddenly everything goes wrong. This happens all the time. The others leave you alone and you have to take care of yourself.

A solution was to create a Foundation that was able to withstand every Change at the Lower level.

This layer was called Middleware. It is situated somewhere between the UP and the DOWN of all the Layers. History has proven that this solution is not helpful.

Everything changes all the time.

I want to give you a Model to understand the complexity of the problem. I use a Horizontal and a Vertical layer, A Matrix. Layered architectures can be mapped on a Matrix, a Cube or a higher dimensional Structure, N-Dimensional Space.

Every component can be described by a point that is connected to N-variables. The links between the components are lines connecting the points.

The first thing we could do is use one dimension to describe the “type” of the component (software, hardware, database, form, etc). If we create a picture of this dimension we have created a System Diagram. There are many types of System Diagrams invented. They were called a Method or a Modeling Language. Every new Method created a “Method War” because the Users always thought that their Method was the Best.

I participated in many activities of the IFIP (International Federation for Information Processing). We tried to find a way to find the “Best Method” to Improve the Practice. It was proven that there was no best method.

Many roads lead to Rome. See “Information Systems Design Methodologies: Improving The Practice“, T.W. Olle, H.G. Sol and A.A. Verrijn-Stuart, (Eds.), North-Holland.

At the end the major Method Wars ended with a Compromise, UML. Compromises are always the worst solution for a problem. UML is a very complicated method.

If we start the Diagram with an Event and we have chosen the right Modeling Language, now called a Programming Language, we are able to “simulate” the System or to “generate” the software. There are many Tools, Modelers, Simulators, Languages and Generators developed.

They also created wars (called Competition) between all kinds of Vendors. In the end many Tools were taken out of the market by the Vendors and the Users got stuck. They were unable to convert the old tools to the new tools or they simply did not take the time to do this. This problem is called the Legacy Problem.

The Market invented something to solve this problem called Reverse Engineering. Reverse Engineering proved to be a failure because the semantics, the meaning, of the software was gone.

When you deconstruct a car and you show an engineer all the parts he knows the parts belonged to a car. When you do this with something nobody ever knew it existed the only engineer that is capable to reconstruct the original is the engineer who constructed it.

When the software is old the original programmer is gone and nobody is able to understand what he was doing. Sometimes the software contains documentation. The programmer has written a Story about the Meaning of the Software. Programmers never took (and take) the time to do this.

I want to get back to N-Dimensional Space. I hope you understand that we when we use enough dimensions we are able to Model Everything.

We are also able to MAP Everything to Everything. Mapping or Converting could solve many problems.

There were Systems on the market that helped you to MAP one Structure to another Structure. An example was Rochade. I implemented Rochade when I was responsible for IT-Architectures and I know Rochade solved many problems with Legacy Systems.

Rochade used something called a Scanner or Parser. A Parser is a “piece of software” that translates a “piece of software” into another “piece of software”. It stored the data of the software (the meta-data) in a “general” format that could be translated to other formats.

When you program in a Software Language the code is translated to another Language. This happens many times until the software reaches the Lowest Level, The Processor or the CPU.

The CPU uses a Cycle to process a very simple language that consists of binary numbers. These numbers are real numbers or operations. The simplest operations are operations on a Set.

The whole concept of the CPU was invented by John von Neumann and is therefore named the Von Neumann Architecture.

The architecture of von Neumann has a big disadvantage called the Von Neumann bottleneck. The CPU is continuously forced to wait.

The Von Neumann Computer is Wasting Time and Energy.

An alternative is the Parallel Architecture. Parallel computing has recently become the dominant paradigm in computer architectures. The main reason is the Rise of the Internet.

The Rise of the Internet started the Fall of Centralized IT-Architectures and Centralized IT-Strategy.

At this moment we need another approach to Manage or Control the sofware-production of a big company.

This approach can be found in the Open Source Movement.

If we use the Matrix Approach we can answer interesting questions.

First I introduce a Rule.

When we increase the amount of Dimensions we are able to make every point and connection between a point Unique. If we do the opposite we are able to make every point and connection The Same.

When people talk about the Reuse of a Component we are looking for a Dimension were some points and their connections are the Same.

I hope you see that it is possible to Reuse Everything and to Reuse Nothing. The Choice is Yours.

This also the Practice in Software. When I discovered that the Year-2000 problem could lead to a disaster I started a research-project with the CWI. The CWI develop a very intelligent parser that could create Software-Maps.

When we studied the maps we saw that some pieces of software came back all the time. These were “citations”. One programmer invented a new construct and others reused the construct all the time. The major difference with the Theory of Reuse was that MANY Parts were the Same.

When you dig deep enough you always find “The Same”.

The CPU is handling binary codes and when you would come from another planet you would not understand why all these zero’s and 1’s are creating a DIVERSITY. They create a DIVERSITY because Something is Interpreting the Sequence. This Something is also a Program. This program uses a Theory about Languages. Most of the time it supposes a Context Free Language. A Context Free Language is a language where the interpretor always moves in one direction. It processes a List.

The Diversity a Computer Produces is based on one long List of Binary patterns. If we could analyze all the possible patterns we could find all the possible software-programs that could be build until Eternity. Because the binary codes can be mapped to the Natural Numbers. We need only one dimension (A line) to classify all the possible software-components in the World.

In 1931 Gödel’s stated the so called incompleteness theorems. He uses the Natural Numbers to prove that a part of our Human Reality cannot be described by a Computer Program.

There is something “left for us” that the Machines cannot take over. This part is related to the Emotions and the Imagination. We cannot Automate them. If we do this we stop Innovation and Commitment.

Now I want to come back to IT-Architectures.

When You start an IT-Architecture Out of Nothing you start with a Small Amount of Dimensions. The world looks very simple to you. When you add detail you have to increase the amount of dimensions. The effect of this is that Everything Changes. New Possibilities arise. If you go on using the Top-Down Approach you will move into a State of Huge Complexity. Always start in the Middle!

At a certain moment You have to move to Reality.This means Programming Software. At that moment You encounter something you never thought of. The Software Legacy!!!!!!!!!!!!

When you increase the Scope of your System and you leave the Boundaries of Your Company (the World of the Internet) the Complexity increases also.At that moment You encounter something you never thought of, Open Source. Millions of Possibilities arise and You don’t Know what to do!

Behind the Software are People. Some of them are creating Small Companies and they are doing things you’re company is also doing but they do it much cheaper and faster.

What to do?

If You Can’t Beat them Join Them.