Cascade – Better Practices for Effective Delivery of Information Systems in a Multi-project Environment

David Wright asked:

Cascade

Better practices for effective delivery of information systems in a multi-project environment. ©

 

David W. Wright

 

Copyright © 2008 by David Wright

This article is written for people who work at companies that have an IT department to support their non-IT business. In comparison, this article is not written for people who work at IT companies, from Google on down to someone selling excel macros. The main reason is that I have not worked for an IT company and, although I have worked with many in customer-vendor relationships, I do not feel qualified to write about something I do not have experience with. However, even companies selling IT need some other IT to help run their business, so perhaps some of my experiences may overlap with theirs.

Given that, let us consider the vast majority of companies that are making automobiles or serving hamburgers or providing investment advice or any and everything else. In those companies you will find a department/division filled with IT ‘professionals’: Programmers, Project Managers, Analysts, Testers, Implementers, Operators, and loads of other specialists like Database Administrators and Network Engineers. (I use professional in quotes because long-standing professions like lawyers and engineers like to protect the use of the word.)

The name of this group has evolved generally to the ‘Information Technology Department’, or IT for short (no ‘D’ at the end); such past names as “Data Processing” or “Management Information Systems” seem to have faded away. As an acronym, “IT” is used interchangeably to name the department and describe the functional scope addressed by that department.

So, as we speed through the first decade of the 21st century and on into the next, what is the common situation that these IT departments face? It likely includes:

- An installed set of information systems that are used by the business to control and report on its operations: There is always more than just one system, and possibly hundreds, many doing the same work as many others. Some may be decades-old, a few may be recent and using fairly new technologies, with the implementations of the rest spread-out over the history of the department since that first ‘computing machine’ was purchased, with the wide range of development and operating technologies which that implies. (How along ago did you read that your mainframe computer was dead? Did it die? Not likely.) The still favourite term for describing this installed base is legacy systems.

- A large back-log of requested changes to that installed base of systems, overlaid with a long list of problems/bugs in the systems that the business is currently ‘working around’ until they ever get fixed.

- An “IT Strategy” that describes in glowing terms how the above two cases will be remedied by moving to some new method or tool or one enterprise system… if the budget and resources can ever be freed up from fixing and changing the current systems

- A senior management group (or steering committee or review board) that is charged with deciding what IT work is approved and carried out: this group may be formal or informal, and may only be one person in the end, the CEO or someone he/she has delegated it to; since IT costs money, the CFO is a common choice.

      In this structure, the head of the IT department (CIO is the hot title, of course) plays a committee secretary and/or facilitator role, who is supposed to assist the rest of management in making their IT choices. This is a difficult role, as the rest of management has a split-view of IT, that most of it is a waste of time and money, except for the work each one wants for their own department/division.

- A large number of current projects being carried out,  that no one can remember when they started and, even if some target date has been set, no reasonable expectation exists that they will end soon.

- An IT staff that has remained the same size in numbers, or has been reduced, while being charged with doing more work than ever: “Work Smart, Not Hard”, “Do More With Less”. Each person is probably assigned to many of those current projects, juggling the work and trying to determine what they should really be working on.

     The overall morale and effectiveness of this staff depends greatly on the skills of their immediate management and more senior managers who provide some level of inspiration; if not, morale plunges and turnover increases.

Sound familiar? Then you have the same experiences as me, and I have written this article for you.

The premise of this article is that the situation described above can change, but it is usually slow-going; or worse, there can be an immediate change when your department is outsourced. In the meantime, what can be done to be “successful” in your average IT department in your average company?

What has to be done is to deliver on all those change requests, and finish those current projects so you can start new ones (there will always be new ones). At risk of sounding like a psychedelic poster, this means accepting the things you can’t change and changing the things you can.

What can’t you change right now?

- The installed base of legacy systems.

- The backlog of change requests and bugs (even if you do manage to deal with a lot of these things, there will be new ones come along to take their place; that is job security).

- Senior management’s’ conflicting priorities for IT.

What can you change (or at least start to change?)

- The structuring and management of the IT projects.

- Overall management of staff by skills/specialties.

- Allocation of staff to the projects.

What follows in the rest of this article is my prescription for this change, supported by some war stories, lessons learned, and lots of ideas to try. Even an ‘average’ company has some unique aspects, so maybe not everything in this book will work for everyone, but I hope to give you enough ideas that some will work. My own fervent belief is that visible success with IT projects may lead to softening/improvement of the things you can’t change right now.

And why should you believe me?

David Wright is a veteran of over 25 years in the IT trenches. He started as a programmer in the mainframe-dominated 80’s, followed by 20 years as a Business Analyst and Architect, supplemented with stints in project management and testing when limited resources required it.

 Mr. Wright has spent time both in operational areas delivering enhanced and new business systems, and in research and development focusing on information system methodologies and tools. The companies he has served in these capacities have ranged from life insurance to express delivery to equipment leasing & financing. In those companies, he has supported both the operational business and supporting functions like Finance, Human Resources, and even IT administration systems.

Sound good enough? If so, let’s get started.

How will we document this? With Principles.

Like you, I have read my fair share of IT books and articles, and each author has to decide how to best present what they are trying to communicate. One way is to describe the problems/opportunities of interest to the reader, then document the new approach or method that solves the problems; the reverse is to document the new approach/method first and then describe what problems it fixes.

I would like to try a mixture of the two, in the context of a set of. This
is not an original idea, I must admit, as many manifestos and proclamations have been used to introduce new ways of things; the Agile Manifesto is well known (http://agilemanifesto.org/),and I am a particular fan of the Business Rules Manifesto (http://www.businessrulesgroup.org/
brmanifesto.htm). These and others like them lay out  the basic reasons for their existence and the value they provide.

Principles can be very effective; the best I have ever seen were created by Mao Tse-tung for the new Chinese Red Army after his first insurrection failed. His principals for protracted/attrition warfare was summed up in “a sixteen character military guide that even an illiterate peasant could understand…

 Enemy advances, we retreat.

 Enemy halts, we harass.

 Enemy tires, we attack.

 Enemy retreats, we pursue.”

Effective words, against which I hope my own do not completely whither in comparison.

 

So what are these new IT principles I propose?

1. There is always more work to be done than people to do it.

2. Projects change the business, so know the overall business first.

3. Use an overall Architecture to describe the business, before and after projects.

4. Pick the right project(s) for the business.

5. Once a project is started, finish it.

6. Specialize – each member of a team assigned to a project should do what they do best for the length of that project.

7. One Architect/Analyst can generate enough work for two Developers and one Tester, structure your project teams in this ratio.

8. It’s the Deliverable (that matters), not the Task.

9. Leave a record of what you have done, so the project will not miss you if you leave.

10. Models are better than text.

11. Partition large projects into 3 month phases, which is the longest period you can plan for without the chance of significant change to priorities, resources, etc.

12. Within the three month phase, parcel work into two-week periods; analyze for 2 weeks, then design and develop for 2 weeks ( 2 developers ), and then test for 2 weeks. When the first 2 weeks of analysis is done, start the next two weeks of analysis in parallel to the design/development; carry on in cascading 2 week periods until the entire project scope has been addressed.

13. Given many medium to small software Deliverables, use architecture to manage and integrate the Deliverables into a complete system.

I consider these a work in progress so, like most inventors of new principles or practices, I have come up with an overall name to encompass them, for now and as they evolve. It is:

Cascade – Better practices for effective delivery of information systems in a multi-project environment. ©

Let’s look briefly at what each principle means or implies; which will serve as introductions to the remaining chapters that cover the principles in more detail.

1.  There is always more work to be done than people to do it.

Bordering on glibness, this principle summarizes the reality of virtually all organizations and activities, not just Information Systems delivery. It implies that a group within an organization charged with delivery of end results will a have a back-log of work not yet done. The existence of such a back-log is in itself not a problem, it reflects the desire of an organization to solve new problems and actively improve itself; the problem arises when it grows perceptively in size from management’s perspective, and the length of time a change item sits on the back-log increases such that it can be measured in numbers of months or years.

So, we must accept that there will be a backlog; fully eliminating it would mean that the delivery group (like IT) becomes redundant, or that the overall organization has stagnated. What must be done is to embrace the back-log; it is IT’s input material and should be managed as such.

2.   Projects change the business, so know the overall business first.

A never-ending discussion in IT circles is about how much IT staff needs to know about the business that the information systems are supporting. It is high-lighted by every want-ad for an IT job that says previous experience in the employer’s industry is mandatory.

Is detailed industry knowledge  and experience absolutely necessary for an IT job?  No.

Can an IT staffer be effective with absolutely no knowledge of their employer’s industry?   No.

As in many situations like this, the ‘Yes’ answer lies somewhere between the two extremes. Like a pendulum, the level of industry knowledge will vary across this spectrum, based on the specific organization, and by the particular IT role; e.g. Analysts need to know more than Programmers, and it justifiably argued that Testers need to know even more than Analysts.

So, some industry knowledge is required. I suggest from experience, however. that several week’s research on an industry is equivalent to several year’s work experience, as much of a person’s experience is rendered out-of-date by industry changes, or is too specific to the company they worked at. This is truer today than ever, as the ubiquitous Web makes information about almost anything available with a text string and few mouse clicks.

As a result, IT needs to know something about the business going into a project, and the willingness to learn more as a project progresses.

The addendum to this discussion, however, is how much does IT need to know about the specific way their employer conducts business when competing in their industry . Again from experience, I suggest that very little or no such knowledge is needed going into a project. The cliché that “a little knowledge is a dangerous thing” applies here. If IT people know too much about the current business, they may be unconsciously constrained when devising new IT solutions by ‘the way things have always been done here.’  In extreme cases, this can lead to an IT staffer having the delusional belief that they know more about the business than the systems users and their management.

Do not fall victim to this belief. IT is about underlying hardware and infrastructure, and the information systems that run on them. The systems’ users and their management — supported by all the strategies, policies, procedures and rules that define and control the business — will know the specifics of their business better than IT; their jobs depend on it.

This is not to imply that all business users & management are all-knowing, or that all businesses operate without duplications or errors, or that there are not things the business doesn’t know yet. In fact, effective use of IT can address many such issues in the operation of a business, but IT and IT people do this in support of the business; IT does not define the business.

So, know your business in order to support your business.

3.   Use Architecture to describe the business, before and after projects.

“Architecture” is becoming a more widely used term associated with Information Technology. The number of adjectives applied to the term seems endless: “Technical Architecture”, “Systems Architecture”, “Business Systems Architecture”, “Enterprise Architecture”, and so on, so it must be important.

Why do we need Architecture?

Architecture is not an end in itself; Architecture exists because things need to be built.

Architecture is required when building anything that is not simple; in its essence, Architecture identifies all the separate components of an end product and how all the components are related and fit together to comprise the whole of the product.

Architecture is layered to capture and present information about the product to different audiences, from initial/high concept to detailed specification.

Applied to IT, a component assembly
approach is dominating the industry, from the OO approach of software development, to real-time use of defined services, as popularized by SOA. Specialized agent software is starting to assist in finding services and brokering between different services to perform transactions collaboratively.< br/>
For the average company using IT, architecture is needed because it needs focused IT functionality to deliver the highest current value, while trying hard to ensure that the function will work (“integrate”) with the next function that is needed.

It should be emphasized that this is Architecture for Information Systems.; there are methods and approaches being promoted for an overall “Business Architecture”, architecture for the whole business, not just IT.  Originating from IT circles, they often look like an IT/IS architecture, which can be confusing. The Architecture in this book is definitely about Information Technology and Systems.

The good news is that you do not need to invent an IT Architecture method for your company. Many authors and vendors have methods available already. When starting out, I recommend investigating/adopting the architecture that started it all, the “Zachman Framework” as developed and enhanced by John Zachman. He devised a matrix framework that cross-references core information concepts against the levels of abstraction that are used by different audiences and participants in delivery of Information Systems.

The key benefit of the framework is that it illustrates how information concepts can be transformed through the levels to produce operating components of the needed Information Systems.

See the framework at zifa.com

Last two points on Zachman:

- The cross points of the Concept columns and Abstraction rows are called “Cells”; each cell will group the methods or documents (artifacts) that describe the content of the cell. Zachman does not specify what artifacts to use, or what methodologies to use to create the artifacts. The things in each cell on the diagram are just suggestions.

Keep this in mind for Principle #10.

- The Framework looks two-dimensional, but it is actually multi-dimensional when artifacts in one cell are cross-referenced to artifacts in other cells, the most obvious example is What vs. How, i.e., what function creates specific occurrences of data.

 

4. Pick the right project(s) for the business.

At any one time, the IT department of an average company is running multiple projects. How did they get started? How were they even defined as a project that needed to be carried out?

No one may actually know. In more chaotic environments, projects can start as a seed of an idea, pick up momentum and resources if a manager or two can see that they will benefit from the project. At some point, the project will bump into another one, usually because they both want the same IT staff or other resources. Strong managers can often come out of these resource conflicts with what they need for their project, while the other managers suffer from their project going on hold or being cancelled. Otherwise, the conflict is escalated until one common, higher manager has to step in and decide who gets what; and this is often the first time the higher manager has even heard about the projects.

I sat in on a senior management committee meeting where the progress to date of a grand project was presented and a request for a budget increase was made to complete the project. The CEO took all this in, and commented that this was very interesting, but given the sizable amounts of money and time expended so far, why the heck had he never heard of this project before? The next week my manager sent me to see the CIO, who charged me with coming up with a new process for defining, approving, and controlling  IT projects, better known these days as ‘Project Governance’.

So, how do you pick the ‘right’ IT projects? First you have to make the choice explicit, avoiding the random start-ups described above; do encourage any and everyone to suggest possible projects. An active strategic and operational planning process will also tend to drive out new projects as senior and middle management look for IT assistance to reach their assigned goals. All these proposed project ideas then become input to a ‘gating’ process, supported by means of valuing the worth of a project like, but not limited to, cost-benefit analyses.

5.   Once a project is started, finish it.

Even with a good process to pick the right projects to execute, there will be a strong if unrecognized tendency to initiate too many projects at once, or initiate more projects before any already underway have been completed. This goes back to the average senior manager’s split view that most IT spending is a waste, except for their own projects. Given several senior managers in an organization expressing these views, a natural reaction is to have at least one project underway for each manager; if most of the IT efforts are applied to projects for only a few managers, the rest will complain or start looking for other options.

However, trying to run too many projects at once ends up pleasing no one, as no project makes any noticeable enough progress to be seen as a success, so the result is that no one is happy with IT’s performance, even IT.

So, projects have to be run such that at least one is completing within each reporting period; this would be quarterly in most companies. In large IT shops with dozens of projects, it may be a percentage you aim for, 10% to 20% of projects completed per period. Given the common situation of limited IT resources, allocating resources to a set of projects has to be guided by a focus on completion, not competing priorities.

6.   Specialize – each member of a team assigned to a project should do what they do best for the length of that project.

This principle supports #5 above. With limited resources, there is another strong tendency to have IT staff ‘wear multiple hats’ on a project, especially the Business Analyst who is asked to also be the Project Manager and/or Lead Tester. Rather than getting more than you are paying for, you get less as an IT Staffer skilled in one role spends more time performing the other roles than an appropriate specialist would, and is distracted from being productive in their primary role.

At any time, and perhaps more so currently, there will be debates about which is better, a generalist or a specialist. I will agree that a strong, experienced generalist who can cover a wide number of tasks is the best resource you can have. However, these people are rare, and the odds of having even one such person in your average IT department are low . So, make sure that your staff is doing what they do best as much as possible as often as possible.

(This is not to say that people may not learn new roles over time, either switching to different roles or aspiring to be that coveted generalist. However, this is the task of overall resource management, overhead which will take up some portion of a person’s productive time. This is why most project management methods usually assume only about 6 out of each 8 hours can be used on actual projects. This reduced availability speaks even more to the value of specialization at a point in time. It also speaks to the value of matrix management, where Project Managers manage the projects and Resource Managers handle the ‘care and feeding’ of the valuable IT staff assigned to projects.)

7.   One Architect/Analyst can generate enough work for two Developers and one Tester, structure your project teams in this ratio.

This is actually one of those “rules of thumb” that have been borne out over time. (The ratio may vary a bit from case to case, like wh
en the experience levels are different across the roles.) This ratio combines with the specialization of principle #6 to form the strong basis for the Cascade effect covered in the later principles.

8.   It’s the Deliverable (that matters), not the Task.

The final deliverable of an IT project is the Information System ready to be used effectively by the Business. If you can jump from ‘Start’ to this final deliverable in one “Task”, then p
ower to you. Some people can do this; most cannot. This is again where a team of specialists is most effective on an average project.

This means that the project work will be divided into many tasks, sub-tasks, etc. . . . Once assigned a task, it is the goal of a specialist to produce a deliverable/result/artifact that can be used by the next specialist to further the progress of the overall project. Unfortunately, this simple idea has been the starting point for literally hundreds of IS delivery methodologies, many which spend an inordinate amount of content explaining how to do a task, how to amass all the tasks in Phases, and often insisting that its way is mandatory for success.

What this can lead to is an over-emphasis on the ‘how’ of IT project tasks, to the detriment of actually completing them with all due speed. Here we see the task that is 90% done for weeks, or the infamous ‘analysis paralysis’ where a project cannot seem to get past Requirements.  Ends do not justify any means, but Ends must be delivered.

9.   Leave a record of what you have done, so the project will not miss you if you leave.

If change is the only constant, then resources on a project will change. The risk in such change is that a person’s contribution to a project will be lost, and that the new person assigned to the project will have to start over. This is a particular risk in “quick and dirty” projects where an operating result is produced, but no one else can understand the code that was produced. However, if the contribution involves producing quality artifacts as described in #8 above, there is always a point-in-time record of what has been accomplished so far, which can be used by new project resources to continue the project with minimal disruption.

 

10.   Models are better than text.

I would like to think that by this point in time, this principle no longer requires justification. It has been at least a few years since I last saw a dense “SRD” or “SDD” document (SYSTEM REQUIREMENTS DOCUMENT, SYSTEM DESIGN DOCUMENT).  I must offer my respect to the many talented people who labored to produce these documents over the decades; these documents were at least a step up from no Requirements or Design artifacts at all.

Consider what a ‘model’ really is in general; it is a representation of a finished product in a scaled down version; engineers have been creating models of what they are going to build for centuries, for such reasons as testing out problems on a small scale, and for presenting a view of the end result to whomever may be paying for it. 

At this point, though, let us abandon any other aspect of physical engineering as an analogy for Information Systems development. Software is different; while tools and bridges and buildings have been created to either extend or protect the physical capabilities of human beings, Information Systems are created to extend the our mental capabilities, to help our thinking.

Given that, software can still be engineered, but it is a different type of Engineering. Software or Information Engineering has existed as a known concept since the 1970’s, although anyone who thought to call themselves a software engineer usually incurred the wrath, or at least disdain, of the established Engineering Disciplines and Schools. I am not an engineer, would not claim to be one, but my understanding is that traditional engineering is now addressing software within its disciplines. In the future, as software becomes even more critical to our well-being and safety, it may be that those who design and create software will have to be accredited engineers, just like the ones who design and build bridges. I think history shows that quite a few early bridges and other structures were prone to collapse before engineering principles started to prevent it. Software is only a half-century old, so even in the age of internet speed and high change, more time will be needed to bring software in line with other long time products.

In the meantime, it should be a goal for all of us in the IT business to adopt useful aspects of engineering to improve the quality of software, and modeling is a key concept to adopt. It almost frightens me that many still promote programming as an art form, that code can be beautiful or exquisite in some way. Well, even the most obscure art needs an audience to appreciate it, and it can’t just be other artists. Software is a product to be used, not admired, so if anything, programmers of the past 50 years have been more like craftsmen, using individual skills and experience to produce useful ‘objects’ for society to use. The problem is that demand continues to out-strip programmer output (remember the backlog!), so improved, repeatable and transferable methods are needed to transform software development from a craft to a true industry.

11.   Partition large projects into 3 month phases, that is the longest period you can plan for without the chance of significant change to priorities, resources, etc.

I was lucky to learn this early in the 90’s as Project Management was getting a higher profile, accompanied by the increased use of Microsoft Project. Other PM tools were in use, but usually in limited numbers; MS Project, on the other hand, was readily available and budding Project Managers thought they could now plan the whole world for months or years in advance.  What actually happened, however, was constant re-planning as the reality of business change and resource turnover always took their toll. As Napoleon said, “A plan is only good until the battle is joined.” After that, one must adapt to the changes that will always come.

My own experience was on a large project that was broken into a dozen pieces, which were planned separately to a target 18 months away, at which point I was asked to integrate them into one plan while resolving resource conflicts. First thing, we found was that MS Project of that time crashed when you reached around a thousand tasks.

It was about this time that some Project Management consultants were brought on to sort out this mess, and where I first heard the above principle. Yes, you need a plan to get started and to control a project over time. You can even sketch out a plan out over many months to see and communicate the big picture; but, do not commit to any target date over 3 months away; odds are you will miss it. This also means you should do the detailed planning only to the next target date, meaning the full Work Breakdown Structure (WBS) and resource assignment.  (At the other end of the detail level, the IBM consultants also recommended that the shortest task in your WBS should be 2 weeks, no less, otherwise you are micro-managing.)

Once you have a detailed plan for three months, and a high-level plan for the rest of the project, you can add more detail to further target dates as the project progresses, as each interim target is reached or on a rolling month-by-month basis. The level of change in any 3 month period will be manageable, much more than over the whole project.

There is a more recent corollary to the three month principle; the age of the mega-project should be over by now.  Any IS project that takes many months or years to deliver the system is destined to fail. Yes, large systems are still needed, but break them into pieces that can be delivered, ideally about every 3 months; longer than that and you start to slide back to the mega-project approach, while shorter than that will not produce enough of the system to be worth delivering to the business. All business works in 3 month quarter cycles anyway, IT should too.

12.   Within the three month phase, parcel work into two-week periods; analyze for 2 weeks, then design and develop for 2 weeks (two developers), and then test for 2 weeks. When the first
2 weeks of analysis is done, start the next two weeks of analysis in parallel to the design/development; carry on in cascading 2 week periods until the e
ntire project scope has been addressed.

OK, a 64 word-long paragraph is pushing the boundary of a ‘Principle’, but this point is the basic building block of Cascade. Are two week periods too aggressive? I think not, based on experience. I find developers and a tester like to work in such quick bursts, as delivering more results faster makes anyone feel more productive and accomplished, and illustrate quickly what works and doesn’t work. However, small bits delivered quickly need to be integrated into an overall solution, which leads to Principle #13.

13.   Given many medium to small software Deliverables, use Architecture to manage and integrate the Deliverables into a complete system.

This is a more specific statement of Principle #3; in Cascade, an Information System Architecture is used to integrate the two week deliverables, until a complete deliverable (component, sub-system) is assembled.

In parallel, a release schedule is a great approach to support delivery. Gather the usable deliverables into timed releases that go into production together. As per Principle #11, a Release each quarter is recommended. The business receives what they are paying for often enough to be of value, but not often enough such that assimilation of change is so frequent it causes chaos.

How do these sound? I know some people will disagree with some of the principles (very strongly in some cases). Some may be interested or incredulous enough to want more details. My intent is to publish a follow-up set of articles, one for each principle. If you want it all sooner, I offer all of it in book form at lulu.com/content/2088656 .

If anyone would like to discuss these Principles with me directly, visit me at //sites.google.com/site/cascadethebook/ and leave a comment.

 

Create a video blog