Chapter 4: Socioorganizational Factors and Abandoned Projects

Software development is a social activity that takes place in complex organizational environments with various interacting social entities making up the project teams. Coordination of the work of the different teams as well as communication both within each team and among the different teams is extremely important to ensure the success of the development process. There are two dimensions to the organizational environment in which the work of the project teams takes place. The first dimension is social and deals with the social interactions among the various organizational entities engaged in the development. The second dimension is organizational and deals with the responsibilities and authority of the organizational structure, in particular the management hierarchy involved in the project. The organizational issues, including behavioral and political authority and the influence of management, all come into focus in shaping the project goals and objectives and in guiding the project to successful completion. Hence almost all the influences of a nontechnical nature that are critical in helping determine the project outcome can be subsumed under the socioorganizational-factor category.

In this chapter we will examine the socioorganizational factors that contribute to abandoned projects, emphasizing the critical influence of this factor category on decisions to abandon development projects. The discussion will proceed on two levels. On the first level I will show the mutual influence of the organizational and social dimensions of the systems development effort with empirical data adapted from earlier studies of abandoned projects by Ewusi-Mensah and Przasnyski (1994, 1991). On the second level, analysis of the influence of both the social and organizational issues will be based on a qualitative analysis of reported cases of abandoned projects in the public domain. I will examine abandoned projects from both public and private organizations. There are, however, no significant differences between the projects that end up being abandoned in public versus private organizations. In general, abandoned software projects share basic characteristics irrespective of their organizational origins.

Empirical Analysis of Abandoned Projects

In two studies of abandoned projects, Zbigniew Przasnyski and I examined the role played by social and organizational issues in decisions to abandon software development projects (see Ewusi-Mensah and Przasnyski 1991, 1994). In this analysis I will discuss software projects within the context of an organizational environment in which the information needs of the organization are expected to be satisfied through the successful implementation of the software. Such software is conventionally referred to as information systems (IS)—that is, software systems designed specifically to address some organizational information requirements for operational and decision-making purposes. In the first study (1991, 83), we found that "all types of IS projects are susceptible to being abandoned—not necessarily just high risk, complex or unstructured ones. The major characteristics of the practice are rooted in organizational behavioral/political issues and, to a lesser extent, in economic and technical issues." This study was followed by the second study (1994) of Fortune 500 companies in the United States. This later study attempted to measure how each of the identified factors—organizational, technical, and economic—contributes to project abandonment. The results discussed in this section are broadly adapted from the Fortune 500 study, which had a larger sample size than the 1991 study and also had the data factor analyzed. Interested readers should refer to the two studies for details of the research protocols and other relevant issues.

Profile of Abandoned Projects

This analysis is derived from the Fortune 500 study (Ewusi-Mensah and Przasnyski 1994). The study questionnaire required the respondent to provide detailed information relating to one typical abandoned project (even if the organization may have experienced more than one abandoned project). Table 4.1 provides a summary of the number of software projects abandoned. Less than a third of the respondents had not experienced any abandoned projects, and about another third indicated that five or more projects were abandoned in the period under investigation, namely, 1982–1986. Table 4.2 describes the type of abandonment experienced, with about two-thirds of the organizations experiencing total abandonment (not including the missing responses). The life-cycle stage during which the project was abandoned is given in table 4.3, and tables 4.4 and 4.5 provide a summary of the abandoned-project expenditures and durations respectively.

Source: Ewusi-Mensah and Przasnyski 1994; adapted with permission

Source: Ewusi-Mensah and Przasnyski 1994; adapted with permission

Source: Ewusi-Mensah and Przasnyski 1994; adapted with permission

Source: Ewusi-Mensah and Przasnyski 1994; adapted with permission

Source: Ewusi-Mensah and Przasnyski 1994; adapted with permission

As shown in table 4.3, more than two-thirds of the projects were abandoned at the design and coding/testing stages of the development process. The design stage exhibited the most instances of abandonment (almost two-fifths), showing how critical this stage is to the successful outcome of a project. The least vulnerable stage in the development process is the planning stage, followed by the analysis stage. Projects abandoned at the design and coding/testing stages of development invariably incur heavy expenditures of resources. The costs tend to be higher because of all the time and effort already expended before the decision to terminate the project is finally made, a decision presumably reached because insufficient progress is being made to satisfy the management stakeholder group's expectations. Table 4.4 illustrates the costs incurred for the various abandoned projects; more than 70 percent of the projects incurred less than $2 million in project costs. The remaining 30 percent of the projects incurred costs ranging from $2–$4 million (i.e., 8%) to as much as $20–$30 million (i.e., about 2%). Table 4.5 depicts the estimated project duration; less than a third of the projects lasted a maximum of six months, and another 30 percent or so lasted from a minimum of thirty-one months to more than three years.

The total number of IS and user personnel involved in the projects is given in table 4.6, where the number of people involved in the projects— both IS personnel and end users—ranged from about one to five for 50 percent of the respondents. More than a quarter of the respondents indicated six to ten IS personnel were involved in the abandoned projects versus just over 16 percent of end users. At the other end of the scale, less than 8 percent of the respondents indicated that at the time of abandonment more than twenty-five IS personnel were engaged in the project. The corresponding number of end users who reported working on abandoned projects was just under 15 percent, or twice the number reported for IS personnel. Table 4.7 shows the total number of departments involved, with the incidence of abandonment increasing when two or more departments are involved in the project. Ewusi-Mensah and Przasnyski (1991, 74) noted this trend and suggested that perhaps the increase in the incidence of abandonment may be attributed to the "extra problems in communication," coordination, and management "produced by increasing the number of departmental stakeholders." Table 4.8 shows the number of project managers working on abandoned projects, with over 50 percent reporting two managers, presumably with each manager in charge of a distinct part of the project.

Source: Ewusi-Mensah and Przasnyski 1994; adapted with permission

Source: Ewusi-Mensah and Przasnyski 1994; adapted with permission

Source: Ewusi-Mensah and Przasnyski 1994; adapted with permission

Finally, table 4.9 gives a classification of the abandoned projects based on the type of decision or task the project was intended to accomplish. It also utilizes the perceptions of senior management with respect to the risks/complexity and benefits the project was anticipated to experience, as well as their perceptions of the strategic significance of the project to the organization. The table indicates that about two-fifths of the projects were considered to be of strategic value to the organization; they were urgently needed (60%) and were also considered to be highly beneficial (64%) to the organization. Less than two-fifths of the projects were considered moderately to highly risky, and less than four-fifths were perceived to be moderately to highly complex. The majority of the projects were of a structured or programmable type, dealing with operational control (more than 70%) and transaction processing (about 60%). Only about a tenth to a quarter of the respondents indicated that their projects were of a tactical control or strategic planning nature. However, more projects of the unstructured, less programmable, and more complex variety were reported to be of a tactical control and strategic planning nature. Thus it can be deduced that the projects reported abandoned by the responding organizations covered the entire spectrum of the variety of tasks or decisions the software systems were expected to support in those organizations.

Source: Ewusi-Mensah and Przasnyski 1994; adapted with permission

Organizational Issues in Abandoned Projects The data from the Fortune 500 study (Ewusi-Mensah and Przasnyski 1994) were factor analyzed for evidence of organizational, technical, and economic factors plus any hidden underlying interactions among them. Factor analysis is a statistical procedure used to determine the existence of structure in a large array of responses or measured variables typically obtained through a survey instrument. The factor analysis creates a minimum number of new variables that are linear combinations of the original variables obtained from the information given by the respondents. The new variables contain most, if not all, of the information in the original variables, with each new variable showing the contribution of the factors made by each of the original variables. By using this technique, one is able to show the total variability of the responses in terms of the new variables or factors (Reyment and Jreskog 1993). The details of the factor-analysis data are discussed in Ewusi-Mensah and Przasnyski 1994. In this chapter I will limit the discussion of the factor-analysis results to social and organizational issues pertaining to decisions to abandon the project. Issues dealing with technical and economic factors are deferred to chapters 5 and 6, respectively.

Table 4.10 provides a summary of all twelve factors that emerged from the analysis; six of these factors can be grouped into a class representing socioorganizational issues based on a review of the data, as shown in table 4.11. These six socioorganizational factors can be placed in two categories representing senior management and organizational matters, and end-user-related issues. The first category deals with management commitment, perceptions, changes, and personnel-related issues in connection with the development process—that is, factors F5 and F7. The third factor in this group, F12, pertains to organizationwide matters such as merger/acquisition and its effect on an ongoing project in the merged or acquired organization. In factor F5 the issue was described as "loss of critical personnel and management changes." Indeed, as was pointed out in the study by Ewusi-Mensah and Przasnyski (1994, 194) "Loss of personnel and management changes are experiences that all IS [i.e., software] development projects deal with in varying degrees." The explanation given was that "once a project loses the support of key management there is a high likelihood that, ceteris paribus, the project may eventually be abandoned." This is because the "loss of key management supportive of the project may be associated with the loss of adequate funding and/or loss of interest on the part of top management to even continue with the project."

Source: Ewusi-Mensah and Przasnyski 1994; adapted with permission

A corollary to this factor (i.e., F5) was provided by factor F7—that is, "management commitment and perceptions" with respect to the project. Here also, the obvious explanation is that once there is "a lack of commitment on the part of senior management to provide resources for the project and/or senior management do not perceive the project as important to the organization, then there is a high likelihood that the project may not be successfully completed." The argument one can make is that the loss of senior management support or commitment typically means that "the project is not viewed favorably," thus making it more difficult to obtain the requisite resources to continue with it, which may ultimately sabotage progress on the project (Ewusi-Mensah and Przasnyski 1994, 195).

The third factor in this first group, factor F12, was the "consequence of merger/acquisition by another company." The explanation offered for this factor is that whenever a company is acquired by or merged with another company, a review of current projects is usually initiated, which may result in the cancellation of projects not deemed favorable in the strategic plans of the new organizational setup. This factor is therefore reflective of management practices in organizations in general. The results of the above analysis are confirmed by other observers who describe senior management commitment as being one of the requisite factors essential to project development success. (See, for example, Standish Group, 1995; Johnson 1995; Cole 1995.)

End-user-related issues dealt with the involvement and participation of this stakeholder group in the development process. Factors F6, F8, and F11 speak directly to these issues. Factor F6, termed "end-user acquiescence," was interpreted to mean that the development team chapter 7 I report on a case study of a failed project—CODIS—from the perspective of the end users and on the role they played in the project's development. In general, the social and organizational issues at the core of systems development efforts are embodied in the significant roles that the two stakeholder groups of end users and senior management are expected to play to ensure the success of the project. Their active participation and commitment to the development process is an accepted fact of good software development practice.

chapter 3. To the extent possible based on the available data, we will try to ascertain which of the factors identified in the framework as critical to abandonment occurred and at what stage(s) of the development life cycle this happened.