Organizing Business Knowledge The Mit Process Handbook [Electronic resources]

2.2 A Framework for Studying Coordination

2.2.1 What Is Coordination?

We all have an intuitive sense of what the word ''coordination''means. When we attend a well-run conference, when we watch a winning basketball team, or when we see a smoothly functioning assembly line we may notice how well coordinated the actions of a group of people seem to be. Often, however, good coordination is nearly invisible, and we sometimes notice coordination most clearly when it is lacking. When we spend hours waiting on an airport runway because the airline can't find a gate for our plane, when the hotel where we thought we had a reservation is fully booked, or when our favorite word processing program stops working in a new version of the operating system, we may become very aware of the effects of poor coordination.

For many purposes, this intuitive meaning is suffcient. However, in trying to characterize a new interdisciplinary area, it is also helpful to have a more precise idea of what we mean by ''coordination.'' Appendix A lists a number of definitions that have been suggested for this term. The diversity of these definitions illustrates the diffculty of defining coordination, and also the variety of possible starting points for studying the concept. For our purposes here, however, it is useful to begin with the following simple definition:

Coordination is managing dependencies among activities.^[1]

This definition is consistent with the simple intuition that, if there is no interdependence, there is nothing to coordinate. It is also consistent with a long history in organization theory of emphasizing the importance of interdependence (e.g., Thompson 1967; Galbraith 1973; Lawrence and Lorsch 1967; Pfeffer 1978; Rockart and Short 1989; Hart and Estrin 1990; Roberts and Gargano 1989).

As the definition suggests, we believe it is helpful to use the word ''coordination''in a fairly inclusive sense. For instance, it is clear that actors performing interdependent activities may have conflicting interests and that what might be called ''political processes''are ways of managing them (e.g., Ciborra 1987; Williamson 1985; Schelling 1960; Kling 1980). Similarly, even though words like ''cooperation,''''collaboration,''and ''competition''each have their own connotations, an important part of each of them involves managing dependencies between activities.^[2]

It should also be clear that coordination, as we have defined it, can occur in many kinds of systems: human, computational, biological, and others. For instance, questions about how people manage dependencies among their activities are central to parts of organization theory, economics, management science, sociology, social psychology, anthropology, linguistics, law, and political science. In computer systems, dependencies between different computational processes must certainly be managed, and, as numerous observers have pointed out, certain kinds of interactions among computational processes resemble interactions among people (e.g., Fox 1981; Hewitt 1986; Huberman 1988a, b; Miller and Drexler 1988; Smith and Davis 1981). To give a sense of the approaches different fields have taken to studying coordination, we summarize in appendix B examples of results about coordination from computer science, organization theory, economics, and biology.

Even though we believe there are more similarities among these different kinds of systems than most people appreciate, there are obviously many differences as well. One of the most important differences is that issues of incentives, motivations, and emotions are usually of much more concern in human systems than in other kinds of systems. In computer programs, for example, the ''incentives''of a program module are usually easy to describe and completely controlled by a programmer. In human systems, on the other hand, the motivations, incentives, and emotions of people are often extremely complex, and understanding them is usually an important part of coordination. Even in human systems, however, analogies with other kinds of systems may help us understand fundamental constraints on coordination and imagine new kinds of organizations that might be especially motivational for people.

2.2.2 Basic Coordination Processes

A primary vehicle for facilitating transfer among these different disciplines is identifying and studying the basic processes involved in coordination: Are there fundamental coordination processes that occur in all coordinated systems? If so, how can we represent and analyze these processes? Is it possible to characterize situations in a way that helps generate and choose appropriate coordination mechanisms for them?

One of the advantages of the definition we have used for coordination is that it suggests a direction for addressing these questions. If coordination is defined as managing dependencies, then further progress should be possible by characterizing different kinds of dependencies and identifying the coordination processes that can be used to manage them.

Table 2.1 suggests the beginnings of such an analysis (for more details, see Malone et al. 1993). For example, one possible kind of dependency between different activities is that they require the same (limited) resources. The table shows that shared resource constraints can be managed by a variety of coordination processes such as 'first-come–first-serve', priority order, budgets, managerial decision, and marketlike bidding. If three job shop workers need to use the same machine, for instance, they could use a simple 'first-come–first-serve'mechanism. Alternatively, they could use a form of budgeting with each worker having pre-assigned time slots, or a manager could explicitly decide what to do whenever two workers wanted to use the machine at the same time. In some cases they might even want to ''bid''for use of the machine and the person willing to pay the most would get it.

The lists of dependencies and coordination processes in table 2.1 are by no means intended to be exhaustive. It is important to note, however, that many specific processes that arise in particular kinds of systems (e.g., design for manufacturability) can be seen as instances of more generic processes (e.g., managing ''usability''constraints between adjacent steps in a process).

In fact we believe that one of the most intriguing possibilities for coordination theory is to identify and systematically analyze a wide variety of dependencies and their associated coordination processes. Such a Handbook of coordination processes could not only facilitate interdisciplinary transfer of knowledge about coordination, it could also provide a guide for analyzing the coordination needs in particular situations and generating alternative ways of fulfilling them (see Malone et al. 1993).

One question that arises immediately is how to categorize these dependencies and coordination processes. Table 2.1 provides a start in this direction. Crowston (1991) suggests a more structured taxonomy based on all the possible relationships between ''tasks''and ''resources.''

To illustrate the possibilities for analyzing coordination processes, we will discuss in the remainder of this section the coordination processes listed in table 2.1 and how they have been analyzed in different disciplines.

Managing Shared Resources Whenever multiple activities share some limited resource (e.g., money, storage space, or an actor's time), a resource allocation process is needed to manage the interdependencies among these activities. Resource allocation is perhaps the most widely studied of all coordination processes. For example, it has received significant attention in economics, organization theory, and computer science.

ECONOMICS Much of economics is devoted to studying resource allocation processes, especially those involving marketlike pricing and bidding mechanisms. As economists have observed, for instance, markets have a number of interesting properties as resource allocation mechanisms (Simon 1981). For one thing, they can be very decentralized: many independent decision makers interacting with each other locally can produce a globally coherent allocation of resources without any centralized controller (e.g., Smith 1776). For another thing, markets have a built-in set of incentives: when all participants in a perfect market try to maximize their own individual benefits, the overall allocation of resources is (in a certain sense) globally ''optimal''(e.g., Debreu 1959).

ORGANIZATION THEORY Organization theory has also paid great attention to resource allocation issues. For instance, control of resources is intimately connected with personal and organizational power: those who control resources have power, and vice versa (e.g., Pfeffer and Salancik 1978). In general, organization theorists emphasize hierarchical resource allocation methods where managers at each level decide how the resources they control will be allocated among the people who report to them (e.g., Burton and Obel 1980a, b). In practice, however, resource allocation in organizations is much more complex than a simple hierarchical model suggests. For instance, managers may try to increase their own power by attracting resources (e.g., employees and money) away from other possible activities (Barnard 1964) or by using their resources in a way that is very suboptimal from the point of view of the whole organization.

How can we choose between different resource allocation methods? Recent work in transaction cost theory addresses part of this question by analyzing the conditions under which a hierarchy is a better way of coordinating multiple actors than a market (e.g., Williamson 1975, 1985). For example, if there are extra costs associated with a market transaction (e.g., extensive legal and accounting work), then the costs of internal transactions within a hierarchical firm may be lower and therefore preferable. A related question involves the conditions under which it is desirable to use marketlike resource allocation mechanisms (such as transfer pricing) within a hierarchical organization (Eccles 1985).

COMPUTER SCIENCE Resource allocation issues also arise in computer systems and much work has been done on these topics (e.g., Cytron 1987; Halstead 1985). For instance, operating systems require algorithms for allocating resources—such as processors and memory—to different processes and for scheduling accesses to input/ output devices, such as disks (e.g., Deitel 1983). As we will see below, there have also already been examples of cross-fertilization of ideas about resource allocation between computer science and other fields. For example, in section 2.3.4, we will see how analogies with human markets have generated novel resource allocation schemes for computer systems.

TASK ASSIGNMENT One very important special case of resource allocation is task assignment, that is, allocating the scarce time of actors to the tasks they will perform. An insight of the approach we are taking here, therefore, is that all the resource allocation methods listed in table 2.1 are potentially applicable for task assignment too.

For instance, in trying to imagine new coordination processes in a human organization, one might consider whether any given situation requiring task assignment could be better managed by managerial decision, by prior assignment according to task type, or by a pricing mechanism. To illustrate the surprising ideas this might lead to, consider Turoff's (1983) suggestion that employees within a large organization should be able to ''bid''for the internal projects on which they wish to work, and that teams could be selected using these bids. There are obviously many factors to consider in determining whether such an arrangement would be desirable in a particular situation, but it is interesting to note that one potential disadvantage—the significantly greater communication required—would be much less important in a world with extensive computer networks.

Managing Producer–Consumer Relationships Another extremely common kind of relationship between activities is a ''producer–consumer''relationship, that is, a situation where one activity produces something that is used by another activity. This relationship clearly occurs in all physical manufacturing processes, for instance, where the output of one step on an assembly line is the input to the next. It also occurs with information whenever one person in an organization uses information from another or when one part of a computer program uses information produced by another.

Producer–consumer relationships often lead to several kinds of dependencies:

Prerequisite constraints. A very common dependency between a ''producer''activity and a ''consumer''activity is that the producer activity must be completed before the consumer activity can begin. When this dependency exists, there must at least be some notification process to indicate to the consumer activity that it can begin. For instance, when an automobile designer delivers a completed drawing of a part to the engineer who will design the manufacturing process for that part, the arrival of the drawing in the engineer's in-box ''notifies''the engineer that her activity can begin.

Managing prerequisite dependencies also often involves explicit sequencing and tracking processes to be sure that producer activities have been completed before their results are needed. For instance, techniques from operations research, such as PERT charts and critical path methods, are often used in human organizations to help schedule large projects with multiple activities and complex prerequisite structures. These and other project tracking systems are also often used by managers to identify activities that are late and then use their authority to ''motivate''the people responsible for the late tasks.

What alternatives can we imagine for managing this dependency? One possibility would be computer-based tracking systems that make it easy for everyone in the project to see status information about all other activities and their dependencies. In this case late tasks would be visible to everyone throughout the project, and ''authoritarian''motivation by managers would become less important.

Sequencing problems arise frequently in computer systems, as well. For instance, one of the key issues in taking advantage of parallel processing computers is determining which activities can be done in parallel and which ones must wait for the completion of others (Arvind and Culler 1986; Holt 1988; Peterson 1977, 1981). Some of these ideas from computer science have also been used to help streamline processes in human organizations by taking advantage of their latent parallelism (e.g., Ellis et al. 1979).

Transfer. When one activity produces something that is used by another activity, the thing produced must be transferred from the ''producer''activity to the ''consumer''activity. Managing this dependency usually involves physical transportation. In this sense physical transportation can be considered a coordination activity, since it involves managing a dependency between a ''producer''activity and a ''consumer''activity. When the thing transferred is information, we usually call the transfer ''communication,''rather than transportation.

In addition to simply transporting things, managing the transfer dependency also often involves storing things being transferred from one activity to another. For instance, one way of managing this aspect of the transfer dependency is to carefully control the timing of both activities so that items are delivered 'just in time'to be used, and no storage is needed. This technique, for example, is becoming increasingly common in manufacturing environments (Schonberger 1982, 1986). A more common approach is maintain an inventory of finished items, ready for the second activity to use, as a buffer between the two activities. Operations researchers, for instance, have developed techniques for determining at what stock levels and by how much to replenish an inventory in order to minimize costs (e.g., the ''economic order quantity''; McClain et al. 1992).

Managing this dependency is also important in certain parts of computer science. For example in parallel processing systems, the rate of execution of processes must sometimes be regulated to ensure that the producer does not overwhelm the consumer, and vice versa (e.g., Arvind et al. 1986). As our framework would suggest, a common approach to this problem is to place a buffer between the two processes and allocate space in the buffer to one process or the other. Network protocols manage similar problems between communicating processes that do not share any memory (Tannenbaum 1981).

Usability. Another, somewhat less obvious, dependency that must often be managed in a producer–consumer relationship is that whatever is produced should be usable by the activity that receives it. One common way of managing this dependency is by standardization, creating uniformly interchangeable outputs in a form that users already expect. This is the approach on assembly lines, for example. Another approach is to ask users what characteristics they want. For instance, in human organizations this might be done by market research techniques such as surveys and focus groups (Kinnear and Taylor 1991).

A third, related, alternative is participatory design, that is, having the users of a product actively participate in its design (Schuler and Namioka 1993). This is a widely advocated approach to designing computer systems, for example, and it is interesting to note that the increasingly common practice of ''concurrent engineering''(Carter and Baker 1991) can also be viewed as a kind of ''participatory design.''In concurrent engineering, people who design a product do not simply hand the design ''over the transom''to those who design its manufacturing process. Instead, they work together concurrently to create designs that can be manufactured more easily.

In computer systems the usability dependency occurs whenever one part of a system must use information produced by another. In general, this dependency is managed by designing various kinds of interchange languages and other standards.

Managing Simultaneity Constraints Another common kind of dependency between activities is that they need to occur at the same time (or cannot occur at the same time). Whenever people schedule meetings, for instance, they must satisfy this constraint.

Another example of this constraint occurs in the design of computer systems in which multiple processes (i.e., instruction streams) can be executed simultaneously. (These systems may have multiple processors or a single processor which is shared between the processes.) In general, the instructions of the different processes can be executed in any order. Permitting this indeterminacy improves the performance of the system (e.g., one process can be executed while another waits for data to be input) but can cause problems when the processes must share data or resources. System designers must therefore provide mechanisms that restrict the possible orderings of the instructions by synchronizing the processes (i.e., ensuring that particular instructions from different streams are executed at the same time; Dubois et al. 1988).

Synchronization primitives can be used to control sharing of data between a producer and consumer process to ensure that all data is used exactly once (the producer–consumer problem) or to prevent simultaneous writes to a shared data item (the mutual exclusion problem). For example, if two processes simultaneously read and then update the same data (e.g., adding a deposit to an account balance), one process might overwrite the value stored by the other.

One example of interdisciplinary transfer involving this concept is the work of Singh and colleagues in using computer science concepts about synchronized interactions to model process in human organizations (Singh and Rein 1992).

Managing Task–Subtask Dependencies

TOP-DOWN GOAL DECOMPOSITION A common kind of dependency among activities is that a group of activities are all ''subtasks''for achieving some overall goal. As we discuss in more detail below, there is a sense in which some overall evaluation criteria or ''goals''are necessarily implied by the definition of coordination. The most commonly analyzed case of managing this dependency occurs when an individual or group decides to pursue a goal, and then decomposes this goal into activities (or subgoals) that together will achieve the original goal. In this case we call the process of choosing the goal selection, and the process of choosing the activities goal decomposition.

For example, the strategic planning process in human organizations is often viewed as involving this kind of goal selection and goal decomposition process. Furthermore an important role for all managers in a traditionally conceived hierarchy is to decompose the goals they are given into tasks that they can, in turn, delegate to people who work for them. There are, in general, many ways a given goal can be broken into pieces, and a long-standing topic in organization theory involves analyzing different possible decompositions such as by function, by product, by customer, and by geographical region (Mintzberg 1979). Some of these different goal decompositions for human organizations are analogous to ways computer systems can be structured (e.g., Malone and Smith 1988).

In computer systems we usually think of the goals as being predetermined, but an important problem involves how to break these goals into activities that can be performed separately. In a sense, for example, the essence of all computer programming is decomposing goals into elementary activities. For instance, programming techniques such as subroutine calls, modular programming, object-oriented programming, and so forth can all be thought of as techniques for structuring the process of goal decomposition (Liskov and Guttag 1986). In these cases the goal decomposition is performed by a human programmer. Another example of goal decomposition in computer systems is provided by work on planning in artificial intelligence (e.g., Chapman 1987; Fikes and Nilsson 1971; Allen et al. 1990). In this case goals are decomposed by a planning program into a sequence of elementary activities, based on knowledge of the elementary activities available, their prerequisites, and their effects.

In some cases techniques for goal decomposition used in computer systems may suggest new ways of structuring human organizations. For example, Moses (1990) suggests that human organizations might sometimes be better off not as strict hierarchies but as multilayered structures in which any actor at one level could direct the activities of any actor at the next level down. This multilayered structure is analogous to successive layers of languages or ''virtual machines''in a computer system (see Malone 1990).

Bottom-up Goal Identification Even though the most commonly analyzed cases of coordination involve a sequential process of goal selection and then goal decomposition, the steps do not necessarily happen in this order. Another possibility, for instance, is that several actors realize that the things they are already doing (with small additions) could work together to achieve a new goal. For example, the creation of a new interdisciplinary research group may have this character. In human systems this bottom-up process of goal selection can often engender more commitment from the actors involved than a top-down assignment of responsibility.

Managing Other Dependencies As noted above, the dependencies discussed so far are only a suggestive list of common dependencies. We believe there are many more dependencies to be identified and analyzed. For instance, when two divisions of a company both deal with the same customer, there is a shared reputation dependency between their activities: what one division does affects the customer's perception of the company as a whole, including the other division. As another example, when several people in the same offce want to buy a new rug, a key problem is not how to allocate the rug, but what color or other characteristics it should have. We might call this, therefore, a shared characteristics dependency.

More generally, there are many types of dependencies between objects in the world that are managed by coordination processes. For instance, an important part of managing the design of complex manufactured products involves managing the dependencies between different subcomponents. At first glance our definition of coordination (as managing dependencies between activities) might appear to omit dependencies between objects that are not activities. We believe, however, that this focus has the advantage of greatly simplifying the analysis of a coordinated situation. In fact it appears that all dependencies that require coordination can be treated this way. For example, dependencies between components matter because they, explicitly or implicitly, affect the performance of some activities (e.g., designing or redesigning the components), and they can therefore be viewed as a source of dependencies between those activities.

In general, as these examples illustrate, there may be many ways of describing different dependencies, coordination processes, and their relationships to each other (e.g., Crowston 1991). We believe that there are many opportunities for further work along these lines.

Summary of Basic Coordination Processes Table 2.2 loosely summarizes our discussion so far by listing examples of how common coordination processes have been analyzed in different disciplines. The key point of this table, and indeed of much of our discussion, is that the concepts of coordination theory can help identify similarities among concepts and results in different disciplines. These similarities, in turn, suggest how ideas can be transported back and forth across disciplinary boundaries and where opportunities exist to develop even deeper analyses.

2.2.3 Example: Analyzing the Task Assignment Process

So far, the examples we have described have mostly involved a single field or analogies that have been transported from one discipline to another. To illustrate the possibilities for developing abstract theories of coordination that can apply simultaneously to many different kinds of systems, let us consider the task assignment process as analyzed by Malone and Smith (Malone 1987; Malone and Smith 1988; see also related work by Baligh and Richartz 1967; Burton and Obel 1980a). As we have described in more detail elsewhere (Malone 1992), these analyses illustrate the kind of interdisciplinary interaction that our search for coordination theory encourages: the models grew originally out of designing distributed computer systems, they drew upon results from operations research, and they led eventually to new insights about the evolution of human organizations.

A Generic Task Assignment Problem Consider the following task assignment problem: A system is producing a set of ''products,''each of which requires a set of ''tasks''to be performed. The tasks are of various types, and each type of task can only be performed by ''server''actors specialized for that kind of task. Furthermore the specific tasks to be performed cannot be predicted in advance; they only become known during the course of the process and then only to actors we will call ''clients.''This description of the task assignment problem is certainly not universally applicable, but it is an abstract description that can be applied to many common task assignment situations. For instance, the tasks might be (1) designing, manufacturing, and marketing different kinds of automobiles, or (2) processing steps in different jobs on a computer network.

Possible Coordination Mechanisms One (highly centralized) possibility for solving this task assignment problem is for all the clients and servers to send all their information to a central decision maker who decides which servers will perform which tasks and then notifies them accordingly. Another (highly decentralized) possibility is suggested by the competitive bidding scheme for computer networks formalized by Smith and Davis (1981). In this scheme a client first broadcasts an announcement message to all potential servers. This message includes a description of the activity to be performed and the qualifications required. The potential servers then use this information to decide whether to submit a bid on the action. If they decide to bid, their bid message includes a description of their qualifications and their availability for performing the action. The client uses these bid messages to decide which server should perform the activity and then sends an award message to notify the server that is selected.

Malone and Smith (Malone 1987; Malone and Smith 1988) analyzed several alternative coordination mechanisms like these, each of which is analogous to a mechanism used in human organizations. In particular, they developed formal models to represent various forms of markets (centralized and decentralized) and various forms of hierarchies (based on products or functions). Then they used techniques from queueing theory and probability theory to analyze trade-offs among these structures in terms of production costs, coordination costs, and vulnerability costs. For instance, they showed that the centralized schemes had lower coordination costs but were more vulnerable to processor failures. Decentralized markets, on the other hand, were much less vulnerable to processor failures but had high coordination costs. And decentralized hierarchies (''product hierarchies'') had low coordination costs, but they had unused processor capacity that led to high production costs.

Applying These Models to Various Kinds of Systems Even though these models omit many important aspects of human organizations and computer systems, they help illuminate a surprisingly wide range of phenomena. For instance, as Malone and Smith (1988) show, the models are consistent with a number of previous theories about human organizational design (e.g., March and Simon 1958; Galbraith 1973; Williamson 1985) and with major historical changes in the organizational forms of both human organizations (Chandler 1962, 1977) and computer systems. These models also help analyze design alternatives for distributed scheduling mechanisms in computer systems, and they suggest ways of analyzing the structural changes associated with introducing new information technology into organizations (section 2.3.2; Crowston et al. 1987; Malone and Smith 1988).

2.2.4 Other Processes Needed for Coordination

In addition to the processes described above for managing specific dependencies, two other processes deserve specific attention: group decision-making and communication. It is sometimes possible to analyze these processes as ways of managing specific dependencies. For instance, communication can be viewed as a way of managing producer–consumer relationships for information. However, because of the importance of these two processes in almost all instances of coordination, we describe them separately here.

Group Decision-Making Many coordination processes require making decisions that affect the activities of a group. For instance, in sharing resources, a group must somehow ''decide''how to allocate the resources; in managing task–subtask dependencies, a group must ''decide''how to segment tasks. In all these cases the alternative ways of making group decisions give rise to alternative coordination processes. For example, any group decision can, in principle, be made by authority (e.g., a ''manager''decides), by voting, or by consensus (resulting from negotiation).

Because of the importance of group decision-making in coordination, answers to questions about group decision making (e.g., Simon 1976; Arrow 1951) will be important for developing coordination theory. For instance, what are the decision-making biases in groups (e.g., Janis and Mann 1977) as opposed to individuals (Kahneman and Tversky 1973)? How do computer-based group decision-making tools affect these processes (e.g., Kraemer and King 1988; Dennis et al. 1988; Kiesler et al. 1984)? Can we determine optimal ways of allocating tasks and sharing information for making group decisions (Miao et al. 1992)? How do (or should) decision-making processes change in situations where both rapid response and high reliability are required (Roberts et al. 1994).

Communication As with decision-making, there is a already a great deal of theory about communication, both from a technical point of view (e.g., Shannon and Weaver 1949) and from an organizational point of view (e.g., Allen 1977; Rogers and Agarwala-Rogers 1976; Weick 1969). One obvious way of generating new coordination processes, for example, is by considering alternative forms of communication (synchronous vs. asynchronous, paper vs. electronic) for all the places in a process where information needs to be transferred.

A coordination framework also highlights new aspects of these problems. For example, when we view communication as a way of managing producer–consumer relationships for information, we may be concerned about how to make the information ''usable.''How, for instance, can actors establish common languages that allow them to communicate in the first place? This question of developing standards for communication is of crucial concern in designing computer networks in general (Dertouzos 1991) and cooperative work tools in particular (e.g., Lee and Malone 1990). The process by which standards are developed is also of concern to economists, philosophers, and others (e.g., Farrell and Saloner 1985; Hirsch 1987).

A related set of questions arises when we are concerned about how a group of actors can come to have ''common knowledge,''that is, they all know something, and they also all know that they all know it. There is a growing literature about this and related questions in fields as diverse as computer science, economics, and linguistics (Halpern 1987; Aumann 1976; Milgrom 1981; Gray 1978; Cohen and Levesque 1991; Shoham 1993).

^[1]This definition was particularly influenced by Rockart and Short (1989) and Curtis (1989). The importance of coordination in this very general sense was perhaps first recognized by Holt (1980, 1983).

^[2]These terms also, of course, have broader meanings. For instance, cooperation usually implies shared goals among different actors, competition usually implies that one actor's gains are another's losses, and collaboration often connotes peers working together on an intellectual endeavor. However, it is sometimes useful to consider all these terms as describing different approaches to managing dependencies among activities, that is, as different forms of coordination.