Piecemeal Growth

Critical success factor for any software development is that everybody understands respects and adheres to Conway's Law: the structure of an organization, and its architecture, are isomorphic. Therefore, make sure the organization is compatible with the product architecture.

The story below, in the form of a pattern language, illustrates a preliminary attempt to use a pattern language to strengthen and tune an organization using feedback and insight. It is essentially a process of repair. Note, perhaps surprisingly, that none of these patterns have fundamental ties to software development. They are applicable to any design activity: any activity where a group of people is building something to solve a problem. They are equally applicable to software services as to building product; to hardware development as to software development.

They are patterns about human nature and human organizations, about the ways that people come together to solve problems. Used patterns are links to the pages describing them (sometimes within the same page).

So, please allow me to take you on a tour that takes you from now to the summer of 2006. Since it is sometimes difficult to reconcile a task or work function with the existing organization of the enterprise, management uses [[Upside Down Matrix Management] to form a Self Selecting Team—from the right roles and people in a way that may cut across the current organizational structure—to successfully deliver a key client proposition, as well as the required IT platform, organization and Excellence Guides for marketing, business, development and operations.

Management supports and facilitates these teams so they can Lead By Example which will motivate other people in the organization to copy the practices to their advantage. It’s success may also radiate to the external market, recognizing it as a great place to work, attracting talent to further embellish your enterprise.

The structure of the current organization artificially reduces the throughput and increases the latency of business processes. Use Coupling Decreases Latency to open communication paths between roles to increase the overall coupling/role ratio, particularly between central process roles. Aim for a Moderate Truck Number (or twinning or pairing) to avoid single point of failures in the organization.

These teams Engage Customers not only to capture Requirements As Stories (using Storyboards), but also during the total time the project runs. These stories are clustered into Minimal Marketable Features, components of intrinsic marketable value, optimizing ROI by creating competitive differentiation, generating revenue, saving costs, projecting a strong brand, and enhancing loyalty.

Early in the project, Size The Schedule provides an estimate of time and effort. Initial estimates are done using T-shirt Costing: Small (days), Medium (weeks), Large (months), eXtra Large (quarters). The degree of financial rigor applied closely aligns business and IT and allows for Incremental Funding, creating cost-effective and agile business, development and production cycles. Management and leaders make sure to Compensate Success on each significant delivery (both client propositions and internal milestones).

Named Stable Bases let projects start from a known position, and continue to known, stable positions. Frameworks As Real Products further strengthens predictability and dependability for business, development and operations to build on. For instance, a global architecture delivers its framework as a whole product, including an SDK, documentation, a reference implementation, a compatibility test suite (functional and non-functional), sample code, and a Community Process.

During development, individuals use a Private World and Private Versioning for speedy and independent development, consolidating into Named Stable Collective Worlds that facilitate Continuous Incremental Integration and Automated Testing, thereby avoiding nasty surprises later in the project and reducing costs across the board while facilitating continuous operation.

From an application architecture point of view, parallelization can be enhanced by Version Controlled Service Interfaces, facilitating cost-effective loosely coupled assembly of appropriately grained system services into valuable client propositions and benefits, addressing their primary needs. At the same time, this Service Oriented Architecture loosely couples to the back office, abstracting from it by using Facades and absorbing any performance or throughput differences.

From the organization point of view, make sure to Decouple Stages. And use Handlers To Connect Systems. The One Week Heartbeat—almost like a system clock—allows the various departments and units to adopt Three Or Four Week Iterations, creating a comfortable rhythm, not only for the entire organization, but also for the market.

These iterations naturally synchronize every quarter (12 + 1 weeks), even leaving a full week for reflection and learning. This quarterly rhythm nicely facilitates seasonal releases (Spring, Summer, Fall, Winter) for eXtra Large releases or changes. They also often comfortably coincide with financial and business planning. Smoother synchronization can be achieved by adopting either a three week or four week across the board. Do whatever feels more natural, and try to avoid unnatural tension by adopting the right rhythm.

Some changes, like migrating existing functionality to a new framework, may even require multiple seasons. Make sure Big Changes Take Less Than One Year.

Management enforces Matching Peer Roles across departments or units. For instance, both the business and the development department each have a dedicated and committed project manager. Lacking parity in this creates undesirable unbalance and may shift responsibilities to the wrong side of the boundary. In all cases, make sure that Responsibilities Engage.

Wherever and whenever necessary, introduce Fire Walls and Gate Keepers to reduce noise and thrashing, and to keep up to speed. Limit the number of roles of each individual to Sixteen Roles Or Less. Furthermore, organize the enterprise so each role has Three To Seven Helpers Per Role in order to avoid over utilization of managers and domain experts.

The schedule allows sufficient time to Build Prototypes, which is an important way of verifying whether our estimates are good, and to mitigate any technological or organizational risks.

During the course of development, if things do not go well, we may have to adjust the schedule, but be sure to Take No Small Slips. This allows us to have Completion Headroom, and lends credibility to a Recommitment Meeting. Using a Work Queue with an Informal Labor Plan helps organize the work.

The work is divided into Development Episodes that coincide with the overall rhythm of the organization. It is important as you do so that the process supports the development work, rather than vice versa. This is Developer Controls Process. Be sure that as a manager, you do not meddle too much and create work for everyone else. Instead, Work Flows Inward.

Make sure Someone Always Makes Progress that the project doesn’t get bogged down or get deadlocked.

If there is a crisis, assign a [{Team Per Task]] to handle it. It may be necessary to Sacrifice One Person to make sure that the rest of the team can progress.

In the case of training new people (which is a distraction), use Day Care.

If things get stuck, unstick them with Interrupt Unjams Blocking. If you do so, Don't Interrupt An Interrupt. The [{Architecture Board]] (sometimes called “Quality Assurance”, or perhaps “Governance”) provides the principles, guidelines and tools that make sure project teams use the right technology, use the right components and use the right processes. It also collects experiences (best practices and solutions to recurring problems in appropriate pattern languages) in order to learn from mistakes so they won’t happen again.

The Architecture Board leads by example by providing Rapid Feedback, clear and comprehensible communication between the business, IT and itself. The architecture board is responsible for both the IT aspects and the organizational aspects. It is a strong center for capturing and sharing knowledge and intellectual property. As such, it fuels Collective Ownership.

The Mercanary Analyst plays a key role since communication is so significantly important for this entity.

Conway’s law

…an Architect and development team are in place. The architecture is fairly well-established.

•••

If the parts of an organization—such as teams, departments, or subdivisions—do not closely reflect the essential parts of the product, or if the relationships between organizations do not reflect the relationships between product parts, then the project will be in trouble.

The system architecture shapes the communication paths in an organization. De facto organization structure shapes formal organization structure. Formal organization structure shapes architecture. Early architectural formulations are only approximations and are unstable. However, there are major rhythms in the architecture that reflect areas of core business competency, and that level of concern is more closely tied to organizational structure than the broader concerns of the whole architectural structure.

Therefore: Make sure the organization is compatible with the product architecture.

At this point in the language, it is more likely that the architecture should drive the organization than vice versa. An organization will have periodic reviews of the architecture, and potentially of project management strategies (see Stand Up Meeting). At each of these meetings (if indeed they are separate) care should be taken to align the structure of the architecture with the structure of the organization, by making piecemeal changes to one or the other.

•••

The organization and product architecture will be aligned. It becomes easier for the pattern Developer Controls Process to succeed.

One reason to let the architecture dominate more than organizational concerns is that the architecture is more often constrained by the problem, and that ties into the core reasons for the existence of the enterprise; see Organization Follows Market, for example.

However, political forces are also powerful and may dominate even over core business needs; however, that usually bodes for serious organizational struggles. The best structure in the long term is one that comes from a threeway alignment between the main structures of the business (domain), the structure of the organization, and the structure of the software.

One approach is to design the major software artifacts around domain analysis considerations and to align the organization with the architecture accordingly; this works best for greenfield projects and where the original design team is small. Another approach is to design the organization around the business needs and let the architecture follow the organization; this is more important in legacy organizations where “expertise tradition” may suggest both organizations and architectures that don’t follow more standard domain analyses.

Of course, all three of these structures must change over time to deal with evolution in the market, technology, and staff, though the fundamental assumptions about relationships between parts are unlikely to change frequently in a successful business.

But even less momentous changes must be dealt with and, more importantly, the project must take opportunities to leverage its growing understanding of the business, of the suitability of specific technologies to support the business, and the organizational and system structures that can support the business.

Much of this pattern language aims at maintaining the project communication essential to the long-term alignment of these structures. Specific patterns like Stand Up Meeting should be viewed as opportunities not only to review the architecture, but to review the organizational structure and business strategies as well.

Gerard Meszaros (formerly of Nortel) notes that you want to bind the organization to the architecture only after the architecture has stabilized. If you bind the organization to the architecture too early, architectural drift will lead to interference between individuals’ domains of control.

On the other hand, Alistair Cockburn points out in Skill Mix that it is sometimes necessary to separate subsystems according to the staff skills you have, since it takes advantage of the skills the team already has. Subsystem By Skill addresses finer team structure with regard to architectural considerations. Deploy Along The Grain or, more specifically, Owner Per Deliverable and Code Ownership are Conway's Law in the small. Use Standards Linking Location to overcome the isolationism of Conway's Law.

The rationale is historical, from Conway’s timeless paper “How do committees invent?”, Conway, Melvin E., Datamation, 14, April, 1968.