Legacy Waterfall and V-Models of software development rely on heavy upfront time investments to specify solutions, integrations or features, usually leading to extensive documentation. These approaches are unsuitable for Agile processes due to their inflexibility, high costs, and inability to adapt to real-time changes.
However, it is often assumed that agile teams work without specifications because they prioritize embracing change and delivering the product over documentation. Well, this is not true. Even agile projects, regardless of their size, still require some level of structure, guidance, governance, and documentation to ensure alignment and prevent chaos. This is particularly true for large-scale projects that involve multiple development squads and hundreds of engineers.
This article provides unfiltered insights into the patterns and methodologies that we have implemented at IBM for some of our largest internal transformation projects.
Why structure, guidance, governance ?
Assuming you are creating a complex product with multiple squads involved, the assumption that agile scrum teams are self-sufficient and that any requirement could be assigned to any of them inevitably leads to the following questions:
- Who is responsible for owning the data model, or at least specific pieces or objects of it?
- How can we align on foundational capabilities such as libraries, utility classes, and services that should be reused?
- For what data elements does the product become the system of record (SoR), and for which is it just a consumer or processor?
- How can we align and standardize upstream and downstream data flows?
- Who is leading and standardizing the digital experience, including questions related to design systems, component libraries, shell, page layouts, and design principles?
- How do we inform users about upcoming or deployed changes?
- How does testing and continuous integration/continuous deployment (CI/CD) work, and how can it be standardized?
- How do we ensure the highest standards of security and compliance are maintained?
This is by no means a complete list, but it is enough to identify a pattern:
Many of the questions mentioned above cannot be answered within the scope of a single squad and instead must be addressed at the program level. This means we need a higher level of oversight above the squads to drive standardization, and we need to establish a governance and guidance process that serves the squads. This process is not intended to slow down development activities, but if applied correctly, it can help reduce churn, optimize outputs, and accelerate development. Overall, this process provides us with guardrails to develop a solution.
Another important consideration is whether to specify requirements upfront or document them after implementation. However, specifying requirements before implementation has significant advantages in terms of execution efficiency.
What is Elaboration ?
Elaboration is the process of defining a solution proposal that typically leads to a lightweight specification document that is used by development squads for execution. The Product Owner leads the elaboration by forming the elaboration team, gathering business-level requirements, discovering and transforming potentially existing legacy capabilities, defining personas and user stories, testing procedures, and working out the solution specification documents together with an assigned Architect.
Elaboration concludes with a successful approval by the Architecture Review Board (ARB), which represents the governance layer I referred to.
The Steps of demand management
- At IBM, we typically use Aha! or Jira to collect incoming demand at the top of the funnel. While short-term corporate objectives can significantly impact demand, our prioritization process primarily revolves around corporate strategy, our core transformation principles, and our emphasis on enhancing the user experience.
- An elaboration usually begins with accepted demand, prioritized by the Workstream leader and assigned to a matching Product Owner.
- The Product Owner forms the elaboration team (details below).
- Depending on complexity, a series of meetings are conducted to research, gather requirements, define personas, write user stories, propose architectural decisions, specify the data model and APIs, provide wireframes, and collect a reasonable level of implementation details. This usually happens over a short period of time, with a few weeks considered the maximum, though there could be exceptions.
- The relevant pieces of information are collected and summarized in a specification document, which can vary in size (see below) and is presented to the Architecture Review Board (ARB). The ARB serves as a checkpoint to review the proposal, verify architectural consistency, and overall test whether the elaboration was conducted properly. The elaboration ends successfully with an ARB approval.
- The Product Owner breaks the specification document into Jira stories and tasks, which are assigned to the members of the development squad for execution.
- Development and testing occur in the squads.
- The feature, capability, or integration goes live, and OCM is applied.
The Elaboration Team
The product owner is responsible for forming the elaboration team, which typically includes representatives from the following roles:
- Workstream (or Portfolio) leader: leads the product’s strategy and defines why a feature is needed or must be changed.
- Product owner: owns the feature, capability, or integration and is responsible for gathering the elaboration team and driving the elaboration. The product owner defines what the feature is.
- Architect: ensures the technical fit of the solution proposal on the product/platform scope and in the context of the related upstream and downstream systems. The architect defines how to implement the feature by ensuring architectural cohesiveness, consistency, and standardization. The architect ensures the feature fits into the bigger picture.
- SMEs: people who know the to-be or as-is solution and can guide the team through the relevant details.
- Design and Research: people who define the user experience of various scenarios the solution is expected to cover. This includes user experience research to understand how the users will ideally work and the production of design wireframes.
- OCM: contributes the necessary steps to inform and enable the users before and after the capability goes live.
Efficient specification based on T-shirt size
At IBM, we use a rough T-shirt sizing to differentiate the size of solution proposals in our recent transformation projects:
- Small – low effort, low complexity, usually attached to an existing capability, roughly one sprint (two weeks) of development work. This type of specification is submitted to the ARB as a Jira Epic with no formal structure.
- Medium – midsize effort and complexity, can be a small new feature, a maximum of two sprints (four weeks) of development work. This type of specification should be submitted as an "Architecture Decision Brief" (ADB) document, which should not exceed five pages of content and usually describes the problem to be solved, its solution, and some implementation details.
- Large – high effort and complexity, usually a new capability or integration with more than two sprints of work. This is the most complex specification document we work with, and it can vary in size between 10–40 pages. The most relevant aspect here is to find the right balance between the time invested and "good enough". The specification should create a proper architectural framework for an implementation and provide consistency, architectural cohesiveness, and standardization. At the same time, it should give enough guidance to development teams to implement the capability.
Specification Document Structure
To provide an understanding of the structure of such a specification document, the following is a general outline:
- Introduction and Executive Summary – this section provides an overview of the feature and describes the "Why" on an executive level.
- Terminology – a glossary that includes a description of any abbreviations used.
- Business Requirements, User Stories, Personas, and Scenarios – this chapter represents the "What." It starts at a high-level with a collection of the requirements, dives deeper into concrete user stories, and finally describes the necessary scenarios in the product, including flow diagrams for the personas involved. This is the most relevant section to bridge the original demand with the implementation details and creates a common understanding of what will be delivered.
- Architecture – this and the next three chapters represent the "How" and describe the solution in technical terms, considering the already existing product, platform, and its ecosystem.
- Object Model – this section includes any new objects or changes to existing objects, and addresses security, compliance, and retention questions, such as sharing of records, field visibility, or field history tracking.
- UI Mockups – for each of the user stories mentioned above, a set of UI mockups is created and provided by design. At the delivery time of the specification document, these do not necessarily need to be pixel-perfect, but clearly need to outline how the UI will enable the specified flows and how new pages fit into the Information Architecture of the product.
- Implementation Details – this is where to define how to configure or develop specific aspects of the implementation to support the development squads. The extent of this chapter can greatly vary and really depends on how much time was available for the elaboration and how profound the skill sets in the development squads are.
- Future Considerations and Open Topics – this chapter distinguishes between what is excluded from the specification and may happen at some later point in time.
Summary
We have observed that investing time in upfront feature specification and creating documents does not necessarily conflict with the Agile methodology. The key factor here is to strive for "good enough".
If we view Software Development as a journey that involves several iterative steps under Agile development, the specification can be regarded as a somewhat blurry vision of the destination. It is crucial to have a clear understanding of where we are headed rather than blindly proceeding.
The specification also functions as a contract among the Workstream leader, Product Owner, Architect, Design, OCM, and Development teams regarding what will be delivered. It establishes a "good enough" level of mutual understanding and ensures that everyone is on the same page.
If you find the process of elaboration a bit too expensive, keep in mind that you can also deliver a specification document in iterations. This is a common situation at the ARBs, and it allows the development teams to start working on chunks of the project while it is still being elaborated.
It’s also worth noting that the assigned Architect and Product Owners always have a seat at the ARBs at IBM. This has been very helpful in preparing ARB submissions, and it means that submissions are not a surprise and are already aligned with the broader team. Our success rate of first-time-pass ARB submissions is over 98%, which shows that the ARB is simply a checkpoint and that the elaboration process happened well before. One could say that the ARB is just the catalyst for the elaboration. In very rare cases, an approval is refused, but only when it becomes clear that no proper elaboration was done, the solution is not well thought through, or lacks critical details. This happens only when the proposing Product Owner is not connected to the broader team.
Therefore, the elaboration process with ARB approval as a checkpoint creates the right mindset in the teams to specify a feature "good enough" for an agile execution.
Thank you.
About Thomas Reinecke – in my roles as Chief Architect on several key IBM-internal transformation projects executed during the last 5 years, I’ve had unique opportunities to co-lead and influence a few of the most complex, comprehensive and impactful business transformations at IBM, with transformation of Support, Sales and the Business Partner Ecosystem being mentioned as examples. Specifying capabilities at a reasonable level was a regular hot topic on all of these projects.
