artificial-intelligence Archives - Page 2 of 2

An IELTS Modernization Case Study

1. Background and Challenge

In 2023, the IELTS modernization programme set out to accelerate delivery and improve quality for a complex, enterprise‑scale IT landscape. The organization adopted SAFe and operated with multiple PODs. There was strong architectural governance and substantial investment in tooling and QA. Yet, the volume of business functionality delivered remained below expectations.

Delivery suffered from long lead times, frequent rework, and low predictability, especially when changes were pushed late in the process. Despite “doing agile” from a process perspective, there was no shared, quantified view of system performance. There was also no coherent operating model aligned with value streams. This made focused improvement difficult.

The AO‑method (Agile Organization method) was selected to:

Diagnose the current operating system using a systemic, quantitative lens.
Design a value‑driven organizational model and transformation backlog.
Validate the new model via controlled experiments with selected PODs.

2. AO‑Method Assessment – AO scorecard baseline

The first AO assessment, conducted in early May 2023, evaluated five dimensions: Deliver, Demand, Capacity, Capability, and Organization.

On a 0–10 scale, IELTS scored:

Deliver: 0
Demand: 3
Capacity: 1
Capability: 1
Organisation: 0

The AO goal state for the coming quarter was defined. It includes Deliver 5, Demand 10, Capacity 10, Capability 10, and Organization 10. This makes the performance gap explicit for both leadership and teams.

Key findings by AO dimension

Deliver: Releases were irregular. They were often tied to big‑bang events. Most PODs delivered into sandbox or test environments rather than production‑like systems. CICD existed in parts of the landscape, but organizational conditions (dependencies, approvals, test environments) prevented continuous delivery.
Demand: Multiple product and feature backlogs co‑existed. They mixed functional and technical items. Business value was not consistently used as the primary prioritization criterion. This led to last‑minute decisions, frequent backlog push into sprints, and weak alignment of team backlogs with programme‑level roadmaps.
Capacity: Teams were not stable. People were frequently disturbed during sprints. Scrum Masters struggled to protect teams from organizational noise. The AO assessment showed that teams were far from the “pizza team” ideal with clear boundaries and stable allocation.
Capability: Developers were largely specialized by component or technology. They did not own work end‑to‑end. This limited self‑commitment and slowed flow. User stories were often too coarse. They lacked acceptance criteria and Definition of Done (DoD). Estimation sessions turned into capacity negotiations.
Organization: Architecture, QA, and support functions were organized as central service providers. They had high approval power. This created long dependency chains and blocked flow. Value streams were not structurally reflected in the organization. PODs often supported other teams instead of owning outcomes.

Kanban and Lean lens

Beyond the scorecard, AO applied a Kanban and Lean waste lens:

Work was not fully visualized end‑to‑end, policies (e.g. DoD, capacity allocation, risk rules) were implicit, and WIP limits were absent or local.
Feedback loops around quality, financial, and flow metrics (CFDs, control charts) were sporadic. Improvement routines (True North, target conditions, PDCA) were weak or missing.
The 8 wastes appeared clearly. Transportation and waiting were evident due to big‑bang releases and late testing. Overprocessing occurred through repeated test cycles. Underused talent was a result of siloed experts and central approvals.

3. Systemic Diagnosis: “Not a Team Problem”

The AO‑method reframed the situation from “teams not performing” to “system not designed for flow and learning”.

Technology‑driven model

The existing working model was heavily technology‑driven:

Architecture and QA were in the lead; development PODs essentially executed their decisions and handled defect backlogs.
Several teams (Hydra, Inception, Transformers, Decepticons) delivered mostly non‑functional or support work, indicating that development teams were not end‑to‑end.
PI planning did not consistently include all relevant teams and functions, which delayed architectural decisions and increased misalignment risk.

The programme behaved like a large, monthly Scrum with strong central control. It did not operate as a network of autonomous, value‑oriented PODs.

Events and ARTIFACTS symptoms

AO uncovered characteristic anti‑patterns around agile ceremonies and artifacts:

Sprint Planning often mixed functional and technical decomposition. The stories were not INVEST. The DoD was not used as the central contract. Developers tended to be told what to do rather than committing to clear, negotiable backlog items.
Sprint Reviews focused on code demos instead of working functionality and UAT, which weakened empirical control of product value.
Retrospectives were high‑level and management‑oriented, producing few concrete experiments for the PODs’ way of working.
Release backlogs and cross‑team dependencies were not clearly visible. Jira was not yet used as a single source of truth across the programme and teams.

From an AO perspective, these symptoms pointed to an operating system incapable of supporting continuous delivery, learning, and local decision‑making.

4. AO‑Based Intervention Design

The next step was to design interventions directly mapped to the AO dimensions and scorecard gaps.

AO scorecard‑driven goals

The AO scorecard was used to define explicit quarterly targets:

Deliver: move from big‑bang/quarterly releases to “deliver every sprint”, leveraging existing CICD capabilities, and progressively move towards DevOps “on‑the‑go”.
Demand: converge onto a functional product backlog, prioritized by business value, with clear PI‑level objectives and roadmaps decomposed into MVPs.
Capacity: stabilize PODs. Reduce disturbances. Strengthen Scrum Masters in their protective and coaching role. This will help move towards true pizza teams.
Capability: improve story slicing, adopt DoD at all levels (story, sprint, release, PI), and build more cross‑functional, end‑to‑end PODs.
Organization: realign the PI organization around value streams. Introduce an Executive Action Team (EAT) and Scrum‑of‑Scrums rhythm. Clarify the role of architecture and testing as enabling partners.

Roll‑out model and AO backlogs

The AO‑method proposed a staged rollout model, “deliver early and often value that matters”, supported by four explicit backlogs.

Vision & Strategy Backlog
- Clarify IELTS vision and PI objectives from a business perspective.
- Define a business plan and a technical plan for PI14, including a PI‑level DoD.
Execution Backlog
- Implement Scrum “by the book” in selected PODs (clear Sprint Goals, ready stories, DoD‑based commitments, protected sprints).
- Ensure usage of Jira, Confluence and Miro as integrated tools supporting transparency and collaboration.
Architecture Backlog
- Shift architecture from lead/approval to consult/enabling, providing clear architectural DoD, platforms, simulators, and stable environments (including pre‑production).
- Reduce reliance on late testing by supporting test‑driven and shift‑left practices in PODs.
Organization Backlog
- Redesign the PI organization around value streams, with architecture and testing each having a dedicated POD and Product Owner.
- Establish governance via EAT and structured PI planning that covers the entire flow from demand to production.

To support this, AO recommended engaging senior agile coaches at both the programme and team levels. They suggested co‑creating a transformation backlog owned by Product Owners, Scrum Masters, and programme leadership.

5. AO in Practice: Panda and Incredibles POC (Swarm)

To demonstrate the viability of the AO‑aligned working model, two PODs—Panda and Incredibles—were selected for an experiment.

Experiment design

The AO‑method prescribed:

Sprint Planning: Stories had to be ready, including description, acceptance criteria, and team‑defined DoD. Non‑ready items were pushed back to the Product Owner. The Sprint Goal was defined collaboratively, and developers committed only to clear, negotiable work.
Daily Scrum: each team member used Jira to show current tasks. They planned the day and highlighted blockers. The team did not accept scope changes except after story slicing.
Sprint Review: focused on working software, inviting stakeholders to inspect and adapt outcomes; code demos were not considered sufficient.
Sprint Retrospective: emphasized concrete changes to the way of working, using Sprint Review and sprint experience as inputs.

Measured impact via AO scorecard

Using the AO scorecard, the experiment showed a significant performance improvement for Panda and Incredibles:

In October 2023, they achieved Deliver 8. They also reached Demand 7 and Capacity 10. Their Capability was 9 and Organization was 0. (Score Max 10 for each dimension).
Compared to the initial May baseline (0–3–1–1–0), this demonstrated substantial improvement, particularly in Deliver, Capacity and Capability.
Other teams, still operating in the old model, lagged behind and continued to suffer from organizational and demand‑side issues.

Sprint 5 for Panda and Incredibles showed increased productivity. This happened when PODs were not disturbed by scope changes. They also had a clear sense of purpose. This provided evidence that AO‑compatible conditions—end‑to‑end scope, DoD‑based commitment, protected sprints—could significantly improve delivery without changing the underlying technology stack.

6. Constraints and ORGANIZATIONAL Response

Despite the encouraging POC results, several AO‑critical changes were not adopted at scale.

PI planning and governance

The PI14 planning event was a missed opportunity:

Many AO recommendations (end‑to‑end PI planning, explicit PI DoD, organization updates to reduce dependencies) were not implemented.
The event resembled a management meeting rather than a true collective planning session focused on reciprocal commitment.
Dependencies and environmental issues raised by PODs remained unresolved, and scope changes continued to be pushed into teams.

From an AO perspective, this indicated that the operating system at leadership and governance level largely remained unchanged. This situation limited the impact of team-level improvements.

Structural risks and recommendation

Given the lack of systemic adoption, the AO assessment concluded that:

Confidence in achieving PI14 goals was below 5/10. Delivery would be possible only at the cost of scope cuts and quality compromises.
Without organizational commitment to the AO‑aligned model, further steps toward an agile organization would be constrained. This commitment is especially crucial for architecture, testing, and PI governance.

The final recommendation was to consider repositioning the supplier teams as a genuine service provider. This includes defining clear performance and quality metrics. These actions should be taken if the client organization did not commit to the AO‑based organizational model. This would allow measurement and transparency even within a suboptimal system.

7. Lessons Learned for AO‑Method

The IELTS case yields several key lessons for practitioners of the AO‑method:

Quantitative AO scorecards create a shared reality.
AO scored Deliver, Demand, Capacity, Capability and Organization over time and across teams. This scoring made systemic issues visible. It transformed subjective complaints into a common fact base for leadership and teams.
End‑to‑end PODs are non‑negotiable.
PODs act as support teams in a technology‑driven structure. As long as this continues, value flow remains fragile. It also remains dependent on central functions. The Panda and Incredibles experiment shows that true end‑to‑end PODs, empowered with DoD and clear scope, can quickly improve performance.
PI planning and governance are part of the operating system.
Without AO‑compatible PI planning (vision, PI DoD, dependency management, reciprocal commitment), team‑level improvements remain local optimizations.
AO‑style experiments are powerful change levers.
Carefully designed experiments with selected PODs provide evidence that the target model works. This approach reduces perceived risk. It also gives leadership a concrete reference.
Organizational will is the ultimate constraint.
AO can diagnose and propose a coherent model. It can demonstrate its effectiveness locally. However, sustainable change requires leadership to adopt the new operating system. It’s not just about team‑level practices.

Twelve months after NorthRiver Bank acquired BrightPay, a fast‑growing fintech, the celebration banners were gone. Internally, people had started calling it “the merger that never landed.”

On paper, the deal made sense. NorthRiver brought capital, licenses, and a broad customer base. BrightPay brought a modern payments platform. They also provided a product team that could ship in weeks instead of years. The investor deck promised “accelerated digital growth” and “seamless omni‑channel experiences.”

Reality was different. Two parallel hierarchies kept running in competition. BrightPay’s engineers were still pushing releases from their own pipeline. At the same time, the bank’s IT insisted every change go through legacy governance. Journeys that were supposed to be “integrated” now had even more handovers. There were relationship managers at NorthRiver, risk at headquarters, product owners at BrightPay, plus at least three steering committees. No one could say who actually owned “digital SME Onboarding.”

Customers felt it first. Time to open a digital business account crept from five days to twelve. Contact‑center complaints about “stuck applications” doubled. Two senior BrightPay product leads left within six months, taking key knowledge with them.

Everyone had an explanation. “Tech is the blocker.” “Compliance is too rigid.” “They don’t understand our culture.” What they did not have was a shared way to see the system. They lacked focus on a few critical flows. No one had clear authority to fix them.

That is when the AO team was called in. They received a brief that sounded simple. Yet, it seemed impossible. “We have to show real integration results in the next two quarters. We must do this without breaking the bank or the fintech.”

1. AO diagnosis: seeing the stuck system

The AO work started with a fast, visual diagnostic rather than another slide‑deck TOM.

Mapped three priority value streams:
- Digital SME Onboarding
- Consumer current‑account opening
- Instant payments incidents.
Drew real sociograms around each stream. These sociograms show who actually talks to whom when something moves or gets stuck. This is not the org chart.
Surfaced “strangled swarms”: informal cross-org groups trying to fix issues. They have no mandate, no budget, and no clear decision rights.

Three patterns emerged:

Fragmented ownership
- At least four groups claimed they “owned” digital SME Onboarding (retail banking, SME business, BrightPay product, central IT).
- None had authority across process, technology, and policy for the end‑to‑end journey.
- Outcomes (time‑to‑open, NPS, churn) had no single accountable group.
Waterfall PMI overlay on top of complex work
- The integration office ran dozens of workstreams (IT, HR, Risk, Products), each with its own plan and RAG status.
- No cross‑functional group owned a specific customer journey or a synergy hypothesis.
- Steering committees saw slides about “integration progress,” but not how real customer cases moved through the merged system.
Culture as a blind spot, not a design input
- Bank leaders talked about “synergies” and “best‑of‑both,” but BrightPay teams experienced a slow absorption into legacy ways of working.
- Bank teams felt overrun by “cowboys” pushing changes fast; fintech engineers felt trapped in endless approvals.
- Culture differences appeared as blame, instead of being used as design material for complementary strengths.

AO interpretation

AO framed this not as “bad people” but as a system that made local optimization rational and end‑to‑end value impossible.

The system rewarded local optimization (protecting functions, minimizing risk) more than shared outcomes.
Informal cross‑org problem‑solving groups existed but were “strangled swarms”: no mandate, no budget, no clear decision rights.
The Target Operating Model existed mainly as a slide‑deck. It did not exist as living agreements about who owns what. It also did not define how decisions are made.

2. AO intervention: swarms and two‑speed TOM

Instead of redesigning the whole organization, AO proposed a minimal evolutionary Target Operating Model anchored on a few high‑value swarms.

2.1 Swarm the value, not the org chart

For the first wave, AO helped leaders choose two integration themes:

Integrated “Digital SME Onboarding”
Unified mobile current account experience.

For each theme, AO set up a cross‑functional swarm:

Members from: BrightPay product & engineering, NorthRiver IT, SME business, risk/compliance, operations, and customer support.
One clearly named outcome (e.g. “Median SME Onboarding time ≤ 5 days with NPS +20”).
Explicit decision rights for this slice: they could change process steps, UI copy, and routing rules. They could also modify some policy constraints within guardrails.
6–8 week integration sprints, each producing a working change in production, not just a design.

AO introduced Swarm‑4 style habits. These include short hypothesis cycles, visible backlog, and Plexus‑like governance. As a result, funding decisions followed evidence from swarm experiments.

2.2 Two‑speed TOM

AO resisted pressure to “fully harmonize” everything at once.

High‑volume, regulated core (e.g. deposit accounting, AML monitoring) stayed on the bank’s slower but stable processes for now.
Customer‑facing digital journeys were moved into AO swarms with a mandate to experiment, even if it meant temporary dual processes.

This two‑speed TOM shifted the question from “When will we be fully integrated?” to “Which flows need integrated behavior now, and how can we evolve the rest safely?”

2.3 Culture in the room

AO designed explicit integration ceremonies:

Monthly “customer walk” reviews occurred. Bank and fintech leaders followed real cases through the system. They heard directly from frontline staff.
Swarm retros that included both HR and risk, making culture and policy trade‑offs discussable, not invisible constraints.

Instead of generic culture workshops, culture was handled as part of everyday decisions. These decisions included how much risk to accept for faster Onboarding. They also involved deciding what service tone to use. Additionally, there were considerations on how to handle fallback when the fintech platform failed.

3. Outcomes and AO‑style metrics

Within two swarm cycles (around 3–4 months), the bank‑fintech system started to behave differently.

Quantitative shifts:

Median SME digital Onboarding time dropped from 12 days to 6, with a clear path to 5 days.
“Where is my application?” contact‑center calls for SME Onboarding fell by ~30% in swarm‑covered segments.
Decision lead‑time for key integration choices (e.g. product variants, UI changes) shrank from 6–8 weeks of committee ping‑pong to under 10 days for the swarm’s scope.

Qualitative shifts:

BrightPay engineers reported “having a real product again” rather than “just being an integration project.”
Bank business leaders described “finally seeing the same picture” when talking about journeys and constraints.
Attrition stabilized in the integrated areas. A few who had considered leaving decided to stay. They changed their minds once they saw the swarms’ mandate was real.

AO encouraged the organization to track a small, stable metric set for each swarm:

Time‑to‑value (from idea to production change affecting customers)
Decision latency on cross‑org items
Defect / incident rate on the integrated journey
Engagement / retention of key roles inside swarms.

The AO Method (Agile Organization/Agile Organizations Method) is Pierre Neis’s own framework. It is distilled from more than a decade of agile coaching. This includes organizational transformation work across many companies, cultures, and contexts.

Core origins

Practice-based patterns: Neis describes AO as emerging from recurring patterns. He observed these while coaching and redesigning organizations of different sizes over many years. He then consolidated these into a coherent model. Later, he wrote a book called “AO.”
Response to “Agile as system dynamic”: AO starts from the hypothesis that “Agile” is not a method. It is a dynamic within a social system (the organization). An “Agile Organization” enables highly efficient human interactions for better response to threats and opportunities.
Organic/anthropomorphic view of organizations: AO was formulated as an answer to the mechanistic, purely structural view of organizations. It favors an organic, anthropomorphic model. In this model, all actors are engaged around a shared purpose. Agility emerges from conditions rather than imposed methods.

Theoretical influences

Complexity and complex adaptive systems: AO integrates complexity science and views organizations as living, adaptive systems. It focuses on emergence, feedback loops, and enabling conditions. It does so rather than using rigid frameworks.
Agile values beyond teams: It extends agile principles. These include individuals and interactions, responsiveness to change, and iterative learning. The extension moves from the team/process level to organization-wide design, leadership, and culture.
Organizational design & systemic thinking: AO is explicitly positioned alongside frameworks like Requisite Organization and the Viable System Model. It draws on systemic organizational theory while emphasizing adaptive structure and human-centric design.

First FORMALIZATION

Initial consolidation (around 2018): Neis publicly consolidated recurring agile patterns into the AO Model around 2018. He presents it as “key agile patterns from a decade of experience.”
Program for coaches, managers, HR: He created a program for agile coaches. It was also designed for managers and HR practitioners. This happened about five years before the 2022 Agile Alliance book listing. It was deliberately decoupled from any single agile methodology. This approach later fed into the AO concepts and patterns.
Books and “New Normal” framing: The Agile Alliance description of “The New Normal – AO Concepts and Patterns of 21st Century Agile Organizations” shows AO being framed as a comprehensive model with five areas (e.g., “Structure is not Organization,” transition phases, five experiences, five work areas) used to guide large-scale transformation.

Distinctive design choices

People-centric, organic growth: AO explicitly promotes “people-centric organic growth.” It is based on pillars such as coherence, cohesion, and simple rules. It also emphasizes avoidance and separation. These pillars reflect its origin in lived transformation work rather than abstract org charts.
Systemic, human-centric method: It positions itself as a systemic, human-centric method for agile transformation. This approach contrasts with more structural or team-bounded agile approaches.
Integration with existing agile practices: AO is often presented as something that extends or complements Scrum. It also enhances other frameworks. AO enables internal startups, swarms, and plexus-like networks inside organizations.

Five experiences or metrics in AO-method

The AO‑method defines five experiences (or experience‑based metrics) as its core measurement dimensions:

Enterprise experience – How the whole enterprise experiences agility. This includes strategy coherence and economic performance. It also covers the ability to respond to threats and opportunities. Lastly, it involves the perceived “agility of the business” at the top level.
Organization experience – This refers to how the internal organization experiences its own design. This includes the clarity of structures and roles, the flow of work, and decision latency. It also assesses how well the organizational setup supports agile dynamics.
People experience – How individuals and teams experience work. It includes engagement, autonomy, mastery, and psychological safety. It also encompasses the everyday feel of collaboration and leadership.
Customer experience – It encompasses how customers perceive the organization’s delivery. This includes value relevance, speed, reliability, and the ease of interacting with the company.
System experience – This refers to the behavior of the overall socio-technical system. It includes aspects such as stability, adaptability, and quality of feedback loops. Additionally, it considers how well the technical and social systems support continuous learning.

AO uses these five experiences as feedback loops. They are not merely output numbers like velocity. The “Experience Metrics Workbook” then breaks each dimension down into concrete indicators and templates.

Five work areas in AO-method

In AO, the “five work areas” are:

The platform – The stable, shared backbone of the organization. It includes core services, enabling functions, governance, and standards. These provide a safe container for experiments and day‑to‑day work. It is where you ensure coherence, basic rules, and support structures.
The Plexus – The dynamic network of relationships, communities, and cross‑cutting coordination mechanisms that connect units beyond the formal hierarchy. This is where knowledge flows, sense‑making, and alignment across silos happen.
Programs – Longer‑lived, mission‑driven streams that coordinate multiple initiatives around strategic themes or products. They give continuity, funding, and direction to clusters of work aligned with strategic outcomes.
Projects – Time‑bounded efforts with a clear goal, scope, and delivery horizon. They organize work that needs focus and structure but does not require a permanent or semi‑permanent stream.
Swarms – Highly adaptive, short‑lived, cross‑functional groups that self‑organize around urgent opportunities or problems. They embody the most fluid, emergent form of collaboration in AO, forming and dissolving quickly as needs arise.