Enterprise Architecture covers a range of practices from business strategy through the structure and organisations of applications to detailed design using enterprise frameworks. While Business, Applications and Technology Architecture are often developed independently to address different stakeholder needs, the value of modelling is enhanced when different views are reconciled through realisation, trace and dependency relationships. In complex organisations, high-level taxonomies carry more influence when they represent the aggregation of application and services, while applications criticality is better understood when related to high-level functions and critical processes.

A Universal Bank example

A universal (covering Retail, Corporate, Investment and Wealth) commissioned a number of initiatives using Sparx Enterprise Architect to model the organisation in distinct repositories

Service Architecture

Focused on common modelling of applications with common layout of charter, requirements; high-level-design, functional and detailed design. Using a common repository allowed for dependencies between application and common frameworks to be highlighted. Detailed design of fifteen thousand application components was captured using UML notation with MDG meta-model customisation customisation for additional properties

Process Architecture

Focused on the processes performed across the organisation from client onboarding, through trading to risk management and financial accounting. The Enterprise Process Model was organised around a hierarchical process taxonomy that provided a Bank on Page process view, with MDG meta-model customisation customisation for reference to common applications, organisational hierarchy and other business taxonomies. Detained process modelling of twenty thousand activities, decision gateways and events was captured using BPMN notation.

Functional Architecture

Focused on the Functional Taxonomy to identify sponsorship for application and services, to coordinate the governance of change with escalation points to mitigate the impact of issues. Functional Architecture provided the current and future state of the business and allowed common capabilities to be identified, and transitioned to standard functions/services.

Domain Architecture

Focused on end-to-end design for specific domains/applications without the imposition of taxonomies or structure; but with the freedom to mix function, data, process, application design as appropriate for each initiative.

Enterprise Architecture

The challenge for Enterprise Architecture was to bring together different views and enrich each viewpoint with context provided by the different models. Rationalisation served to bridge the gaps between the viewpoints and provide fresh impetus to keep models current as design moved through implementation to maintenance. Rationalisation faced a number of challenges:

• Size and complexity of models/metamodel-customisation, tested and exceeded the capability XMI transfer of repositories when projects did not share the same package hierarch, but had dependencies between projects
• The need to retain domain-specific viewpoints, so that different aspects were not referenced inappropriately (such as a Customer Actor, component, table or class being referenced when Customer process lane was needed)
• The need for MDG meta-model customisation to continue to evolve to meet domain presentation needs
• Inability to coordinate/schedule the suspension of modelling activities to allow consolidation
• Performance of huge models for interactive modelling activities

Value

While the impediment to enterprise rationalisation were considerable, the potential benefit of requirement traceability and enterprise lineage were also considerable. Sparx Enterprise Architect is almost unique in its ability to model an organisation from high-level Enterprise Architecture and detailed application design though to real-time interaction of timing sensitive application. Sparx Enterprise Architect allows an interbank-interface class to be related to the strategic function, requirement and data that is needed for regulatory reporting.

Solution

The initial solution was a batch database ETL process, intended as a one-time load, but evolved into the need for an Enterprise Hub to provide continuous replication between models. Today, the Enterprise Hub provides continuous replication between domain-specific models, and an enterprise repository, applying viewpoint rules to changes:

1. Status filters to prevent unfinished changes being replicated, and to separate as-is and to-be viewpoints
2. Content filters to prevent application or database content being replicated to a consolidated process viewpoint

Enterprise Hub also allows for globally distributed repositories with local performance

In Data Lineage we highlighted that data lineage can be captured using UML <<trace>> relationships and field-level lineage can be projected from trace + common attribute names. This blog is concerned with using those relationships to automatically derive systems lineage.

Application Landscape

A core deliverable for any mature Enterprise Architecture practice is the application landscape that shows applications within the content of the platforms and layers that they are built upon, clustered so that collaborators appear next to each other. Applications are connected by <<Flow>> relationships that highlight the data that is passed between them. Often only the high-level flows are used, but the detail flows inform the layout of the diagram

Service/Message/Event Buses

A spider-web of flows between systems (where the number of flows between application is a factorial of the number of applications) is often used to demonstrate the complexity of the Current-State compared to the simplicity of a service/message bus in the target state. Services-Buses compound the difficulty of mapping lineage because it becomes common to highlight input/output as message and presume that interpretation of data is usage problem – this is erroneous:

• Generalising at the landscape, pushes the actual architecture work down to the applications and converts the landscape from a diagram to picture.
• Over generalisation mitigates against analysis of availability or exception escalation.
• Automatic lineage projects this as a complex dependency graph where every system is potentially dependant on every other system. This is not a deficiency of the lineage graph, but highlights the need to be more specific.

Event Buses

Quality Event buses derive from the market-data pub/sub pattern rather than the message-queue pattern because they are time-ordered and newer events always supersede prior events. In lineage terms events busses appear to extenuate the service bus complexity problem, but can highlight design faults because the presence of event-lineage in an externally facing interface highlights data-leakage rather than complexity – fixing the leakage fault removes the complexity.

Deriving Enterprise Lineage

In object oriented analysis and design it is axiomatic that if [A] inherits from [B] and [B] inherits from [C] then [A] also inherits from [C] (even if [B] completely replaces the behaviour of [C]). Appling inheritance semantics to data trace [A] <- [B] <- [C] implies [A] <- [C]. In a trading scenario it is common to take the closing-price of one geographical market, and never use it because the opening-price of the next geographic market is available before we use it, but in 1% of cases (holidays/disasters) an opening price is not available and the prior closing-price is used.

The diagram show how enterprise lineage can be inferred from trace relationships. Although Reporting consumes Fact from the data-warehouse, FI Trade is inferred to be in the Fact lineage because Fact has a trace relationship t FI trade. It will be demonstrated that the enterprise lineage can be automatically recursive derived from the flow relationships and the data trace relationships. With enterprise lineage, lineage does not need to be confined to components but can be extended to Actors (Bloomberg in the example); code classes/interfaces and even use-cases and business process for manual sourcing. Examining the lineage at the Reporting component boundary highlights (without consideration of internal systems complexity that Bloomberg contributes to Price and that Fact includes an element of manual data-entry. From a governance/regulatory perspective, it is not necessary to see the detailed working to know that price-sourcing and operational-risk need to be considered. Demonstration that lineage is automatically derived from detailed flows means that a detailed audit is not required to confirm the veracity of lineage summaries.

Price lineage generated automatically from above diagram

["SimpleBank.Component.Reporting", 
    [
        ["SimpleBank.Component.Data Warehouse",
            [
                ["SimpleBank.Component.Message Bus", 
                    [
                        ["SimpleBank.Component.MarketData",
                            ["SimpleBank.Class Diagram.MarketData.Bloomberg"]
                        ]
                    ]
                ],
                "SimpleBank.Component.FI Trading", 
                "SimpleBank.Component.EQ Trading"
            ]
        ]
    ]
]

Fact lineage generated automatically from above diagram

["SimpleBank.Component.Reporting", 
    [   
        ["SimpleBank.Component.Data Warehouse", 
            [   
                ["SimpleBank.Component.Message Bus", 
                    ["SimpleBank.Component.FX Trading"]],
                ["SimpleBank.Component.FI Trading", 
                    [
                        ["SimpleBank.Use Case.Execute Swap OTC", 
                            ["SimpleBank.Process.Execute Trade"]]
                    ]
                ], 
                "SimpleBank.Component.EQ Trading"
            ]
        ]
    ]
]

Conclusion

An alternative approach to data-lineage highlighted that <<trace>> relationships are superior when your objective is to provide lineage analysis rather than design. Using trace for data-lineage is an enabler for enterprise lineage. Enterprise Data-Lineage is well suited to enterprise architecture because it separates the solution domain from the enterprise domain where lineage analysis becomes an additional part architecture review and feedback to solutions architects and designers is addressed through improvement proceses. The next article will show how lineage is generated automatically through an Enterprise Hub