Stephen Channell

Stephen Channell

Independent Enterprise Architect and software developer who as been using Sparx EA since 7.0, and implemented  Sparx EA at a number of Investment Banks and ProCloud + WebEA

Thursday, 24 September 2020 02:52

Enterprise Architecture scalability



Enterprise Architecture covers a range of practices from business strategy through the structure and organisations of applications to detailed design using enterprise frameworks. While Business, Applications and Technology Architecture are often developed independently to address different stakeholder needs, the value of modelling is enhanced when different views are reconciled through realisation, trace and dependency relationships. In complex organisations, high-level taxonomies carry more influence when they represent the aggregation of application and services, while applications criticality is better understood when related to high-level functions and critical processes.

A Universal Bank example

A universal (covering Retail, Corporate, Investment and Wealth) commissioned a number of initiatives using Sparx Enterprise Architect to model the organisation in distinct repositories

Service Architecture

Focused on common modelling of applications with common layout of charter, requirements; high-level-design, functional and detailed design. Using a common repository allowed for dependencies between application and common frameworks to be highlighted. Detailed design of fifteen thousand application components was captured using UML notation with MDG meta-model customisation customisation for additional properties

Process Architecture

Focused on the processes performed across the organisation from client onboarding, through trading to risk management and financial accounting. The Enterprise Process Model was organised around a hierarchical process taxonomy that provided a Bank on Page process view, with MDG meta-model customisation customisation for reference to common applications, organisational hierarchy and other business taxonomies. Detained process modelling of twenty thousand activities, decision gateways and events was captured using BPMN notation.

Functional Architecture

Focused on the Functional Taxonomy to identify sponsorship for application and services, to coordinate the governance of change with escalation points to mitigate the impact of issues. Functional Architecture provided the current and future state of the business and allowed common capabilities to be identified, and transitioned to standard functions/services.

Domain Architecture

Focused on end-to-end design for specific domains/applications without the imposition of taxonomies or structure; but with the freedom to mix function, data, process, application design as appropriate for each initiative.

Enterprise Architecture

The challenge for Enterprise Architecture was to bring together different views and enrich each viewpoint with context provided by the different models. Rationalisation served to bridge the gaps between the viewpoints and provide fresh impetus to keep models current as design moved through implementation to maintenance. Rationalisation faced a number of challenges:

  • Size and complexity of models/metamodel-customisation, tested and exceeded the capability XMI transfer of repositories when projects did not share the same package hierarch, but had dependencies between projects
  • The need to retain domain-specific viewpoints, so that different aspects were not referenced inappropriately (such as a Customer Actor, component, table or class being referenced when Customer process lane was needed)
  • The need for MDG meta-model customisation to continue to evolve to meet domain presentation needs
  • Inability to coordinate/schedule the suspension of modelling activities to allow consolidation
  • Performance of huge models for interactive modelling activities


While the impediment to enterprise rationalisation were considerable, the potential benefit of requirement traceability and enterprise lineage were also considerable. Sparx Enterprise Architect is almost unique in its ability to model an organisation from high-level Enterprise Architecture and detailed application design though to real-time interaction of timing sensitive application. Sparx Enterprise Architect allows an interbank-interface class to be related to the strategic function, requirement and data that is needed for regulatory reporting.


The initial solution was a batch database ETL process, intended as a one-time load, but evolved into the need for an Enterprise Hub to provide continuous replication between models. Today, the Enterprise Hub provides continuous replication between domain-specific models, and an enterprise repository, applying viewpoint rules to changes:

  1. Status filters to prevent unfinished changes being replicated, and to separate as-is and to-be viewpoints
  2. Content filters to prevent application or database content being replicated to a consolidated process viewpoint

Enterprise Hub also allows for globally distributed repositories with local performance


In Data Lineage we highlighted that data lineage can be captured using UML <<trace>> relationships and field-level lineage can be projected from trace + common attribute names. This blog is concerned with using those relationships to automatically derive systems lineage.

Application Landscape

A core deliverable for any mature Enterprise Architecture practice is the application landscape that shows applications within the content of the platforms and layers that they are built upon, clustered so that collaborators appear next to each other. Applications are connected by <<Flow>> relationships that highlight the data that is passed between them. Often only the high-level flows are used, but the detail flows inform the layout of the diagram

Service/Message/Event Buses

A spider-web of flows between systems (where the number of flows between application is a factorial of the number of applications) is often used to demonstrate the complexity of the Current-State compared to the simplicity of a service/message bus in the target state. Services-Buses compound the difficulty of mapping lineage because it becomes common to highlight input/output as message and presume that interpretation of data is usage problem – this is erroneous:

  • Generalising at the landscape, pushes the actual architecture work down to the applications and converts the landscape from a diagram to picture.
  • Over generalisation mitigates against analysis of availability or exception escalation.
  • Automatic lineage projects this as a complex dependency graph where every system is potentially dependant on every other system. This is not a deficiency of the lineage graph, but highlights the need to be more specific.

Event Buses

Quality Event buses derive from the market-data pub/sub pattern rather than the message-queue pattern because they are time-ordered and newer events always supersede prior events. In lineage terms events busses appear to extenuate the service bus complexity problem, but can highlight design faults because the presence of event-lineage in an externally facing interface highlights data-leakage rather than complexity – fixing the leakage fault removes the complexity.

Deriving Enterprise Lineage

In object oriented analysis and design it is axiomatic that if [A] inherits from [B] and [B] inherits from [C] then [A] also inherits from [C] (even if [B] completely replaces the behaviour of [C]). Appling inheritance semantics to data trace [A] <- [B] <- [C] implies [A] <- [C]. In a trading scenario it is common to take the closing-price of one geographical market, and never use it because the opening-price of the next geographic market is available before we use it, but in 1% of cases (holidays/disasters) an opening price is not available and the prior closing-price is used.

The diagram show how enterprise lineage can be inferred from trace relationships. Although Reporting consumes Fact from the data-warehouseFI Trade is inferred to be in the Fact lineage because Fact has a trace relationship t FI trade. It will be demonstrated that the enterprise lineage can be automatically recursive derived from the flow relationships and the data trace relationships. With enterprise lineage, lineage does not need to be confined to components but can be extended to Actors (Bloomberg in the example); code classes/interfaces and even use-cases and business process for manual sourcing. Examining the lineage at the Reporting component boundary highlights (without consideration of internal systems complexity that Bloomberg contributes to Price and that Fact includes an element of manual data-entry. From a governance/regulatory perspective, it is not necessary to see the detailed working to know that price-sourcing and operational-risk need to be considered. Demonstration that lineage is automatically derived from detailed flows means that a detailed audit is not required to confirm the veracity of lineage summaries.

Price lineage generated automatically from above diagram
        ["SimpleBank.Component.Data Warehouse",
                ["SimpleBank.Component.Message Bus", 
                            ["SimpleBank.Class Diagram.MarketData.Bloomberg"]
                "SimpleBank.Component.FI Trading", 
                "SimpleBank.Component.EQ Trading"
Fact lineage generated automatically from above diagram
        ["SimpleBank.Component.Data Warehouse", 
                ["SimpleBank.Component.Message Bus", 
                    ["SimpleBank.Component.FX Trading"]],
                ["SimpleBank.Component.FI Trading", 
                        ["SimpleBank.Use Case.Execute Swap OTC", 
                            ["SimpleBank.Process.Execute Trade"]]
                "SimpleBank.Component.EQ Trading"


An alternative approach to data-lineage highlighted that <<trace>> relationships are superior when your objective is to provide lineage analysis rather than design. Using trace for data-lineage is an enabler for enterprise lineage. Enterprise Data-Lineage is well suited to enterprise architecture because it separates the solution domain from the enterprise domain where lineage analysis becomes an additional part architecture review and feedback to solutions architects and designers is addressed through improvement proceses. The next article will show how lineage is generated automatically through an Enterprise Hub


Cephei.QL is a large project, consisting of over two thousand classes and twenty thousand closures. It has been implemented in F# because parallel executing needs to be functionally immutable to avoid co-mutation of data on separate threads, but is also a simpler problem when type inference can be implied in many places.

Whilst there is a huge amount of code that needs to be produced; with some exceptions, the pattern of mapping an underlying library to a higher level abstraction is common


This blog is about how model-driven architecture can be applied to the problem of creating a library that integrates a number of underlying sources. Traditionally Architecture modelling tools approach this problem by translating classes to a metadata rich source where the use of templates allows the implementation to be derived at runtime.

For Cephei.QL, code generation has been used because:

  • The target language (F#) is not supported by the current generation of software engineering tools.
  • The permutations are more complex than is supported by templates and would require a runtime code generator to be used.
  • Runtime generation is not appropriate for high-performance production environments.
  • The target library is intended to provide recipes for specific implementation, that can tailored to specific problems.

While this example is concerned with generation of abstractions over a large financial library (library -> cell models -> serialisation -> Excel integration), the problem is the same as implementing data-structures that need to be mapped from Javascript though application tiers to data persistence


Sparx Enterprise Architect is the leading tool for software engineering because of the wide range languages supported and tools for Model-Driven-Architecture.. In this scenario it is also extremely strong because the underlying repository supports SQL queries and therefore rich database integration.

EA.Gen.Model is a .NET class library (that we developed years ago) that uses the Entity Data Model to provide a high-level object-graph view of the underlying Sparx Repository, with Linq Queries There are many templating technologies based on the principles for web-server pages that generate HTML for browsers, but the one we use is T4 Text Templates because it can be used within a Build Server (TFSTeamcityJenkins, etc) without having to configure open access.


The full Visual Studio 2019 code for Cephei.Gen is available from GitHub, but includes the C++ code generator that was previously used to wrap QuantLib before switching to the C# port QLNet. The NetModel/Class.cs provides the data model for classes, and NetQL/ provides template for the bulk of the Cephei.QL code.

After reverse engineering the source code into Sparx, the code generator can be run to generate F# code.


For a wide variety of problems, model driven architecture can be used to accrue greater value from the models we create during systems design, increasing productivity, and reducing errors introduced when coded by han

Friday, 22 May 2020 19:33

Enterprise Hub

Pleased to announce availability of the “Enterprise Hub”, an server-side replication hub that complements Sparx Enterprise Architect and Pro-Cloud in the large-scale enterprise space where a single large repository for Business/Systems/Technology/Data across many functional domains is not practical.

The Hub automate the replication and reconciliation of many models for downstream Stakeholder viewpoints and current/target state perspectives without the effort Export/Import.