Displaying items by tag: python

Saturday, 06 February 2021 15:00

Automatic creation of system model with Neo4j

Introduction

Frequently there are legacy systems/applications with no previous documentation, and the lack of it makes maintenance and further development of those systems/applications stagnant and demanding. This particular article outlines the ability of graphs (created through the Neo4j database) to automate the creation of documentation in the form of ArchiMate model in Sparx Enterprise Architect or Archi.

We will create an ArchiMate model for the legacy application that will describe basic entities, such as screens, fragments, roles, files and relations between them. An HTML version of the automatically generated ArchiMate model is available here. It's an export from Sparx Enterprise Architect to HTML.

In a next article, we use the graph to identify:

  • application communities
  • communities where fragments are used
  • communities with fewer screens than roles
  • the missing role setting in a linked screen
  • unused parts of the application
  • risk business processes

Example

An illustrative example of two screens and its relations to other screens, fragments, roles and files is below: An illustrative example of the relation between two screens and other fragments, roles and files is below:

Screen example

Even if there is only the source code of the application, there is a way (by source code scrapping) to obtain all required information. The source code scrapping should be implemented accordingly to the specific application. However, it is not covered in this article.

The example of an outcome from the scrapping for one screen (Screen_414) in a json format is below:

Screen example

The json definition of the screen (Screen_414) and its relations to screens, fragments, roles and files is possible to transform to a graph (nodes and relationships):

Screen example

The graph (Screen_414) and its nodes and relationships are possible to represent as an Archimate model. There are two steps:

  • an export from graf to ArchiMate® 3.1 Model Exchange File definition format
  • an import ArchiMate® 3.1 Model Exchange File definition to Sparx Enterprise Architect or Archi

Screen example

 

The example of automatically created documentation for the screen (Screen_414) in Sparx Enterprise Architect:

Database schema

Solution architecture

The solution is a mix of the following technologies, tools and standards:

  • Python
  • Jupyter Notebook
  • Neo4j (Neo4j Sandbox)
  • Neo4j Bloom
  • Neo4j APOC Library
  • Graph Data Science Playground
  • Cypher
  • ArchiMate
  • ArchiMate® 3.1 Model Exchange File definition
  • Sparx Enterprise Architect
  • Archi
  • VB script

Database schema

Description of architecture components
 
ComponentDescription
Legacy application A system/application for which the documentation will be created.
Source code scrapping An implemented functionality. The file as an outcome from the application source code scrapping (in json format) containing information about screens, fragments, roles, files and relations between them. The application code scrapping isn't covered in this article.

The anonymized file is available here: screenList_20210105_anonymized.json

Python / Neo4j The Python script in Jupyter Notebook using a graph database Neo4j.

There is a possibility to use Sandbox and create new Neo4j database in less than 60 seconds. No download required.

Import definition to Neo4j Part of the Python script. The script imports data from file screenList_20210105_anonymized.json to graph database Neo4j.
Generate ArchiMate model definition Part of the Python script. The script creates valid file in format ArchiMate® 3.1 Model Exchange File definition:

- screen_model.xml

The ArchiMate Model Exchange File Format is a standard file format that can be used to exchange ArchiMate models between tools that create or interpret ArchiMate models. The scope of the standard is restricted to the elements and the relationships of an ArchiMate model. It excludes features that are vendor specific.

Generate model diagram definition Part of the Python script. The script creates files with the definition of diagrams:

- screens.csv

- roles.csv

- fragments.csv

- links.csv

Sparx Enterprise Architect Sparx Enterprise Architect is a visual platform for designing and constructing software systems, business process modeling, and more generalized modeling purposes.
Create ArchiMate model A given functionality of Sparx Enterprise Architect. We have a valid ArchiMate® 3.1 Model Exchange File.

It is possible to import generated file screen_model.xml into a prepared Enterprise Architect project as a UML model.

To import an Open Exchange Format file, in Ribbon select the item "Specialize > Technologies > ArchiMate > Import Model Exchange File". Click on the Import button to import the selected Model Exchange file into the specified Package.

Create ArchiMate model diagrams An implemented functionality by VB script #1 generate_archimate_diagrams.vbs in the prepared Enterprise Architect project. The VB script generates diagrams using the definition in files:

- screens.csv

- roles.csv

- fragments.csv

- links.csv

To run the VB script, select context menu item "Specialize->Scripts->#1 generate_archimate_diagrams"

Archi Archi is an open-source modelling toolkit to create ArchiMate models and sketches.
Create ArchiMate model A given functionality of Archi.

It is possible to import a generated file screen_model.xml and create a new ArchiMate model.

To import an Open Exchange Format file, select the menu item "File->Import->Open Exchange XML Model...". Select and open the required XML file. This will create a new ArchiMate model in the Models Tree.

 

Create ArchiMate model diagrams in Sparx Enterprise Architect

Magic how to automatically generate diagrams in Sparx Enterprise Architect is in the VB script #1 generate_archimate_diagrams.vbs. The VB script generates diagrams using the definition in files:

  • creates a new diagram,
  • inserts defined elements
  • formats the layout of the elements in the diagram.

For example by following diagram definition:

DiagramName;NodeKeys;Documentation

Role_1269;['NEOID_3e19fa29-bae1-48da-88bc-d1ffa17c4a8a', 'NEOID_9f0d17d0-9076-452b-90cd-c64fc6892f60'];The diagram describes the role Role_1269.

the following diagram is generated:

Database schema

 

Graph database schema

The graph consists of 343 screens, 159 fragments, 36 roles and 465 files. A visual scheme of our graph database is given below.

Database schema

Node labels

Node LabelsDescriptionProperties with examples
Screen The screen of application.

"path": "/WEB-INF/tiles/adm/Screen_414.jsp"

"fileName": "Screen_414.jsp"

"code": "Screen_414"

"documentation": "Screen ...

"name": "Screen_414"

"link": "ADM033"

"alias": "Title of Screen_414"

"id": "NEOID_cfabe9f8-ff6c-4ee6-ab19-ae0bdc7f192e"

"class": "ApplicationComponent"

"group": "ADM"

Fragment The fragment belongs to the screen or other fragments.

"path": "/tiles/com/fragments/Fragment_1297.jsp"

"fileName": "Fragment_1297.jsp"

"documentation": "Fragment ... "

"name": "Fragment_1297"

"id": "NEOID_0fdcc7a5-1889-4fb6-aa62-94bf5128f840"

"class": "ApplicationComponent"

File The file where the screen or the fragment source code is defined.

"name": "Fragment_1315.jsp"

"path": "/tiles/com/fragments/Fragment_1315.jsp"

"id": "NEOID_04ff5f05-a207-4b8d-a1e8-928f187822e1"

"class": "DataObject"

"documentation": "File ..."

Role The role that restricts access to the screen.

"name": "Role_1259"

"id": "NEOID_44086a73-aa07-423e-a119-449e12e7ff9b"

"class": "DataObject"

"documentation": "Role ... "

Relationship Types

Relationship TypesDescription
FlowRelationships Relation represents the possible transition from one screen to another (name: 'LINK_TO').
CompositionRelationships Relation represents that fragment is part of the screen or another fragment (name: 'HAS_FRAGMENT').
AssociationRelationships Relation represents that screen or fragment source code is defined in the file (name: 'HAS_FILE').
Relation represents a role which controls access to the screen (name: 'HAS_ROLE').

 

Description of how to generate documentation

  1. Create new Neo4j database (https://neo4j.com/sandbox/)
  2. Set Python script parameters:
    • boltUrl
    • password
    • exportPath
  3. Run the Python script and generete files:
    • screen_model.xml
    • screens.csv
    • roles.csv
    • fragments.csv
    • links.csv
  4. Import generated file screen_model.xml into a prepared Enterprise Architect project as a model. To import an Open Exchange Format file, in Ribbon, select the item "Specialize > Technologies > ArchiMate > Import Model Exchange File". Click on the Import button to import the selected Model Exchange file into the specified Package.
  5. Run a script in Sparx Enterprise Architect. To run the VB script, select context menu item "Specialize->Scripts->#1 generate_archimate_diagrams". Diagrams are generated using the definition in files:
    • screens.csv
    • roles.csv
    • fragments.csv
    • links.csv
 
Published in Tutorials
Tagged under