Displaying items by tag: domain model

Subscribe to this RSS feed

Monday, 15 July 2019 05:12

How does a code generator enable large-team parallel development?

One of our biggest challenges at Parallel Agile is explaining how two apparently unrelated aspects of our business are actually very closely related. Specifically, how our domain driven database and API generator (aka Parallel Agile CodeBot) enables the Parallel Agile Process, which allows large teams of developers to collaborate more efficiently and breaks the “two pizza rule” on team size.

To understand how the CodeBot enables large-scale parallel development we need to go back to Brooks’ Law, the widely believed postulate that you can’t accelerate a software project by throwing more developers at it. Fred Brooks identified some key reasons why this has been (for the most part) a truism for the last 50 years. Two of the most important reasons are:

1) Communication. When you add new developers to a software project, somebody has to explain to them how the existing code is structured, and the new developers each have to experience a learning curve before they can be productive. Generally it’s the experienced developers (who are already productive) that have to take time away from their own work to train the new folks. So adding developers can actually slow you down.

2) Integration. If you have a large team of developers working (relatively) independently, it’s going to be difficult to make all of the pieces plug together and play well as an integrated whole piece of software.

It’s important to remember that Fred Brooks developed his postulate while managing a large mainframe operating project (OS/360), which was introduced in 1964. That’s a long time ago. There were no UML modeling tools, no REST APIs, no NoSQL databases. So it’s not unreasonable to ask the question 55 years later whether any modern technical innovations might mitigate some of the underlying factors that have made Brooks’ Law seem to be inviolable for such a long time. That answer turns out to be Yes – making a domain model executable mitigates both the communication and integration problems and enables large-team development to work a lot better.

While I’ve never been a big fan of Brooks’ Law, when we started the research that led to Parallel Agile, there was no particular intent of trying to disprove it. It started back in 2014 when Barry Boehm and Supannika Mobasser invited me to give a guest lecture on ICONIX Process in their CS577 Software Engineering class at the University of Southern California. Around this time I had developed an interest in geofencing, and had in mind to develop a geofenced mobile app that delivered coupons to your phone when you got physically close to the business that was offering the coupons.

LBA use cases

When I accepted the invitation to guest lecture I decided to ask if they would like me to grade a couple of UML homework assignments in addition to guest lecturing. They agreed so I split my Location Based Advertising project into 47 low level use cases and assigned one use case per student as homework (writing prototype code for the use case was optional extra credit).

To make a long story short, we were all surprised that the 47 independently developed use cases actually integrated into a cohesive system. I can clearly recall sitting in Barry’s office saying “you know, that’s not supposed to be possible” and Barry agreeing “yes, I know”. When we started investigating how the successful integration happened it seemed that we had made some fortunate choices of using a NoSQL database (Cassandra) and a REST API (Node JS), and giving all the students a code template that implemented the database access functions (aka CRUD functions) for a Cassandra collection that they could clone for their part of the database.

LBA Domain Model 2

It turned out that having the domain model in place made communication across the large (47 student) team go pretty well, and having all of the database access functions available through a REST API allowed different pieces of the system (iOS app, Android app, web app) to integrate together with ease. And when we studied the effort numbers it turned out that each of the 47 use cases took about 4 days to develop – a day on requirements analysis, a day on design, and two days of coding.

One of the student from that project joined the PhD program and implemented a code generator that turned domain models into Mongo DB and Node JS, and after using this successfully on several other projects we decided to start Parallel Agile.

So, to sum it up…having a visual model of the problem domain makes it easy for everybody to get on the same page about what’s being built. Generating your database from this domain model, and having a uniform set of functions to Create, Read, Update and Delete items in the database makes it easy for everybody’s code to integrate together. And that’s how a code generator enables large-scale parallel development.

Published in Tutorials

Tagged under

Automatically trace elements to domain model in Enterprise Architect

This script will search the text in the comments, scenario’s and linked document of the selected elements for terms in the domain model. If it finds a match it will create a trace from your selected element to the element in the domain model.

The idea is that when we write descriptions and scenario’s we often use terms already defined in the domain model, so it would be good if we had a real trace from the element in the domain model and the element using it in its description.

Link to domain model example

But it is hard to keep the descriptions and scenario’s in sync with the traces to the domain model. This script does the hard part for us and keeps those traces in sync. All you need to do is type your description and run the script. The script will find the referenced elements in the domain model and create a trace link to it. It will also remove any automatic traces that are not valid anymore.

Next to the Name, Notes, and the Linked Document, this script will also search the scenarios, both theScenario Description, as the structured Scenario Steps. In case of a Scenario Step the script will insert to the name of the domain model element in the Uses column. Link to domain model scenario

Note that the script is case sensitive. That is the reason why RoomType does not appear in the uses column. Enterprise Architect itself isn’t that picky, and it underlines Roomtype anyway because it finds a relationship to RoomType.

In order to execute the script you have to select either one or more elements in the project browser, or select a package in the project browser.

Then select Script|Link To Domain Model

Free download

The script

In the first part we ask the user to select the domain model package.
If you want to use this script on a particular model you may want to replace that by a hard coding the Domain models GUID into the script.
Since this script may run for a while on a large model, we also create an output tab so we can inform the user of our progress.

sub main
    'let the user select the domain model package
    dim response
    response = Msgbox ("Please select the domain model package", vbOKCancel+vbQuestion, "Selecte domain model")
    if response = vbOK then
         
        dim domainModelPackage as EA.Package
        set domainModelPackage = selectPackage()
        if not domainModelPackage is nothing then
            Repository.CreateOutputTab "Link to Domain Model"
            Repository.ClearOutput "Link to Domain Model"
            Repository.EnsureOutputVisible "Link to Domain Model"
            'link selected elements to the domain model elements
            linkSelectionToDomainModel(domainModelPackage)
            'tell the user we are done
            Repository.WriteOutput "Link to Domain Model", "Finished!", 0
        end if
    end if
end sub

Next we create a dictionary of all classes in domain model package and all of the sub-packages recursively.

'first get the pattern from all the classes in the domain model
dim dictionary
Set dictionary = CreateObject("Scripting.Dictionary")
 
'create domain model dictionary
'tell the user what we are doing
Repository.WriteOutput "Link to Domain Model", "Creating domain model dictionary", 0
addToClassDictionary domainModelPackage.PackageGUID, dictionary

Based on this dictionary we can create one large Regular Expression that will tell us which of the terms in the dictionary are used in a certain text.

' and prepare the regex object
dim pattern
'create the pattern based on the names in the dictionary
pattern = createRegexPattern(dictionary)
Dim regExp  
Set regExp = CreateObject("VBScript.RegExp")
regExp.Global = True  
regExp.IgnoreCase = False
regExp.Pattern = pattern

Then we check what is selected in the project browser. If the user has selected one or more elements we start process those elements. If a package is selected then we process the elements in the selected package.

' Get the type of element selected in the Project Browser
dim treeSelectedType
treeSelectedType = Repository.GetTreeSelectedItemType()
' Either process the selected Element, or process all elements in the selected package
select case treeSelectedType
    case otElement
        ' Code for when an element is selected
        dim selectedElements as EA.Collection
        set selectedElements = Repository.GetTreeSelectedElements()
        'link the selected elements with the 
        linkDomainClassesWithElements dictionary,regExp,selectedElements 
    case otPackage
        ' Code for when a package is selected
        dim selectedPackage as EA.Package
        set selectedpackage = Repository.GetTreeSelectedObject()
        'link use domain classes with the elements in the selected package
        linkDomainClassesWithElementsInPackage dictionary, regExp,selectedPackage
    case else
        ' Error message
        Session.Prompt "You have to select Elements or a Package", promptOK
end select

Here’s the complete code for the script.

option explicit
 
!INC Local Scripts.EAConstants-VBScript
 
'
' Script Name: Link to Domain Model
' Author: Geert Bellekens
' Purpose: Link elements with classes in the domain model based on their name.
' Date: 15/11/2015
'
 
 
sub main
    'let the user select the domain model package
    dim response
    response = Msgbox ("Please select the domain model package", vbOKCancel+vbQuestion, "Selecte domain model")
    if response = vbOK then
         
        dim domainModelPackage as EA.Package
        set domainModelPackage = selectPackage()
        if not domainModelPackage is nothing then
            Repository.CreateOutputTab "Link to Domain Model"
            Repository.ClearOutput "Link to Domain Model"
            Repository.EnsureOutputVisible "Link to Domain Model"
            'link selected elements to the domain model elements
            linkSelectionToDomainModel(domainModelPackage)
            'tell the user we are done
            Repository.WriteOutput "Link to Domain Model", "Finished!", 0
        end if
    end if
end sub
 
'this is the actual call to the main function
main
 
function linkSelectionToDomainModel(domainModelPackage)
    'first get the pattern from all the classes in the domain model
    dim dictionary
    Set dictionary = CreateObject("Scripting.Dictionary")
     
    'create domain model dictionary
    'tell the user what we are doing
    Repository.WriteOutput "Link to Domain Model", "Creating domain model dictionary", 0
    addToClassDictionary domainModelPackage.PackageGUID, dictionary
     
    'tell the user what we are doing
    Repository.WriteOutput "Link to Domain Model", "Interpreting dictionary", 0
    ' and prepare the regex object
    dim pattern
    'create the pattern based on the names in the dictionary
    pattern = createRegexPattern(dictionary)
    Dim regExp  
    Set regExp = CreateObject("VBScript.RegExp")
    regExp.Global = True  
    regExp.IgnoreCase = False
    regExp.Pattern = pattern
     
    ' Get the type of element selected in the Project Browser
    dim treeSelectedType
    treeSelectedType = Repository.GetTreeSelectedItemType()
    ' Either process the selected Element, or process all elements in the selected package
    select case treeSelectedType
        case otElement
            ' Code for when an element is selected
            dim selectedElements as EA.Collection
            set selectedElements = Repository.GetTreeSelectedElements()
            'link the selected elements with the 
            linkDomainClassesWithElements dictionary,regExp,selectedElements 
        case otPackage
            ' Code for when a package is selected
            dim selectedPackage as EA.Package
            set selectedpackage = Repository.GetTreeSelectedObject()
            'link use domain classes with the elements in the selected package
            linkDomainClassesWithElementsInPackage dictionary, regExp,selectedPackage
        case else
            ' Error message
            Session.Prompt "You have to select Elements or a Package", promptOK
    end select
end function
 
'this function will get all elements in the given package and subpackages recursively and link them to the domain classes
function linkDomainClassesWithElementsInPackage(dictionary,regExp,selectedPackage)
    dim packageList 
    set packageList = getPackageTree(selectedPackage)
    dim packageIDString
    packageIDString = makePackageIDString(packageList)
    dim getElementsSQL
    getElementsSQL = "select o.Object_ID from t_object o where o.Package_ID in (" &amp; packageIDString &amp; ")"
    dim usecases
    set usecases = getElementsFromQuery(getElementsSQL)
    linkDomainClassesWithElements dictionary,regExp,usecases
end function
 
 
function linkDomainClassesWithElements(dictionary,regExp,elements)
    dim element as EA.Element
    'loop de elements
    for each element in elements
        'tell the user what we are doing
        Repository.WriteOutput "Link to Domain Model", "Linking element: " &amp; element.Name, 0
        'first remove all automatic traces
        removeAllAutomaticTraces element
        'match based on notes and linked document
        dim elementText
        'get full text (name +notes + linked document + scenario names + scenario notes)
        elementText = element.Name
        elementText = elementText &amp; vbNewLine &amp; Repository.GetFormatFromField("TXT",element.Notes)
        elementText = elementText &amp; vbNewLine &amp; getLinkedDocumentContent(element, "TXT")
        elementText = elementText &amp; vbNewLine &amp; getTextFromScenarios(element)
        dim matches
        set matches = regExp.Execute(elementText)
        'for each match create a «trace» link with the element
        linkMatchesWithelement matches, element, dictionary
        'link based on text in scenariosteps
        dim scenario as EA.Scenario
        'get all dependencies left
        dim dependencies
        set dependencies = getDependencies(element)
        'loop scenarios
        for each scenario in element.Scenarios
            dim scenarioStep as EA.ScenarioStep
            for each scenarioStep in scenario.Steps
                'first remove any additional terms in the uses field
                scenarioStep.Uses = removeAddionalUses(dependencies, scenarioStep.Uses)
                set matches = regExp.Execute(scenarioStep.Name)
                dim classesToMatch 
                set classesToMatch = getClassesToMatchDictionary(matches, dictionary)
                dim classToMatch as EA.Element
                for each classToMatch in classesToMatch.Items
                    if not instr(scenarioStep.Uses,classToMatch.Name) &gt; 0 then
                        scenarioStep.Uses = scenarioStep.Uses &amp; " " &amp; classToMatch.Name
                    end if
                    'create the dependency between the use case and the domain model class
                    linkElementsWithAutomaticTrace element, classToMatch
                next
                'save scenario step
                scenarioStep.Update
                scenario.Update
            next
        next
    next
end function
 
function linkMatchesWithelement(matches, element, dictionary)
    