Page Comparison

In the current implementation, ACM supports multi-participant with same supported element Type but different participantId, so they need different properties file.

...

Participant replicas can be a kubernetes StatefulSets that consume different properties file with unique consumer groups and unique UUIDs/replica (participants with same UUIDs have different replica number).

The StatefulSet uses the SPRING_CONFIG_NAME environment variable pointing to the spring application properties file unique to each of the participant replica.

Each of the properties file with the names pod-0.yaml, pod-1.yaml is mounted to the volumes. And the SPRING_CONFIG_NAME variable can be set to /path/to/$HOSTNAME.yaml to use the corresponding

properties file. 

By this approach the participant can have multiple replicas to work with a shared data.

env:
- name: HOSTNAME
  valueFrom:
    fieldRef:
      fieldPath: metadata.name

- name: SPRING_CONFIG_NAME
  value: /path/to/${HOSTNAME}.yaml

For example considering the http participant replica, ${HOSTNAME} will be "policy-http-ppnt-0" and "policy-http-ppnt-1" and their 
corresponding properties files with the names "http-ppnt-0.yaml" and "http-ppnt-1.yaml" is volume mounted. 

Note: In a scenario of two participants in replicas (we are calling "policy-http-ppnt-0" and "policy-http-ppnt-1"), ACM-Runtime will assignee as before any composition definition in prime time to specific participant based of supported element definition type. All participants will receive the same messages and store same data, so all participants are synchronized with ACM-R. Into all messages from ACM-R to participants will be present the replica number to indicate what participant will do the job (like prime or deploy). Example ACM-R send deploy message with replica 0, so "policy-http-ppnt-0" save the new instance and deploy and "policy-http-ppnt-1" just save that instance. When "policy-http-ppnt-0" send outProperties, then ACM-R and "policy-http-ppnt-1" receive the message and save that properties. When "policy-http-ppnt-0" has completed the deploy, the send the message and then then ACM-R and "policy-http-ppnt-1" receive the message and save the result. No issue if ACM-R send an undeploy message with replica 1, because all applications are synchronized.

Changes in Participant:

• Register, Status and Unregister message have to contain the replica number
• store data from messages from ACM-R if the participantId is matching and enable the actions only if the participantId and replica number are matching with the message
• store data (outProperties, and action completed) from messages from participants with same participantId and different replica
• implement time-out to stop the process if is running out of time

...

• Multiple participant replicas possible - it can deal with messages across many participants
• All participants should have same group-id in kafka
• All should have the same participant-id.

Solution 4: Distributed Cache

Issues:

Disadvantages of DB use

1. The Participant as a concept envisages an extremely lightweight mechanism, which can be deployed as a very lightweight plugin into the runtime entity implementing a participant
2. Introducing a database into the participant means that participant becomes a heavyweight plugin with cross participant and participant/acm-r configuration required
3. There are multiple sources of truth for participant and element data, the ACM-R DB and the this new participant DB have duplicated data
4. Participant configuration is no longer transparent to the participant implementing component, having a DB for each participant forces orthogonal configuration across participant implementing components
5. Configuration of components implementing participants becomes much more complex, the components must manage the participant replication
6. The replication mechanism enforces the use of an additional 3pp, the database

Solution 4: Distributed Cache

Issues:

• Not persistent - if the application that handles cache server restarts - data is lost.
• Approval issues - with Redis, Etcd, Search Engine.

...

1. We will preserve Participant Design backward compatibility, there is no change to the participant intermediary interface for participant implementations
2. Participant version backward compatibility will not be preserved because we need to pass replica information in the registration and  operational messages, all participants will have to be upgraded to the new version.
3. The REST API that returns participant information will be updated to include replica information
4. ACM-R is made HA so that it itself can scale
5. We can use Kafka load balancing on the participants and get the load balancing functionality for nothing
6. A new Kafka topic is used for synchronization

Optimal Solution:

After analysis, it is clear that the best solution to use is number 3.

• Arbitrary number of participants possible
• DB migrator upgrades older versions
• Restart scenario not applicable anymore. Could be removed.
• Approval not an issue - postgres already used by acm.
• DB will be created automatically - as are required tables.

Older participant versions support (Regression)

...

2. A new Kafka topic is used for synchronization

Optimal Solution:

After analysis, it is clear that the best solution to use is number 5.