In the current implementation, ACM supports multi-participant with same supported element Type but different participantId, so they need different properties file.
...
Participant replicas can be a kubernetes StatefulSets that consume different properties file with unique consumer groups and unique UUIDs/replica (participants with same UUIDs have different replica number).
The StatefulSet uses the SPRING_CONFIG_NAME environment variable pointing to the spring application properties file unique to each of the participant replica.
Each of the properties file with the names pod-0.yaml, pod-1.yaml is mounted to the volumes. And the SPRING_CONFIG_NAME variable can be set to /path/to/$HOSTNAME.yaml to use the corresponding
properties file.
By this approach the participant can have multiple replicas to work with a shared data.
env:
- name: HOSTNAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: SPRING_CONFIG_NAME
value: /path/to/${HOSTNAME}.yaml
For example considering the http participant replica, ${HOSTNAME} will be "policy-http-ppnt-0" and "policy-http-ppnt-1" and their
corresponding properties files with the names "http-ppnt-0.yaml" and "http-ppnt-1.yaml" is volume mounted.
Note: In a scenario of two participants in replicas (we are calling "policy-http-ppnt-0" and "policy-http-ppnt-1"), ACM-Runtime will assignee as before any composition definition in prime time to specific participant based of supported element definition type. All participants will receive the same messages and store same data, so all participants are synchronized with ACM-R. Into all messages from ACM-R to participants will be present the replica number to indicate what participant will do the job (like prime or deploy). Example ACM-R send deploy message with replica 0, so "policy-http-ppnt-0" save the new instance and deploy and "policy-http-ppnt-1" just save that instance. When "policy-http-ppnt-0" send outProperties, then ACM-R and "policy-http-ppnt-1" receive the message and save that properties. When "policy-http-ppnt-0" has completed the deploy, the send the message and then then ACM-R and "policy-http-ppnt-1" receive the message and save the result. No issue if ACM-R send an undeploy message with replica 1, because all applications are synchronized.
Changes in Participant:
- Register, Status and Unregister message have to contain the replica number
- store data from messages from ACM-R if the participantId is matching and enable the actions only if the participantId and replica number are matching with the message
- store data (outProperties, and action completed) from messages from participants with same participantId and different replica
- implement time-out to stop the process if is running out of time
...
- The concept of participant replicas into ACM-R and the Participant Intermediary
- A lightweight mechanism for replica update between replicas and ACM-R. Every time a replica changes its data, the change is reported to ACM-R and ACM-R updates all other replicas for that participant
The diagram above depicts data management as implemented today in ACM-R. ACM-R maintains the "source of truth" for all compositions and their elements in the ACM-R database.
...
Therefore today, for the three types of data above, the ACM-R database has the state of the participant. This means that creation of participant replicas is rather trivial. We leave the current data handling mechanism in place, introduce the concept of participant replicas in ACM-R, and introduce data synchronization across participant replicas.
We introduce a participant replication table in the ACM-R database, which is an index used to record the replicas that exist of each participant. When the replica of a component implementing a participant is created (by Kubernetes or otherwise), the participant intermediary in the component registers with ACM-R as usual and as it does today. The only difference is that the participant intermediary will send the participant ID and the replica number. ACM-R is updated to accept registrations from participants with the same Participant ID and different replica numbers. When the first replica for a certain participant registers, ACM-R will handle this registration exactly as it does today and will add the replica as the single replica that exists for this participant. When the next replica registers, ACM-R will recongnise that this is a second replica for a participant that already exists and will record this as a replica. Rather than priming this replica, ACM-R will copy all the data from the first replica to this replica. The registration of further replicas will continue to follow this pattern.
During normal operation where ACM-R receives and executes requests towards participants, ACM-R will use Participant Load Balancing to select a replica using a round-robin algorithm and execute the operation on that replica. When the operation finishes, ACM-R will synchronize the data from the replica that executed the operation to all the other replicas using Participant Synchronization.
If ACM-R is informed by a replica that an Implementing Component changed composition element properties, Participant Synchronization synchronizes these changes to all other Participant Intermediary replicas.
In this solution:
...