...
Chameleon - Subsystem that processes spike events from the real-time graph updates and formats the events, entry point to gallifrey/time machine, and enforces/formats the requests into the format needed by Gallifrey/Time Machine
Gallifrey/Time Machine - Subsystem that makes & retrieves historical assertions
...
Resources > Gizmo > Synapse > Chameleon > Gallifrey/Time Machine > Historical Champ > Historical DB Cluster
I feel Chameleon could be bypassed for this flow, treating chameleon's functionality as just the dmaap history processor, and synapse would call gallifrey/tm directly. This way if chameleon goes down, gallifrey/tm could still service historical GET requests.
Resources > Gizmo > Synapse > Chameleon > Gallifrey/Time Machine > Historical Champ > Historical DB Cluster
...
If the timestamp was sent on a non-singular node call then we would return a message stating that this functionality is not supported.
Gallifrey/Time Machine API Spec:
Type | URI | Query Params | Description | Champ Interaction |
GET | relationship/<ID> | t-k=Timestamp that specifies knowledge ie: when we received the assertion in Gallifrey/TM t-t=Timestamp that specifies a time assertion made by the client for when the change (update/add/delete) occurred in the network | Retrieve a relationship by ID | Champ needs to handle accepting a relationship id and the timestamp to run a historical query on the graph db A subgraph strategy will be used in champ to filter on the relative timestamp provided. meta=[true/false] if true, payload retrieved will hold the metaproperties at t-k or t-t |
GET | relationship/<ID>/lifespan NOT in scope for Casablanca - decided on call 5/23/2018 | None | Retrieve all the timestamps for create, update, delete operations against this relationship | Champ would be called to retrieve the lifespan on the relationship with meta=true and t-k=lifespan to retrieve all the metaproperties on the relationship |
GET | entity/<ID> | t-k=Timestamp that specifies knowledge ie: when we received the assertion in Gallifrey/TM t-t=Timestamp that specifies a time assertion made by the client for when the change (update/add/delete) occurred in the network | Retrieve an entity by ID | Champ needs to handle accepting an entity id and the timestamp to run a historical query on the graph db retrieving the asserted state. Default not sending back metaproperties, if metaproperties are needed a parameter would need to be sent to champ. A subgraph strategy will be used in champ to filter on the relative timestamp provided. meta=[true/false] if true, payload retrieved will hold the metaproperties at t-k |
GET | entity/<ID>/lifespan NOT in scope for Casablanca - decided on call 5/23/2018 | None | Retrieve all the timestamps for create, update, delete operations against this entity | Champ would be called to retrieve the lifespan on the entity with meta=true and t-k=lifespan to retrieve all the metaproperties on the entity |
PUT | relationship/<ID> | actor=name of the system making the assertion changes-only=[true|false] if true, gallifrey/tm will actually determine what has changed between the PUT payload and the most recent set of assertions for the relationship. If false, the entire PUT body will be considered as a new set of assertions whether something has changed or not. create=[true|false] if true, Gallifrey/TM assumes that this is a create request, if false it assumes it is an update t-t=Timestamp that specifies a time assertion made by the client for when the change (update/add/delete) occurred in the network | Asserts that a relationship is to be created or updated (depending on the query parameters that are passed in). This API appends new assertions against the specified relationship. Generated in Gallifrey/TM t-k=Timestamp that specifies knowledge ie: when we received the assertion in Gallifrey/TM (why are we generating this here? if there is a maintenance issue our timings would be out of sync with when these took place in the real-time db) | create = false (changes-only true (execute diff)/false(assume everything changed)) When an assertion is being made without a network timestamp gallifrey/tm will call champ requesting the relationship with it's most current metaproperties. Gallifrey/TM would then adjust the metaproperties (of the updated properties) and would send the payload back to champ with a new current state and an updated previous state's metaproperties. Champ would override it's current metaproperty (for the updated/deleted properties) with the old and current metaproperties sent from Gallifrey/TM. For added properties, they would be added directly with the metaproperties sent from Gallifrey/TM. When an assertion is being made with a network timestamp gallifrey/tm will call champ requesting the relationship with all of its metaproperties. Gallifrey/TM would then insert the new assertion where appropriate (and adjust neighboring metaproperties) and send the modified payload back to champ for a replace. create = true POST - this would be a new create and Gallifrey/TM would pass the metaproperties on each of it's property values and on the relationship itself |
PUT | entity/<ID> | actor=name of the system making the assertion changes-only=[true|false] if true, gallifrey/tm will actually determine what has changed between the PUT payload and the most recent set of assertions for the entity. If false, the entire PUT body will be considered as a new set of assertions whether something has changed or not. create=[true|false] if true, Gallifrey/TM assumes that this is a create request, if false it assumes it is an update t-t=Timestamp that specifies a time assertion made by the client for when the change (update/add/delete) occurred in the network | Asserts that an entity is to be created or updated (depending on the query parameters that are passed in). This API appends new assertions against the specified entity. Generated in Gallifrey/TM t-k=Timestamp that specifies knowledge ie: when we received the assertion in Gallifrey/TM | create = false (changes-only true (execute diff)/false(assume everything changed)) When an assertion is being made without a network timestamp gallifrey/tm will call champ requesting the entity with it's most current metaproperties. Gallifrey/TM would then adjust the metaproperties (of the updated properties) and would send the payload back to champ with a new current state and an updated previous state's metaproperties. Champ would override it's current metaproperty (for the updated/deleted properties) with the old and current metaproperties sent from Gallifrey/TM. For added properties, they would be added directly with the metaproperties sent from Gallifrey/TM. When an assertion is being made with a network timestamp gallifrey/tm will call champ requesting the entity with all of its metaproperties. Gallifrey/TM would then insert the new assertion where appropriate (and adjust neighboring metaproperties) and send the modified payload back to champ for a replace. create = true POST - this would be a new create and Gallifrey/TM would pass the metaproperties on each of it's property values and on the entity itself |
DELETE | relationship/<ID> | actor=name of the system making the assertion t-t=Timestamp that specifies a time assertion made by the client for when the change (update/add/delete) occurred in the network | Asserts that a relationship has been deleted. Generated in Gallifrey/TM t-k=Timestamp that specifies knowledge ie: when we received the assertion in Gallifrey/TM | Gallifrey/TM would request the latest relationship from champ and would set all of it's properties dbEndTimes to t-k along with the dbEndTime on the relationship itself. |
DELETE | entity/<ID> | actor=name of the system making the assertion t-t=Timestamp that specifies a time assertion made by the client for when the change (update/add/delete) occurred in the network | Asserts that an entity has been deleted. Generated in Gallifrey/TM t-k=Timestamp that specifies knowledge ie: when we received the assertion in Gallifrey/TM | Gallifrey/TM would request the latest entity from champ and would set all of it's properties dbEndTimes to t-k along with the dbEndTime on the entity itself. |
...
Type | URI | Query Params | Description |
GET | relationship/<ID> | t-k=Timestamp that specifies knowledge ie: when we received the assertion in Gallifrey/TM t-t=Timestamp that specifies a time assertion made by the client for when the change (update/add/delete) occurred in the network | Retrieve a relationship by ID |
GET | entity/<ID> | t-k=Timestamp that specifies knowledge ie: when we received the assertion in Gallifrey/TM t-t=Timestamp that specifies a time assertion made by the client for when the change (update/add/delete) occurred in the network | Retrieve an entity by ID |
...
Between Chameleon->Gallifrey/TM, the following calls are made:
...
Open question: what happens in the event that db edge rules change etc. (migrations)
When To Record T-K value (Live DB vs Historical DB)
Below is an explanation of the thoughts around when to record the t-k value, examples explaining outcome of storing at live vs historical DB and their pros/cons.
| View file | ||||
|---|---|---|---|---|
|
GUI Mocks
New integrated functionality (updates for history)
...