...
...
...
...
...
...
...
...
...
...
REMOVED IN RELEASE G - NO LONGER SUPPORTED
MirrorMaker is a stand-alone tool for copying data between two Apache Kafka clusters. It is a Kafka consumer and producer hooked together. Data is read from topics in the origin cluster and written to a topic with the same name in the destination cluster. You can run many such mirroring processes to increase throughput and for fault-tolerance
DMaaP Message Router offers the following REST API to create a mirror maker process, update a mirror maker process etc.
Mirror Maker
| Expand |
|---|
|
POST /mirrormakers/create This API will create a Mirror Maker. Required AAF Permissions : org.onap.dmaap.mr.mirrormaker|*|admin Request Payload:| Parameter Name | Req'd | |
|---|
| createMirrorMaker | Y | Mirror Maker Request Object | | name | Y | Unique Name for the MirrorMaker A-Z 1-0 only | | consumer | Y | Consumer hostname/IP Address | | producer | Y | Producer hostname/IP Address | | numStreams | Optional | Number of consumer threads to start. (If not provided default is 1) | | whitelist | Optional | Topic names to be whitelisted. This can be provided later using createWhitelist API | | enablelogCheck | Optional |
Enable error log monitoring on the MirrorMaker thread. Default values is False. If not provided error log is not monitored. |
{
"createMirrorMaker":
{
"name": "centraltoedge",
"consumer": "10.12.6.98:30491",
"producer": "10.12.5.108:30491",
"numStreams": "3",
"whitelist": "org.onap.dmaap.mr.mrtesttopic",
"enablelogCheck": "true"
}
}
Response Payload:{
"listMirrorMaker": {
"name": "centraltoedge",
"producer": "10.12.5.108:30491",
"whitelist": "org.onap.dmaap.mr.mrtesttopic",
"enablelogCheck": true,
"consumer": "10.12.6.98:30491",
"status": "RESTARTING",
"numStreams": 3
}
}
Error Messages: | Error Reson | Error Message |
|---|
| No AAF Mirror Maker Admin permission | No Mirror Maker Admin permission. | | Creating new MirrorMaker with existing MirrroMaker name | MirrorMaker <MM Name> already exists | | Incorrect payload | This is not a CreateMirrorMaker request. Please try again. | | Consumer field/value missing | Please provide Consumer host:port details | | Producer field/value missing | Please provide Producer host:port details | | Incorrect naming convention | Mirror Maker name can only contain alpha numeric | | Incorrect JSONIncorrect JSON object. | Incorrect JSON object.Could not parse JSON. Please correct the JSON format and try again. | | If Mirror Maker agent is down or it takes more than (right now it is configured 15 seconds) | listMirrorMaker is not available, please make sure MirrorMakerAgent is running |
|
...
| Expand |
|---|
|
POST /mirrormakers/delete This API is used to delete an existing MirrorMaker. Required AAF Permissions: org.onap.dmaap.mr.mirrormaker|*|admin Request:Request Payload:| Parameter Name | Req'd | Description |
|---|
| deleteMirrorMaker | Y | Mirror Maker Request Object | | Name | Y | Existing MirrorMaker name |
{
"deleteMirrorMaker":
{
"name":"centraltoedge2"
}
}
Response Payload:
"listMirrorMaker": [
{
"name": "centraltoedge",
"producer": "10.12.5.108:30491",
"whitelist": "org.onap.dmaap.mr.mrtesttopic",
"enablelogCheck": true,
"consumer": "10.12.6.98:30491",
"status": "RUNNING",
"numStreams": 3
}
],
"messageID": "99310"
}
Error Messages:| Error Reason | Error Message |
|---|
| No AAF Mirror Maker Admin permission | No Mirror Maker Admin permission. | | Incorrect MirrorMaker name that does not exist | MirrorMaker <MM Name> does not exist | | Incorrect payload | This is not a DeleteMirrorMaker request. Please try again. | | Incorrect JSON | Incorrect JSON object.Could not parse JSON. Please correct the JSON format and try again. | | If Mirror Maker agent is down or it takes more than (right now it is configured 15 second) | listMirrorMaker is not available, please make sure MirrorMakerAgent is running. |
|
WhiteList
| Expand |
|---|
|
POST /mirrormakers/createwhitelist This API is used to add a new topic to the whitelist. AAF permissions required: org.onap.dmaap.mr.mirrormaker|*|user , org.onap.dmaap.mr.topicFactory|:org.onap.dmaap.mr.topic:org.onap.dmaap.mr|create Request Payload:| Parameter Name | Req'd | Description |
|---|
| name | Y | Mirror Maker name | | namespace | Y | a namespace | | whitelistTopicName | Y | a topic name that contains only alpha numeric value |
{
"name":"centraltoedge",
"namespace":"org.onap.dmaap.mr",
"whitelistTopicName":"org.onap.dmaap.mr.mrtesttopic2"
}
Response Payload:{
"listMirrorMaker": [
{
"name": "centraltoedge",
"producer": "10.12.5.108:30491",
"whitelist": "org.onap.dmaap.mr.mrtesttopic,org.onap.dmaap.mr.mrtesttopic2",
"enablelogCheck": true,
"consumer": "10.12.6.98:30491",
"status": "RESTARTING",
"numStreams": 3
}
],
}
Error Messages:| Error Reason | Error Message |
|---|
| No AAF Mirror Maker User permission | No Mirror Maker User permission. | | No AAF Mirror Maker User Create permission (User should have create topics permission ont he namespace listed) | No Mirror Maker User Create permission. | | Incorrect JSON | Incorrect JSON object.Could not parse JSON. Please correct the JSON format and try again. | | Request contain topic that already exist | The topic already exist. | | If the namespace in the topic does not match with the namespace in the request | The namespace of the topic does not match with the namespace you provided. | | If Mirror Maker agent is down or it takes more than (right now it is configured 15 second) | listWhiteList is not available, please make sure MirrorMakerAgent is running | | Incorrect payload | This is not a createWhitelist request. Please try again. |
|
...
| Expand |
|---|
|
POST /mirrormakers/deletewhitelist This API removes a topic from whitelist. AAF permissions required: org.onap.dmaap.mr.mirrormaker|*|user , org.onap.dmaap.mr.topicFactory|:org.onap.dmaap.mr.topic:org.onap.dmaap.mr|create Request Payload:| Parameter Name | Req'd | Description |
|---|
| name | Y | Mirror Maker name | | namespace | Y | a namespace | | whitelistTopicName | Y | a topic name that contains only alpha numeric value |
{
"name":"centraltoedge",
"namespace":"org.onap.dmaap.mr",
"whitelistTopicName":"org.onap.dmaap.mr.mrtesttopic2"
} Response Payload:{
"listMirrorMaker": [
{
"name": "centraltoedge",
"producer": "10.12.5.108:30491",
"whitelist": "org.onap.dmaap.mr.mrtesttopic",
"enablelogCheck": true,
"consumer": "10.12.6.98:30491",
"status": "RESTARTING",
"numStreams": 3
}
],
}
Error Messages:| Error Reason | Error Message |
|---|
| No AAF Mirror Maker User permission | No Mirror Maker User permission. | | No AAF Mirror Maker User Create permission | No Mirror Maker User Create permission. | | Incorrect JSON | Incorrect JSON object.Could not parse JSON. Please correct the JSON format and try again. | | Request contain topic does not exist | The topic does not exist. | | If Mirror Maker agent is down or it takes more than (right now it is configured 15 second) | listWhiteList is not available, please make sure MirrorMakerAgent is running | | Incorrect payload | This is not a DeleteAllWhitelist request. Please try again. |
|
Recommendations:
1) Topic partitions must be same in both producer and consumer Kafka clusters
2) For better performance numStreams should match to the topic partitions