We will create a POC for Dublin release to prove the concept and usefulness of this project.
POC Scope
- DataLake Feeder delivery as service under DCAE (dcaegen2/services/dl-handler/dl-feeder)
- DLFeeder will support pluggable sources and datastore support
- DMAAP/Kafka as primary source (option to poll kafka broker for new topic?)
- Druid/Couchbase/Elasticsearch/Hadoop datastore support
- ConfigBingingService integration to fetch deployment/run-time configuration
- DLAdmin delivery as service under DCAE (dcaegen2/services/dl-handler/dl-admin). Will integrate the DL Admin UI into DCAE dashboard, which is built on portal SDK and serves as UAM for DCAE users/operation.
- Integrate DLAdmin into DCAE-Dashboard (See note below)
Other extension cases (stretch goal)
- RDMS datastore support
- Support other input data stream beside Dmaap/kafka
- Integrate with PNDA/datastore
- DCAE Platform integration for dynamic topic configuration
Deliverable
- DL Feeder container
- DL Admin GUI container
- Documentation (wiki and readme files) to setup/configure above
- Instruction for Test storage and UI setup and DL configuration management to demo data flow and visualization
Resources (To be updated)
China Mobile:
- Guobiao Mo
- Tao Shen
- Binbin Tian
- Man Liu
QCT
- Karl Chiang
- Kate Hsuan
- Ekko Chang
IBM
Driptaroop Das
