We will create a POC for Dublin release to prove the concept and usefulness of this project. As DataLake infrastructure itself will be considered outside of ONAP scope, the POC will focus on delivering one of key component of DL-Feeder as service under DCAE project.

Architecture

Image Added

POC Scope

• DataLake Feeder and DLAdmin delivery as service services under DCAE  (dcaegen2/services repository)
   

  • dcaegen2/services

    └── components

        ├── dl-handler

  /

  •     │   ├── dl-admin
        │   └── dl-feeder

  )

• DLFeeder will support pluggable sources and datastore support
  • DMAAP/Kafka as primary source (option to poll kafka broker for new topic?)
  • Druid/Couchbase/Elasticsearch/Hadoop datastore support
• ConfigBingingService integration to fetch deployment/run-time configurationDLAdmin delivery as service under DCAE (dcaegen2/services/dl-handler/dl-admin).  Will integrate
• Alternate option will be to leverage k8s Configmap.  
• Integrate the DL Admin UI into DCAE dashboard, which is built on portal SDK and serves as UAM for DCAE users/operation.
• Integrate DLAdmin into DCAE-Dashboard (See note below)

...

Other extension cases (stretch goal)

• RDMS datastore support
• Support other input data stream beside Dmaap/kafka
• Integrate with PNDA/datastore
• DCAE Platform integration for dynamic topic configuration
• Logging project integration

Deliverable

• DL Feeder container
• DL Admin GUI container
• Documentation (wiki and readme files) to setup/configure above
• Instruction for Test storage and UI setup and DL configuration management to demo data flow and visualization

...

• Karl Chiang
• Kate Hsuan
• Ekko Chang
• May Lin

IBM

• Driptaroop Das

...