The goal of this DCAE project is to provide the PNDA platform as a deployment option that delivers a big-data analytics platform as part of DCAE.
Overview
Overview presentation of DCAE-PNDA-Overview.pdf.
High level summary of tasks:
Installation of PNDA within DCAE:
|
Health-Check (PNDA to integrate with DCAE health check):
|
Enable Application Deployment on PNDA via DCAE:
|
Data Integration (Enable PNDA to receive data from DCAE collectors like VES, etc.)
|
Applications for PNDA
|
Release related information
Casablanca M3 Milestone for PNDA integration into DCAE:
- Support for HDFS API,
- VES data available in HDFS
- Support for Spark Streaming API
- Support for Spark Batch API
- Jupyter Notebook
PNDA 5.0 Components versions
The source of truth regarding versions is available in the PNDA 5.0 release note.
| Component | Version | |
|---|---|---|
| Kafka | 1.1.0 | |
| Kafka Manager | 1.3.3.15 | |
| PNDA Deployment Manager | XXX | |
| PNDA Package Repository | XXX | |
| PNDA Console | XXX | |
| Gobblin | 0.11.0 | |
| Flink | 1.4.2 | |
| Knox | 1.1.0 | |
| HortonWorks | 2.6.5 | |
| Hadoop | 2.7.3 | |
| HBase | 1.1.2 | |
| Hive | 2.1.0 | |
| Spark | 1.6.3 | |
| Spark | 2.3.0 | |
| Oozie | 4.2.0 | |
| Grafana | 5.1.3 | |
| OpenTSDB | 2.3.0 | |
| Consul | 1.0.3 | |
| Jupyter | 4.2.1 | |
pnda API's
As part of the ongoing dcae integration with the pnda data platform, here are some pointers defining the provided pnda API’s:
- Platform Data Management: https://github.com/pndaproject/platform-data-mgmnt/blob/develop/data-service/README.md
- Platform Deployment Manager https://github.com/pndaproject/platform-deployment-manager#api-documentation
- Platform Package Repository https://github.com/pndaproject/platform-package-repository#repository-api
List of JIRA tickets associated with PNDA for DCAE - Casablanca
List of JIRA tickets associated with PNDA for DCAE - Backlog