DCAE Resource Optimization with Helm transformation
- Vijay Kumar
Introduction
DCAE Transformation was multi-release intiative to simplify DCAE architecture and align with Cloudnative function/technologies to manage DCAE services. With completion of DCAE Architecture transformation to Helm, several of platform components has been retired in Jakarta release. Following page capture the resource optmization/saving made with this feature.
Realization Path
Honolulu release (Phase 1)
REQ-479 Introduction of Helm SVC template for DCAE components; migrate bootstrap services to helm (DCAEGEN-2488); Cloudify remained primary orchestration for dynamic deployments (MOD/CLAMP flows)
Istanbul release (Phase 2)
REQ-685 Extend DCAE SVC template to provide more generic capabilities; migrate all remaining DCAE svc to helm and change default DCAE Svc deployment to Helm (instead of Cloudify)
Jakarta release (Phase 3)
REQ-1038 Integrate MOD to support helm flows; remove Cloudify & platform components from ONAP/DCAE deployments. Extend SVC template to support add-on capabilities and complete component CBS SDK migration for removing Consul (and CBS) dependency
Architecture
Deployment Optimization
With platform dependency removed completely in Jakarta release, there is significant improvement in microservice deployment. DCAE component dependencies are identified within respective DCAE charts and install of DCAE bootstrap can now be completed under 3 min (for locally available images)
Istanbul: Platform component/Handlers required to be up & running prior to DCAE Services component deployment
Jakarta: MS deployment are uniquely handled and controlled via independent charts
Resource Optimization
Below table captures resource saving with DCAE transformation completed in Jakarta release
Deployment | Istanbul | Jakarta | Overall Resource Savings | ||
|---|---|---|---|---|---|
CPU (milliunits) | Memory (mebibytes) | CPU (milliunits) | Memory (mebibytes) | ||
DCAE Bootstrap | ~48 | ~3300 | - | - | CPU utilization reduction 100% Memory utilization reduction 100% |
DCAE Bootstrap + DCAE Microservices | ~60 | ~4500 | ~15 | ~1470 | CPU utilization reduced by ~75% Memory utilization reduced by ~73% |
DCAE Bootstrap + DCAE Microservices + DCAE MOD | ~85 | ~6700 | ~40 | ~3500 | CPU utilization reduced by ~63% Memory utilization reduced by ~53% |
(kubectl top stats below)
DCAE Jakarta release deployment can be done on single node/cluster (8 core and 16GB) along with other ONAP component dependencies (e.g DMAAP, AAF)
Stats Report
UNH Lab
NAME | CPU (CORES) | MEMORY (bytes) |
dev-dcae-hv-ves-collector-779c4b87-phz5r | 1m | 218Mi |
dev-dcae-mongo-0 | 10m | 73Mi |
dev-dcae-ms-healthcheck-646697d498-l6r6j | 1m | 16Mi |
dev-dcae-prh-85ddc47c75-vrt4p | 1m | 371Mi |
dev-dcae-tcagen2-6db79b7869-f82cv | 1m | 549Mi |
dev-dcae-ves-collector-5bd78756d5-jfvcn | 1m | 243Mi |
dev-dcaemod-db-primary-65c8645fbd-mg28l | 2m | 69Mi |
dev-dcaemod-db-replica-84874765bd-nmcnj | 1m | 32Mi |
dev-dcaemod-designtool-6678fcd64f-fps2f | 12m | 1241Mi |
dev-dcaemod-distributor-api-9f4df985c-nhm5h | 1m | 25Mi |
dev-dcaemod-genprocessor-7c6955779c-zfcz9 | 1m | 142Mi |
dev-dcaemod-healthcheck-54d7f78f5d-dmb29 | 1m | 15Mi |
dev-dcaemod-nifi-registry-74f678df8-27kbg | 2m | 486Mi |
dev-dcaemod-onboarding-api-6d8ff449f4-x5zr2 | 1m | 46Mi |
dev-dcaemod-runtime-api-654d5b7b48-sf6vr | 1m | 221Mi |
37 | 3747 | |
dev-dcae-bootstrap-6b8f8f6446-z5jj9 | 0m | 9Mi |
dev-dcae-cloudify-manager-6846d9546c-xz242 | 33m | 1849Mi |
dev-dcae-config-binding-service-656b67b67d-88fb2 | 2m | 84Mi |
dev-dcae-dashboard-ff89ddcd4-sf7bg | 1m | 502Mi |
dev-dcae-dashboard-pg-primary-7bfdfc95c7-lfcmh | 1m | 66Mi |
dev-dcae-dashboard-pg-replica-b5947dc8f-bwsjb | 1m | 39Mi |
dev-dcae-db-primary-677847f688-q44kl | 1m | 46Mi |
dev-dcae-db-replica-7479f9f568-cltpc | 1m | 38Mi |
dev-dcae-deployment-handler-56cb87f596-ct5lk | 1m | 27Mi |
dev-dcae-healthcheck-6b78f9f9d7-jfwrf | 1m | 16Mi |
dev-dcae-hv-ves-collector-779c4b87-phz5r | 2m | 218Mi |
dev-dcae-inv-pg-primary-869dfdb946-gcm9x | 1m | 55Mi |
dev-dcae-inv-pg-replica-54685968db-8d7fm | 1m | 43Mi |
dev-dcae-inventory-api-6c8cf94f57-bfpzq | 2m | 258Mi |
dev-dcae-mongo-0 | 9m | 73Mi |
dev-dcae-ms-healthcheck-646697d498-l6r6j | 1m | 16Mi |
dev-dcae-prh-85ddc47c75-vrt4p | 1m | 371Mi |
dev-dcae-tcagen2-6db79b7869-f82cv | 1m | 549Mi |
dev-dcae-ves-collector-5bd78756d5-jfvcn | 1m | 243Mi |
dev-dcaemod-db-primary-65c8645fbd-mg28l | 2m | 73Mi |
dev-dcaemod-db-replica-84874765bd-nmcnj | 1m | 32Mi |
dev-dcaemod-designtool-6678fcd64f-fps2f | 13m | 1241Mi |
dev-dcaemod-distributor-api-9f4df985c-nhm5h | 1m | 25Mi |
dev-dcaemod-genprocessor-7c6955779c-zfcz9 | 1m | 142Mi |
dev-dcaemod-healthcheck-54d7f78f5d-dmb29 | 1m | 15Mi |
dev-dcaemod-nifi-registry-74f678df8-27kbg | 2m | 486Mi |
dev-dcaemod-onboarding-api-6d8ff449f4-x5zr2 | 2m | 46Mi |
dev-dcaemod-runtime-api-654d5b7b48-sf6vr | 1m | 221Mi |
85 | 6783 |
Windriver Lab
NAME | CPU (CORES) | MEMORY (bytes) |
dev-dcae-config-binding-service-58b88c78b-pv5wl | 6m | 90Mi |
dev-dcae-healthcheck-5f5584d777-x4kxd | 1m | 15Mi |
dev-dcaemod-db-primary-844dc85bf8-krjl9 | 15m | 62Mi |
dev-dcaemod-db-replica-6bd7445778-5gh6r | 2m | 34Mi |
dev-dcaemod-designtool-887b6c995-qb6rh | 23m | 1005Mi |
dev-dcaemod-distributor-api-fd8b49589-p6fh5 | 1m | 24Mi |
dev-dcaemod-genprocessor-5586f8645c-4g9fp | 3m | 165Mi |
dev-dcaemod-healthcheck-688bd8cff6-gcsg9 | 1m | 15Mi |
dev-dcaemod-nifi-registry-5789ccdc74-kbl7c | 2m | 500Mi |
dev-dcaemod-onboarding-api-bd54688c8-m4whc | 6m | 47Mi |
dev-dcaemod-runtime-api-757bbb6489-cxfnq | 1m | 264Mi |
61 | 2221 | |
dev-dcae-bootstrap-5c46b685b5-g8x8m | 0m | 41Mi |
dev-dcae-cloudify-manager-766f86ff88-kzbgc | 129m | 2540Mi |
dev-dcae-config-binding-service-58b88c78b-hkkvd | 4m | 79Mi |
dev-dcae-dashboard-7b4bdf4d4c-7sd6q | 158m | 354Mi |
dev-dcae-dashboard-pg-primary-7668779cf5-9kw4l | 1m | 50Mi |
dev-dcae-dashboard-pg-replica-57bbd47f55-4sgfx | 2m | 32Mi |
dev-dcae-db-primary-7b698bfc66-4zfzs | 1m | 50Mi |
dev-dcae-db-replica-c669b8446-qbvjd | 3m | 28Mi |
dev-dcae-deployment-handler-5c967554bc-8h2sw | 1m | 28Mi |
dev-dcae-healthcheck-5f5584d777-2pls7 | 1m | 8Mi |
dev-dcae-inv-pg-primary-7d95b5499b-m9d2l | 1m | 50Mi |
dev-dcae-inv-pg-replica-57788fcb6d-cgh7h | 1m | 33Mi |
dev-dcae-inventory-api-648db5bfbb-n2ldv | 1m | 213Mi |
dev-dcaemod-db-primary-844dc85bf8-krjl9 | 21m | 74Mi |
dev-dcaemod-db-replica-6bd7445778-5gh6r | 2m | 34Mi |
dev-dcaemod-designtool-887b6c995-qb6rh | 22m | 1005Mi |
dev-dcaemod-distributor-api-fd8b49589-p6fh5 | 1m | 24Mi |
dev-dcaemod-genprocessor-5586f8645c-4g9fp | 2m | 165Mi |
dev-dcaemod-healthcheck-688bd8cff6-gcsg9 | 1m | 16Mi |
dev-dcaemod-nifi-registry-5789ccdc74-kbl7c | 2m | 501Mi |
dev-dcaemod-onboarding-api-bd54688c8-m4whc | 13m | 47Mi |
dev-dcaemod-runtime-api-757bbb6489-cxfnq | 1m | 264Mi |
368 | 5636 |