Overview
| Project Name | MUSIC |
|---|---|
| Target Release Name | Dublin |
| Project Lifecycle State | Incubation |
| Participating Company | AT&T, Intel, Netcracker, Verizon, Windriver (in lexical order) |
Scope
What is this release trying to address?
MUSIC was released in the ONAP Beijing release and provides a service with recipes that individual ONAP components and micro-service can use for state replication, consistency management and state ownership across geo-distributed sites. This is a crucial component enabling ONAP components to achieve S3P in terms of resiliency both within and across sites (platform-maturity resiliency level 3).
In this release we plan to provide to address the following items:
- MUSIC as a service: while MUSIC was consumed internally by components in the Beijing release, in Dublin we intend to provide MUSIC as an independent multi-site clustered service
- Enable automated failure detection and consistent failover across sites for ONAP components using MUSIC through the PROM recipe. It will require no change to the code of the ONAP components and just a few scripting/configuration steps to achieve single-step automated failover while ensuring that the new leader/owner has access to the latest state information.
- Provide the design to make MUSIC a fully sharded, scale out system, where as many ONAP sites/component replicas can be added as required for performance. The significant technical challenge is to eliminate the need for Zookeeper and build MUSIC completely based on Cassandra while preserving all its guarantees. We expect this change to improve both deployabiity (just one tool – Cassandra) and performance (initial benchmarks indicate a factor of at least 4-5 times in terms of throughput). This is a crucial precursor for its use in edge computing and as the state management service for a federated ONAP.
- Provide the design to allow MUSIC to support database (RDBMS) clustering across sites using the mdbc recipe wherein ONAP components that require it can continue using a SQL database within a site while using MUSIC is as the underlying transport layer across sites, with much better performance than standard solutions like Gallera clustering.
- Continued adherence to ONAP S3P requirements in Dublin
Use Cases
- Targeted goal for Dublin
- Stretch goal for Dublin: SDN-C will use the MUSIC PROM recipe for automated and consistent failover across sites.
Minimum Viable Product
MUSIC service that can serve the geo-redundancy needs of ONAP HAS and ONAP Portal while satisfying the platform maturity requirements for the Dublin release.
Functionalities
Epics
Stories
Longer term roadmap
In the long term we hope that MUSIC will be common, shared state-management system for all ONAP components and micro-services to manage geo-redundancy. For example, we envisage the use of MUSIC for multi-site state management in SO (to store Camunda state across sites), <SDN-C, AppC> (to store ODL related state across sites) , A&AI (to store its graph data) and most other ONAP components that need to manage state across sites. Further, we envision that these services will use the MUSIC recipes (mdbc, prom, musicCAS, musicQ) to achieve the goal of a multi-site active-active federated ONAP solution.
Release Deliverables
Indicate the outcome (Executable, Source Code, Library, API description, Tool, Documentation, Release Note...) of this release.
| Deliverable Name | Deliverable Description |
|---|---|
| Source code and REST API | The entire source code for the MUSIC service and the PROM recipe and the corresponding API to access it. |
| Compilation scripts | Script to generate the MUSIC war and jar file that can be deployed in the Apache Tomcat webserver |
| Installation guide | Document that describes how MUSIC can be installed in containers |
| Tool description | Complete description of the inner workings of MUSIC and how it performs state management |
| Basic Benchmarks | Basic performance benchmarks for the MUSIC operations |
| API documentation | REST API documentation in Swagger |
| Test Cases | Junit test cases covering sufficient parts of MUSIC code |
| Design Document for MUSIC Federation | Provide the design to make MUSIC a fully sharded, scale out system, where as many ONAP sites/component replicas can be added as required for performance. |
| Design Document for RDBMS Support | Provide the design to allow MUSIC to support database (RDBMS) clustering across sites using the mdbc recipe wherein ONAP components that require it can continue using a SQL database within a site while using MUSIC is as the underlying transport layer across sites, with much better performance than standard solutions like Gallera clustering. |
Sub-Components
NA.
Architecture
High level architecture diagram
In this figure the ONAP components targeted for R2 are ONAP HAS and Portal (stretch goal).
MUSIC will be available as a common service like DMaap or AAF as shown in the red, oblong box below:
Platform Maturity
Referring to CII Badging Security Program and Platform Maturity Requirements, fill out the table below by indicating the actual level , the targeted level for the current release and the evidences on how you plan to achieve the targeted level.
| Area | Actual Level | Targeted Level for current Release | How, Evidences | Comments |
|---|---|---|---|---|
| Performance | 1 | 1 | This file shows basic performance benchmarks performed for MUSIC on a 10 node cluster. |
|
| Stability | 1 | 1 | As shown in this file, our experimental runs were all over 1 hour. |
|
| Resiliency | 2 | 2 | Within each container we have scripts that will detect failure of MUSIC and restart it. However, if the entire container fails, we will need OOM to bring it up. |
|
| Security | 2 | 2 |
|
|
| Scalability | 1 | 1 | Among the MUSIC components [tomcat, zookeeper, cassandra], new MUSIC nodes with the tomcat and cassandra can be added seamlessly to scale the cluster (MUSIC itself is state-less). Zookeeper nodes ideally should not be scaled since there are major performance implications. However, this can be done with reconfiguration. |
|
| Manageability | 1 | 1 | Using EELF with logback as the logging provider. |
|
| Usability | 1 | 1 | Use SWAGGER for the REST API and Installation Docs. Will need to enhance and update the documentation. |
|
API Incoming Dependencies
AAF
API Outgoing Dependencies
API this project is delivering to other projects (OOF and Portal in this release)
| API Name | API Description | API Definition Date | API Delivery date | API Definition link (i.e.swagger) |
|---|---|---|---|---|
| MUSIC API | The REST API used to store state and manage access to it through a locking service. | Done. | Done. | https://gerrit.onap.org/r/#/c/31127/2/swagger.json |
| PROM API | The REST API used to acess PROM | TBD. | TBD. | TBD. |
Third Party Products Dependencies
Third Party Products mean products that are mandatory to provide services for your components. Development of new functionality in third party product may or not be expected.
List the Third Party Products (OpenStack, ODL, RabbitMQ, ElasticSearch,Crystal Reports, ...).
| Name | Description | Version |
|---|---|---|
| Cassandra | Higly-available key-value store that will maintain state. | 3.2 |
| Zookeeper | Distributed coordination service used to provide the locking service for MUSIC. | 3.4.6 |
| Tomcat | Web-server that will host the MUSIC code and support the REST API. | 9 |
Testing and Integration Plans
The following type of tests will be ensured for MUSIC in this release:
- Unit tests: Junit test cases will be added incrementally as part of code delivery ensuring static code analysis as recommended by common SW delivery practices. MUSIC is written in Java, so coverage target percentage will be evaluated along project. Unit Tests will be automated (as part of the build), so that execution will happen before every MUSIC delivery. This includes writing test cases for the new components and those for the existing ones not yet covered under unit testing. A python based Unit Testing framework will be integrated into the project, binding all developers to start with Unit Tests when developing each module.
- Functional Test cases: MUSIC will be tested according to the functionality committed to the Beijing release with a specific focus on the functionality utilised by ONAP HAS and ONAP Portal for MUSIC-based state management.
- End to End Test cases: MUSIC will be integrated within the whole ONAP components architecture and dedicated set of E2E test cases will be setup. Purpose, scope and details on E2E testing will be planned/defined subsequently by working with the Integration team, and for the test cases in the scope of this release.
Gaps
None identified so far.
Known Defects and Issues
None identified so far.
Risks
None identified so far.
Resources
Udated the Resources Committed to the Release centralized page.
Release Milestone
The milestones are defined at the Release Level and all the supporting project agreed to comply with these dates.
Team Internal Milestone
This section is optional and may be used to document internal milestones within a project team or multiple project teams. For instance, in the case the team has made agreement with other team to deliver some artifacts on a certain date that are not in the release milestone, it is erecommended to provide these agreements and dates in this section.
It is not expected to have a detailed project plan.
| Date | Project | Deliverable |
|---|---|---|
| To fill out | To fill out | To fill out |
Documentation, Training
- Highlight the team contributions to the specific document related to he project (Config guide, installation guide...).
- Highlight the team contributions to the overall Release Documentation and training asset
- High level list of documentation, training and tutorials necessary to understand the release capabilities, configuration and operation.
- Documentation includes items such as:
- Installation instructions
- Configuration instructions
- Developer guide
- End User guide
- Admin guide
- ...
Note
The Documentation project will provide the Documentation Tool Chain to edit, configure, store and publish all Documentation asset.
Other Information
Vendor Neutral
If this project is coming from an existing proprietary codebase, ensure that all proprietary trademarks, logos, product names, etc. have been removed. All ONAP deliverables must comply with this rule and be agnostic of any proprietary symbols.
Free and Open Source Software
FOSS activities are critical to the delivery of the whole ONAP initiative. The information may not be fully available at Release Planning, however to avoid late refactoring, it is critical to accomplish this task as early as possible.
List all third party Free and Open Source Software used within the release and provide License type (BSD, MIT, Apache, GNU GPL,... ).
In the case non Apache License are found inform immediately the TSC and the Release Manager and document your reasoning on why you believe we can use a non Apache version 2 license.
Each project must edit its project table available at Project FOSS.
Charter Compliance
The project team comply with the ONAP Charter.