Project Name: Distributed KV Store for configuration settings
Project description:
Requirements:
- Currently, the configuration properties are stored in files. Each ONAP project maintains its settings in its own configuration files. Some of the challenges & requirements are :
- Any update to the configuration settings require restart of the service. This could lead to ONAP service unavailability for some time. Requirement is that any configuration update should not lead to restart of ONAP services.
- There is good amount of duplication information among the files in various ONAP projects. Due to this, any change in this common information requires updating multiple files, which is error prone.
- In future releases, ONAP is going to bring up services with multiple instances for high availability. Any change in the configuration should be reflected in all service instances and also the configuration should be consistent in all instances.
- Configuration store itself should be distributed for high availability of configuration data.
- Configuration change should be easier for administrator - GUI facility to update the configuration.
- When there is a cluster of service instances, there can be a need for one of the instances to become a leader. That is, leader election could be a requirement for services.
- Yet times, some operations need to be atomically performed and hence there could be a requirement for distributed lock.
Distributed KV Store project is expected to address above requirements. It is expected to provide following functionality
- Centralization of configuration
- Common configuration settings without duplication
- Configuration update without the need for restarting services
- GUI/CLI to update the configuration settings
- Auto notification of services when there is a configuration change.
- Reliable and distributed data storage.
- Leadership selection
- Distributed lock
Pictorial representation of Distributed KV Store with consuming services:
API Reference for the KV store:
Scope:
Functionality :
- Ability to store configuration settings using reliable and distributed KV storage systems.
- Ability to provide leadership election among micro services.
- Ability to provide semaphore across service instances of ONAP services.
- Ability to modify the configuration settings
- Ability to inform active ONAP services of configuration changes.
- Ability to read the changed configuration
- config-seed micro service to load default values of configuration settings, if settings are not present in the distributed KV storage.
- Bootstrap phase functionality where each service upon startup reads the configuration settings from distribtued KV storage.
Interface/API:
- RESTful API to set/read/update the configuration settings, leadership election and take/release distributed lock.
- Long poll for near instant configuration notifications.
- Client packages:
- For configuration store, both Java and Python have abstraction alrady over consul and hence there is no special service-level package is required.
- Python and Java based client package to simplify leadership election and distirbuted locks. This package wraps various operations required to elect the leader. Also, this client package abstracts various operations required to create lock, take and release the lock.
Architecture Alignment:
- How does this project fit into the rest of the ONAP Architecture?
- Configuration-seed micro service is expcted to be a common service.
- consul from MSB can be used as Distributed key store
- Boot strap phase is expected to be part of each service
- What other ONAP projects does this project depend on?
- It does not depend on any existing ONAP projects.
- Are there dependencies with other open source projects?
- Consul (http://consul.io). Reasons for choosing consule:
- Well maintained.
- Used by many projects (e.g EdgexFoundry, vault project)
- Good support for Java libraries (For Java based services, cfg4j is normally used to read the configuration. cfg4j has consul backend, python-envconsul package is useful for python based services)
- It is alredy being used by ONAP (as part of MSB) as service-registry and service-discovery.
- Consul (http://consul.io). Reasons for choosing consule:
Other Information:
- Distributed KV Store : https://github.com/hashicorp/consul
- configuration-seed : https://github.com/edgexfoundry/core-config-seed (But suggest to implement this in golang, ideas can be taken from this project).
Link to tests:
https://git.onap.org/music/distributed-kv-store/tree/src/dkv/api/endpointViews_test.go
Key Project Facts:
Primary contact : Srinivasa Addepalli, Shashank Kumar Shankar
Facts | Info |
|---|---|
| PTL (first and last name) | |
| Jira Project Name | |
| Jira Key | |
| Project ID | |
| Link to Wiki Space |
Release Components Name:
Note: refer to existing project for details on how to fill out this table
Components Name | Components Repository name | Maven Group ID | Components Description |
|---|---|---|---|
distributed-kv-store | music/distributed-kv-store | org.onap.music.distributed-kv-store | Source code to read initial configuration date into KV Store. |
Resources committed to the Release:
Note 1: No more than 5 committers per project. Balance the committers list and avoid members representing only one company.
Note 2: It is critical to complete all the information requested, that will help to fast forward the onboarding process.
Role | First Name Last Name | Linux Foundation ID | Email Address | Location |
|---|---|---|---|---|
| PTL | ||||
| Committers | Shashank Kumar Shankar | shashank.kumar.shankar | shashank.kumar.shankar@intel.com | Portland, OR |
| Contributors | ||||
Distributed KV Store Functionality Freeze Checklist:
| Practice Area | Checkpoint | Yes/No | Evidence - Comment | How to? |
|---|---|---|---|---|
| Product Management | Are all provisional APIs interface (stub) been defined (at beta-quality level)? | Yes | https://git.onap.org/music/distributed-kv-store/tree/swagger.json | |
| Is there a final list of externally consumable APIs available? | Yes | https://git.onap.org/music/distributed-kv-store/tree/swagger.json | ||
For all completed Sprints, have Sprint Backlog Stories been marked as "Done" in Jira? | Yes | |||
Are all tasks associated with the completed Sprint Backlog Stories been marked as "Done" in Jira? | Yes | https://jira.onap.org/secure/RapidBoard.jspa?rapidView=113&view=detail&selectedIssue=MUSIC-9&quickFilter=273 | ||
| If applicable to your project, has your team been able to clear the project' blockers? If not provide status on the plan to close the blocker(s). | Yes | https://jenkins.onap.org/view/music/job/music-distributed-kv-store-master-verify-golang/ | ||
| What new features or changes to existing features in this project scope need to be communicated to VNF Providers? List the changes in the Evidence tab. | NA | |||
| If yes to the previous question, have these been communicated to the VNF Requirements project? | NA | |||
| Release Management | Have all source code files been updated with License Apache 2 header? | Yes | Specific rules and instruction are available in ONAP wiki. | |
| Has the year format in copyright header of all source code files been updated? (Rules for new files created in 2018 and existing files modified in 2018 are different) | NA | Guidance on year format | ||
| In case source code can't be edited, has a "License.txt" file been placed at the root directory for which the license is applicable? | NA | Guidance for source code file that can't be edited | ||
| (a) Has the Project Team added appropriate license and copyright notices to all ONAP source code and documentation files, where possible for the particular file format? | Yes | |||
| (b) Has the Project Team reviewed and understood the most recent license scan reports from the LF, for both (a) licenses within the codebase and (b) licenses for third-party build time dependencies? | Yes | |||
| For both (a) and (b) questions, have all high priority non-Project Licenses been either removed, planned for removal before code freeze, or escalated as likely exception requests? | Yes | |||
Have all API projects dependencies been captured? | Yes | https://git.onap.org/music/distributed-kv-store/tree/swagger.json | ||
| Development | For new projects approved for this release, have all defined repositories source code been placed into Gerrit? | Yes | https://gerrit.onap.org/r/gitweb?p=music/distributed-kv-store.git;a=summary | For evidences, provide link(s) to Gerrit repos by providing the URL as shown in this example |
| Has the project team reach the Automated Unit Test Code Coverage expectation? (Refer to artifacts available in Sonar) | NA | Not applicatible to Golang projects. | ||
| Is there any binaries (jar, war, tar, gz, gzip, zip files) in Gerrit project repository? | No | Refer to CI Development Best Practices | ||
| Could you ensure that all proprietary trademarks, logos, product names, company name, etc. have been removed? All ONAP deliverables must comply with this rule and be agnostic of any proprietary symbols. | Yes | |||
| Is there any pending commit request older than 36 business hours in Gerrit? | No | |||
| Have all the Jenkins jobs successfully passed (Merge-Jobs)? | Yes | https://jenkins.onap.org/view/music/job/music-distributed-kv-store-master-verify-golang/ | ||
| Are all snapshot binaries available in Nexus? | In Progress | |||
| Integration and Testing | Have functional test cases been documented in wiki? | Yes | https://wiki.onap.org/display/DW/Distributed+KV+Store#DistributedKVStore-Linktotests: | |
| Have you implemented in Jenkins at least 1 functional test case for each of the project repository? | Yes | https://git.onap.org/music/distributed-kv-store/tree/src/dkv/api/endpointViews_test.go | As an example (provided by Integration Team) | |
| Has the project code successfully passed the Build process? | Yes | https://jenkins.onap.org/view/music/job/music-distributed-kv-store-master-merge-golang/ | Goal is to ensure your project latest commits have not broken the build. | |
| Documentation | Has the team identified and outlined the set of documentations to be delivered in this Release? | Yes |
Test Coverage Screenshot:
