Content Comparison

...

• Repository: https://gerrit.onap.org/r/#/admin/projects/dcaegen2/collectors/hv-ves
• Changes: https://gerrit.onap.org/r/#/q/project:dcaegen2/collectors/hv-ves

Background

HV-VES -HV collector has been proposed, based on a need to process high-volumes of data generated frequently by a large number of NFs. The driving use-case is the 5G RAN, where it is expected that up to 10k NF instances report the data, per DCAE platform deployment. The network traffic generated in simulations - based on 4G BTS Real-Time PM data has shown, that GPB serialization is 2-3 times more effective, than JSON serialization utilized in VES collector.

Results have been published within ONAP presentation in Casablanca Release Developer Forum:  Google Protocol Buffers versus JSON - 5G RAN use-case - comparison

The goal of the collector is to support high volume data. It uses plain TCP connections tunneled in SSL/TLS. Connections are stream-based (as opposed to request-based) and long running. Payload is binary-encoded (currently we are using Google Protocol Buffers). HV-VES uses direct connection to DMaaP's Kafka. All these decisions were made in order to support high-volume data with minimal latency.

...

• Do not block. Use non-blocking libraries. Do not use block* Reactor calls inside the core of the application.
• Pay attention to memory usage.
• Do not decode the payload - it can be of a considerable size. The goal is to direct the event into a proper Kafka topic. The routing logic should be based only on VES Common Header parameters.
• All application logic should be defined in hv-collector-core module and tested on a component level by tests defined in hv-collector-ct. The core module should have a clean interface (defined in boundary package: api and adapters).
• Use Either functional data type when designing fail-cases inside the main Flux. Using exceptions is a bit like using goto + it adds some performance penalty: collecting stack trace might be costly but we do not usually need it in such cases. RuntimeExceptions should be treated as application bugs and fixed.

Stories