Page Comparison

Hazelcast - distributed in-memory data grid (IMDG) platform, reliable for processing and storing data efficiently.

...

• Distributes data, partitions (both primary and backup replicas) across all cluster members

  • Back up counts are ignored in a single instance setup

  • Default number of partitions 271

• Automatically creates backups at partition level which are distributed across all cluster members (back up count is configurable)

• Members have core count * 20 threads that handle requests

• Single replica of each partition is created

  • One of these replica is called primary and others are backups

  • Partition table information is sent and update across the members by default every 15 seconds, this can be configured with the ff:

    • hazelcast.partition.table.send.interval

• For maps and queues

  • one synchronous back up, zero asynchronous back up

  • Binary in memory format

Hazelcast Topologies

There are two available Hazelcast Topologies (Embedded, Client/Server).

...

Standard Collections (AP)

• Maps

  • synchronous backup

    • changes to a map waits to finish update on the primary and all other backups before operation continues

  • asynchronous backup

    • changes to a map does not wait  to finish update on the primary and all other backups before operation continues

• Queues

• Lists

• Sets

• Ringbuffers

...

• Cluster size depends on multiple factors for application

• Adding more members to the cluster

  • Capacity for CPU-bound computation rises

  • Use of Jet stream processing engine helps this

  • Adding members increases the amount of data that needs to be replicated.

  • Can improve fault tolerance or performance (if minimum recommended (1 per application instance) is not enough)

• Fault tolerance recommendation

  • At least 3 members or n+1

...

• One instance of hazelcast per application instance

  • though for fault tolerance, we might need 3 ?

• Use map.set() on maps instead of map.put() if you don’t need the old value.

  • This eliminates unnecessary deserialization of the old value.

  • map.putIfAbsent() - inserts only if key doesn't exist

  • map.set() - inserts or updates unconditionally

• Consider use of Entry processor of updating map entries in bulk (i.e. get then set)

• Changing to OBJECT In-Memory format for specific maps

  • applies entry processor directly on the object

  • useful for queries

  • No serialization/deserialization is performed

• Enabling in-memory back up reads

  • Allowing member to check its own back up instead of calling the owner of the partition or the primary copy

  • Can be useful to reduce latency and improve performance

  • Needs back ups to be synchronised

• Back Pressure mechanism in hazelcast

  • acts like a traffic controller 

  • helps prevent system overload

  • helps prevent crashes due to excessive operation invocations

  • Limits the number of concurrent operations and periodically syncs async backups

  • Needs to be configured as it is disabled by default

    • configure to be enabled

    • configure max concurrent invocations per partition

• Controlling/Configuring partitions

  • Putting data structures in one partition

    • reduces network traffic

    • improves data access performance

    • Simplifies data management

  • Using custom partition strategies based on requirements (Partitioning strategy)

• Reading Map Metrics

Backups in Hazelcast

• Hazelcast does not support backup redistribution, meaning if the back up configuration of a running cluster is changed then the existing backups are unaffected. The cluster needs to be restarted to take effect of the new configuration.

  • For example: The following cluster is configured to have the default backup count of 1, when the backup count is increased to 2, no new backups will be created. If the backup count is decreased to 0, A2 will still exist but may not be valid.

2 types of data backups

By default, hazelcast structures are configured to have 1 synchronous back up and 0 asynchronous back up.

Synchronous backups

• operations are blocked until all backups are copied

• default count is 1

• sets by “.setBackupCount(1)”

Asynchronous (async) backups

• operations proceed without waiting for backup completion

• best for performance over immediate consistency

• sets by “.setAsyncBackupCount(1)”

Different scenarios

Proposal scenario for CPS-NCMP:

Synchronous backup count = 0

Async backup count = 1

• allows one backup count to be created asynchronously, meaning it won’t block operations while the backup is being created

Map Backups

• Memory needed for map is “M + B(M)” wherein M is the memory used by primary data and B(M) is the number of backups

• Enabling In-Memory backup reads

  • allows local members to access data from their own backups than relying on the primary owner of data.

  • sets by “.setReadBackupData(true)”

Scenario with in memory backup reads enabled:

1. Local backup read attempt → 1a. Local backup contains data → 1b. Returns data

2. Local back up read attempt → 2a. Local backup does contain data → 2b. Fetches data from primary source → 2c. Returns data

Hazelcast data structures overview

• Hazelcast data structures have options for using synchronous operations or asynchronous operations.

• Synchronous operations blocks the calling thread until the operation is completed.

• Asynchronous operations allows the calling thread to continue executing while the operation is processed in the background.

Here are some of the operation examples…

IMap

IQueue

ISet

CPS-NCMP Hazelcast data structures