Is there better sharding functionality for int64 keys in golang?

Question content

I am using concurrent mapping from this repository and I can choose the key type when creating the mapping using newwithcustomshardingfunction . I just need to provide my own sharding function for the int64 key, which is what I'm using here.

I'm also using the latest version of go where I can use generics, so I decided to use concurrent-map by implementing my own sharding functionality, key is int64.

import (
    cmap "github.com/orcaman/concurrent-map/v2"
)

func shardingFunc(key int64) uint32 {
    return uint32(key) // TODO - create a better sharding function that does not rely on how uint32 type conversion works
}

func main() {
    testMap := cmap.NewWithCustomShardingFunction[int64, *definitions.CustomerProduct](shardingFunc)
    // ... use the map ...
}

I want to know if my sharding function is ok for int64 keys or should I have a better sharding function? I don't want to have a situation where I get an index out of range error or any other problem.

Correct answer

The sharding function is a hash function. This function should evenly distribute the key over the 32-bit space.

If the lower four bytes of your init64 value are evenly distributed, then uint32(key) will be used as the slicing function.

uint32(key) An example of a bad choice is when the low byte has a constant value. For example, if the key value is something like 0x00010000, 0x00020000, ..., then uint32(key) will evaluate to zero. This is not a uniform distribution.

If you don't know how the int64 key is distributed, it's better to use all bits of the key in the sharding function. Here's one using xor:

func shardingFunc(key int64) uint32 {
    return uint32(key) ^ uint32(key >> 32) 
}

The above is the detailed content of Is there better sharding functionality for int64 keys in golang?. For more information, please follow other related articles on the PHP Chinese website!