Why does my redis cache only return partial data

php Editor Banana When using redis cache, you may encounter a common problem: Why does my redis cache only return part of the data? This problem may have multiple causes, including improper cache settings, cache key name conflicts, cache expiration, etc. Before solving this problem, we need to carefully check the code and configuration to ensure that the cache is functioning properly. This article will analyze the possible causes for you and provide solutions to help you solve this problem and ensure that the redis cache returns complete data.

Question content

This function checks the implementation of postgres and redis cache When I make a get request, the first result returns all the data, but when I make the next result, some fields of the data are missing

func (usr *UserImplementation) GetAllUsers(ctx context.Context) ([]models.User, error) {

    cachedUsers, err := databaseapi.Redis_CacheDB_Api()
    if err != nil {
        return nil, fmt.Errorf("error connecting to Redis cache: %v", err)
    }

    // pipe := cachedUsers.TxPipeline()

    cachedData, err := cachedUsers.Get(ctx, "users").Result()
    if err != nil && err != redis.Nil {
        return nil, fmt.Errorf("error retrieving cached data: %v", err)
    }

    if cachedData != "" {
        // Data found in the cache, return the cached data
        var cachedUsers []models.User
        err := json.Unmarshal([]byte(cachedData), &cachedUsers)
        if err != nil {
            return nil, fmt.Errorf("error unmarshaling cached data: %v", err)
        }
        return cachedUsers, nil
    }

    users, err := usr.pg.Postgres_DB_Api().DB.GetAllUsers(ctx)
    if err != nil {
        return nil, fmt.Errorf("error: %v", err.Error())
    }

    cacheData, err := json.Marshal(users)
    if err != nil {
        return nil, fmt.Errorf("error marshaling data for caching: %v", err)
    }

    expiration := time.Hour

    err = cachedUsers.Set(ctx, "users", string(cacheData), expiration).Err()
    if err != nil {
        return nil, fmt.Errorf("error caching data: %v", err)
    }

    return models.DatabaseUsersToUsers(users), nil

}

This is my user structure: type user struct { id uuid.uuid json:"id" Name string json:"first_name" Last name string json:"last_name" Other name string json:"other_name" Username string json:"user_name" Password string json:"password" Email string json:"email" Profile image string json:"profile_image" Status string json:"status" isadmin bool json:"is_admin" Role string json:"role" Gender string json:"Gender" Phone number string json:"phone_number" createdat time.Time json:"created_at" updatedat time.Time json:"updated_at" }

Solution

I put together a small example to try and help you. As a premise, I've simplified your example just to provide what's important here. Since some parts of the program haven't been shared yet, I have to make some guesses. If this example doesn't work, please let me know what's missing and I'll update my answer. Let's start with my command to run postgres/redis locally using docker.

set up

The command I used is:

1. docker run -d -p 54322:5432 -e postgres_password=postgres postgres
2. docker run -d --name redis-stack -p 6379:6379 -p 8001:8001 redis/redis-stack:latest

Now, let’s switch to code.

program

package main

import (
    "context"
    "encoding/json"
    "fmt"
    "time"

    "github.com/redis/go-redis/v9"
    "gorm.io/driver/postgres"
    "gorm.io/gorm"
)

type user struct {
    ID        int    `json:"id"`
    FirstName string `json:"first_name"`
    LastName  string `json:"last_name"`
}

func main() {
    // 1. instantiate clients
    dsn := "host=localhost port=54322 user=postgres password=postgres"
    db, err := gorm.Open(postgres.Open(dsn))
    if err != nil {
        panic(err)
    }
    redisClient := redis.NewClient(&redis.Options{
        Addr:     ":6379",
        Password: "",
        DB:       0,
    })

    // 2. automigrate objects & seed dummy data
    db.AutoMigrate(&user{})
    db.Create(&user{ID: 1, FirstName: "John", LastName: "Doe"})
    db.Create(&user{ID: 2, FirstName: "Suzy", LastName: "White"})

    // 3. attempt to retrieve from cache
    var users []user
    cachedUsers, err := redisClient.Get(context.Background(), "users").Result()
    if err != nil && err != redis.Nil {
        panic(fmt.Errorf("err retrieving cached data: %v", err))
    }
    if cachedUsers != "" {
        if err := json.Unmarshal([]byte(cachedUsers), &users); err != nil {
            panic(fmt.Errorf("err while unmarshaling data: %v", err))
        }
        fmt.Println("users taken from Redis")
        for _, v := range users {
            fmt.Println(v)
        }
        return
    }

    // 4. retrieve from the DB
    if err := db.Model(&user{}).Find(&users).Error; err != nil {
        panic(fmt.Errorf("err while retrieving from DB: %v", err))
    }

    // 5. set the key within the cache
    rawUsers, err := json.Marshal(users)
    if err != nil {
        panic(fmt.Errorf("err while marshaling users: %v", err))
    }
    if err := redisClient.Set(context.Background(), "users", rawUsers, time.Minute*15).Err(); err != nil {
        panic(fmt.Errorf("err while setting key in cache: %v", err))
    }
    fmt.Println("users taken from DB")
    for _, v := range users {
        fmt.Println(v)
    }
}

Let’s take a closer look at each section (divided by numbered comments):

1. There is not much to say in client initialization. We initialized the client pointing to the local instance
2. Then, we set up the database with some dummy data
3. We try to get data from the redis instance. If they are found, we print them out and terminate the program
4. If the data is not found in the cache, then we get them from the database
5. Finally, if we are getting data from the database, it is safe to assume that we should also put them in the cache

In both cases, we use the same fields to get the same amount of data. So if you stick with this example and adapt it to your model and project type (mine wasn't a web project) you should be fine. If you still have problems please let me know. Thanks!

The above is the detailed content of Why does my redis cache only return partial data. For more information, please follow other related articles on the PHP Chinese website!