Example sharing of js development database

The front-end often needs to save some data, and the saving here refers to long-term storage. The previous idea was to save the data in the cookie, or save the key in the cookie, and save other data on the server. These scenes have many uses and are very mature and easy to use. But I still want data that can be stored locally for a long time, similar to a database or web sql.

The new generation of browsers basically support local databases. Just use it directly when you need it. If it doesn't work, you can still use Storage to make do.

What if I need the functionality of a database and there is no compatible storage method? Do I need to encrypt and store these things locally? I want to save a lot of things?

Currently I did encounter this situation when using ReactNative. I need to store a lot of data locally. Someone said, wouldn't it be better to use SQLite directly?

Okay, absolutely fine. I am just developing it from a front-end attitude here. What if some students don’t need SQLite, but just need a simple way to store a large amount of data?

There may be many usage scenarios, the bottom layer of the implementation can be replaced at will, and even the implementation method can be written at will. Here I am building an informal, front-end-friendly data repository based on the attitude of the front-end creating the world.

Underlying Storage

The usage scenario here is ReactNative, so I use RN's AsyncStorage.

Convert all data to be saved into objects and into strings. The core idea here is serialization. Store all data as strings.

import { AsyncStorage } from 'react-native';

exports.setItem = async (key, value) => {    let item = JSON.stringify({
        v: value
    });    return await AsyncStorage.setItem(key, item);
}

When reading, a conversion is also required to convert the string into the originally stored data.

exports.getItem = async (key) => {    let item = await AsyncStorage.getItem(key);    if (!item) {        return null;
    }    return JSON.parse(item).v || null;
}

What needs special treatment is the acquisition of the list. RN has an API that returns multiple pieces of data based on multiple keys. It returns an array object. Array number 0 is the key value of data storage, and sequence number 1 is the specific string of data storage.

exports.getlist = async (keys) => {    let list = await AsyncStorage.multiGet(keys);
    list = list.map(item => JSON.parse(item[1]).v || null);    return list;
}

Let’s also take out a few other methods used. Nest one more layer here, keeping the same formation as above.

exports.removeItem = async (key) => await AsyncStorage.removeItem(key);

exports.removeItems = async (keys) => await AsyncStorage.multiRemove(keys);

exports.allKeys = async () => await AsyncStorage.getAllKeys();

Underlying Optimization

The above is just a simple implementation. If there are no special requirements, it will be almost the same. However, those who want to go further can consider starting to optimize.

For example, optimize the speed of json conversion. When we use the JSON object method to convert, there is actually a process of determining the numerical type. If we define the data type in advance. There is no need to judge again when converting.

You can define a model object and pre-define the fields required by this table. You can check how Sequelize is defined. It is very simple to do this according to the way of relational row database.

//用户对象const users = db.define('t_users', {
    id: {
        type: Sequelize.INTEGER,
        primaryKey: true,
    },    //用户名
    username: {
        type: Sequelize.STRING
    },    //密码
    pwd: {
        type: Sequelize.STRING
    },    //状态
    status: {
        type: Sequelize.INTEGER
    },    //昵称
    nickname: {
        type: Sequelize.STRING
    },    //token
    token: {
        type: Sequelize.STRING
    },
    create_time: {
        type: Sequelize.TIME
    }
}, {
    freezeTableName: true,
    timestamps: false,
});

Implementing storage

Let’s refer to the implementation of relational database here.

First you need to divide the tables and databases. In this way, you can pay less attention to this information when storing data, and focus on data operations.

constructor(tableName = "table", db = "db") {        //检查库,表是否存在
        //初始化索引表
        this.db = db;        this.tableName = tableName;        this.tableKey = db + "_" + tableName;        this.init();
    }

Store them separately inside the current object. When creating the object, you can create different operation methods based on different libraries and tables. Class is used here, and each table corresponds to a separate object.

Since we are using the storage method provided by RN, the addition and update here are actually the same method. When adding, a unique ID will be created based on the current timestamp, and this ID will be used as the key to store it in the database. Therefore, there is no need to store the ID separately when using it. However, if you feel that the ID is different from what you need, you can also define an ID yourself to store the key value.

//添加和更新
    async add(data = {}) {        if (data.constructor !== Object) return;        if (!data._id)data._id = uuid();
        await setItem(this.tableKey + "_" + data._id, data);        return data;
    }

A separate method of obtaining based on ID is provided when obtaining. What is considered here is that it is very simple and convenient to obtain by ID. Some data can be read quickly, and there is no need to query line by line.

/**
 * 通过id查询
 * @param {*} id 
 */async getById(id) {    if (!id) return {};    return await getItem(this.tableKey + "_" + id);
}

Compared to querying based on id, fuzzy query is indeed very slow. If it is not really needed, it is better not to use this fuzzy query. Here is a custom query method that can determine whether this row of data is needed based on the returned object. You can also add the top parameter to limit the number returned. Using this parameter can also improve performance when there is a lot of data.

/**
 * 通过过滤方法查询
 * @param {*} fn 
 */async get(fn, top = 0) {    let keys = await allKeys();    if (keys.length == 0) return [];    if (top > 0 && keys.length > top) keys.length = top;    const listkey = keys.filter(item => item.indexOf(this.tableKey + "_") === 0);    if (listkey.length == 0) return [];    let list = await getlist(listkey);
    list = list.filter(item => fn(item));    return list;
}

Finally, add the deletion and clearing methods, and a simple deletion library is completed.

 /**
 * 删除
 * @param {*} id 
 */async delete(id) {    if (!id) return {};
    await removeItem(this.tableKey + "_" + id);
}/**
 * 清空表
 */async clear() {    let keys = await allKeys();    const listkey = keys.filter(item => item.indexOf(this.tableKey + "_") === 0);    if (listkey.length == 0) return;
    removeItems(listkey);
}

When using it, you only need to create the object and then call it where needed. It is simple and convenient to use, and after optimization, it can even be used as client-side redis.

//初始化数据库let db=new JSDB();//添加一个自定义数据db.add({name:"test",key:"abc"});//根据id获取数据db.getById("1223467890");//根据条件查询数据db.get(d=>d.name==="test");//删除对应的数据db.delete("1223467890");//情况所有数据db.clear()

Optimize creation

The first thing to optimize is the creation of objects. Each object creation actually consumes a lot of money. Wouldn't it be great if this consumption could be reduced?

Here we draw on the concept of database pool and implement an object pool method. There is no direct return after the object is created, it has to go through the pool operation.

Put the object into the pool and reset it to an empty object when the page is destroyed. You don't have to create a new one the next time you request one. You can use it by directly assigning the name of the table or library. The memory is unchanged and makes me laugh a bit.

Optimizing Query

It is still very troublesome to read Stroage every time you query, especially this operation is an asynchronous operation and requires sending a message to the native side.

We can store the last read data in a variable. If we need to use this line of data next time, there is no need to read it again. This can easily provide reading speed.

This method can continue to be optimized. Limit the amount of data saved in variables to prevent the amount from exceeding the APP limit. You can also make a logical judgment on the storage time limit, put frequently used data in it, and find opportunities to delete data that are not commonly used.

Using this method can also optimize the effectiveness of data in variables and reduce the memory size occupied by variables. However, try not to use a timer in the implementation method. You can consider using a trigger type. The deletion action is triggered when the conditions are met.

Optimize writing

As mentioned above, variables need to be put in when reading to increase the reading speed. By the way, we thought, can the writing speed also be improved?

We put the data to be stored in a temporary variable. If the time we set or the data length is checked and the amount we set is checked, a save operation will be triggered.

It should be noted here that the variables used to save data and the variables used when saving should be used interchangeably to prevent data loss.

For example: use variable 1 when saving. Before writing to the database, change the object to be saved to variable 2, then read the data of variable 1 and store it in the database. This is double buffered writing.

Of course, you still need to determine the exit event of the APP. If the APP exits, please be sure to save the content in the variable once, otherwise all your hard work will be lost.

---

Okay, a simple database is completed. If you want to use it, you can first search for the react-native-jsdb library on npm. I have put the first part of the implementation on npm, and subsequent optimizations will be fully open sourced.

npm warehouse address

Article address

The front-end still needs to save some data in many cases, and the storage here refers to long-term storage. The previous idea was to save the data in the cookie, or save the key in the cookie, and save other data on the server.

These scenes have many uses and are very mature and easy to use. But I still want data that can be stored locally for a long time, similar to a database or web sql.

The new generation of browsers basically support local databases. Just use it directly when you need it. If it doesn't work, you can still use Storage to make do.

What if I need the functionality of a database and there is no compatible storage method? Do I need to encrypt and store these things locally? I want to save a lot of things?

Currently I did encounter this situation when using ReactNative. I need to store a lot of data locally. Someone said, wouldn't it be better to use SQLite directly?

Okay, absolutely fine. I am just developing it from a front-end attitude here. What if some students don’t need SQLite, but just need a simple way to store a large amount of data?

There may be many usage scenarios, the bottom layer of the implementation can be replaced at will, and even the implementation method can be written at will. Here I am building an informal, front-end-friendly data repository based on the attitude of the front-end creating the world.

Underlying Storage

The usage scenario here is ReactNative, so I use RN's AsyncStorage.

Convert all data to be saved into objects and into strings. The core idea here is serialization. Store all data as strings.

import { AsyncStorage } from 'react-native';

exports.setItem = async (key, value) => {    let item = JSON.stringify({
        v: value
    });    return await AsyncStorage.setItem(key, item);
}

When reading, a conversion is also required to convert the string into the originally stored data.

exports.getItem = async (key) => {    let item = await AsyncStorage.getItem(key);    if (!item) {        return null;
    }    return JSON.parse(item).v || null;
}

What needs special treatment is the acquisition of the list. RN has an API that returns multiple pieces of data based on multiple keys. It returns an array object. Array number 0 is the key value of data storage, and sequence number 1 is the specific string of data storage.

exports.getlist = async (keys) => {    let list = await AsyncStorage.multiGet(keys);
    list = list.map(item => JSON.parse(item[1]).v || null);    return list;
}

Let’s also take out a few other methods used. Nest one more layer here, keeping the same formation as above.

exports.removeItem = async (key) => await AsyncStorage.removeItem(key);

exports.removeItems = async (keys) => await AsyncStorage.multiRemove(keys);

exports.allKeys = async () => await AsyncStorage.getAllKeys();

Underlying Optimization

The above is just a simple implementation. If there are no special requirements, it will be almost the same. However, those who want to go further can consider starting to optimize.

For example, optimize the speed of json conversion. When we use the JSON object method to convert, there is actually a process of determining the numerical type. If we define the data type in advance. There is no need to judge again when converting.

You can define a model object and pre-define the fields required by this table. You can check how Sequelize is defined. It is very simple to do this according to the way of relational row database.

//用户对象const users = db.define('t_users', {
    id: {
        type: Sequelize.INTEGER,
        primaryKey: true,
    },    //用户名
    username: {
        type: Sequelize.STRING
    },    //密码
    pwd: {
        type: Sequelize.STRING
    },    //状态
    status: {
        type: Sequelize.INTEGER
    },    //昵称
    nickname: {
        type: Sequelize.STRING
    },    //token
    token: {
        type: Sequelize.STRING
    },
    create_time: {
        type: Sequelize.TIME
    }
}, {
    freezeTableName: true,
    timestamps: false,
});

Implementing storage

Let’s refer to the implementation of relational database here.

First you need to divide the tables and databases. In this way, you can pay less attention to this information when storing data, and focus on data operations.

constructor(tableName = "table", db = "db") {        //检查库,表是否存在
        //初始化索引表
        this.db = db;        this.tableName = tableName;        this.tableKey = db + "_" + tableName;        this.init();
    }

将它们分开存储在当前对象内部，在创建对象的时候就可以根据不同的库、表创建不同的操作方法。这里使用的是class，每个表都对应一个单独的对象。

由于我们使用的是RN提供的存储方法，所以这里的添加和更新其实是一个方法。在添加的时候会根据当前时间戳创建一个唯一id，使用这个id作为key存储在数据库中。所以在使用的时候不需要再单独存入id，不过如果你觉得这个id跟你需要的有差别也可以自己定义一个id来作为key值存储。

//添加和更新
    async add(data = {}) {        if (data.constructor !== Object) return;        if (!data._id)data._id = uuid();
        await setItem(this.tableKey + "_" + data._id, data);        return data;
    }

在获取的时候单独提供了一个根据id获取的方式。这里考虑的是通过id获取非常的简单方便，对于某些数据完全可以快速读取，没必要一行一行的去查询。

/**
 * 通过id查询
 * @param {*} id 
 */async getById(id) {    if (!id) return {};    return await getItem(this.tableKey + "_" + id);
}

相对于根据id查询来说，模糊查询确实很慢，如果不是真实需要，还是不要使用这种模糊查询的好。这里提供了一个自定义查询的方法，可以根据返回的对象判断是否需要这行数据。同时也可以添加top参数来限制返回的数量。使用这个参数也可以在数据很多的时候提高性能。

/**
 * 通过过滤方法查询
 * @param {*} fn 
 */async get(fn, top = 0) {    let keys = await allKeys();    if (keys.length == 0) return [];    if (top > 0 && keys.length > top) keys.length = top;    const listkey = keys.filter(item => item.indexOf(this.tableKey + "_") === 0);    if (listkey.length == 0) return [];    let list = await getlist(listkey);
    list = list.filter(item => fn(item));    return list;
}

最后把删除和清空的方法加上，这样一个简单的删除库就完成了。

 /**
 * 删除
 * @param {*} id 
 */async delete(id) {    if (!id) return {};
    await removeItem(this.tableKey + "_" + id);
}/**
 * 清空表
 */async clear() {    let keys = await allKeys();    const listkey = keys.filter(item => item.indexOf(this.tableKey + "_") === 0);    if (listkey.length == 0) return;
    removeItems(listkey);
}

使用的时候只需要创建对象，然后在需要的地方调用即可。使用起来简单又方便，再加上优化之后的情况甚至可以当成客户端的redis来使用。

//初始化数据库let db=new JSDB();//添加一个自定义数据db.add({name:"test",key:"abc"});//根据id获取数据db.getById("1223467890");//根据条件查询数据db.get(d=>d.name==="test");//删除对应的数据db.delete("1223467890");//情况所有数据db.clear()

优化创建

首先要优化的就是对象的创建。每个对象创建其实都是一个很大的消耗，如果能把这个消耗降低岂不是美滋滋！

这里我们借鉴数据库池的概念，实现一个对象池的方法。在对象创建之后并没有直接返回，要在经过池的操作。

将对象放入池内，并在页面销毁的时候重置为一个空对象。下次请求创建的时候就不必再创建新的了。直接赋值表、库的名称就可以使用了。内存毫无变化，并且有点想笑。

优化查询

每次查询都需要去读Stroage还是很麻烦的，尤其这个操作是异步操作，是需要发消息到native端的。

我们可以将上次读取的数据先存在一个变量中，如果下次还需要使用这行数据，就完全不需要再去读取了。这样就可以很简单的提供读取速度。

这个方式还可以继续优化。将变量中保存的数据限制数量，防止数量太多超过了APP的限制。还可以将这个保存的时限做一个逻辑判断，常使用的数据放在里面，不常用的就找机会删除。

使用这种方式也可以优化变量中数据的有效性，减少变量占用内存的大小。不过实现的方式尽量不要使用定时器的形式，可以考虑使用触发式的。在条件满足的时候再触发删除动作。

优化写入

上面提到读取的时候需要放入变量来提高读取速度。我们顺便想到写入的速度是不是也可以提高啊？

我们将要存的数据放在临时的变量里，如果查过我们设置的时间或者数据长度查过我们设置的数量就触发一次保存操作。

这里要注意，保存数据的变量和存入时候使用的变量要交替使用，防止出现丢数据的情况。

比如：存的时候使用变量1，在写到数据库之前，将要存的对象改成变量2，然后读取变量1的数据并存入数据库中。这就是双缓存写入。

当然还是要判断一次APP的退出事件的，如果APP退出了，请一定要把变量中的内容保存一次，不然你的心血就全丢了。

---

好了，一个简单的数据库就完成了。想要使用的可以先在npm上搜索react-native-jsdb这个库。我已经将第一部分实现放在了npm上，后续的优化也会满满的开源出来的。

The above is the detailed content of Example sharing of js development database. For more information, please follow other related articles on the PHP Chinese website!