Use MongoDB database to store product information

A basic functional module of e-commerce business is to store product information with rich categories. Various products have different characteristics and parameters. MongoDB’s flexible document model is very suitable for this type of business. This article mainly introduces how to use MongoDB to store product classifications. information.

Relational database solutions

The above problems can also be solved using traditional relational databases, such as the following solutions

For For different products, create different tables

For example, two products, music albums and movies, have some common attributes, but also have many unique attributes. You can create two different tables with different attributes. schema.

CREATE TABLE `product_audio_album` (
    `sku` char(8) NOT NULL,
    ...
    `artist` varchar(255) DEFAULT NULL,
    `genre_0` varchar(255) DEFAULT NULL,
    `genre_1` varchar(255) DEFAULT NULL,
    ...,
    PRIMARY KEY(`sku`))
...
CREATE TABLE `product_film` (
    `sku` char(8) NOT NULL,
    ...
    `title` varchar(255) DEFAULT NULL,
    `rating` char(8) DEFAULT NULL,
    ...,
    PRIMARY KEY(`sku`))
...

The main problem with this approach is that

1. For each new product category, a new table needs to be created

2. Application developers must explicitly distribute requests to the corresponding tables for querying. Querying multiple products at one time is cumbersome to implement

All products are stored in a single table

CREATE TABLE `product` (
    `sku` char(8) NOT NULL,
    ...
    `artist` varchar(255) DEFAULT NULL,
    `genre_0` varchar(255) DEFAULT NULL,
    `genre_1` varchar(255) DEFAULT NULL,
    ...
    `title` varchar(255) DEFAULT NULL,
    `rating` char(8) DEFAULT NULL,
    ...,
    PRIMARY KEY(`sku`))

Store all products in a table. This table contains the attributes required by all products. Different products set different attributes as needed. This method makes product query relatively simple and allows one query to span multiple products. products, but the disadvantage is that it wastes a lot of space.

Extract public attributes, multi-table inheritance

CREATE TABLE `product` (
    `sku` char(8) NOT NULL,
    `title` varchar(255) DEFAULT NULL,
    `description` varchar(255) DEFAULT NULL,
    `price`, ...
    PRIMARY KEY(`sku`))
CREATE TABLE `product_audio_album` (
    `sku` char(8) NOT NULL,
    ...
    `artist` varchar(255) DEFAULT NULL,
    `genre_0` varchar(255) DEFAULT NULL,
    `genre_1` varchar(255) DEFAULT NULL,
    ...,
    PRIMARY KEY(`sku`),
    FOREIGN KEY(`sku`) REFERENCES `product`(`sku`))
...
CREATE TABLE `product_film` (
    `sku` char(8) NOT NULL,
    ...
    `title` varchar(255) DEFAULT NULL,
    `rating` char(8) DEFAULT NULL,
    ...,
    PRIMARY KEY(`sku`),
    FOREIGN KEY(`sku`) REFERENCES `product`(`sku`))
...

The above scheme extracts the common attributes of all products and stores the public attributes in one table for each product Create a new table according to your own needs, and only store information unique to the product in the new table.

Entity Attribute Values ​​form storage

All data is stored in the form of 3-tuples. This solution actually uses the relational database as KV storage. The model Simple, but not very convenient for complex queries.

##ENTITY                                                                                                                00e8da9b title                                                                                

sku_00e8da9b                                                                              …                     … MongoDB solution

MognoDB is different from relational databases in that it has no schema, the document content can be customized very flexibly, and it can make good use of the above product classification storage. Requirements: Store product information in a collection, and document content can be customized for different products in the collection.

For example, a music album can have a document structure similar to the following

{
  sku: "00e8da9b",
  type: "Audio Album",
  title: "A Love Supreme",
  description: "by John Coltrane",
  asin: "B0000A118M",
  shipping: {
    weight: 6,
    dimensions: {
      width: 10,
      height: 10,
      depth: 1
    },
  },
  pricing: {
    list: 1200,
    retail: 1100,
    savings: 100,
    pct_savings: 8
  },
  details: {
    title: "A Love Supreme [Original Recording Reissued]",
    artist: "John Coltrane",
    genre: [ "Jazz", "General" ],
        ...
    tracks: [
      "A Love Supreme Part I: Acknowledgement",
      "A Love Supreme Part II - Resolution",
      "A Love Supreme, Part III: Pursuance",
      "A Love Supreme, Part IV-Psalm"
    ],
  },
}

while a movie can be stored as

{
  sku: "00e8da9d",
  type: "Film",
  ...,
  asin: "B000P0J0AQ",
  shipping: { ... },
  pricing: { ... },
  details: {
    title: "The Matrix",
    director: [ "Andy Wachowski", "Larry Wachowski" ],
    writer: [ "Andy Wachowski", "Larry Wachowski" ],
    ...,
    aspect_ratio: "1.66:1"
  },
}

所有商品都拥有一些共同的基本信息，特定的商品可以根据需要扩展独有的内容，非常方便； 基于上述模型，MongoDB 也能很好的服务各类查询。

查询某个演员参演的所有电影，并按发型日志排序

db.products.find({'type': 'Film', 'details.actor': 'Keanu Reeves'}).sort({'details.issue_date', -1})

上述查询也可以通过建立索引来加速

db.products.createIndex({ type: 1, 'details.actor': 1, 'details.issue_date': -1 })

查询标题里包含特定信息的所有电影

db.products.find({
    'type': 'Film',
    'title': {'$regex': '.*hacker.*', '$options':'i'}}).sort({'details.issue_date', -1})

可建立如下索引来加速查询

db.products.createIndex({ type: 1, details.issue_date: -1, title: 1 })

扩展

当单个节点无法满足海量商品信息存储的需求时，就需要使用MongoDB sharding来扩展，假定大量的查询都是都会基于商品类型，那么就可以使用商品类型字段来进行分片。

db.shardCollection('products', { key: {type: 1} })

分片时，尽量使用复合的索引字段，这样能满足更多的查询需求，比如基于商品类型之后，还会经常根据商品的风格标签来查询，则可以把商品的标签字段作为第二分片key。

db.shardCollection('products', { key: {type: 1, 'details.genre': 1} })

如果某种类型的商品，拥有相同标签的特别多，则会出现jumbo chunk的问题，导致无法迁移，可以进一步的优化分片key，以避免这种情况。

db.shardCollection('products', { key: {type: 1, 'details.genre': 1, sku: 1} })

加入第3分片key之后，即使类型、风格标签都相同，但其sku信息肯定不同，就肯定不会出现超大的chunk。