search
HomeDatabaseMysql TutorialHBase实战系列3—搭建ThriftServer实时监控系统

背景: 在hbase应用中,如果使用C++来访问HBase,往往通过ThriftServer进行数据的读写,ThriftServer服务的状况直接影响了应用服务的体验。因此,在HBase ThriftServer之上的Metrics系统、以及实时监控系统,可以第一时间发现服务质量变化以及相关问题,同时

背景:

在hbase应用中,如果使用C++来访问HBase,往往通过ThriftServer进行数据的读写,ThriftServer服务的状况直接影响了应用服务的体验。因此,在HBase ThriftServer之上的Metrics系统、以及实时监控系统,可以第一时间发现服务质量变化以及相关问题,同时,良好的监控系统,也有助于服务的完善。

ThriftServer实时监控系统的挑战

1)ThriftServer的服务具有分散性。我们一般为不同的应用启动多个ThriftServer,那么如果我们想多维度统计分析某个应用的统计状况,就会比较困难。例如:百川抓取平台,随机使用多个ThriftServer进行读写操作,单个ThriftServer的实时请求状况可以很容易分析,但是如果想看整个百川平台对于HBase集群的整体压力,就必须实时地从不同的ThriftServer中收集相关读写请求的数据,然后还得将分散到不同节点上的数据,进行聚合和汇总。显然,这样工作或者可以通过中心统一的key-value store,或者通过分布式流数据处理平台。

2)报警平台采用分散式还是中心式的问题。ThriftServer是一个个体,不同的应用会把多个ThriftServer汇总在一起。那么我们如何对ThriftServer出现的问题进行报警是最有效和最健壮的,成为了设计实时监控系统需要考虑的重要问题之一。

3)?如何让监控信息可靠、并且容忍短时间错误。实时监控系统有一个特点就是,它数据是根据时间为横坐标,因此数据量较大,在整个系统中,整个系统不应该因部分Metrics没有处理,而影响到整体的监控平台的处理效果。

4)需要满足动态扩展性。因为ThriftServer依赖的HBase是动态可扩展的,那么ThriftServer监控系统也应该满足一定的扩展性,并提供足够强劲的Availability。

ThriftServer实时监控系统的架构与设计

为了更好地解决ThriftServer实时监控系统带来的挑战,该设计使用了两大特色的设计:

1)分散式报警

2)使用OpenTSDB收集实时数据。

架构图如下。

thriftserver-real-monitor

具体的设计如下:

1)每个ThriftServer都会启动一个Metrics线程,提供ReadRequestNum、WriteRequestNum、errorRequestNum的统计与汇报。具体设计思想来源于HBase Metrics和HBase Metrics的相关参数解读。这对应于上图中的实时统计信息。

2)分散式报警。分散式报警比集中报警的粒度更细,而且嵌入到每个ThriftServer中,没有中心控制平台的单点故障的问题。对于嵌入ThriftServer代码内部的不灵活性,这里利用HBase Metrics提供的配置文件,可以灵活配置相关报警的选项和Threshold。

3)中心收集平台。这里选用OpenTSDB主要考虑如下几点:

  • OpenTSDB可以实现分布式部署。
  • 支持tag模式,可以实现不同的统计需求
  • 支持Rate、Sum、Max、Min等统计,可以实现多个ThriftServer实时收集数据的汇总。
  • 提供了GnuPlot的实时绘图功能。
  • 数据容器依赖于Hbase,通过AsyncHbase实现数据的异步入库。

4)随机选择OpenTSDB Server,实现一定的load balance。并且需要实时监控Telnet连接的可用性,在连接错误时,实现OpenTSDB Server的动态切换。

本系列文章属于Binos_ICT在Binospace个人技术博客原创,原文链接为http://www.binospace.com/index.php/hbase-series-3-build-thriftserver-real-time-monitoring-system/,未经允许,不得在网上转载。

相关文章:

http://www.binospace.com/index.php/hbase-combat-series-1-compression-and-coding-techniques/

http://www.binospace.com/index.php/hbase-combat-series-2-region-monitoring/

From Binospace, post HBase实战系列3—搭建ThriftServer实时监控系统

文章的脚注信息由WordPress的wp-posturl插件自动生成


Copyright © 2008
This feed is for personal, non-commercial use only.
The use of this feed on other websites breaches copyright. If this content is not in your news reader, it makes the page you are viewing an infringement of the copyright. (Digital Fingerprint:
)
Statement
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
Explain the InnoDB Buffer Pool and its importance for performance.Explain the InnoDB Buffer Pool and its importance for performance.Apr 19, 2025 am 12:24 AM

InnoDBBufferPool reduces disk I/O by caching data and indexing pages, improving database performance. Its working principle includes: 1. Data reading: Read data from BufferPool; 2. Data writing: After modifying the data, write to BufferPool and refresh it to disk regularly; 3. Cache management: Use the LRU algorithm to manage cache pages; 4. Reading mechanism: Load adjacent data pages in advance. By sizing the BufferPool and using multiple instances, database performance can be optimized.

MySQL vs. Other Programming Languages: A ComparisonMySQL vs. Other Programming Languages: A ComparisonApr 19, 2025 am 12:22 AM

Compared with other programming languages, MySQL is mainly used to store and manage data, while other languages ​​such as Python, Java, and C are used for logical processing and application development. MySQL is known for its high performance, scalability and cross-platform support, suitable for data management needs, while other languages ​​have advantages in their respective fields such as data analytics, enterprise applications, and system programming.

Learning MySQL: A Step-by-Step Guide for New UsersLearning MySQL: A Step-by-Step Guide for New UsersApr 19, 2025 am 12:19 AM

MySQL is worth learning because it is a powerful open source database management system suitable for data storage, management and analysis. 1) MySQL is a relational database that uses SQL to operate data and is suitable for structured data management. 2) The SQL language is the key to interacting with MySQL and supports CRUD operations. 3) The working principle of MySQL includes client/server architecture, storage engine and query optimizer. 4) Basic usage includes creating databases and tables, and advanced usage involves joining tables using JOIN. 5) Common errors include syntax errors and permission issues, and debugging skills include checking syntax and using EXPLAIN commands. 6) Performance optimization involves the use of indexes, optimization of SQL statements and regular maintenance of databases.

MySQL: Essential Skills for Beginners to MasterMySQL: Essential Skills for Beginners to MasterApr 18, 2025 am 12:24 AM

MySQL is suitable for beginners to learn database skills. 1. Install MySQL server and client tools. 2. Understand basic SQL queries, such as SELECT. 3. Master data operations: create tables, insert, update, and delete data. 4. Learn advanced skills: subquery and window functions. 5. Debugging and optimization: Check syntax, use indexes, avoid SELECT*, and use LIMIT.

MySQL: Structured Data and Relational DatabasesMySQL: Structured Data and Relational DatabasesApr 18, 2025 am 12:22 AM

MySQL efficiently manages structured data through table structure and SQL query, and implements inter-table relationships through foreign keys. 1. Define the data format and type when creating a table. 2. Use foreign keys to establish relationships between tables. 3. Improve performance through indexing and query optimization. 4. Regularly backup and monitor databases to ensure data security and performance optimization.

MySQL: Key Features and Capabilities ExplainedMySQL: Key Features and Capabilities ExplainedApr 18, 2025 am 12:17 AM

MySQL is an open source relational database management system that is widely used in Web development. Its key features include: 1. Supports multiple storage engines, such as InnoDB and MyISAM, suitable for different scenarios; 2. Provides master-slave replication functions to facilitate load balancing and data backup; 3. Improve query efficiency through query optimization and index use.

The Purpose of SQL: Interacting with MySQL DatabasesThe Purpose of SQL: Interacting with MySQL DatabasesApr 18, 2025 am 12:12 AM

SQL is used to interact with MySQL database to realize data addition, deletion, modification, inspection and database design. 1) SQL performs data operations through SELECT, INSERT, UPDATE, DELETE statements; 2) Use CREATE, ALTER, DROP statements for database design and management; 3) Complex queries and data analysis are implemented through SQL to improve business decision-making efficiency.

MySQL for Beginners: Getting Started with Database ManagementMySQL for Beginners: Getting Started with Database ManagementApr 18, 2025 am 12:10 AM

The basic operations of MySQL include creating databases, tables, and using SQL to perform CRUD operations on data. 1. Create a database: CREATEDATABASEmy_first_db; 2. Create a table: CREATETABLEbooks(idINTAUTO_INCREMENTPRIMARYKEY, titleVARCHAR(100)NOTNULL, authorVARCHAR(100)NOTNULL, published_yearINT); 3. Insert data: INSERTINTObooks(title, author, published_year)VA

See all articles

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

Video Face Swap

Video Face Swap

Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Tools

MantisBT

MantisBT

Mantis is an easy-to-deploy web-based defect tracking tool designed to aid in product defect tracking. It requires PHP, MySQL and a web server. Check out our demo and hosting services.

Dreamweaver Mac version

Dreamweaver Mac version

Visual web development tools

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)

PhpStorm Mac version

PhpStorm Mac version

The latest (2018.2.1) professional PHP integrated development tool

WebStorm Mac version

WebStorm Mac version

Useful JavaScript development tools