search
HomeDatabaseMysql Tutorial 生产环境下Hadoop大集群安装与配置+DNS+NFS

《生产环境下Hadoop大集群安装与配置+DNS+NFS》一环境LinuxISO:CentOS-6.0-i386-bin-DVD.iso32位JDKversion:1.6.0_25-eaforlinuxHad..

《生产环境下Hadoop大集群安装与配置+DNS+NFS》

一 环境

Linux ISO:CentOS-6.0-i386-bin-DVD.iso     32位   

JDK version:"1.6.0_25-ea"   for  linux

Hadoop software version:hadoop-0.20.205.0.tar.gz   for  linux

VMware® Workstation   Version: 7.0.0 build-203739

我的linux虚拟机配置   master   slave1   slave2    信息如下

主机名


 

IP


 

节点名


 

备注


 

h1


 

192.168.2.102


 

master


 

namenode和jobtracker


 

h2


 

192.168.2.103


 

slave1


 

datanode和tasktracker


 

H4


 

192.168.2.105


 

slave2


 

datanode和tasktracker


 

DNS服务器安装与配置参考链接,感谢yanggw8071   mikeliu   chizeng

1.      

2.      

3.      

二 Bind安装DNS服务器并成功解析主机名

1.说明一下,我把DNS服务器安装到h1节点(master上)然后对h1 h2 h4 节点的主机名进行解析

2.需要h1节点可以上网,因为我们需要使用yum方式安装DNS的rpm包,美国空间,上网方式如下

CentOS需要能连入网络,这个比较简单,可以参考《NOSQL系列-memcached安装管理与repcached高可用性》,这篇文章在一开始就介绍了虚拟机如何连接上网

3.原来我们使用的hosts文件来解析主机名,现在可以不使用了直接删除即可,我呢直接改名吧,香港服务器,这样留着备用

[root@h1 etc]# mv /etc/hosts /etc/hosts_20130126

4.有的筒子使用了webmin软件GUI方式配置了DNS服务器,这种方式是挺方便的界面化人性化,但我用惯了命令行(码农的归宿)所以直接使用rpm方式来安装DNS了

5.检查一下bind包有没有,这也是采用chroot机制

[root@h1 ~]# rpm -q bind

package bind is not installed

[root@h1 ~]# rpm -q bind-chroot

package bind-chroot is not installed

这里显示还没有安装,那么我们就需要yum方式安装啦

6.现在使用yum安装bind 和 bind-chroot 软件包

[root@h1 ~]# yum -y install bind bind-chroot

省略。。。。。。

Dependencies Resolved

=======================================================================================

Package     Arch     Version                 Repository       Size

=======================================================================================

Installing:                安装这2个包

bind        i686     32:9.8.2-0.10.rc1.el6_3.6   updates         4.0 M

bind-chroot  i686     32:9.8.2-0.10.rc1.el6_3.6   updates         70 k

Updating for dependencies:  升级2个依赖包

bind-libs    i686      32:9.8.2-0.10.rc1.el6_3.6   updates         882 k

bind-utils  i686     32:9.8.2-0.10.rc1.el6_3.6     updates         180 k

Transaction Summary

=======================================================================================Install       2 Package(s)

Upgrade    2 Package(s)

Installed:

bind.i686 32:9.8.2-0.10.rc1.el6_3.6      bind-chroot.i686 32:9.8.2-0.10.rc1.el6_3.6                                          

Dependency Updated:

bind-libs.i686 32:9.8.2-0.10.rc1.el6_3.6   bind-utils.i686 32:9.8.2-0.10.rc1.el6_3.6                                         

Complete!

到此bind 和 bind-chroot包安装完成

[root@h1 ~]# rpm -q bind

bind-9.8.2-0.10.rc1.el6_3.6.i686

[root@h1 ~]# rpm -q bind-chroot

bind-chroot-9.8.2-0.10.rc1.el6_3.6.i686

哈哈 现在都已经安装好了

7. yum install bind*还有三个包需要安装

[root@h1 etc]# yum install bind*

Dependencies Resolved

=======================================================================================

Package     Arch     Version                 Repository       Size

=======================================================================================

Installing:                安装这3个包

bind-devel   i686    32:9.8.2-0.10.rc1.el6_3.6     updates         376 k

bind-dyndb-ldap i686  1.1.0-0.9.b1.el6_3.1        updates          63 k

bind-sdb   i686   32:9.8.2-0.10.rc1.el6_3.6       updates          305 k

Transaction Summary

=======================================================================================I

Install       3 Package(s)

Upgrade     0 Package(s)

Installed:

bind-devel.i686 32:9.8.2-0.10.rc1.el6_3.6              

bind-dyndb-ldap.i686 0:1.1.0-0.9.b1.el6_3.1              

bind-sdb.i686 32:9.8.2-0.10.rc1.el6_3.6            

Complete!

现在都安装完毕了

8.修改named.conf文件,目录/etc/named.conf

[root@h1 etc]# vim /etc/named.conf        修改2处,标红显示

options {

        listen-on port 53 { any; };       //把127.0.0.1 修改成 any

        listen-on-v6 port 53 { ::1; };

        directory       "/var/named";

        dump-file       "/var/named/data/cache_dump.db";

        statistics-file "/var/named/data/named_stats.txt";

        memstatistics-file "/var/named/data/named_mem_stats.txt";

        allow-query     { any; };      //localhost 修改成 any

        recursion yes;

        dnssec-enable yes;

        dnssec-validation yes;

        dnssec-lookaside auto;

        /* Path to ISC DLV key */

        bindkeys-file "/etc/named.iscdlv.key";

        managed-keys-directory "/var/named/dynamic";

};

logging {

        channel default_debug {

                file "data/named.run";

                severity dynamic;

        };

};

zone "." IN {

        type hint;

        file "named.ca";

};

include "/etc/named.rfc1912.zones";

include "/etc/named.root.key";

9. 修改named.rfc1912.zones文件,目录/etc/named.rfc1912.zones

配置正向反向配置文件

[root@h1 etc]# vim /etc/named.rfc1912.zones     在这个文件末尾添加如下内容,红色是注意的地方

zone "leonarding.com" IN {

        type master;

        file "leonarding.com.zone";       注意这是你自己设置的域名,要牢牢记住

        allow-update { none; };

};

zone "2.168.192.in-addr.arpa" IN {

        type master;

        file "2.168.192.in-addr.zone";      这是反向配置文件

        allow-update { none; };

};

10.创建leonarding.com.zone和2.168.192.in-addr.zon 文件

[root@h1 etc]# cd /var/named          进入这个目录

[root@h1 named]# cp -p named.localhost leonarding.com.zone

[root@h1 named]# cp -p named.localhost 2.168.192.in-addr.zone

对这2个文件进行修改,修改成新的文件,这2个文件就是区域文件,一定要写对了,否则会报错

添加正向文件并且配置

[root@h1 named]# vim leonarding.com.zone

$TTL 86400

@    IN    SOA  h1.leonarding.com. chizk. (

     0 ; serial (d. adams)

     1D ; refresh

     1H ; retry

     1W ; expiry

     3H ) ; minimum

@    IN    NS  h1.leonarding.com.

h1.leonarding.com. IN A 192.168.2.102

h2.leonarding.com. IN A 192.168.2.103

h4.leonarding.com. IN A 192.168.2.105

添加反向文件并且配置

[root@h1 named]# vim 2.168.192.in-addr.zone

$TTL 86400

@    IN    SOA  h1.leonarding.com. chizk. (

     0 ; serial (d. adams)

     1D ; refresh

     1H ; retry

     1W ; expiry

     3H ) ; minimum

@    IN    NS  h1.leonarding.com.

102 IN PTR h1.leonarding.com.

103 IN PTR h2.leonarding.com.

105 IN PTR h4.leonarding.com.

修改正向文件和反向文件属组,使用chgrp即可

[root@h1 named]# chgrp named leonarding.com.zone

[root@h1 named]# chgrp named leonarding.com.rev

11.修改/etc/resolv.conf 添加DNS域名服务器ip

h1(master)

[root@h1 named]# vim /etc/resolv.conf

nameserver 192.168.2.102          在最后面追加此条即可,就是我的master机器ip

说明我们是把master机器作为DNS域名解析服务器,

h2(slave)

[root@h2 sysconfig]# vim /etc/resolv.conf

nameserver 192.168.2.102

h4(slave)

[root@h4 .ssh]# vim /etc/resolv.conf

nameserver 192.168.2.102

添加DNS域名服务器ip,当有域名或主机名解析的时候,就知道去哪台机器上进行映射了

12.启动DNS服务器

[root@h1 named]# service named start

启动  named:                            【确定】

13.使用nslookup命令测试域名解析

测试之前关闭防火墙

[root@h1 named]# service iptables status

Iptables:未运行防火墙

正向解析域名->IP

[root@h1 named]# nslookup h1.leonarding.com

Server:          192.168.2.102                    指域名服务器ip地址

Address:       192.168.2.102#53                     ip+端口号,DNS默认53端口

Name:   h1.leonarding.com                    域名解析

Address: 192.168.2.102                       映射ip

[root@h1 named]# nslookup h2.leonarding.com

Server:          192.168.2.102                    指域名服务器ip地址

Address:       192.168.2.102#53                     ip+端口号,DNS默认53端口

Name:   h2.leonarding.com                    域名解析

Address: 192.168.2.103                       映射ip

[root@h1 named]# nslookup h4.leonarding.com

Server:          192.168.2.102                    指域名服务器ip地址

Address:       192.168.2.102#53                    ip+端口号,DNS默认53端口

Name:   h4.leonarding.com                    域名解析

Address: 192.168.2.105                       映射ip

反向解析IP->域名

[root@h1 named]# nslookup 192.168.2.102

Server:          192.168.2.102

Address:       192.168.2.102#5

102.2.168.192.in-addr.arpa     name = h1.leonarding.com.

[root@h1 named]# nslookup 192.168.2.103

Server:          192.168.2.102

Address:       192.168.2.102#53

103.2.168.192.in-addr.arpa     name = h2.leonarding.com.

[root@h1 named]# nslookup 192.168.2.105

Server:          192.168.2.102

Address:       192.168.2.102#53

105.2.168.192.in-addr.arpa     name = h4.leonarding.com.

我们还可以使用host命令做解析测试

[root@h1 named]# host h2.leonarding.com

h2.leonarding.com has address 192.168.2.103

[root@h1 named]# host 192.168.2.103

103.2.168.192.in-addr.arpa domain name pointer h2.leonarding.com.

Statement
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
Explain the InnoDB Buffer Pool and its importance for performance.Explain the InnoDB Buffer Pool and its importance for performance.Apr 19, 2025 am 12:24 AM

InnoDBBufferPool reduces disk I/O by caching data and indexing pages, improving database performance. Its working principle includes: 1. Data reading: Read data from BufferPool; 2. Data writing: After modifying the data, write to BufferPool and refresh it to disk regularly; 3. Cache management: Use the LRU algorithm to manage cache pages; 4. Reading mechanism: Load adjacent data pages in advance. By sizing the BufferPool and using multiple instances, database performance can be optimized.

MySQL vs. Other Programming Languages: A ComparisonMySQL vs. Other Programming Languages: A ComparisonApr 19, 2025 am 12:22 AM

Compared with other programming languages, MySQL is mainly used to store and manage data, while other languages ​​such as Python, Java, and C are used for logical processing and application development. MySQL is known for its high performance, scalability and cross-platform support, suitable for data management needs, while other languages ​​have advantages in their respective fields such as data analytics, enterprise applications, and system programming.

Learning MySQL: A Step-by-Step Guide for New UsersLearning MySQL: A Step-by-Step Guide for New UsersApr 19, 2025 am 12:19 AM

MySQL is worth learning because it is a powerful open source database management system suitable for data storage, management and analysis. 1) MySQL is a relational database that uses SQL to operate data and is suitable for structured data management. 2) The SQL language is the key to interacting with MySQL and supports CRUD operations. 3) The working principle of MySQL includes client/server architecture, storage engine and query optimizer. 4) Basic usage includes creating databases and tables, and advanced usage involves joining tables using JOIN. 5) Common errors include syntax errors and permission issues, and debugging skills include checking syntax and using EXPLAIN commands. 6) Performance optimization involves the use of indexes, optimization of SQL statements and regular maintenance of databases.

MySQL: Essential Skills for Beginners to MasterMySQL: Essential Skills for Beginners to MasterApr 18, 2025 am 12:24 AM

MySQL is suitable for beginners to learn database skills. 1. Install MySQL server and client tools. 2. Understand basic SQL queries, such as SELECT. 3. Master data operations: create tables, insert, update, and delete data. 4. Learn advanced skills: subquery and window functions. 5. Debugging and optimization: Check syntax, use indexes, avoid SELECT*, and use LIMIT.

MySQL: Structured Data and Relational DatabasesMySQL: Structured Data and Relational DatabasesApr 18, 2025 am 12:22 AM

MySQL efficiently manages structured data through table structure and SQL query, and implements inter-table relationships through foreign keys. 1. Define the data format and type when creating a table. 2. Use foreign keys to establish relationships between tables. 3. Improve performance through indexing and query optimization. 4. Regularly backup and monitor databases to ensure data security and performance optimization.

MySQL: Key Features and Capabilities ExplainedMySQL: Key Features and Capabilities ExplainedApr 18, 2025 am 12:17 AM

MySQL is an open source relational database management system that is widely used in Web development. Its key features include: 1. Supports multiple storage engines, such as InnoDB and MyISAM, suitable for different scenarios; 2. Provides master-slave replication functions to facilitate load balancing and data backup; 3. Improve query efficiency through query optimization and index use.

The Purpose of SQL: Interacting with MySQL DatabasesThe Purpose of SQL: Interacting with MySQL DatabasesApr 18, 2025 am 12:12 AM

SQL is used to interact with MySQL database to realize data addition, deletion, modification, inspection and database design. 1) SQL performs data operations through SELECT, INSERT, UPDATE, DELETE statements; 2) Use CREATE, ALTER, DROP statements for database design and management; 3) Complex queries and data analysis are implemented through SQL to improve business decision-making efficiency.

MySQL for Beginners: Getting Started with Database ManagementMySQL for Beginners: Getting Started with Database ManagementApr 18, 2025 am 12:10 AM

The basic operations of MySQL include creating databases, tables, and using SQL to perform CRUD operations on data. 1. Create a database: CREATEDATABASEmy_first_db; 2. Create a table: CREATETABLEbooks(idINTAUTO_INCREMENTPRIMARYKEY, titleVARCHAR(100)NOTNULL, authorVARCHAR(100)NOTNULL, published_yearINT); 3. Insert data: INSERTINTObooks(title, author, published_year)VA

See all articles

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

Video Face Swap

Video Face Swap

Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Tools

DVWA

DVWA

Damn Vulnerable Web App (DVWA) is a PHP/MySQL web application that is very vulnerable. Its main goals are to be an aid for security professionals to test their skills and tools in a legal environment, to help web developers better understand the process of securing web applications, and to help teachers/students teach/learn in a classroom environment Web application security. The goal of DVWA is to practice some of the most common web vulnerabilities through a simple and straightforward interface, with varying degrees of difficulty. Please note that this software

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

Safe Exam Browser

Safe Exam Browser

Safe Exam Browser is a secure browser environment for taking online exams securely. This software turns any computer into a secure workstation. It controls access to any utility and prevents students from using unauthorized resources.

SublimeText3 Linux new version

SublimeText3 Linux new version

SublimeText3 Linux latest version

mPDF

mPDF

mPDF is a PHP library that can generate PDF files from UTF-8 encoded HTML. The original author, Ian Back, wrote mPDF to output PDF files "on the fly" from his website and handle different languages. It is slower than original scripts like HTML2FPDF and produces larger files when using Unicode fonts, but supports CSS styles etc. and has a lot of enhancements. Supports almost all languages, including RTL (Arabic and Hebrew) and CJK (Chinese, Japanese and Korean). Supports nested block-level elements (such as P, DIV),