Hadoop也可以运行在一个伪分布模式,每个Hadoop守护进程运行在一个单独的Java程序的一个节点。 伪分布模式需要在单机模式基础上进行配置。 在/software/hadoop/etc/hadoop/ 目录下 core-site.xml,hdfs-site.xml,mapred-site.xml,yarn-site.xml等文件。 下面
Hadoop也可以运行在一个伪分布模式,每个Hadoop守护进程运行在一个单独的Java程序的一个节点。
伪分布模式需要在单机模式基础上进行配置。
在/software/hadoop/etc/hadoop/ 目录下
core-site.xml,hdfs-site.xml,mapred-site.xml,yarn-site.xml等文件。
下面对以上文件进行配置。
一、etc/hadoop/core-site.xml
包含了hadoop启动时的配置信息。
<code><span>configuration</span>> <span>property</span>> <span>name</span>>fs.defaultFS<span><span>name</span>></span> <span>value</span>>hdfs://localhost:9000<span><span>value</span>></span> <span><span>property</span>></span> <span><span>configuration</span>></span></code>
二、etc/hadoop/hdfs-site.xml
用来配置集群中每台主机都可用,指定主机上作为namenode和datanode的目录。
<code><span>configuration</span>> <span>property</span>> <span>name</span>>dfs.replication<span><span>name</span>></span> <span>value</span>>1<span><span>value</span>></span> <span><span>property</span>></span> <span><span>configuration</span>></span></code>
三、etc/hadoop/mapred-site.xml
默认情况下,/usr/local/hadoop/etc/hadoop/文件夹下有mapred.xml.template文件,我们要复制该文件,并命名为mapred.xml,该文件用于指定MapReduce使用的框架
<code><span>configuration</span>> <span>property</span>> <span>name</span>>mapreduce.framework.name<span><span>name</span>></span> <span>value</span>>yarn<span><span>value</span>></span> <span><span>property</span>></span> <span><span>configuration</span>></span></code>
四、etc/hadoop/yarn-site.xml
包含了MapReduce启动时的配置信息。
<code><span>configuration</span>> <span>property</span>> <span>name</span>>yarn.nodemanager.aux-services<span><span>name</span>></span> <span>value</span>>mapreduce_shuffle<span><span>value</span>></span> <span><span>property</span>></span> <span><span>configuration</span>></span></code>
五、格式化hdfs
<code> hdfs namenode -<span>format</span> </code>
只需要执行一次即可,如果在hadoop已经使用后再次执行,会清除掉hdfs上的所有数据。
六、启动Hadoop
经过上文所描述配置和操作后,下面就可以启动这个单节点的集群
执行启动命令:
<code> sbin/<span><span>start</span>-dfs.sh </span></code>
执行该命令时,如果有yes /no提示,输入yes,回车即可。
接下来,执行:
<code>sbin/<span><span>start</span>-yarn.sh </span></code>
执行完这两个命令后,Hadoop会启动并运行。
执行 jps
命令,会看到Hadoop相关的进程。
浏览器打开 http://localhost:50070/,会看到hdfs管理页面。
浏览器打开 http://localhost:8088,会看到hadoop进程管理页面。
七、WordCount验证
dfs上创建input目录
<code>bin/hadoop fs <span>-mkdir</span> <span>-p</span> input</code>
把hadoop目录下的README.txt拷贝到dfs新建的input里
<code>hadoop fs -copyFromLocal README<span>.txt</span> input</code>
运行WordCount
<code>hadoop jar share/hadoop/mapreduce/sources/hadoop-mapreduce-examples-<span>2.5</span><span>.2</span>-sources<span>.jar</span> org<span>.apache</span><span>.hadoop</span><span>.examples</span><span>.WordCount</span> input output</code>
执行过程:
运行完毕后,查看单词统计结果
<code>hadoop fs <span>-cat</span> output<span>/*</span></code>

MySQL uses a GPL license. 1) The GPL license allows the free use, modification and distribution of MySQL, but the modified distribution must comply with GPL. 2) Commercial licenses can avoid public modifications and are suitable for commercial applications that require confidentiality.

The situations when choosing InnoDB instead of MyISAM include: 1) transaction support, 2) high concurrency environment, 3) high data consistency; conversely, the situation when choosing MyISAM includes: 1) mainly read operations, 2) no transaction support is required. InnoDB is suitable for applications that require high data consistency and transaction processing, such as e-commerce platforms, while MyISAM is suitable for read-intensive and transaction-free applications such as blog systems.

In MySQL, the function of foreign keys is to establish the relationship between tables and ensure the consistency and integrity of the data. Foreign keys maintain the effectiveness of data through reference integrity checks and cascading operations. Pay attention to performance optimization and avoid common errors when using them.

There are four main index types in MySQL: B-Tree index, hash index, full-text index and spatial index. 1.B-Tree index is suitable for range query, sorting and grouping, and is suitable for creation on the name column of the employees table. 2. Hash index is suitable for equivalent queries and is suitable for creation on the id column of the hash_table table of the MEMORY storage engine. 3. Full text index is used for text search, suitable for creation on the content column of the articles table. 4. Spatial index is used for geospatial query, suitable for creation on geom columns of locations table.

TocreateanindexinMySQL,usetheCREATEINDEXstatement.1)Forasinglecolumn,use"CREATEINDEXidx_lastnameONemployees(lastname);"2)Foracompositeindex,use"CREATEINDEXidx_nameONemployees(lastname,firstname);"3)Forauniqueindex,use"CREATEU

The main difference between MySQL and SQLite is the design concept and usage scenarios: 1. MySQL is suitable for large applications and enterprise-level solutions, supporting high performance and high concurrency; 2. SQLite is suitable for mobile applications and desktop software, lightweight and easy to embed.

Indexes in MySQL are an ordered structure of one or more columns in a database table, used to speed up data retrieval. 1) Indexes improve query speed by reducing the amount of scanned data. 2) B-Tree index uses a balanced tree structure, which is suitable for range query and sorting. 3) Use CREATEINDEX statements to create indexes, such as CREATEINDEXidx_customer_idONorders(customer_id). 4) Composite indexes can optimize multi-column queries, such as CREATEINDEXidx_customer_orderONorders(customer_id,order_date). 5) Use EXPLAIN to analyze query plans and avoid

Using transactions in MySQL ensures data consistency. 1) Start the transaction through STARTTRANSACTION, and then execute SQL operations and submit it with COMMIT or ROLLBACK. 2) Use SAVEPOINT to set a save point to allow partial rollback. 3) Performance optimization suggestions include shortening transaction time, avoiding large-scale queries and using isolation levels reasonably.


Hot AI Tools

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Undress AI Tool
Undress images for free

Clothoff.io
AI clothes remover

Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Article

Hot Tools

SecLists
SecLists is the ultimate security tester's companion. It is a collection of various types of lists that are frequently used during security assessments, all in one place. SecLists helps make security testing more efficient and productive by conveniently providing all the lists a security tester might need. List types include usernames, passwords, URLs, fuzzing payloads, sensitive data patterns, web shells, and more. The tester can simply pull this repository onto a new test machine and he will have access to every type of list he needs.

Dreamweaver CS6
Visual web development tools

SAP NetWeaver Server Adapter for Eclipse
Integrate Eclipse with SAP NetWeaver application server.

SublimeText3 Linux new version
SublimeText3 Linux latest version

SublimeText3 Mac version
God-level code editing software (SublimeText3)
