未加星标

Starting Flume agents by using the Ambari web interface

字体大小 | |
[数据库(综合) 所属分类 数据库(综合) | 发布者 店小二03 | 时间 2017 | 作者 红领巾 ] 0人收藏点击收藏

The following directions detail the manual installation of software into IBM Open Platform for Apache Hadoop. These directions, and any binaries that may be provided as part of this article (either hosted by IBM or otherwise), are provided for convenience and make no guarantees as to stability, performance, or functionality of the software being installed. Product support for this software will not be provided (including upgrade support for either IOP or the software described). Questions or issues encountered should be discussed on the BigInsights StackOverflow forum or the appropriate Apache Software Foundation mailing list for the component(s) covered by this article.

Apache Flume can be used to efficiently collect, aggregate, and move large amounts of log data from many different sources to a centralized data store. Ambari provides an intuitive, easy-to-use Hadoop management web interface backed by its RESTful APIs. You can use the Ambari web interface to configure Flume and to start, stop, or monitor Flume agents.

The Flume agents that are started by Ambari are run by user flume, and the file directory that is used in agent configuration must be accessible to user flume. The log files are located in /var/log/flume on the Flume node.

This article shows you how to run Flume agents from Ambari on an IBM Open Platform (IOP) 4.2 cluster.

Starting Flume agents from Ambari Log in to Ambari and verify that the services are running. Click the Flume service; the Summary page displays the overall status. Click the Configs tab to edit the Flume configuration.
Starting Flume agents by using the Ambari web interface
The Configs page has two sections: “flume.conf” for Flume agent configuration, and “Advanced flume-env” for the flume-env.sh file. Expand “flume.conf” which, by default, contains only one comment line: # Flume agent config. Add your agent configuration to this field. Click Save and then restart the Flume service (click Restart) to enable the changes to take effect.
Starting Flume agents by using the Ambari web interface
On the Summary page, you should now be able to see that the newly defined agents are running. You can start or stop the flume agent for a specific host from the drop-down list next to the corresponding Running button. You can also restart, start, or stop the Flume service from the Service Actions menu, or monitor various metrics for your channel in the Metrics section.
Starting Flume agents by using the Ambari web interface
Using configuration groups to start Flume agents with different configurations

The example agents in the previous section run on all hosts. In most cases, Flume flow is complex, with different agents running across the Flume hosts. From the Ambari web interface, you can divide hosts into different configuration groups. Each configuration group can be associated with a specific configuration file for Flume agents. This section shows you how to create configuration groups for Flume agents.

On the Flume Configs page, the default group contains all of the Flume hosts, and all hosts in this default group have the same Flume agent running.
Starting Flume agents by using the Ambari web interface
To create a configuration group, click Manage Config Groups -> +. Type a name and description for the new group. Click OK.
Starting Flume agents by using the Ambari web interface
Select the new group (in this example, “group1″), then click + under the empty host name box.
Starting Flume agents by using the Ambari web interface
Select hosts to include in the configuration group. Click Save.
Starting Flume agents by using the Ambari web interface
Create other configuration groups, as appropriate.
Starting Flume agents by using the Ambari web interface
You can change the configuration values for a group. For example, select group1 from the group list, click + in the flume.conf section, and then add the Flume agent configuration details.
Starting Flume agents by using the Ambari web interface
Starting Flume agents by using the Ambari web interface
Expand the Advanced flume-env section, click +, and change flume-env values, as appropriate. You can update the memory parameter or add customized JAR files to the class path.
Starting Flume agents by using the Ambari web interface
Similarly, you can change the configuration values for another group, such as group2, for example.
Starting Flume agents by using the Ambari web interface
On the Summary page, you can now see that the hosts in different configuration groups have different Flume agents running.
Starting Flume agents by using the Ambari web interface

本文数据库(综合)相关术语:系统安全软件

主题: HadoopRESTIBM
分页:12
转载请注明
本文标题:Starting Flume agents by using the Ambari web interface
本站链接:http://www.codesec.net/view/523034.html
分享请点击:


1.凡CodeSecTeam转载的文章,均出自其它媒体或其他官网介绍,目的在于传递更多的信息,并不代表本站赞同其观点和其真实性负责;
2.转载的文章仅代表原创作者观点,与本站无关。其原创性以及文中陈述文字和内容未经本站证实,本站对该文以及其中全部或者部分内容、文字的真实性、完整性、及时性,不作出任何保证或承若;
3.如本站转载稿涉及版权等问题,请作者及时联系本站,我们会及时处理。
登录后可拥有收藏文章、关注作者等权限...
技术大类 技术大类 | 数据库(综合) | 评论(0) | 阅读(92)