13518219792
这篇文章主要介绍了Flume怎样安装部署，具有一定借鉴价值，感兴趣的朋友可以参考下，希望大家阅读完这篇文章之后大有收获，下面让小编带着大家一起了解一下。
环境：
1.flume-ng-1.6.0-cdh6.7.0.tar.gz
2.jdk 1.8
1.下载安装包并解压
wget http://archive.cloudera.com/cdh6/cdh/5/flume-ng-1.6.0-cdh6.7.0.tar.gz
tar -xvf flume-ng-1.6.0-cdh6.7.0.tar.gz
rm flume-ng-1.6.0-cdh6.7.0.tar.gz
mv apache-flume-1.6.0-cdh6.7.0-bin flume
2.配置环境变量
vi /etc/profile
export FLUME_HOME=/opt/software/flume
exportPATH=.:$FLUME_HOME/bin:$KAFKA_HOME/bin::$ZOOKEEPER_HOME/bin:$PATH
source /etc/profile
3. 配置flume-env.sh文件
cd /opt/software/flume/conf/
cp flume-env.sh.template flume-env.sh
vi flume-env.sh
export HADOOP_HOME=/opt/software/hadoop
export JAVA_HOME=/usr/java/jdk1.8.0_45
4.版本验证
flume-ng version
5.Flume部署示例
cd /opt/software/flume/bin/
vi netcat.conf
# Name the components on this agent
a1.sources = r1
a1.sinks = k1
a1.channels = c1
# Describe/configure the source
a1.sources.r1.type = netcat
a1.sources.r1.bind = localhost
a1.sources.r1.port = 44444
# Describe the sink
a1.sinks.k1.type = logger
# Use a channel that buffers events in memory
a1.channels.c1.type = memory
a1.channels.c1.capacity = 1000
a1.channels.c1.transactionCapacity = 100
# Bind the source and sink to the channel
a1.sources.r1.channels = c1
a1.sinks.k1.channel = c1
6.运行FlumeAgent，监听本机的44444端口
flume-ng agent -c conf -f netcat.conf -n a1 -Dflume.root.logger=INFO,console
[root@hadoop001 bin]# flume-ng agent -c conf -f netcat.conf -n a1 -Dflume.root.logger=INFO,console
Info: Including Hadoop libraries found via (/opt/software/hadoop-2.6.0-cdh6.7.0/bin/hadoop) for HDFS access
Info: Excluding /opt/software/hadoop-2.6.0-cdh6.7.0/share/hadoop/common/lib/slf4j-api-1.7.5.jar from classpath
Info: Excluding /opt/software/hadoop-2.6.0-cdh6.7.0/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar from classpath
Info: Including Hive libraries found via (/opt/software/hive-1.1.0-cdh6.7.0) for Hive access
.......
18/03/25 17:52:39 INFO instrumentation.MonitoredCounterGroup: Component type: CHANNEL, name: c1 started
18/03/25 17:52:39 INFO node.Application: Starting Sink k1
18/03/25 17:52:39 INFO node.Application: Starting Source r1
18/03/25 17:52:39 INFO source.NetcatSource: Source starting
18/03/25 17:52:40 INFO source.NetcatSource: Created serverSocket:sun.nio.ch.ServerSocketChannelImpl[/127.0.0.1:44444]
7.打开另外一个client
telnet localhost 44444
[root@hadoop001 ~]# telnet localhost 44444
Trying ::1...
telnet: connect to address ::1: Connection refused
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
111
OK18/03/25 18:11:15 WARN source.NetcatSource: Client sent event exceeding the maximum length
18/03/25 18:12:50 INFO sink.LoggerSink: Event: { headers:{} body: 31 31 31 0D 111. }
8.完毕
感谢你能够认真阅读完这篇文章，希望小编分享的“Flume怎样安装部署”这篇文章对大家有帮助，同时也希望大家多多支持创新互联，关注创新互联-成都网站建设公司行业资讯频道，更多相关知识等着你来学习!
