Zookeeper日志设置和清理
第二步:还需要改${zkhome}/bin/zkEnv.sh, 注意加粗处, 这时日志已经可以成功按照你设置的目录进行输出了# Define some default values that can be overridden by system properties zookeeper.root.logger=INFO,ROLLINGFILE zookeeper.console.threshold=INFO zookeeper.log.dir=. zookeeper.log.file=zookeeper.log zookeeper.log.threshold=DEBUG zookeeper.tracelog.dir=. zookeeper.tracelog.file=zookeeper_trace.log # # ZooKeeper Logging Configuration # # Format is "
(, )+ # DEFAULT: console appender only log4j.rootLogger=${zookeeper.root.logger} # Example with rolling log file #log4j.rootLogger=DEBUG, CONSOLE, ROLLINGFILE # Example with rolling log file and tracing #log4j.rootLogger=TRACE, CONSOLE, ROLLINGFILE, TRACEFILE # # Log INFO level and above messages to the console # log4j.appender.CONSOLE=org.apache.log4j.ConsoleAppender log4j.appender.CONSOLE.Threshold=${zookeeper.console.threshold} log4j.appender.CONSOLE.layout=org.apache.log4j.PatternLayout log4j.appender.CONSOLE.layout.ConversionPattern=%d{ISO8601} [myid:%X{myid}] - %-5p [%t:%C{1}@%L] - %m%n # # Add ROLLINGFILE to rootLogger to get log file output # Log DEBUG level and above messages to a log file log4j.appender.ROLLINGFILE=org.apache.log4j.DailyRollingFileAppender //按天日记轮转 log4j.appender.ROLLINGFILE.Threshold=${zookeeper.log.threshold} log4j.appender.ROLLINGFILE.File=${zookeeper.log.dir}/${zookeeper.log.file} # Max log file size of 10MB #log4j.appender.ROLLINGFILE.MaxFileSize=10MB # uncomment the next line to limit number of backup files #log4j.appender.ROLLINGFILE.MaxBackupIndex=10 log4j.appender.ROLLINGFILE.layout=org.apache.log4j.PatternLayout log4j.appender.ROLLINGFILE.layout.ConversionPattern=%d{ISO8601} [myid:%X{myid}] - %-5p [%t:%C{1}@%L] - %m%n # # Add TRACEFILE to rootLogger to get log file output # Log DEBUG level and above messages to a log file log4j.appender.TRACEFILE=org.apache.log4j.FileAppender log4j.appender.TRACEFILE.Threshold=TRACE log4j.appender.TRACEFILE.File=${zookeeper.tracelog.dir}/${zookeeper.tracelog.file} log4j.appender.TRACEFILE.layout=org.apache.log4j.PatternLayout ### Notice we are including log4j's NDC here (%x) log4j.appender.TRACEFILE.layout.ConversionPattern=%d{ISO8601} [myid:%X{myid}] - %-5p [%t:%C{1}@%L][%x] - %m%n
第三步：设置zookeeper.out zookeeper.out由nohup的输出，也就是zookeeper的stdout和stdeer输出，修改bin/zkServer.sh 文件：#!/usr/bin/env bash # Licensed to the Apache Software Foundation (ASF) under one or more # contributor license agreements. See the NOTICE file distributed with # this work for additional information regarding copyright ownership. # The ASF licenses this file to You under the Apache License, Version 2.0 # (the "License"); you may not use this file except in compliance with # the License. You may obtain a copy of the License at # # http://www.apache.org/licenses/LICENSE-2.0 # # Unless required by applicable law or agreed to in writing, software # distributed under the License is distributed on an "AS IS" BASIS, # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. # This script should be sourced into other zookeeper # scripts to setup the env variables # We use ZOOCFGDIR if defined, # otherwise we use /etc/zookeeper # or the conf directory that is # a sibling of this script's directory ZOOBINDIR="${ZOOBINDIR:-/usr/bin}" ZOOKEEPER_PREFIX="${ZOOBINDIR}/.." if [ "x$ZOOCFGDIR" = "x" ] then if [ -e "${ZOOKEEPER_PREFIX}/conf" ]; then ZOOCFGDIR="$ZOOBINDIR/../conf" else ZOOCFGDIR="$ZOOBINDIR/../etc/zookeeper" fi fi if [ -f "${ZOOCFGDIR}/zookeeper-env.sh" ]; then . "${ZOOCFGDIR}/zookeeper-env.sh" fi if [ "x$ZOOCFG" = "x" ] then ZOOCFG="zoo.cfg" fi ZOOCFG="$ZOOCFGDIR/$ZOOCFG" if [ -f "$ZOOCFGDIR/java.env" ] then . "$ZOOCFGDIR/java.env" fi if [ "x${ZOO_LOG_DIR}" = "x" ] then ZOO_LOG_DIR="/data/zookeeper/outlogs/" fi if [ "x${ZOO_LOG4J_PROP}" = "x" ] then ZOO_LOG4J_PROP="INFO,ROLLINGFILE" fi if [ "$JAVA_HOME" != "" ]; then JAVA="$JAVA_HOME/bin/java" else JAVA=java fi #add the zoocfg dir to classpath CLASSPATH="$ZOOCFGDIR:$CLASSPATH" for i in "$ZOOBINDIR"/../src/java/lib/*.jar do CLASSPATH="$i:$CLASSPATH" done #make it work in the binary package #(use array for LIBPATH to account for spaces within wildcard expansion) if [ -e "${ZOOKEEPER_PREFIX}"/share/zookeeper/zookeeper-*.jar ]; then LIBPATH=("${ZOOKEEPER_PREFIX}"/share/zookeeper/*.jar) else #release tarball format for i in "$ZOOBINDIR"/../zookeeper-*.jar do CLASSPATH="$i:$CLASSPATH" done LIBPATH=("${ZOOBINDIR}"/../lib/*.jar) fi for i in "${LIBPATH[@]}" do CLASSPATH="$i:$CLASSPATH" done #make it work for developers for d in "$ZOOBINDIR"/../build/lib/*.jar do CLASSPATH="$d:$CLASSPATH" done #make it work for developers CLASSPATH="$ZOOBINDIR/../build/classes:$CLASSPATH" case "`uname`" in CYGWIN*) cygwin=true ;; *) cygwin=false ;; esac if $cygwin then CLASSPATH=`cygpath -wp "$CLASSPATH"` fi #echo "CLASSPATH=$CLASSPATH"
第四步：快照事物日志 在使用zookeeper过程中，我们知道，会有dataDir和dataLogDir两个目录，分别用于snapshot和事务日志的输出（默认情况下只有dataDir目录，snapshot和事务日志都保存在这个目录中),通过zoo.cfg配置文件可配置：_ZOO_DAEMON_OUT="$ZOO_LOG_DIR/zookeeper.out" case $1 in start) echo -n "Starting zookeeper ... " if [ -f $ZOOPIDFILE ]; then if kill -0 `cat $ZOOPIDFILE` > /dev/null 2>&1; then echo $command already running as process `cat $ZOOPIDFILE`. exit 0 fi fi nohup $JAVA "-Dzookeeper.log.dir=${ZOO_LOG_DIR}" "-Dzookeeper.root.logger=${ZOO_LOG4J_PROP}" \ -cp "$CLASSPATH" $JVMFLAGS $ZOOMAIN "$ZOOCFG" > "$_ZOO_DAEMON_OUT" 2>&1 < /dev/null &
#snapshot file dir dataDir=/data/zookeeper/data #tran log dir dataLogDir=/data/zookeeper/log二、日志清理 正常运行过程中，ZK会不断地把快照数据和事务日志输出到这两个目录，并且如果没有人为操作的话，ZK自己是不会清理这些文件的，需要管理员来清理，这里介绍4种清理日志的方法。在这4种方法中，推荐使用第一种方法，对于运维人员来说，将日志清理工作独立出来，便于统一管理也更可控。毕竟zk自带的一些工具并不怎么给力，这里是社区反映的两个问题： https://issues.apache.org/jira/browse/ZOOKEEPER-957 http://zookeeper-user.578899.n2.nabble.com/PurgeTxnLog-td6304244.html 第一种，也是运维人员最常用的，写一个删除日志脚本，每天定时执行即可：
#!/bin/bash
#snapshot file dir
dataDir=/data/zookeeper/data/version-2
#tran log dir
dataLogDir=/data/zookeeper/log/version-2
#zk log dir
logDir=/data/zookeeper/outlog/
#Leave 60 files
count=60
count=$[$count+1]
ls -t $dataLogDir/log.* | tail -n +$count | xargs rm -f
ls -t $dataDir/snapshot.* | tail -n +$count | xargs rm -f
ls -t $logDir/zookeeper.log.* | tail -n +$count | xargs rm -f以上这个脚本定义了删除对应三个目录中的文件，保留最新的60个文件，可以将他写到crontab中，设置为每天凌晨2点执行一次就可以了。
第二种，使用ZK的工具类PurgeTxnLog，它的实现了一种简单的历史文件清理策略，可以在这里看一下他的使用方法：http://zookeeper.apache.org/doc/r3.4.3/api/index.html ，可以指定要清理的目录和需要保留的文件数目，简单使用如下：java -cp zookeeper.jar:lib/slf4j-api-1.6.1.jar:lib/slf4j-log4j12-1.6.1.jar:lib/log4j-1.2.15.jar:conf org.apache.zookeeper.server.PurgeTxnLog第三种，对于上面这个Java类的执行，ZK自己已经写好了脚本，在bin/zkCleanup.sh中，所以直接使用这个脚本也是可以执行清理工作的。 第四种，从3.4.0开始，zookeeper提供了自动清理snapshot和事务日志的功能，通过配置 autopurge.snapRetainCount 和 autopurge.purgeInterval 这两个参数能够实现定时清理了。这两个参数都是在zoo.cfg中配置的：-n
autopurge.purgeInterval: 24*2 autopurge.snapRetainCount: 2autopurge.purgeInterval 这个参数指定了清理频率，单位是小时，需要填写一个1或更大的整数，默认是0，表示不开启自己清理功能。 autopurge.snapRetainCount 这个参数和上面的参数搭配使用，这个参数指定了需要保留的文件数目。默认是保留3个。 参考：http://nileader.blog.51cto.com/1381108/932156
