将查日志的工作从运维部转至项目部

• 记日志
  • 日志的级别 → debug/notice/warning
  • 记在哪 → rsyslog 本地记录 + 远程集中记录¹⁾
    • rsyslog 如何在 both 本地 and 远端记录? → central_logging
    • 如何在(PHP)程序中调用 rsyslog 接口? → php syslog()
    • 如何把 php 提交的日志记录在恰当的文件里? → 配置详解之 RULES
• 读日志
  • 什么工具好?
    • Adiscon LogAnalyzer - syslog web viewer, analysis and reporting tool
    • Graylog2
      • rsyslogd + graylog2 日志管理方案(1)
  • 日志的意义 → 程序中要有关于日志的文档
  • 搜索和过滤

Rsyslog is an enhanced multi-threaded syslogd with a focus on security and reliability. Among others, it offers support for on-demand disk buffering, reliable syslog over TCP, SSL, TLS and RELP, writing to databases, email alerting. It is a drop-in replacement for syslogd.

http://wiki.debian.org/Rsyslog

Rsyslog has a modular architecture. It enables functionalities to be added dynamically through these modules. The modules are categorized as:

• Input Modules – Used to gather messages from various sources
• Output Modules – Used to write the messages to various places ( file, socket etc.. )
• Parser Modules – Used to parse the message content

######################
	MODULES
######################
 
$ModLoad imuxsock
$ModLoad imklog
 
######################
	Directives
######################
# Set the default permissions for all log files. 
 
$FileOwner root
$FileGroup adm
$FileCreateMode 0640
$DirCreateMode 0755
 
######################
	RULES
######################
mail.info    /var/log/mail.info
mail.warn    /var/log/mail.warn
mail.err     /var/log/mail.err
daemon.*     /var/log/daemon.log

Every rule line consists of two fields, a ‘selector field’ and an ‘action field’. The selector field is divided into two, ‘facilities & priorities’. Action specifies what action must be taken for the matched rule.

syslog 基本的 selectors 说明如下:

1. facility.priority: a facility and a priority, separated by a period ('.'). Messages refer to a facility (auth, authpriv, daemon, cron, ftp, lpr, kern, mail, news, syslog, user, uucp, local0, … , local7 ) and are assigned a severity (debug, info, notice, warning, warn (same as warning), err, error (same as err), crit, alert, emerg, panic (same as emerg)) by the sender of the message.
   1. 自定义程序使用 LOCAL1 比较合适, logging - Which program defaults uses syslog local[0-7] facilities? - Server Fault

      LOCAL0 is used by postgresql
      LOCAL2 is used by sudo
      LOCAL3 is used by some versions of SpamAssassin
      LOCAL4 is used by default by slapd (OpenLDAP server)
      LOCAL5 is sometimes used by the Snort IDS
      LOCAL7 is used for boot messages on Fedora 12

2. >=: default is that all messages of the specified priority and higher are logged according to the given action.
3. *.*: An asterisk ('*') stands for all facilities or all priorities.
4. ,: You can specify multiple facilities with the same priority pattern in one statement using the comma (',') operator.
5. ;: Multiple selectors may be specified for a single action using the semicolon (';') separator.

rsyslog 对基本的 selectors 做了如下扩展:

1. =: You may precede every priority with an equals sign ('=') to specify only this single priority and not any of the above.
2. <: You may also (both is valid, too) precede the priority with an exclamation mark ('!') to ignore all that priorities, either exact this one or this and any higher priority.

rsyslog 还提供了 Property-Based Filters 和 Expression-Based Filters 两种更高级的条件筛选

Property-Based Filters 的表达式形如:

:property, [!]compare-operation, "value"

• property 的完整列表见 The Property Replacer
• compare-operation 包括 contains, isempty, isequal, startswith, regex, ereregex. 需注意 startswith 测试总不符合期望, google 后发现反映此问题的 bug lauchpad Bug #479592, 还未解决

c-like, 详见 RainerScript,

• Syslog
• www.rsyslog.com/doc/rsyslog_conf_filter.html
• http://www.rsyslog.com/doc/rsyslog_recording_pri.html
• http://www.rsyslog.com/doc/rsyslog_conf_examples.html
• http://wiki.rsyslog.com/index.php/Configuration_Samples

• 普通文件 文件及父级目录不需提前建立, rsyslog 会自动建立

  *.*     /var/log/file.log # log to a file with RFC3339 timestamps

• Remote machine

  *.* @192.168.0.1 // via UDP
  *.* :omrelp:192.168.0.1:2514 // RELP will prevent message loss

• 显示给某(些)用户

  *.emerg :omusrmsg:root,user1,user2
  *.emerg :omusrmsg:*

• 不记录

  *.*   ~      # discards everything.

更多见 man 5 rsyslog.conf

/etc/rsyslog.conf

# provides support for local system logging
$ModLoad imuxsock 
 
# provides kernel logging support (previously done by rklogd)
$ModLoad imklog
 
# provides UDP syslog reception. For TCP, load imtcp.
$ModLoad imudp
 
# For TCP, InputServerRun 514
$UDPServerRun 514
 
$template REMOTE,"/var/log/gstation/%fromhost-ip%.log"
if $fromhost-ip != '127.0.0.1' then ?REMOTE

需注意 rsyslog 用户($FileOwner syslog, $FileGroup adm)需对 log 目录有写权限

/etc/rsyslog.conf

$ModLoad imuxsock
 
$ModLoad imklog
 
$IncludeConfig /etc/rsyslog.d/*.conf
 
# Provides UDP forwarding. The IP is the server's IP address
*.* @192.168.1.1:514 
 
# Provides TCP forwarding. But the current server runs on UDP
# *.* @192.168.1.1:514

在服务器/客户端都已配置好, 且重启后, 可在客户端 sudo echo 'hello' , 若设置成功, 则在客户端的 /var/log/auth.log³⁾, 及服务器的 /var/log/192.168.1.2/syslog.log 中都会有相应记录.

另外, 由于 /etc/rsyslog.d/50-default.conf 中有规则 auth,authpriv.* /var/log/auth.log, 所以本地远程都有 log.

How to Setup Rsyslog Remote Logging on Linux (Central Log Server)

没什么用

A sample template will look like:

$template mytemplate “Text-Before %msg% Text-After\n”

The above template will log the message “This is hello from rsyslog” as:

Text-Before This is hello from rsyslog Text-After

详见: Bug #523610 “rsyslogd spins CPU on some kernels” : Bugs : “rsyslog” package : Ubuntu

2 种解决办法:

1. 关闭 rsyslog 某模块: sed -i -e 's/^\$ModLoad imklog/#\$ModLoad imklog/g' /etc/rsyslog.conf
2. 安装老版本 rsyslog