Linux – rsyslog server template consideration for multiple remote hosts

linuxrsyslog

Linux Destro is : Centos 6.8

Below is what the rsyslog.conf looks like:

# cat /etc/rsyslog.conf

$ModLoad imuxsock # provides support for local system logging (e.g. via logger command)
$ModLoad imklog   # provides kernel logging support (previously done by rklogd)
#$ModLoad immark  # provides --MARK-- message capability
# Provides UDP syslog reception
$ModLoad imudp
$UDPServerRun 514
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
$template server-dc, "/scratch/rsyslog/%HOSTNAME%/messages.log"
authpriv.*   ?server-dc
*.info,mail.none,authpriv.none,cron.none   ?server-dc
$IncludeConfig /etc/rsyslog.d/*.conf

#*.info;mail.none;authpriv.none;cron.none                /var/log/messages
#*.*                                                     @elk-rsyslog:514

Point for me to understand:

1)

$template server-dc, "/scratch/rsyslog/%HOSTNAME%/messages.log"
enter code here

So, in the above line /scratch/rsyslog is a Directory path on the rsyslog server where we are forwarding all the logs from the unix remote hosts which crearts a directory structure following a message file like:

/scratch/rsyslog/remot1-Ser1/messages.log
/scratch/rsyslog/remot1-Ser2/messages.log

So, this works okay, However at the same time we have few network devices as well which are forwarding the network logs into the same location in a below format:

/scratch/rsyslog/Sep/messages.log
/scratch/rsyslog/Oct/messages.log

In the above case of network logs its creating a Directory by month name following a message file So, I'm looking for a way in rsyslog to define a different path for network logs as such /scratch/rsyslog/network so the network logs can be collected into a Separate Folder, reason behind this is, i'm processing these logs to Elasticsearch hence i'm using wildcard for unix logs as /scratch/rsyslog/*/messages.log but this also includes the network logs being wildcard(*) called.

So, Is there a way i can say if the logs are coming from particular remote host or IP should go to a particular Folder in rsyslog server?

2) Here in $template server-dc, "/scratch/rsyslog/%HOSTNAME%/messages.log" what is login for putting server-dc as this is a another Hostname, Is this gets any purpose or we can change this to something relevant considering above requirement.

3) Same server-dc also been included into another config parameters as well.

Desired

Would simply like to know if there is way i can Forward the logs for like systems , network, firewall into the separate distinct Directories like..

1 – /scratch/rsyslog/system/%HOSTNAME%/messages.log

2 – /scratch/rsyslog/network/%HOSTNAME%/messages.log

3 – /scratch/rsyslog/firewall/%HOSTNAME%/messages.log

Best Answer

You can have any number of templates, and test incoming messages for their hostname or ip address. If your hostnames are well-structured, for example all "systems" start with "sys" such as sys10 and sysabc, then the number of tests can be reduced.

For example,

$template mysystems,"/scratch/rsyslog/system/%HOSTNAME%/messages.log"
$template mynets,"/scratch/rsyslog/network/%HOSTNAME%/messages.log"
$template myfirewalls,"/scratch/rsyslog/firewall/%HOSTNAME%/messages.log"

if $fromhost startswith "sys" then -?mysystems
& stop
if $fromhost startswith "net" then -?mynets
& stop
if $fromhost startswith "fw" then -?myfirewalls
& stop

The & stop line stops the message that matches the previous line from being treated further. You can test the ip address with, for example,

if $fromhost-ip startswith "192.168." then -?mynets

If you want to keep the *.info,... filter, you can modify the above, for example, 

if $fromhost startswith "sys" then {
  *.info,mail.none,authpriv.none,cron.none   -?mysystems
  & stop
}

Note, however, that if you want to not log some items, you should really do this filtering at the sender, not at this end of the network. It is just wasting network bandwidth to send messages that you then filter out and throw away.

See the extensive rsyslog documentation, noting which version you have.

Related Question