Real Monitoring and alarm

LogMon monitors logs collected in real time and and generates alarms for suspicious event
LogMon GUI(Xlogmaster) can be used for looking at log files in real time.
more about Xlogmaster is given in 'Using Xlogmaster' section.

Log Archival

Logs collected are moved to NAS on periodic basis.
LogMon is provided with NFS mount point. moving log file on NAS helps
to keep disk utilization low.

Using of LogMon.

1. Collecting logs sent by TailLog
   
   
   $>logrotate.pl [port no] [path/to/logfile] [directory/to/which/logfile/to/be/rotated/]
   
   
   TailLog sends log on UDP port specified in configuration file. logrotate.pl can read that port
   and collect the logs and write into logfile. every day at 00:00:00 logfile will be rotated and
   will go to specified directory with year and month hierarchy and time extention
   (e.g. file
   av_imss rotated on 3 apr 2006 might be placed as <dir>/2006/04/av_imss-20060404-000003)
   
2. Collecting logs from CheckPoint FireWall
   
   
   $>fw1-loggrabber -l /path/to/lea_R54.conf -c /path/to/fw1-loggrabber_R54.conf
   
   
   LogMon provides wrapper to run this command, assuring that even if the connection is broken
   for some reason command will be called again so that log collection will continue
   
3. Collecting logs from Cisco IDS
   
   $>cisco_redirect.pl
   
   cisco_redirect.pl uses configuration file to know about details about Cisco IDS server
   
   details are given in 'Configuring LogMon' section
   
   
4. Starting log collection from all devices
   
   $>call_logrotate.pl
   
   'call_logrotate.pl' uses configuration file to know about
   
   • for which devices logging is to started.
   • which devices uses TailLog client.
   • if TailLog then details about port no., file name,rotation directory.
   • if fw1-loggrabber then fw1-loggrabber's configuration files
   • if Cisco IDS then Cisco IDS's conf file

   as CiscoPix FireWall logs are collected using syslog log
   collection will start at system start.
   

5. Generating report
   
   
   $>log_process.pl [OPTIONS] [DEVICE] [INPUT DIR] [OUTPUT FILE] [FROM TIME] [TO TIME]
   
   
   options can be '-daily' or '-agg' depending upon how report is to be generated
   
   Device name can be any from this list
   
   av_imss, av_iwss, av_symantec1, av_symantec2, ids_rs, ids_cisco, fw_pix, fw_r55, fw_r54
   
   input directory should be upto device level depth
   for '-daily' option,and upto final level for '-agg' option.
   
   output file is a report file in which report generated will be written.
   
   from time and to time ,here u can specify duration for which report is to be generated.
   Time can be provided in any format.
   
   e.g.
   
   
   $>log_process.pl -daily ids_rs /opt/logmon/ROTATED/ids/ids_rs/
   /opt/logmon/REPORTS/ids/ids_rs/DAILY/2006/07/ids_rs-20060709 2006070900:00:00 2006070923:59:59
   
   
   here actual log file will be at /opt/logmon/ROTATED/ids/ids_rs/<year>/<month>/
   
   log_process.pl will select appropriate log files
   
   $>log_process.pl -agg fw_pix /var/tmp/pixtempdir/ /opt/logmon/REPORTS/fw/fw_pix/WEEKLY/2006/fw_pix-13 "4 June 2006 00:00:00" "10 June 2006 23:00:00"
   
   
   
6. Generating reports for all devices
   
   $>call_logprocess.pl
   
   call_logprocess.pl uses configuration files to know about details like
   
   • where are the log files
   • where to put reports generated
   • whether weekly report are to be generated

   For the daily reports time duration will be taken previous days 24 hours.
   
   i.e. if fired on 24 march 2006 then time duration taken will be from "23 march 2006 00:00:00" to "23 march 23:59:59"
   
   For weekly reports duration will be from "17 march 2006 00:00:00" to "23 march 2006 23:59:59"

7. Send mail from LogMon
   
   $>mail.pl [options]
   
   Options can be any from following list
   
   • -smtp_server
   • -from
   • -to
   • -cc
   • -bcc
   • -subject
   • -body_header
   • -body_content
   • -body_footer
   • -footer
   • -attach

   options like to,cc,bcc and attach can take multiple values and should be provided in comma
   separated form
   
   unspecified options value will be taken from configuration file
   
   more will be in 'Configuring LogMon' section
   
   mail.pl is mainly used for sending reports and alerts to concern person
   

8. Send generated reports
   
   $>call_mail.pl
   
   'call_mail.pl' will send previous days reports to address specified in configuration file
   

Configuring of LogMon.

• chkpt_ignore_pattern_file [30]
• chkpt_r54_protected_ip_file [31]
• chkpt_r55_protected_ip_file [32]
• cisco_ids_severity_file [33]
• device_port.conf [34]
• error_constants.pm [35]
• false_positive_source_ip [36]
• log_paths.pm [37]
• log_process_constants.pm [38]
• mail_conf [39]
• name_for_ip.pm [40]
• pix_conf [41]
• pix_conf_parent_file [42]
• pix_errcodes [43]
• rs_ids_severity_file [44]

Here is the details about each file

1. chkpt_ignore_pattern_file
   
   This file contains patterns in log file for which log entry is to be ignored.
   
   e.g. sample entry can be like this,
   
   action=accept
   
   This will ignore all entries for which action is accept.
2. chkpt_r54_protected_ip_file
   
   This file contains entry about ip address and its name, for which detailed
   information is to be provided in report file.
   
   Regular expression is also supported
   
   i.e.entry like this
   
   172.16.*.* local LAN
   
   will produce detailed information about all ip addresses in 172.16. range.
3. chkpt_r55_protected_ip_file
   
   This file contains entry about ip address and its name,
   for which detailed information is to be provided in report file.
   
   Regular expression is also supported
   
   i.e.entry like this
   
   172.16.*.* local LAN
   
   will produce detailed information about all ip addresses in 172.16. range.
   
   Only difference in report of CheckPoint-R55 and CheckPointR54 is
   
   protected check is done against Destination IP in cake of CheckPoint-R55 whereas
   
   for CheckPoint-R54 protected check is done against Source IP.
4. cisco_ids_severity_file
   
   This file contains 'severity signature' pair.
   
   In Cisco IDS logfile each entry has severity defined ,but adding entry in this file
   for some signature will cause severity defined in logfile to be replaced with what is
   mentioned in this file.
   
   e.g.
   
   For signature 'Half-open sys' severity in log file may be 'High' but
   adding entry in this file like
   
   'low Half-open Syn'
   
   will cause is to be displayed 'low' in prepared report file.
5. rs_ids_severity_file
   
   This file contains 'severity signature' pair and works in exactly same fashion as it works
   for Cisco IDS.
   
6. device_port.conf
   
   This file contains entries like
   
   <device_name>=<port_no> or
   
   <device_name>=n/a
   
   This file will be used to know about the port no. to be used for
   collecting logs from TailLog client.
   
   Devices which do not use TailLog client for those it would be n/a i.e not applicable
   
   Note: Make sure that port no. given in TailLog client's configuration file running on
   respective device matches with what is given here.
   
7. error_constants.pm
   
   This file contains information about error messages that are to be displayed when user
   fires wrong command line for 'log_process.pl'.
   
   user don't need to change/edit this file.
   
8. false_positive_source_ip
   
   This file contains 'Source IP Destination IP' pair for which alert generated in Cisco
   IDS will be considered as false positive events.
   
   Report generated will be equipped with comment column for this events.
   
   Regular expression is supported for IP
   
   e.g. adding entry like this
   
   130.1.3.* 195.1.*.*
   
   will cause all events for with source address from 130.1.3 network and destination
   address from 195.1 network, to be considered as false positive events.
9. log_paths.pm
   
   This file contains information about all the parameters like
   
   path of all configuration files mentioned here
   
   path of all scripts used by LogMon
   
   path of all directories used by LogMon
   
   user don't need to edit this file.
10. log_process_constants.pm
    
    This file is perl module and contains all constants used.
    
    user don't need to edit this.
    
11. mail.conf
    
    This file contains all the details required for sending mail from LogMon server.
    
    Using 'mail.pl' which includes following parameters.
    • smtp_server
    • from
    • to
    • cc
    • subject
    • body_header
    • body_content
    • body_footer
    • footer
    • attach

    User can specify value of these parameter in this file or in command line.
    
    Unspecified parameters value will be taken from this file.
    

12. name_for_ip.pm
    
    It's always easy and better for one to know machine by its name rather than ip address.
    
    This file contains name to be used in report for ip's encountered in log file.
    
    Format of this file is like hash map where 'ip' is key and 'its name' would be the value.
    
13. pix_conf, pix_conf_parent_file, pix_errorcodes
    
    These files are used by logprocess.pl in preparation of Cisco Pix FireWall report
    user don't need to edit these files.

Using Xlogmaster.

System Services and Packages Used

• Syslog
• NTP
• Perl-5.8
• fw1-loggrabber-1.11.1
• TailLog-1.5
• Net::RDEP
• Xlogmaster-1.6.0

Syslog

As described in 'Working of LogMon' section logs of Cisco PIX FireWall are collected
using Syslog

as this logs are coming from remote machine some changes are required in file named
'/etc/sysconfig/syslog'

default syslog configuration will not be able to receive logs from remote machine to do such provision
file '/etc/sysconfig/syslog' needs to be changed

initially line will be as shown below

SYSLOGD_PARAMS=""

changing it to

SYSLOGD_PARAMS="-r"

will enable remote logging and system will be able to receive logs sent by remote machine

Time Synchronization

One of the important thing in process of log collection and report generation
is time synchronization

NTP (Network Time Protocol) is utility to synchronize two Unix machine

adding following entries in file '/etc/ntp.conf' of any device machine will make device
machine to refer time from LogMon machine periodically
server <LogMon machine IP>
where as LogMon itself refers time from Time Server in USA

entries of file '/etc/ntp.conf' of LogMon looks like this

     server 129.6.15.28
     server 129.6.15.29
     server 132.163.4.101
     server 132.163.4.102
     

To synchronize MS-Windows machine with LogMon machine LogMon uses software called 'AboutTime'

Giving LogMons IP as Network Time Server will make Windows machine to refer time from LogMon

Here is a typical client session. In this example, AboutTime reports a 3 millisecond difference between the local computer's clock and the network time server.

Here is AboutTime's own four-server "traffic" display. It shows that one of the server protocols is unavailable (red), and another is responding to a request (yellow).

Perl-5.8

Perl is the programming language used for writing scripts. These scripts runs as daemons
and collects logs from various devices and generate reports.
www.perl.com [46] is official perl home page run by O'Reilly.

fw1-loggrabber-1.11.1

FW1-Loggrabber is a command-line tool to grab log files from Checkpoint FW-1 remotely using
Checkpoints LEA (Log Export Api), which is one part of Checkpoints OPSEC API.
More information can be found at
fw1-loggrabber site [47]

TailLog-1.5

TailLog is a command line utility to collect logs from remote MS-Windows machine.
TailLog can monitor a folder in which log files resides and send UDP packets as logfile is
growing, on desired machine and port.

Binary and source code are included with this documentation CD.

Net::RDEP

Remote Data Exchange Protocol (RDEP) is a protocol designed by Cisco Systems in order to
exchange Intrusion Detection System events, configuration, log, and control messages. This
protocol is supported at least the Cisco IDS version 4.0.



This implementation only supports the collection of events.



The events are retrieved in a format, also developed by Cisco, referred to as Intrusion
Detection Interaction and Operations Messages (IDIOM), which is an XML document. A simple
module for handling these documents is included with this distribution (XML::Idiom).



RDEP supports two methods for retrieving events: an event query and an event
subscription. Both methods use SSL to query the RDEP server and retrieve the events. The
event query method will retrieve all the events in a given time range. No connection is
maintained in anyway. The event subscription, however, does maintain a connection and will
support multiple "gets" to continue to retrieve events as they are available.



See www.cisco.com [48]for more information in RDEP or IDIOM.

Xlogmaster-1.6.0

Xlogmaster allows you to monitor all your log files and all
devices that allow being read by cat. Furthermore it supplies
several filters that will do things from raising/hiding lines in the
text display to starting programs to send you a paging message about a
possible security breach.

If used wisely Xlogmaster can make it very easy to keep track of
everything that happens on your system and it can increase your systems
security by quite a lot.

Directory Structure and Naming.

Making Dumb Terminal.

you want to make Home machine is dumb terminal for Office machine



steps are as follows

1. on Office machine

   1. edit file "/etc/opt/kde3/share/config/kdm/kdmrc"
      
      [xdmcp]
      
      Enable=true
      
      port=177
      

   2. edit file "/etc/X11/xdm/xdm-config"
      
      Comment out this line
      
      "DisplayManager.requestPort: 0"
      
      and make it like
      
      "!DisplayManager.requestPort: 0"
      

2. on Home machine

   edit file "/etc/inittab"
   
   add line
   
   7:123:respawn:/usr/X11R6/bin/X -query [Office_ip] >/var/tmp/xserver.log 2>&1 tty7
   
   at proper place number 7 may

now you should see login prompt of Office machine on your Home machine

if not, configure firewall of Office machine to allow request from Home machine

Log Archival.

Shutting down and Starting LogMon.

Shutting down LogMon

LogMon can be turned down using following comamnd

$>shutdown -h now

to restart LogMon server use fillowing comamnd

$>shutdown -r now

to run this command super user privilege are required.
User should close all LogMon GUI (Xlogmaster) application before putting LogMon down.

Starting LogMon

All the services e.g. NTP, Syslog, LogMon daemon and Xlogmaster should automatically
start on startup of LogMon

few tests are required to check this. More about this in 'How To/Trouble Shooting' section

How To/FAQ.

1. I want to setup TailLog client on my machine
   

   get the TailLog binary from this CD or any other resources
   
   which include other machines on which TailLog is running
   
   prepare a folder on disk containing these three files
   
   1. TailLog.exe
   
   2. TailLog.cfg
   
   3. TailLog.bat
   
   
   
   bat file should do two things
   
   1. go to folder containing TailLog.exe and TailLog.cfg
   
   cd <Folder_name>
   
   2. fire command to start TailLog
   
   TailLog.exe TailLog.cfg
   
   
   
   now add this bat file in 'system Scheduled Tasks' to run on startup using standard procedure
   configure TailLog to run forever in scheduled task (default is to run for 72
   hours).
   now TailLog is ready to run on startup. To start TailLog now right click on TailLog
   entry in Scheduled Task list and select 'run'
   

   Configuring TailLog
   
   edit the TailLog.cfg according to need
   

   Parameter	description
   version	version of TailLog Currently 1.5
   watchDirName	Folder in which Log files resides on server
   tag_name	something to identify which device e.g. ids_rs for RealSecure IDS
   pipe_flag	give 1, to enable pipeLog mode
   command_to_execute	in pipeLog mode this command will be executed and output will be sent to server
   file_pattern	extention of log files e.g. *.log
   diagnostic_tag	any thing but must start with '###'
   diagnostic_flag	keep 1, to enable diagnostic messages on,recommended 1
   skip_flag	keep 1, to get current entries, recommended 1
   check_interval	time interval to check update in log file, recommended value is 5
   send_attempts	TailLog will try to send a packet for maximum this times, recommended value is 100
   attempts_for_mark	number of times to check standard output before sending mark signal,used in pipeLog mode
   send_summary_interval	Summary packet will be sent if no updates are found for this many seconds, recommended value is 20
   ServerName	IP of LogMon Server
   ServerPort	Port number on which to send packets,make sure that one instance of 'logrotate.pl' is listening on this port
   SleepInterval	in milliseconds,time for which TailLog will sleep after sending last packet,recommended value is 2000
   Debug	Flag to log any debugging message from TailLog.keep 0

2. TailLog is giving error saying 'mswinsck.ocx' not found when started.

   file 'mswinsck.ocx' is included with TailLog source and binary.
   
   copy this file in 'C:\windows\system32\'
   
   now go to command prompt and register this ocx file by firing this command
   
   C:\> regsvr32 C:\windows\system32\mswinsck.ocx

3. I want to install AboutTime on my machine.

   Get executable from this CD or download from
   AboutTime Site [49]
   double click on .exe file and installation will start automatically.
   
   any further details can be found at this site.
   

4. I am receiving Alerts from LogMon saying 'Logfile not updated'

   may be logging is stopped for this device
   
   if the device is windows machine (IMSS, IWSS, Symantec, IDS RS) check whether
   TailLog client is running
   
   using following procedure
   
   1. Right click on empty area of task bar
   
   2. click on Task Manager
   
   3. click on Processes
   
   4. select checkbox to see processes from users
   
   5. see if TailLog is present in that list
   
   6. if not restart the TailLog using Scheduled Task.
   
   
   
   if device is checkpoint FireWall
   
   then restart the loggrabber using this command.
   
   $> fw1_restart.pl
   
   this script fired automatically once a day to reestablish the connection so that connection
   with CheckPoint FireWall at most one day old.

5. I am receiving Alerts from LogMon saying 'NTP Server not responding'

   see how many servers(IP) alert is about
   
   there are total four NTP servers from which LogMon is taking time
   
   if it is about 1,2 or 3 servers then this alert can be treated as informational
   alert

6. I am not able to receive daily report mails from LogMon

   See if internal mail system is working fine
   
   if yes then check the SMTP servers IP given in mail.conf file in CONFIGURATION
   directory.
   more detailed information can be found at logfile at LOGMON_LOGS directory,mail.log
   file
   
   if mail was sent from LogMon this file must have entry about mail sent details.

7. I am receiving blank reports from LogMon

   see if any entries were there in log file for that day for that device.
   if log file consist of only diagnostic messages (for Windows based machine)
   
   then it might be the case that TailLog is observing wrong Logfile or wrong
   Directory
   
   for any other device you should receive 'Log file not updated' alert before blank
   report mail
   

8. I want to check whether all services (NTP, Syslog, LogMon daemon) are running

   to check NTP, fire this command on console
   
   $> /etc/init.d/xntpd status
   
   result should show you 'running'
   

   to check Syslog, fire this command on console
   
   $> /etc/init.d/syslog status
   
   result should show you 'running'
   
   
   you need superuser privilege to check this
   

   if result is 'unused' fire command like
   
   $> /etc/init.d/xntpd start
   
   $> /etc/init.d/syslog start
   
   
   
   to see LogMon daemon running or not best method will be to see whether all log files
   are updating or not
   
   another method is see see file 'device_port.conf' in 'CONFIGURATION' directory
   which contains port numbers on which 'logrotate.pl' is listening now fire command
   
   $> fuser <port_no>/udp
   
   if this command shows you some process ID then it means that port is in use. To see
   which process is using this port fire this command
   
   $> ps -p <PID>
   
   this command should show 'logrotate.pl' in command column
   
   if not then that it means that log collection for this port's device is stopped
   
   and you should receive alert from LogMon saying 'Log file not updated'
   

9. I want to see disk space utilization

   following command will show the disk space used
   
   $> df -kh
   

10. I want to see system performance
    

    following command will show systems performance in real time
    
    $> top
    
    use q to exit

Appendix.

TailLog Client pseudo code

TailLog pseudo code is provided with source of TailLog Client included in this CD

click
here [50]
to view TailLog pseudo code

schema for device log file

Here is the description about each field

• access-group :
  
  this field is found in Cisco PIX firewall, and contains information about which
  group is considered while taking action. Various possible values are "inside",
  "inside_access_in", "outside" etc..
  
  
• action :
  
  action taken for encountered attack, can be 'deleted', 'quarantine' for antivirus
  and 'accept' or 'drop' for firewall
  
  
• date_time :
  
  date and time of encountered log event, different devices have this value in different format,
  LogMon converts it into single format.
  
  
• dest_addr :
  
  This field is found in all security devices. And tells about destination for
  which the log entry was generated.
  
  
• dest_nat_addr :
  
  This field is found in Checkpoint firewall and tells about natted IP address of
  destination.
  
  
• dest_port :
  
  This field is found in firewall and IDS and tells about port number on
  destination for which log entry was generated
  
  
• direction :
  
  This field from Cisco PIX firewall, and tells about from direction in which
  packet was traveling eg. inside to out side or inside to inside.
  
  
• facility :
  
  PIX firewall contains field called facility, each log entry contains numeric
  facility and is much similar to severity field.
  
  
• file :
  
  this field is found in antivirus and contains information about filename found
  to be infected
  
  
• interface :
  
  This field is found in Cisco PIX firewall. tells the interface on which packet
  was found and further action was taken
  
  
• intruder_addr :
  
  This field is found in RS IDS, This IP is the IP from which the attack was
  originated
  
  
• intruder_port :
  
  This field is found in RS IDS, This port number is the port number on intruder
  machine
  
  
• method :
  
  This field is found in IWSS method can be protocol used when virus was found.
  Possible values are 'http', 'https' etc..
  
  
• name :
  
  name is name of virus found in case of Antivirus and signature name in case of
  IDS
  
  
• origin :
  
  origin is address of checkpoint server from which the log entry is originated
  this field is found only in checkpoint (R55, R54)
  
  
• product :
  
  this field contains information about Checkpoint firewall product. Not related
  to actual log entry.
  
  
• protocol :
  
  this field is found in firewall and tells about which protocol the
  packet encountered was using
  
  
• reason :
  
  there is no explicit entry in PIX firewall named reason but, PIX firewall logs
  contains information about why the packet was accepted or dropped. This field is
  much similar with rule field in checkpoint firewall.
  
  
• received_by :
  
  This field is found in IMSS log file and contains information about receivers
  email address.
  
  
• received_from :
  
  This field is found in IMSS log file and contains information about senders
  email address.
  
  
• rule :
  
  rule according to which action was taken, in numerical
  representation for checkpoint firewall only
  
  
• severity :
  
  severity of received event, different possible values are 'high', 'medium', 'low',
  or 'alert', 'warning', 'informational' and like.
  
  
• shunned_host :
  
  this field is found in Cisco IDS. When any signature is detected from some host
  for which shunning is requested, IDS sends shunning request to firewall so that
  packets coming from this host.
  
  
• src_addr :
  
  address from where the attack was originated can be email address in case of IMSS and
  can be IP address foe IDS and firewall
  
  
• src_nat_addr :
  
  this field is found in checkpoint firewall logs and contains information about
  natted address of source of packet encountered.
  
  
• src_port :
  
  this filed is found in firewall and IDS and contains port number on source from
  where packet was sent.
  
  
• timezone_offset :
  
  value which needs to be added in date_time to get time in IST
  
  
• user_id :
  
  this field is found in IWSS logs and is actually IP address of machine from
  which request for URL was sent.
  
  
• username
  
  this field is found log entries of Cisco PIX firewall.
  
  eg. Login denied from <ip/port> to <ip/port> for user "<username>"
  
  
• victim_addr :
  
  for whom the attack was intended can be email address in case of IMSS and
  IP address in case if IDS and firewall
  
  
• victim_port :
  
  port number of victim
  
  

sample logs/reports

here are sample reports generated and log entries

click on link to view the file