Difference between revisions of "LogAnalyzer Setup"

From The Linux Source
Jump to: navigation, search
m (→‎Recreating mySQL database)
m
 
(22 intermediate revisions by the same user not shown)
Line 1: Line 1:
−
Note: this documents rsyslog >=5.8.x and loganalyzer >=4.1.5
+
PARENT PAGE LINK: [[LogAnalyzer]]
  
−
===  Server mySQL Setup ===
 
−
1. Install mySQL module
 
−
# yum install mysql-server rsyslog-mysql php php-mysql
 
  
−
2. Start mySQL and change root password
+
Note: this document has been used with the following loganalyzer versions: 3.0.2 (and a few before 3.0.2), 4.1.5
−
# chkconfig mysqld on
+
−
# service mysqld start
+
−
# mysql_secure_installation
+
−
answer Y to all of the questions and set root password (usually set to same as root for system login)
+
  
−
3. Create tables in mySql<br>
+
=== Prerequisites ===
−
Note: this relies on already completing step 2 of Web Interface setup (below)
+
  Rsyslog - any version that supports/includes the mySQL module or has the rsyslog-mysql rpm available
−
  # mysql -p </usr/share/doc/rsyslog-mysql-*/createDB.sql
+
MySQL/MariaDB - See [[MySQL/MariaDB Setup]] to make sure you have MySQL or MariaDB set up/running
 +
Apache - any version, but please use something recent, or the version included with a distro that is still receiving updates
 +
PHP - most versions, but please use something recent, or the version included with a distro that is still receiving updates
  
−
4. Remove unneeded users, create new mySql user and grant proper privs;
+
===  Software Setup ===
−
  # mysql -p mysql
+
1. Install rsyslog and PHP mySQL modules
−
> delete from user where Host!='localhost';
+
  # yum install rsyslog-mysql php-mysql
−
> delete from user where User!='root';
+
−
> grant insert on Syslog.* to 'syslog-insert'@'localhost' identified by 'somepwA';
+
−
> flush privileges;
+
  
−
===  Server rsyslog Setup ===
+
2. Get the latest loganalyzer package (http://loganalyzer.adiscon.com/) and copy to /tmp
−
1. Save original version of rsyslog.conf
+
 
−
Note: do not overwrite if rsyslog.conf-original already exists, it should already be there from the company image, so this step is normally skipped
+
3. Uncompress and move to standard RedHat/CentOS compatible 3rd party software location (/usr/share)<br>
−
  # cp -p /etc/rsyslog.conf /etc/rsyslog.conf-original
+
NOTE: 4.1.5 is probably not the version being installed, please use the version number being  installed in place of 4.1.5
 +
# mkdir /tmp/work-syslog ; cd /tmp/work-syslog/
 +
# tar xzvf /tmp/loganalyzer-4.1.5.tar.gz
 +
# mkdir /usr/share/loganalyzer-4.1.5 /usr/share/doc/loganalyzer-4.1.5
 +
  # mv loganalyzer-4.1.5/* /usr/share/doc/loganalyzer-4.1.5/
 +
# mv /usr/share/doc/loganalyzer-4.1.5/src/* /usr/share/loganalyzer-4.1.5/
 +
# mv /usr/share/doc/loganalyzer-4.1.5/doc/* /usr/share/doc/loganalyzer-4.1.5/
 +
# rm -rf /usr/share/doc/loganalyzer-4.1.5/doc /usr/share/doc/loganalyzer-4.1.5/src /usr/share/loganalyzer-4.1.5/doc
 +
# ln -s /usr/share/loganalyzer-4.1.5 /usr/share/loganalyzer
 +
# touch /usr/share/loganalyzer-4.1.5/config.php
 +
# chown apache.apache /usr/share/loganalyzer-4.1.5/config.php
 +
 
 +
===  Rsyslog Server Setup ===
 +
1. set up user and grant proper perms;
 +
# mysql -p mysql
 +
mysql> grant insert on Syslog.* to 'syslog-insert'@'localhost' identified by 'somepwA';
 +
mysql> flush privileges;
  
−
2. Update /etc/rsyslog.conf
+
2. Update /etc/rsyslog.conf, make sure the following lines are added<br>
−
Make sure the following is enabled/uncommented;
+
Add the following to the end of the ModLoad/MODULES section
−
# Use traditional timestamp format
+
−
$ActionFileDefaultTemplate RSYSLOG_TraditionalFileFormat
+
−
# backward compatibility layer added the following directive
+
−
$ModLoad imudp
+
−
# backward compatibility layer added the following directive
+
−
$UDPServerRun 514
+
−
Add the following line to end of ModLoad/MODULES section;
+
 
  # enable mySql plugin/module
 
  # enable mySql plugin/module
 
  $ModLoad ommysql
 
  $ModLoad ommysql
−
Add the following line to the beginning of the logging/RULES section (before #kern.* line);
+
Add the following to the beginning of the logging/RULES section (before #kern.* line)
 
  # log all to mySql
 
  # log all to mySql
 
  *.*      :ommysql:127.0.0.1,Syslog,syslog-insert,somepwA
 
  *.*      :ommysql:127.0.0.1,Syslog,syslog-insert,somepwA
  
−
3. Restart rsyslog service
+
3. Create database/tables
 +
# mysql -p </usr/share/doc/rsyslog-mysql-*/createDB.sql
 +
 
 +
4. Restart rsyslog service
 
  # service rsyslog restart
 
  # service rsyslog restart
  
−
=== Web Interface setup ===
+
=== Apache Setup ===
−
1. Get latest loganalyzer package (http://loganalyzer.adiscon.com/)
+
NOTE: this assumes this is on the company image, which creates most of the needed dirs/files/configs)<br>
−
 
+
1. setup doc root<br>
−
2. Uncompress and move to proper RedHat/CentOS compatible locations<br>
+
−
NOTE: 3.0.2 is probably not the version you're installing, please use the version you're installing in place of 3.0.2
+
−
# mkdir /tmp/work-syslog ; cd /tmp/work-syslog/
+
−
# tar xzvf loganalyzer-3.0.2.tar.gz
+
−
# mkdir /usr/share/loganalyzer-3.0.2 /usr/share/doc/loganalyzer-3.0.2
+
−
# mv loganalyzer-3.0.2/* /usr/share/doc/loganalyzer-3.0.2/
+
−
# mv /usr/share/doc/loganalyzer-3.0.2/src/* /usr/share/loganalyzer-3.0.2/
+
−
# mv /usr/share/doc/loganalyzer-3.0.2/doc/* /usr/share/doc/loganalyzer-3.0.2/
+
−
# rm -rf /usr/share/doc/loganalyzer-3.0.2/doc /usr/share/doc/loganalyzer-3.0.2/src /usr/share/loganalyzer-3.0.2/doc
+
−
# touch /usr/share/loganalyzer-3.0.2/config.php
+
−
# chown apache.apache /usr/share/loganalyzer-3.0.2/config.php
+
−
 
+
−
3. apache setup (we are assuming this is on a company image, which creates most of the needed files/configs)<br>
+
−
3a. setup doc root<br>
+
 
NOTE: syslog1 is the short hostname (hostname -s) of the system you are installing on
 
NOTE: syslog1 is the short hostname (hostname -s) of the system you are installing on
 
  # cd /home/httpd/syslog1/
 
  # cd /home/httpd/syslog1/
 
  # rm -rf public_html
 
  # rm -rf public_html
 
  # ln -s /usr/share/loganalyzer-3.0.2 public_html
 
  # ln -s /usr/share/loganalyzer-3.0.2 public_html
−
3b. setup vhost file<br>
+
2. setup vhost file<br>
−
add the following to /etc/httpd/conf/vhost-ssl.d/0-syslog1 (hostname -s) after ServerAdmin and before proxy section
+
add the following to /etc/httpd/conf/vhost-ssl.d/0-syslog1 (hostname -s) after ServerAdmin and before proxy section<br>
−
  <Directory /home/httpd/syslog1/public_html>
+
Note: enabling the use of a symlink is done for the dir in which the symlink resides
 +
  <Directory /home/httpd/syslog1>
 
     Options FollowSymLinks
 
     Options FollowSymLinks
 
  </Directory>
 
  </Directory>
−
3c. enable php
+
3. enable php
 
  # cp -p /etc/httpd/conf.d/php.conf /etc/httpd/conf.d-run/
 
  # cp -p /etc/httpd/conf.d/php.conf /etc/httpd/conf.d-run/
−
3d. start up or restart apache and make sure it starts on boot
+
4. start up or restart apache (and make sure it starts on boot)
 +
ENT 7
 +
# systemctl start httpd
 +
# systemctl enable httpd
 +
BEFORE Ent 7
 
  # service httpd restart
 
  # service httpd restart
 
  # chkconfig httpd on
 
  # chkconfig httpd on
  
−
===  Web Interface mySQL setup ===
+
===  Web Interface Setup ===
−
1. setup user & permissions
+
1. setup user and permissions
 
  # mysql -p
 
  # mysql -p
−
  > grant select, update, insert, create, drop, alter on Syslog.* to 'syslog-read'@'localhost' identified by 'somepwB';
+
  mysql> grant select, update, insert, create, drop, alter on Syslog.* to 'syslog-read'@'localhost' identified by 'somepwB';
−
  > flush privileges;
+
  mysql> flush privileges;
  
 
2. web interface configuration<br>
 
2. web interface configuration<br>
Line 105: Line 100:
  
 
===  Web Interface Fix ===
 
===  Web Interface Fix ===
 +
There was an issue (in older vers), that appeared more than once (possibly due to apache or php settings), where a config was populated but was not usable due to missing values. The following needed to be set in this situation to configure these settings for their default values<br>
 
1. fix config.php<br>
 
1. fix config.php<br>
 
1a. edit config.php
 
1a. edit config.php
Line 119: Line 115:
 
  $CFG['EnableIPAddressResolve'] = 1;
 
  $CFG['EnableIPAddressResolve'] = 1;
  
−
===  Purging mySQL database ===
+
===  Client Rsyslog Setup ===
−
This rsyslog configuration writes everything to a mySql database, as well as to the standard log files. The OS has the logrotate process to manage the log files, but if you wish to manage or periodically purge the mySql data, you can set up the following cron job (thanks to Michael Meckelein for posting this in a forum):<br>
+
configure clients to connect to this central syslog server [[Rsyslog Client]]
−
NOTE: you may want to setup another user which has delete permissions (like syslog-purge) to do the periodic cleanup for this cron process
+
−
mysql -u syslog-read -p somepwA -e “delete from SystemEvents where ReceivedAt < date_add(current_date, interval -28 day)” Syslog
+
  
 
===  Recreating mySQL database ===
 
===  Recreating mySQL database ===
−
If you have to recreate mySQL data from scratch<br>
+
If you have to recreate mySQL data from scratch (due to corruption, or other issues)<br>
 
1. make sure mysql is down
 
1. make sure mysql is down
 +
ENT 7
 +
# systemctl stop mariadb
 +
BEFORE Ent 7
 
  # service mysqld stop
 
  # service mysqld stop
  
−
2. wipe out all the data (make sure this is what you want to do, otherwise google how to repair innodb data)
+
2. wipe out all the data (make sure this is what you want to do, otherwise if you are wiping it out due to running out of diskspace or database corruption, google how to repair mysql/mariadb innodb databases)
 
  # rm -rf /var/lib/mysql/* /var/lib/mysql/.my*
 
  # rm -rf /var/lib/mysql/* /var/lib/mysql/.my*
  
−
3. rebuild mysql & loganalyzer tables<br>
+
3. restart mysql
−
Rerun the following steps from above<br>
+
ENT 7
−
  Server mySQL Setup; steps 2-4<br>
+
# systemctl start mariadb
−
  Web Interface mySQL setup; step 1
+
  BEFORE Ent 7
 +
  # service mysqld start
  
−
===  Client rsyslog Setup ===
+
4. purge unneeded users (link)
−
1. /etc/rsyslog.conf, add @sys.log.server.ip lines to logging section, ex;
+
−
# Log all kernel messages to the console.
+
−
# Logging much else clutters up the screen.
+
−
#kern.*                                                /dev/console
+
−
+
−
# Log anything (except mail) of level info or higher.
+
−
# Don't log private authentication messages!
+
−
*.info;mail.none;authpriv.none;cron.none                /var/log/messages
+
−
+
−
# The authpriv file has restricted access.
+
−
authpriv.*                                              /var/log/secure
+
−
+
−
# Log all the mail messages in one place.
+
−
mail.*                                                  -/var/log/maillog
+
−
+
−
# Log cron stuff
+
−
cron.*                                                  /var/log/cron
+
−
+
−
# Everybody gets emergency messages
+
−
*.emerg                                                *
+
−
+
−
# Save news errors of level crit and higher in a special file.
+
−
uucp,news.crit                                          /var/log/spooler
+
−
+
−
# Save boot messages also to boot.log
+
−
local7.*                                                /var/log/boot.log
+
−
+
−
# centralized logging
+
−
*.* @172.160.135.160:514
+
  
−
2. Restart rsyslog service
+
5. rebuild mysql & loganalyzer tables<br>
−
  # service rsyslog restart
+
Rerun the following steps from above
 +
Rsyslog Server Setup; step 1
 +
Server mySQL Setup; steps 2-4
 +
  Web Interface Setup; step 1

Latest revision as of 12:44, 12 June 2017

PARENT PAGE LINK: LogAnalyzer


Note: this document has been used with the following loganalyzer versions: 3.0.2 (and a few before 3.0.2), 4.1.5

Prerequisites

Rsyslog - any version that supports/includes the mySQL module or has the rsyslog-mysql rpm available
MySQL/MariaDB - See MySQL/MariaDB Setup to make sure you have MySQL or MariaDB set up/running
Apache - any version, but please use something recent, or the version included with a distro that is still receiving updates
PHP - most versions, but please use something recent, or the version included with a distro that is still receiving updates

Software Setup

1. Install rsyslog and PHP mySQL modules 

# yum install rsyslog-mysql php-mysql

2. Get the latest loganalyzer package (http://loganalyzer.adiscon.com/) and copy to /tmp

3. Uncompress and move to standard RedHat/CentOS compatible 3rd party software location (/usr/share)
NOTE: 4.1.5 is probably not the version being installed, please use the version number being installed in place of 4.1.5 

# mkdir /tmp/work-syslog ; cd /tmp/work-syslog/
# tar xzvf /tmp/loganalyzer-4.1.5.tar.gz
# mkdir /usr/share/loganalyzer-4.1.5 /usr/share/doc/loganalyzer-4.1.5
# mv loganalyzer-4.1.5/* /usr/share/doc/loganalyzer-4.1.5/
# mv /usr/share/doc/loganalyzer-4.1.5/src/* /usr/share/loganalyzer-4.1.5/
# mv /usr/share/doc/loganalyzer-4.1.5/doc/* /usr/share/doc/loganalyzer-4.1.5/
# rm -rf /usr/share/doc/loganalyzer-4.1.5/doc /usr/share/doc/loganalyzer-4.1.5/src /usr/share/loganalyzer-4.1.5/doc
# ln -s /usr/share/loganalyzer-4.1.5 /usr/share/loganalyzer
# touch /usr/share/loganalyzer-4.1.5/config.php
# chown apache.apache /usr/share/loganalyzer-4.1.5/config.php

Rsyslog Server Setup

1. set up user and grant proper perms; 

# mysql -p mysql
mysql> grant insert on Syslog.* to 'syslog-insert'@'localhost' identified by 'somepwA';
mysql> flush privileges;

2. Update /etc/rsyslog.conf, make sure the following lines are added
Add the following to the end of the ModLoad/MODULES section 

# enable mySql plugin/module
$ModLoad ommysql

Add the following to the beginning of the logging/RULES section (before #kern.* line) 

# log all to mySql
*.*       :ommysql:127.0.0.1,Syslog,syslog-insert,somepwA

3. Create database/tables 

# mysql -p </usr/share/doc/rsyslog-mysql-*/createDB.sql

4. Restart rsyslog service 

# service rsyslog restart

Apache Setup

NOTE: this assumes this is on the company image, which creates most of the needed dirs/files/configs)
1. setup doc root
NOTE: syslog1 is the short hostname (hostname -s) of the system you are installing on 

# cd /home/httpd/syslog1/
# rm -rf public_html
# ln -s /usr/share/loganalyzer-3.0.2 public_html

2. setup vhost file
add the following to /etc/httpd/conf/vhost-ssl.d/0-syslog1 (hostname -s) after ServerAdmin and before proxy section
Note: enabling the use of a symlink is done for the dir in which the symlink resides 

<Directory /home/httpd/syslog1>
   Options FollowSymLinks
</Directory>

3. enable php 

# cp -p /etc/httpd/conf.d/php.conf /etc/httpd/conf.d-run/

4. start up or restart apache (and make sure it starts on boot) 

ENT 7
# systemctl start httpd
# systemctl enable httpd
BEFORE Ent 7
# service httpd restart
# chkconfig httpd on

Web Interface Setup

1. setup user and permissions 

# mysql -p
mysql> grant select, update, insert, create, drop, alter on Syslog.* to 'syslog-read'@'localhost' identified by 'somepwB';
mysql> flush privileges;

2. web interface configuration
2a. go to the new URL (configured in; Web Interface setup, step 3a) in a browser, you will get the following message; 

Error, main configuration file is missing!

2b. click 'here' on the following message; 

Click here to Install Adiscon LogAnalyzer!

2c. click 'next' on the following page; 

Step 1 - Prerequisites

2d. click 'next' on the following page; 

Step 2 - Verify File Permissions

2e. click 'next' on the following page; 

Step 3 - Basic Configuration

2f. Fill in the following and click 'next'; 

Source Type: MYSQL Native
Database Name: Syslog
Database Tablename: SystemEvents
Database User: syslog-read
Database Password: somepwB

2g. Click 'here' on the following message; 

Step 8 - Done
Click here to go to your installation.

Web Interface Fix

There was an issue (in older vers), that appeared more than once (possibly due to apache or php settings), where a config was populated but was not usable due to missing values. The following needed to be set in this situation to configure these settings for their default values
1. fix config.php
1a. edit config.php 

# vi /usr/share/loganalyzer-3.0.2/config.php

1b. set ViewMessageCharacterLimit 

$CFG['ViewMessageCharacterLimit'] = 80;

1c. set ViewStringCharacterLimit 

$CFG['ViewStringCharacterLimit'] = 30;

1d. set ViewEntriesPerPage 

$CFG['ViewEntriesPerPage'] = 50;

1e. set ViewEnableDetailPopups 

$CFG['ViewEnableDetailPopups'] = 1;

1f. set EnableIPAddressResolve 

$CFG['EnableIPAddressResolve'] = 1;

Client Rsyslog Setup

configure clients to connect to this central syslog server Rsyslog Client

Recreating mySQL database

If you have to recreate mySQL data from scratch (due to corruption, or other issues)
1. make sure mysql is down 

ENT 7
# systemctl stop mariadb
BEFORE Ent 7
# service mysqld stop

2. wipe out all the data (make sure this is what you want to do, otherwise if you are wiping it out due to running out of diskspace or database corruption, google how to repair mysql/mariadb innodb databases) 

# rm -rf /var/lib/mysql/* /var/lib/mysql/.my*

3. restart mysql 

ENT 7
# systemctl start mariadb
BEFORE Ent 7
# service mysqld start

4. purge unneeded users (link)

5. rebuild mysql & loganalyzer tables
Rerun the following steps from above 

Rsyslog Server Setup; step 1
Server mySQL Setup; steps 2-4
Web Interface Setup; step 1
Retrieved from "http://thelinuxsource.org/index.php?title=LogAnalyzer_Setup&oldid=230"