The Big Brother Systems and Network Monitor ------------------------------------------------------------------------ Installation and Configuration Manual Version 1.3 - Nov 07th 1999 ------------------------------------------------------------------------ 1.0 Quick and Dirty Install for Big Brother 2.0 Configuring the BB hosts file 2.1 The etc/bb-hosts file 2.2 Basic format of etc/bb-hosts 2.3 etc/bb-hosts special directives 3.0 Customizing your BB installation 3.1 The runbb.sh script 3.2 Configuring the etc/bbdef.sh file 3.3 Localizing programs paths 3.4 Creating BB clients 3.5 Configuring the Web Stuff 3.6 History 3.7 Adding your own tests 4.0 Configuring the Notification feature 4.1 A Little History 4.2 Setting up notification 4.3 Creating notification rules 4.4 Acknowledging notification messages 4.5 Manual notification (Web based) 4.6 SMS notification 4.7 Using kermit 4.8 Adding a custom notification 4.9 Notification/acknowledgement logs 5.0 Miscellaneous Big Brother configuration options 5.1 Customizing the HTMLized status logs 5.2 Specifying filesystem specific values in the disk test 5.3 Enabling security 5.4 Inserting notes about your hosts 5.5 Customizing the footer for bb.html/bb2.html 5.6 Using bbnet to test TCP services 5.7 Sending custom one line status with bb 5.8 Changing the 'purple' interval 5.9 Specifying processes to look for 6.0 Other documentation sources ------------------------------------------------------------------------ This Installation and Configuration Guide is Copyright 1997-1999 by The MacLawran Group Inc. This document may be reproduced, so long as it is kept in its entirety and in its original format. ------------------------------------------------------------------------ ------------------------------------------------------------------------ Section 1: Quick and Dirty Install for Big Brother From the directory you extracted the BB archive: 1. cd ./install, and run ./bbconfig where OS-NAME is freebsd, solaris, linux, hpux9, hpux, sunos, aix, ultrix, osf, netbsd, freebsd, redhat, bsdi, sco3, sco, irix, unixware 2. cd ../src type "make" then type "make install" 3. cd etc edit bb-hosts, put your host entries in there. This is the core of Big Brother. Then edit bbdef.sh to set alarm levels and things. Edit bbwarnsetup.cfg and bbwarnrules.cfg to set up the notification feature. If you want to use fully qualified domain name hosts then make sure you set FQDN=TRUE in etc/bbdef.sh. If this is the first host you install and intend to use it as the display/notification server as well as the server that test the network services then your bb-hosts should contain this line: xxx.xxx.xxx.xxx this.host.name # BBDISPLAY BBPAGER BBNET 4. execute ./bbchkcfg.sh - Checks the bbdef.sh/bbinc.sh/bbsys.sh source scripts for invalid entries execute ./bbchkhosts.sh - Checks the bb-hosts file for errors 5. ln -s $BBHOME/www /WWW/bb (where /WWW is the Document Root dir). Make sure the permissions are correct. Configure your web server for this directory if need be. 6. cd .. ./runbb.sh start examine the BBOUT file for any errors 7. This will have enabled you to have a display/pager server set up. Point your browser to the /bb path of your web server to view the results. At this point no clients are running. Define all of your hosts in the etc/bb-hosts file and when you have all of your hosts defined in etc/bb-hosts then use install/bbclient to create a tarball for BB clients of the same OS/HW type. If you have different OS/HW platforms then reinstall BB, as a client, on each one (then use the bbclient to create a tarball for each identical OS/HW client) and don't forget to copy your master bb-hosts file to it. Run through the install procedure to make sure that the clients are also installed properly (bbchkcfg.sh/bbchkhosts.cfg) If you're new to BB, we suggest that you subscribe to the BB mailing list. To do so, follow these instructions: Send an e-mail to MajorDomo@taex001.tamu.edu In the body of the e-mail message (not the Subject line), place the statement: subscribe bb Nick Silberstein has made an archive of the Big Brother mailing list available. It can be found at the URL: http://www.tpdinc.com/~bb/ When you have finished installing BB on all of your monitored hosts, you might want to check out the FTP archive that contains many contributed scripts. Check it out at ftp://ftp.deadcat.net/pub/BB ------------------------------------------------------------------------ Section 2: Configuring the Big Brother hosts file 2.1 The etc/bb-hosts file The etc/bb-hosts file controls where Big Brother looks for things and the actions that are taken. The format is identical to the standard /etc/hosts file, except with additional directives for Big Brother. 2.2 Basic format of etc/bb-hosts Lines are of the format: IP-ADDR HOSTNAME # DIRECTIVES IP-ADDR: XXX.XXX.XXX.XXX HOSTNAME: host.domain.com if FQDN="TRUE" host if FQDN="" FQDN (display with Fully Qualified Domain Name) is a variable that you set in the etc/bbdef.sh file. Directives the Big Brother knows about are: BBDISPLAY This host displays the HTML results You need a Web server on this host BBPAGER This host acts as the notification server You can have multiple BBDISPLAYs & BBPAGERs. Just put the BBDISPLAY/BBPAGER directive on the config line for each host that acts as BBDISPLAY/BBPAGER. BBNET Indicates that this host checks the ip network services Many hosts can act as a BBNET. If you do have more than one BBNET, make sure that the bb-hosts file only contains the hosts that you will test against. You must be very careful because if a host is tested by many BBNETs then the status display for the services on that host will only show that last test status. You may get errors for that host even though it shows green, it's just that another BBNET send a status message afterwards and that message returned OK. On the other hand you may want to have tests from different location :) http://www_path The host is tested for http connections using the www_path https://www_path The host is tested for https connections using the www_path (requires lynx) Note that you can specify multiple URL by joining the URLs with '|': http://www_path|http://www_path1 or by specifying them individually: http://www_path http://www_path1 ftp Test for ftp service smtp Test for smtp server pop3 Test host for pop3 server telnet Check telnet service ssh Test ssh server nntp Test nntp news server Check /etc/services for proper service name especially pop3 (sometimes referred as pop-3) These directives names MUST appear in /etc/services Any text based service can be checked by putting the /etc/services name as the directive and adding in the list of services in bb-network.sh The '!' can be prefixed to a service such it will be checked that the service is NOT running (!ftp). dns Checks for name resolution server noping Don't do ping test for this host noconn Don't do ping test for this host and don't generate a clear dot dialup If host is down then display clear button otherwise display a green status You can add your own directives that you access through an external test. Known directives from user contributed tests: oracle/fping/trap/sybase/... These contributions can be found on the FTP archive. This is a sample bb-hosts file: # # BIG BROTHER HOSTS FILE # 192.168.110.102 bobo # BBDISPLAY BBPAGER ftp smtp pop3 192.168.110.95 admin # http://admin/ 192.168.110.108 mynet # BBNET nntp! bobo is the display and notification server. mynet is the host that runs ip network services checks it will check the ftp/smtp/pop3 services on bobo, then the http service on admin and finally it makes sure that nntp is NOT running on mynet You can also group hosts within a seperate HTML table. They are two group tags defined: group and group-compress. Results can also be saved to a specific page using the 'page' directive. It is best to have the master bb-hosts file identical on all BB server and client hosts but not required (unless you have multiple BBNETs, see previous comment) Changes to the etc/bb-hosts file are automatic. You do not have to restart BB to initialize the changes. 2.3 etc/bb-hosts special directives 2.3.1 Grouping on the display The Web display may be broken into tables to create a more aesthetic and sensible display. It's also much faster to load small tables, than one giant table. So consider grouping your etc/bb-host entries logically and separating them using the "group" or "group-compress" directives. group Intranet Servers The group directive defines a block of hosts to be grouped in the same HTML table. All hosts lines following the group directive, until a new group/group-compress is defined, belong to that group. The text that follows the directive is the title given to the HTML table. Note that you can embed HTML code in the title: Italic / H3, but use with caution. group-compress California Servers The group-compress is identical to the "group" directive except it will only display services (columns) containing data for that group. 2.3.2 DHCP hosts (no fixed IP address) 0.0.0.0 dhcphost.domain # The 0.0.0.0 indicates that this host is a DHCP host and that the BBNET host will not try to run the connection test on that host. 2.3.3 Modem Banks dialup modem-bank 204.19.116.20 4 The dialup directive (not to be confused with the other dialup tag which displays a clear button if the host is down) is used to specify connectivity for a bank of modems. The 2nd parameter is the name to be displayed on the display page. The 3rd argument is the starting IP address of the modem bank. The last argument is the number of modems on that bank. 2.3.4 Summary Lines summary quebec.mtl 255.255.255.255 http://cafe.domain.com/bb/ This indicates to the BBDISPLAY machine that summary information about the state of this display is to be forwarded to the IP address noted on the line. The summary can be sent to more than one parent machine. You can send the generated bb.html/bb2.html or subpages created with the 'page' directive to any other BBDISPLAY host(s). 2.3.5 Display results in an HTML subpage page nyrouters New York Routers BB always creates two HTML pages: bb.html/bb2.html. When the page directive is encountered, BB will then create a new HTML page until another page and will save the output from that point on to that page. It will keep a pointer to the new page in bb.html. The first argument is the directive, the second one is the name of the page (i.e. nyrouters.html) and the remaining arguments is the caption that will appear in bb.html when it creates the link to that subpage. N.B. summaries and dialup directives will always appear on bb.ht/b2.html and not in any HTML subpages ------------------------------------------------------------------------ Section 3: Customizing your BB installation 3.1 The runbb.sh script The runbb.sh script is the master BB script. It sets up the BB environment with the values from the configuration files. You MUST set the BBHOME variable to the directory path where you have installed BB. Without it, BB will not run properly. If you create external monitoring agents then you can copy them to the $BBHOME/ext directory and define them in the BBEXT variable. cd $BBHOME (to your BB install directory) To start BB: ./runbb.sh or ./runbb.sh start To stop BB: ./runbb.sh stop To restart BB: ./runbb.sh restart If you ever have to add a script that runs as a deamon use the following template to add it in runbb.sh: echo "Starting snmptrapd" echo "Starting snmptrapd" >> BBOUT MIBFILE=$BBHOME/cmu/etc/mib.txt export MIBFILE $BBHOME/cmu/bin/snmptrapd -B | $BBHOME/bin/bb $BBDISP - & set `$PS | $GREP snmptrapd | $GREP -v "$GREP"` > /dev/null BBPID="$BBPID $1" This is from an SNMP trapd extension available at the FTP archive site (ftp://ftp.deadcat.net/pub/BB). You'll notice that the the PID is added to $BBPID, This way, you'll be able to stop the extension at the same time as BB when executing "runbb.sh stop". 3.2 Configuring the etc/bbdef.sh file etc/bbdef.sh is where you configure how Big Brother reacts to a variety of situations that may arise. You can establish levels for warnings(yellow), panics(red) values. Please note that any modifications to this file does not come into effect until the next restart of BB. It is best to stop/start BB immediately after this file has been modified. Use FQDN (fully qualified domain naming) FQDN="" - Host names rendered on the BB status pages will only include the host name, not including the domain name. FQDN="TRUE" - Host names rendered on the BB status pages will include the host name and the domain name. Which program to use to check the HTTP connectivity: LYNX="$BBHOME/bin/bbnet" - This indicates where bbnet (a C program that checks the IP services) is located. LYNX="/usr/local/bin/lynx -dump -head" - Use lynx if you need to check password protected pages. Please use the proper directory path as per you host installation. NONETPAGE defines which IP services not to notify on if an error is encountered. Include the services as they are named in the etc/bb-hosts file. This is only valid on a host defined as a BBNET host. BBTMP indicates the location of the BB temporary directory. You may want to locate it outside of the BB structure. The DFWARN and DFPANIC variables contain the % levels at which the disk test will fail. By default DFWARN is 90% and DFPANIC is 95%. These values can be overriden by values in the etc/bb-dftab file described in section 5.2. CPUWARN and CPUPANIC are default levels based on the load average. CPUWARN is set at 150 and CPUPANIC at 300. These values are the load average (from uptime) multiplied by 100. You can define the processes for which you want to monitor if they are up and running. You do this by setting the PROCS and PAGEPROC variables. The processes included in PROCS will lead to a yellow condition if one the processes is down. Processes in PAGEPROC will generate a red condition if a process is down. MSGS and PAGEMSGS are used to monitor error messages in the system logs. MSGS contain keywords to look for in the system logs. If a keyword is found then this will create a yellow condition and if the keyword is in the PAGEMSGS variable then it will be upgraded to a red condition. You can also discard certain messages by defined portions of that message in the IGNMSGS variable. You can define multiple messages by delimiting each message by a ';'. BBWEBHTMLLOGS="http://bb4.com/bb/html/" This variable defines the path to the HTMLized status logs. This path is concatenated with the name of the HTMLized status file and is appended to the e-mail notification message for a quickpath to the error information. The PAGELEVELS variable indicates to the 'bb' process at which level(s) it should send a notification message. The 'bb' program is used to send status messages and if the status level of the message is found in the PAGELEVELS variable then a notification message is also sent to the BBPAGER host. This is the default value. It can be overridden by the pagelevels: token in the etc/bbwarnsetup.cfg file. The PAGELEVELSMAIL contains the color level(s) on which only e-mail is to be sent. The color level must also be found in PAGELEVELS. The WEBHISTORY defines if you want to use the web-based history display script. This script gives you a 24hr HTML chart and the last 50 status changes. NOPINGCLEAR lets you decide if you want clear dots for host that have the noping directive. If you set to TRUE then clear dots will be displayed for hosts with the noping as well as hosts with IP address 0.0.0.0 (DHCP hosts or hosts accessed only with its full name). You can use the noconn directive in conjunction with NOPINGCLEAR to select which hosts you don't want a clear dot. 3.2 Localizing programs paths Sometimes the OS specific programs paths found in the installation might be wrong for a particular host(s) OS. Change the paths in the etc/bbsys.local file. The variables contained in this file have all default values from the bbsys.OS file. The original bbsys.local is copied from the etc/bbsys.OS file during installation. 3.4 Creating BB clients Creating BB clients is very simple: cd $BBHOME cd install ./bbclient must be found in the etc/bb-hosts file. This will create a tarball for the client that is the same OS/HW platform type as the host on which the bblient command is executed. The tarball is found inthe directory above $BBHOME. Don't forget to reset the PROCS and PAGEPROC variables in etc/bbdef.sh on the client after you've untarred the archive such that the correct processes are defined. If you have different OS/HW platforms then install BB, as a client, on each OS/HW platform (then use the bbclient to create a tarball for each identical OS/HW client) and don't forget to copy your master bb-hosts file to it. Run through the install procedure to make sure that the clients are also installed properly(bbchkcfg.sh/bbchkhosts.cfg) 3.5 Configuring the Web Stuff All Web-based things live in the $BBHOME/www directory. In order for the bb stuff to work correctly, this directory must be linked into your Web site somewhere. I suggest something like the following: ln -s /home/sean/bb/www /WWW/bb (where /WWW is the Document Root dir of your web server). You should then be able to access BB with the URL http://.../bb/bb.html where ... is your webserver address. You also have to configure the script which generates the web-based history page if you set the WEBHISTORY variable to TRUE in etc/bbdef.sh. This page is accessible from the "History" button when you click on a colored dot in one of the display pages (bb.html/bb2.html). You must copy the web/bb-hist.sh script to your /cgi-bin directory and set the BBHOME and BBWEB variables in the script. Make sure the script has the proper permissions to execute: system and web server permissions. If you want only a text based history display then set the WEBHISTORY variable to "FALSE" in etc/bbdef.sh. By default the CGIBINWEBHISTORY (bbdef.sh) points to /cgi-bin but if your install has a different CGI directory then change it here or your history feature won't work. If you intend to enable manual notification/acknowledgement ("PAGE/ACK"button in bb.html/bb2.html) then copy the web/bb-ack.sh script to you /cgi-bin directory and set the BBHOME and BBWEB variables in the script. Make sure the script has the proper permissions to execute: system and web server permissions. Acknowledgments generate a log entry in acks/acklog file. 3.6 History Each time a status log is received it is checked against its previous status to see if it has changed. If it has, then the duration of the previous status is recorded and a new line is appended to the history file with the timestamp of the new status. There a history file for each host.service (status log) received. The history files are located in the www/hist directory. Here's a sample: Sun Jan 24 18:30:45 1999 green 917220645 232068 Wed Jan 27 10:58:33 1999 red 917452713 1560 Wed Jan 27 11:24:33 1999 green 917454273 Notice that last line doesn't have a duration because it is the current status. It will inserted only when the status changes. If you do not wish to have historical data then rename www/hist to something else or remove it. 3.7 Adding your own tests You can easily add your own tests. Start with the template available at ext/template, add your code. Look at bb-local.sh, bb-network.sh for example on how to send data to BB. Then in runbb.sh, specify the name of your script in the BBEXT variable. Restart BB and your test should be running. But before you use it within BB, I suggest you test it for errors by using the method: cd /home/bb (or wherever your BB is located) BBHOME=/home/bb export BBHOME . ./etc/bbdef.sh cd ext ./yourexternaltest Look for errors, fix them, rerun your test until you're satisfied, then update runbb.sh. Note that all temporary files should be created in $BBTMP, make sure you remove them after use. Also, remember that you don't have to deal with sending notification messages, the 'bb' process send a 'page' type message to the BBPAGER host when the status color is found in the PAGELEVELS variable defined in bbdef.sh When it's ready don't forget to update the svcerrlist token in the bbwarnsetup.cfg file on your BBPAGER host. You must assign a numeric code to your column name. ------------------------------------------------------------------------ Section 4: Configuring the Notification feature (BBWARN) 4.1 A little History In the Beginning :), BB could only e-mail and notify by pager. The configuration required that all BB hosts (servers and clients) have the PAGER variable set with the recipients of the notifications. They were no possibilities of customizing the notification feature. This led to the: The Pager Protection Act of 1997 ================================ Use of BB has been linked to the untimely death of at least one pager in Texas. In order to protect pagers from their users, a notification scheme has been created to allow to specify a delay between the notifications. But this wasn't enough. BB netizens got restless and finally a solution was delivered by Robert-Andre Croteau, and described in an article in SysAdmin magazine. It was called BBWARN and it was good. It allowed an admin to specify rules which enable him/her to get notified based on the source of the error (host) and the service involved and also based on the day/time. Different rules could be set up and different recipients could be specified depending on the characteristics of the problem (host/service/day/time). Numerous changes have been made and various configuration options are available to the BB admin(s). Notification methods You can currently get notified by email, numeric pager and SMS communication device. Users have reported custom notification by alpha-numeric pagers and other PCS devices (some are available on the FTP archive or a search of the mailing list archives will lead you to them) To use the email feature, your BBPAGER host (in bb-hosts) must have an email program. This is not a problem on UNIX: the mail (mailx) program is available. Sending a notification message to a numeric pager requires kermit and a modem. The path of kermit should be setup in the KERMIT variable of bbsys.sh/bbsys.local variable. 4.2 Setting up notification All of the BB notification configuration is specified in the etc/bbwarnsetup.cfg file. Here are the various options that can be set. Note that instructions are included in the configuration file as comments. All tokens in the etc/bbwarnsetup.cfg file must be defined on the BBPAGER host. ***** The pagelevels: token must be defined on all BB hosts (servers/clients) because the 'bb' executable sends the notification and that is done locally on each client host. ***** bbwarn: Set to "TRUE" is you want to enable the notification feature svcerrlist: This contains a list of service/code pair. The service name is the column found in the HTML display and the code is the value displayed on a numeric message. If you add a custom check then the column name must have a corresponding code. ignforall: Is a regular expression that is used to temporarely disable notification a host.service combination(s) e.g. .*.cpu|.*.msgs|host11.* Don't notify for any cpu/msgs errors and any errors concerning host11. N.B. This feature can be duplicated but the ! rule (below). pagehelpcode: Numeric code to use when a user sends a manual notification. See section 4.5. ttyline: List of modems devices. You can specify more than one. prefix: Prefix to use when dialing out suffix: Suffix to use when dialing out (like a hangup) pagelevels: At which color level to send a notification. This token MUST be set on every host. If it is modified then make you restart bb. pagerecovered: Set to TRUE if you want to be notified when a problem has been fixed. pagetype: Defines how the pager delay is handled. There are 4 choices (RCPT|EVENT|HOST|GROUP). RCPT: the recipient is not notified until pager delay expires. EVENT: the recipient is not notified for a particular host.service combination until the pager delay expires. HOST: the recipient is not notified for a particular host until the pager delay expires. GROUP: the recipient is not notified for a particular host.service within the same etc/bb-hosts group combination until the pager delay expires. pagemaster: Recpient(s) to receive an e-mail notification if a page notification could not be sent. pageaddhtmlpath: Set to TRUE if you want the HTML path of the status log to be appended to an e-mail notification. If this is set to TRUE, make sure BBWEBHTMLLOGS variable is set correctly in etc/bbdef.sh. cfgdelim: entry delimiter in the etc/bbwarnrules.cfga briefrcpt: define the recipients that should receive a brief notifications message. hg-xxxxxxx: You can have multiple hg-xxxxxxx tokens that define a group of hosts/devices. So you can create a hg-routers and list your routers. These tokens are then specified in the bbwarnrules.cfg file in the 1st/2nd column instead of having to type all of them in all rules lines. pg-yyyyyyy: You can have multiple pg-yyyyyyy tokens that define a group of recipients. These tokens are expanded in the rules lines inthe recipients definition On the client make sure that PAGELEVELS is set (etc/bbdef.sh) and on the server, if appropriate, that PAGELEVELSMAIL(etc/bbdef.sh) is set. Refer to section 3.2 for moer details. 4.3 Creating notification rules You define the notification rules in the etc/bbwarnrules.cfg file Rules are written in the following format: hosts;exhosts;services;exservices;day;time;recipients hosts: match on these hosts (* is a wildcard for all hosts) exhosts: exclude these hosts services: match on these services (* is wildcard for all hosts) exservices: exclude these services day: 0-6 (sunday-saturday) time: 0000-2359 recipients: email address, numeric pager, sms number, qpage recipients are defined with qp- sendpage recipients are defined with sp- There's also a special format of the rule line: !hosts;exhosts;services;exservices;day;time;recipients If a rule line starts with ! the event thats matches the rule line will disable notification to any recipient defined on the rule line for that event. If the recipients field is '*' then no notification will occur for that event. Here's an example !*;;*;;*;*;* This will in effect disable all notifications and render useless any other rule that you have defined :) !*;;*;;*;*;robert@localhost 9999999 This will remove robert@localhost and 9999999 from the list of recipients for the current error if they were defined on a rule(s) that matched the event. The value set by "cfgdelim" (in bbwarnsetup.cfg) is your filed delimiter. In this case the value is ';' Even though egrep regular expressions are allowed do not use the .* construct, just use '*'. It will be replaced with .* in the regexp. It's just that * is more readable than .* Also the default "pagedelay" value (see bbwarnsetup.cfg) which indicates how long before the next notification occurs can be overridden on the rule line for a specific recipeint by appended the time value to the recipient: recipient:XX where XX is the value in minutes. For examples please refer to the etc/bbwarnrules.cfg and the etc/bbwarnsetup.cfg file for complete information. Escalation To escalate a notification, you use the following format for the recipient: recipient:^XX[-YY] XX is the initial wait before sending the notification YY is the delay for each subsequent call. If it is not specified then pagedelay value is used. An escalation can only be acknowledged by the recipient. Initial delay An initial delay can be specified when configuring a recipient. The format is: recipient:~XX[-YY] XX is the initial delay before sending the notification YY is the delay for each subsequent call. If it is not specified then pagedelay value is used. An initial delay can be reset if a recipient acknowledges a notification for all recipients of that notification (see below). 4.4 Acknowledging a notification When an admin is notified, the admin is always sent an acknowlegement tag with the message. This tag number is seven digits: First five digits are a unique number and the last two digits are the recipient's ID for that notification (recipient's ID can change, it is valid only for the current notification) valid In an e-mail notification, the ack number is in the subject and the body of the message: in the subject it is found at the beginning !BB - XXXXXXX! and in the body of the message, it is at the beginning of the body delimited by [XXXXXXX]. In a numeric page, the acknowledgement is always after the BB numeric code. It is the last 7 digits of the numeric message. NOTE, if your pager doesn't support more than 21 digits, then you're out of luck, you'll be missing some digits. You could always try to shorten the ack number but that is left as an exercise for the reader ;-). If you get the first 5 digits then you can always use recipient 99 to acknowledge for all recipients but that may not fly at your site especially if you have multiple recipients. 4.5 Manual notification (Web based "PAGE/ACK" button) You can enable the manual notification feature and let the user reach you by pager/e-mail if they can't get a hold of you. If this is enabled, then you'll receive a numeric message starting with 911 (pagecode in etc/bbwarnsetup.cfg) followed by the phone number to call. Copy the web/bb-page.sh script to your cgi-bin directory and make sure all permissions are correct. You can setup who gets called when, by adding a rule(s) in the etc/bbwarnrules.cfg. The pseudo host to use is notify-admin*. There's an example in the config file. 4.6 SMS notification This a user custom notification scheme for SMS devices. What follows is the README file for his custom hack. Last updated: 1997-01-11 by jaclu@ibk.se This section describes the usage of a kermit script that sends SMS messages. It is used as a mail -> SMS gateway, and also for the "Big Brother" network monitor(obviously). The kermit script for SMS can be found in etc/sms.scr My operator uses NOKIA server software, hopefully you can use this also on Motorola servers, without to much hazzle. Terms SMS Short Message Service SMSC Short Message Service Center CIMD Computer Interface to Message Distribution Syntax kermit sms.scr device file number [number [number [...]]] device is modem device like /dev/cua0 The file should contain the message, only first line of file will be sent, and it MUST be terminated with LF, otherwise the kermit script won't be able to extract the message. SMS messages can only be 160 chars, and since nobody would want a lengthly message on a 8 character display, I haven't bothered to handle message splitting. If message is too long, it is truncated. If one or more SMS phonenumbers can be specified, the message will be sent to each one of them. Configuration I have done this the easy way, that is I start CIMD with faked checksums, if anybody manages to generate correct checksums, please mail me. Log in to SMSC You have to set two variables in sms.scr: \%b phonenumber to you operators SMSC \%c login to access CIMD For Europolitan (Swedish operator) you can use: \%b 46708222902 \%c cimd3 # (use faked checksums) This starts the CIMD server, and from now we must ACK all responses. Don't forget to include your modem prefix to \%b!! I don't use the "prefix" token from BB, since I also use this script as a mail->SMS gateway. If you are going to use this, you must anyhow configure this script, so I hope you don't mind. Start a CIMD session The first step in a CIMD session is to identify yourself You have to set two variables in sms.scr: \%d CIMD account \%e CIMD password Unfourtunately, I cant give you theese values ;) Now you have configured your script and can start to use it! Send to all recipients BB calls the script one time for each recipient, that's a bit of a waste, but it would just be to much work to get around. If you call it manually or from sendmail, you can use multiple recipients. More info The (english) spec of NOKIAS CIMD implementation can be found on: http://www.europolitan.se/europolitan/fick/tjanst/cimdspec.htm If you happen to know swedish, you can read more on Europolitans implementation on: http://www.europolitan.se/europolitan/fick/tjanst/faqcimd.htm Feedback Please let me know if you use this with a non NOKIA server, or if you fix real checksums! Jacob Lundqvist (SMS:+46-708-555 456) Thanks Jacob !!! 4.7 Using kermit By default, BB expects kermit 5 to be installed on your BBPAGER host. On certain installations it is V6 and up that is installed. If that's you case then you must (might) modify the bin/bb-page1.sh script: change numeric.scr to numeric-k6.scr in the script. But, try with numeric.scr at first and if it doesn't then replace with numeric-k6.scr. Don't forget to set KERMIT in bbsys.sh. 4.8 Adding a custom notification procedure 4.9 Notification/acknowledgement logs You'll find logs of who got notified for what reasons, logs about recoveries, who got a recovery message and logs about who acknowledged what notification. They are all located in the www/acks directory: acklog notifications.log recoveries.log recoverymsgs.log ------------------------------------------------------------------------ Section 5: Miscellaneous Big Brother configuration options 5.1 Customizing the HTMLized status logs If you don't like the blinking gifs, you can change them to non-blinking by copying over the files found in www/gifs. The non-blinking gifs start with the nb- prefix. You can change Sean's mean face in the bb.html/bb2.html and HTMLized status logs by replacing the bb.gif file. You can change the header/footer for the HTMLized status pages. The files are located in the /web directory. Note that your customized version can contain special tags which can be replaced by values: &BBDATE: current date/time &BBBACKGROUND: with the current background color (red/green/yellow/clear/purple) &BBRELDATE: Release date of the current version &BBREL: Current version number &BBCOLOR: Color of this status file &BBHOST: Name of host of current status file &BBSVC: Service name of current status file. You can add tags by modifying the bbd.c file. Every time a status log is received by bbd, an HTML version of the status log is created by prefixing the status with the header file and suffixed with the footer file. You can put tags also in the status logs: &red &green &yellow &clear &purple These tags are replaced with a corresponding image source tag for the HTMLized version. Just like the bb-local.sh script does for the disk and procs tests. So, if you create your own custom check, enter these tags if you want to see a colored dot in the HTMLized status. NOTE: If you do not wish to have HTMLized status files, then rename the www/html directory and the main BB display pages will use the plain text status logs which are located in the www/logs directory. 5.2 Specifying filesystem specific values in the disk test You can define warning and panic levels for specific filesystems. They are defined in the etc/bb-dftab file. This file contains definitions for all hosts all in a single file. You just need to redistribute across your hosts. The format is as follows: [hostname]:/filesystem:warning level:panic level example: /:85:95 www.maclawran.ca:/oradata:98:99 / uses 85% for warning level and 95% for panic, this is On www.maclawran.ca /oradata uses 98% for warning and 99% for panic All other filesystems use the default values define int the etc/bbdef.sh file. Use the etc/bb-dftab.INFO file as a starting point. 5.3 Enabling security When BB is installed, it accepts connections from any hosts. You can specify from which hosts only to accept status logs using the etc/security file. You can define individual hosts or networks in the etc/security file. Using this format: 192.168.1.0 192.168.2.125 10.0.0.0 Accept from network 192.168.1.0, for 192.168.2.125 host and the 10.0.0.0 network. 5.4 Inserting notes about your hosts You can setup HTML links on the host name in the bb.html/bb2.html pages to point to an information page for that particular host. If a file exists in the www/notes directory that matches the system name as displayed on the bb pages, then they are linked into both the bb.html page and the bb2.html summary page. These files can end in .htm, .html, or nothing :) 5.5 Customizing the footer for bb.html/bb2.html bb.html/bb2.html are the HTML status pages generated every 5 minutes. You can modify the footer of it by modifying the www/notes/footer file. NOTE: The link to bb4.com MUST be kept in the footer page as per your agreement of the license agreement. 5.6 Using bbnet to test TCP services You can test any text based TCP services using bbnet. To do so add the test in bb-network.sh. If you need just a basic test, then just add the service to the list of other services. If you need to send a special command to the service daemon, then add a case statement like the imap test. Except change the textmsg variable to the text to be sent to the daemom. 5.7 Sending custom one line status with bb The bb executable used to send statuses/notifications to the BBDISPLAY/BBPAGER can also be used by a custom check using standard input: custom_check | $BB $BBDISP - The custom_check program just needs to send a single line status to stdout (use the iexact format of status files found in www/logs). This is very useful if the custom check program is a daemon. 5.8 Changing the 'purple' interval By default, BB will send a 'purple' status to indicate that a status has not been received in the last 30 minutes (default). You can change this value to a more suitable value by modifying PURPLEDELAY in etc/bbdef.sh 5.9 Specifying processes to look for You can define processes to check for existence. They are defined in the etc/bb-proctab file. This file contains definitions for all hosts all in a single file. You just need to redistribute this file across your hosts. The format is as follows: [hostname]:process list for yellow : process list for red example: localhost:smtp:httpd www.maclawran.ca:!oracle "sleep 30":smtp !process - check that host does not exists "sleep 30" - use '"' when you want to check on a process that spwans multiple words Use the etc/bb-proctab.INFO file as a starting point. If bb-proctab exists, then it overrides the PROCS/PAGEPROCS values in bbdef.sh ------------------------------------------------------------------------ Section 6: Other documentation sources $BBHOME/README $BBHOME/LICENSE $BBHOME/README.INSTALL $BBHOME/README.CHANGES $BBHOME/install/README $BBHOME/src/README