|
The receivedIP program suite is for auditing the IP
addresses in e-mail "Received: " headers, and contains parsers
and high speed database query programs to validate IP
addresses in e-mail. The suite consists of three programs:
bsearchtext,
binary search a text file database for character
string(s). (Suitable for inclusion in procmail(1)
scripts.) bsorttext,
binary sort an unordered text file of character
strings. (Suitable for inclusion in procmail(1)
scripts.) receivedIP,
list the IP addresses in "Received: " records in the headers
of e-mail files. (Suitable for building databases.) receivedIPdb,
validate the IP addresses in "Received: " records in the
header of e-mail against IP addresses in a database file.
(Suitable for inclusion in procmail(1) scripts.) receivedIPdbdedup,
dedup the IP addresses in database files(s) against a
database file. (Suitable for building databases.) receivedIPdbrm,
remove the IP addresses in database files(s) against a
database file. (Suitable for building databases.) receivedIPdbusort,
sort the IP addresses in a database file. (Suitable for
building databases.) receivedAddressdb,
validate the e-mail addresses in "Received: " records in the
header of e-mail against e-mail addresses in a database
file. (Suitable for inclusion in procmail(1)
scripts.) receivedTodb,
validate the e-mail addresses in "To: " or "Cc: " records in
the header of e-mail against e-mail addresses in a database
file. (Suitable for inclusion in procmail(1)
scripts.) receivedMSGIDdb,
validate the "Message-ID: " records in the headers of e-mail
files against domain names in a database file. (Suitable for
inclusion in procmail(1) scripts.) receivedUnknowndb,
validate the domain addresses in "Received: " records with
unknown RDNS in the headers of e-mail files against e-mail
addresses in a database file. (Suitable for inclusion in
procmail(1) scripts.) receivedIPforgedb,
validate the domain IP addresses in "Received: " records in
the header of e-mail against IP addresses in a database
file. (Suitable for inclusion in procmail(1)
scripts.) hsearchtext,
hash search an unordered text file of character
strings. (Suitable for inclusion in procmail(1)
scripts.) bsearchbody,
compare domain names and IP addresses found in the body of
an e-mail against a database file. (Suitable for inclusion
in procmail(1) scripts.) tolower,
convert all characters in a text file to lower
case. (Suitable for building databases.) toupper,
convert all characters in a text file to upper
case. (Suitable for building databases.) extract.addresses,
a shell script to extract IP addresses from a hierarchal
e-mail archive of spam. (Suitable for building
databases.) receivedIP.validate,
a not so elegant shell script for validating a receivedIP
database of IP addresses against the black lists available
on the Internet. (Suitable for building databases.)
Availability
The receivedIP program suite is available at http://www.johncon.com/john/archive/receivedIP.tar.gz
and distributed as source code, at no charge.
Installation
Download the source tape archive, receivedIP.tar.gz gunzip receivedIP.tar.gz tar xvf receivedIP.tar cd receivedIP make installation in your system is non-fascist; install
the executables, nroff sources to the man pages, and catman
pages, where you want
Usage
A few lines of code are worth a thousand words. See the
extract.addresses shell script for an example of how the
programs work together to maintain an "Received: " header IP
address black list. To use the IP address database, copy the
file made by extract.addresses to ~/.procmail.reject, and put
the following code fragment in your ~/.procmailrc file:
:0
* ? test -f "${HOME}/.procmail.reject"
* receivedIPdb "${HOME}/.procmail.reject"
{
# Whatever you want to do with e-mail that comes from
# black listed machines goes here.
}
The receivedIP program suite is used in the Stochastic
UCE Detection procmail(1) script, which
is very effective at reducing the amount of commercial e-mail
received by users.
Some Procmail How To/Cookbook Examples
If you need to figure out the recipient address
(i.e., the envelope to,) of an e-mail, howto-to.txt
offers some alternative ways. If you need to detect whether an e-mail is from a
daemon or agent, howto-daemon.txt
is a good way of doing it. If you need to detect whether an e-mail has forged
addresses, howto-forgery.txt
offers some suggestions. Or, you could:
:0
* !? /usr/local/bin/receivedIPforgedb -r m thedomain.com\
"${HOME}/.procmail.local"
{ FORGED=true }
where "${HOME}/.procmail.local" is a Unix flat file
database, (i.e., in lexical order,) of the IP addresses used
in the domain. If you use Qmail
as your MTA, there are some very useful ways of using exit
codes from procmail
scripts listed in howto-qmail-procmail.txt. Sometimes extracting all the "from" addresses in a
message is helpful, and howto-addresses.txt
offers many ways to do it. If you need to defang messages with MIME
attachments, howto-defang.txt
is a way. An e-mail address kill database can be helpful and howto-kill.txt
shows a way to keep messages from people you don't want to
hear from out of your inbox, (this is sometimes called
black listing.) There are various ways of constructing databases of
known e-mail addresses that should be accepted for delivery
into your inbox, howto-known.txt
shows some ways, (this is sometimes called white
listing. If you want to know if a message is from someone in
the local domain, howto-local.txt,
shows a one statement way of finding out. If you are troubled with the bounces generated by
spam with malformed return headers, howto-malformed.txt,
shows a way of handling it. If you want to detect executable attachments in a
message, howto-virus.txt,
shows a way of doing it. If you want to detect if a message has a potentially
malicious attachment, (like a virus or worm,) the procmail
fragment
is an effective way-and has its own web page at Quarantining
Malicious Outlook Attachments. A good way of handling unsolicited commercial e-mail,
(UCE, or spam,) using the techniques outlined in this
section can be found at Stochastic
UCE Detection page, and some general MTA security
considerations can be found on the Securing
E-Mail page.
License
A license is hereby granted to reproduce this software
source code and to create executable versions from this source
code for personal, non-commercial use. The copyright notice
included with the software must be maintained in all copies
produced.
THIS PROGRAM IS PROVIDED "AS IS". THE AUTHOR PROVIDES NO
WARRANTIES WHATSOEVER, EXPRESSED OR IMPLIED, INCLUDING
WARRANTIES OF MERCHANTABILITY, TITLE, OR FITNESS FOR ANY
PARTICULAR PURPOSE. THE AUTHOR DOES NOT WARRANT THAT USE OF
THIS PROGRAM DOES NOT INFRINGE THE INTELLECTUAL PROPERTY
RIGHTS OF ANY THIRD PARTY IN ANY COUNTRY.
So there.
Copyright © 1992-2007, John Conover, All Rights
Reserved.
Comments and/or problem reports should be addressed to:
- john@email.johncon.com
- http://www.johncon.com/john/
- http://www.johncon.com/ntropix/
- http://www.johncon.com/ndustrix/
- http://www.johncon.com/nformatix/
- http://www.johncon.com/ndex/
|
