Several people in this mailing list have asked for such a piece of
software as this.  Thought it might be fun to implement.  Enjoy!

This software can protect (?) an entire LAN or enterprise network
rather than one user's machine and you can use it with your current
proxy (and caching) Web server -- without having to patch it -- if you
already have one as part of your firewall.  It should work with any
standard proxying httpd.  You should be able to convert and load
any list of "obscene" or otherwise "offensive" (by your definition)
sites (URLs) into it's stoplist file.

The alpha software is available as a gzipped tar file via URLs:

 ftp://sparky.cs.yale.edu/pub/long/src/network/security/wwwblock-1.4.tar.gz
 http://sparky.cs.yale.edu/pub/long/src/network/security/wwwblock-1.4.tar.gz

...........................................................
CHANGES in wwwblock 1.4

Enhancements added by Brian Rogers <brogers@integctr.com> :

1.	The ALLOWFILE and CENSORFILE can now take filename 
	extension patterns of the form:  

	*.csh
	*.script
	*.sh

2.	If you define FILTER_JAVASCRIPT in the Makefile and then
	build the software it will strip Javascript from streaming HTML
	between <SCRIPT> and </SCRIPT> effectively disabling it.
...........................................................
README for wwwblock 1.2

wwwblock is a Web request blocker implemented as an application proxy
run from "inetd" which front-ends a real "proxy" (and possbily also
"caching") Web server.  It blocks access by using an "allow" list, or stop list (or both) of URL prefixes kept in text files.

wwwblock 1.2 is alpha code.  No guaruntees or liability assumed.  It
has only been tested under SunOS 4.1.* and Solaris 2.*.  Send bug
reports to me ( Morrow.Long@Yale.EDU ).  Let me know what you think...

In spite of what you might think this program was not funded by the
Communications Decency Act, but was an idea I had to protect my primary
school children.  Afterwards I thought it might be useful for other
environments (K-12 schools, some companies, etc.).

A nice feature about this fairly simple program is that it is more
selective than some solutions some sites have set up in their routers
(ie. block all access to specified IP addresses) and less unwieldy than
entering a long list of hosts to be "denied" in an access control list
(ACL) for a TCP wrapper, but it is still fairly powerful and simple
since a whole class of URLs -- including different ports and
file/pathnames -- can be enabled or blocked with a single line containing the
common prefix (ie. http://www.xxx.com).  Or you can just block port
8080 ( http://www.xxx.com:8080 ) or port 80 ( you need to specific
http://www.xxx.com:80/ and http://www.xxx.com/ to do this).  To
completely block a site you need to specify all of the nicknames a host
is known by as well as any numeric IP addresses it possesses) -- see
the sample "censured-urls" file for examples.

Since a browser user will have to set their browser to use it as an
http proxy you can run wwwblock at port 80 (if you are already running
a proxy/caching Web server at port 80 you can move it to another port
such as 8080 and then set WWW_PROXY_SERVER to localhost and
WWW_PROXY_PORT to 8080) or any port of your choosing.  Here is an
/etc/inetd.conf entry:

This filtering proxy server agent requires a caching/proxy Web server
(ie. CERN, Netscape or Harvest) to connect to and relay all passed HTTP
requests through.  The caching/proxy Web server can be on the same host
at a different port (ie. localhost), in the DMZ or even out on the
Internet.  Ideally the clients of the wwwblock proxy should not have
any other direct access (ie. they cannot access port 80 on any host
outside the local network ) so that they are forced to go through the
wwwblock server which you have configured for them (ie.  they cannot
make any "end-runs" around it).

If you install wwwblock as part of a firewall, but on a machine which
is directly IP accessible from the Internet (ie. on a bastion host
machine in the DMZ) you should probably put a TCP wrapper in front of
it so that only internal network clients can use it (to prevent
Internet clients from using your proxy/caching server).  A TCP wrapper
is not necessary for wwwblock if is installed so that it is only
accessible by internal network clients.

wwwblock can be installed anywhere in the filesystem, though the directory
/usr/local/etc/ is suggested.  If you install the file containing the
list of blocked URLs (prefixes) as a file named anything other than
/usr/local/etc/censored-urls you need to set CENSORED_URLS_FILE to the
filename.  If the file ALLOWED_URLS_FILE (usually /usr/local/etc/allowed-urls)
exists then only those URLs beginning with the prefixes found in it are
allowed.  The CENSORED_URLS_FILE still works in conjunction with an
ALLOWED_URLS_FILE so that you can filter out subsets of the allowed URL
prefixes.

Here are the defaults you may want to change:

#define	ALLOWED_URLS_FILE	"/usr/local/etc/allowed-urls"
#define	CENSORED_URLS_FILE	"/usr/local/etc/censored-urls"
#define	WWW_PROXY_SERVER	"localhost"
#define	WWW_PROXY_PORT		8080

Examples:

To compile for Solaris 2.*:	

  cc -DALLOWED_URLS_FILE=\"/etc/allowed-urls\"	\
	-DCENSORED_URLS_FILE=\"/etc/censored-urls\"	\
	-DWWW_PROXY_SERVER=\"http-proxy.sub.dom.com\"	\
	-DWWW_PROXY_PORT=8081				\
	-o wwwblock wwwblock.c -lsocket -lnsl


To compile for SunOS   4.*:

  cc -DALLOWED_URLS_FILE=\"/etc/allowed-urls\"	\
	-DCENSORED_URLS_FILE=\"/etc/censored-urls\"        \
        -DWWW_PROXY_SERVER=\"http-proxy.sub.dom.com\"   \
        -DWWW_PROXY_PORT=8081     -o wwwblock wwwblock.c


TODO for the future:

	*	Needs documentation!
	*	Check return codes from all library and system calls!
	*	Run lint on it!
	*	Canonicalize hostnames so all forms don't have to be listed?
	*	If the "censored-urls" file becomes very long it would
		probably be useful to speed up URL prefix matching by 
		maintaining a sorted list (yuck!) suitable for binary
		searching or by "hashing" the URL prefix file.
	*	Add more logging, although many might resent as an
		invasion of their privacy having their requests for URLs
		at blocked sites recorded...


- H. Morrow Long
  Morrow.Long@Yale.EDU

   Sat Feb 17 23:50:42 EST 1996
...........................................................