We are definitely leaving the realm of easy. But this is doable right on your Windows 95/98 computer. For free (or cheap.)

This is going to be more of a road map than a tutorial, because of the wide scope of the material. Nevertheless, there should be enough here to get you going.

Introduction

Now I'm going to tell you how get to the next step beyond "static" HTML web pages. They are all well and good, but they don't do anything. In order to do any kind of processing from your web page, like offering a text search feature, you have to use CGI techniques.

CGI stands for Common Gateway Interface, which basically means the way that your HTML code interfaces with some program that does the actual processing work (as far as we are concerned.) The program could be implemented in a number of different languages, but one of the best for this is a language named Perl.

Typically, your HTML code will have some special HTML tags that the web server interprets as a signal to start a program, in our case, a Perl program. Then the Perl program goes off and does the processing and creates a new web page "on the fly" that contains the results of the processing.

The CGI part is easy--you just have to learn a few more new HTML tags. The really long detour is into learning Perl. But there are shortcuts and workarounds.

More Details

I'm assuming that you aren't running a real web server of your own and that you are having your web pages hosted on somebody else's server. In which case, the first thing you have to do is check your user's guide to see if CGI (and Perl) are supported at all. This might be your first road block, because not all web hosting services support CGI.

If they don't, then about your only alternative is to find a web hosting service that supports CGI. The good news is that there are a lot of affordable web hosting services out there that do, and this might be a good excuse to move to one of them. And pick up a real domain name while you are at it.

Running CGI and Perl on your own computer

Because it is very difficult to debug CGI programs that are running on a server on another computer, it is easier to learn how to do this by having everything running on your own computer. First you need to download and install two free programs.

You need to be running a suitable web server program on your own computer. You don't normally need to run a server to develop HTML, because the browser will display the HTML pages just fine, but you do in order to run Perl programs. This is where the CGI part come into play.

Then you need to install the Perl Language.

Installing the Xitami web server

I use the Xitami server for Windows 95/98. This is available from the Xitami web site. Go to the download page and choose the Self Installing Exe version for Win 95/NT standard. Download the .exe file, then close your browser and run the program you just downloaded.

All installed? Then start the server by clicking on the "Xitami" icon and your browser. Go to address http://127.0.0.1/ (This is a "special" address that refers to your own local machine.) If it is running, you should see a "Welcome to Xitami" web page. Good. Then the general idea is to substitute your own web pages for this "dummy" one and then start wrangling with Perl. (However, don't delete the "Welcome to Xitami" page too quick, because it has a link to the Xitami documentation. Best to just rename the "default.htm" that is installed in the \xitami\webpages to something like "xitami.htm" so you don't lose it.)

The HTML that the web server displays resides in a directory on the computer that the web server is running. (Usually.) The Xitami default for this is c:\xitami\webpages

Installing Perl

You want ActiveState Perl for Win32, available from the ActiveState web page. (Is often busy. Try SoftSeek and try the ftp.linux site.) Download Perl for Win32 . Once again, close your browser (and any other programs that are running. Then run the .exe you just downloaded. After it gets done installing, you will have to re-boot because it adds the path to the Perl program to your autoexec.bat file and you need to re-boot for this to take effect.

All re-booted? Now you should test the Perl language all by itself.

Open a DOS Window and change to the directory that you installed Perl in. Use Notepad to create a short "Hello World" Perl program:

printf "Hello World\n"

Save it as "hello.pl". Then type perl hello.pl (Enter) It should print "Hello World" on the screen (in the DOS window.)

Actually, this should work in any directory because Perl is supposed to now be on your path. Change to some unrelated directory and try running Perl. If it doesn't work, then check your autoexec.bat file and make sure that Perl is on your path. Typically, c:\perl\bin; should appear somewhere on your path. If it doesn't, then edit autoexec.bat's path statement and add it. Save and reboot.

Getting Perl to work with Xitami

Xitami defaults to only running cgi scripts (i.e., Perl programs) in its cgi-bin directory. It is a lot easier if you remove this limitation so that you can run Perl scripts in any directory.

To change Xitami's configuration: First start Xitami, if needed, then right click on the Xitami icon on the taskbar. Then click on "Setup" and you should get into the Browser Based Configuration window. Then choose "Normal Configuration." Change "Main CGI directory" and "CGI URLs start with" to the "/" (forward slash) character.

Then go to the CGI page. Make sure that CGI is enabled. Also enable "Match /cgibin anywhere in URL."

If Xitami refuses to let you into the browser based configuration then you have to make some changes in the authorization configuration files.

Change to c:\xitami and use Notepad to create two files:

1. defaults.cfg:

   [Security]
       filename=defaults.aut
   
   

2. defaults.aut:

   [/Admin]
       admin=verysecret
   
   

Now use userid 'admin' and password 'verysecret'. (From the Xitami 'FAQ.')

Stop Xitami and then restart it so that it reads your changes.

Xitami's browser based configuration sometimes acts a bit weird. To be safe, terminate Xitami then restart it and re-enter the Setup Browser Based Configuration and go back to the screens that you just changed and see if the changes are still there. If not try again. Generally do not use your browser's back button when in the Xitami browser based configuration. Instead, use the navigation aids that are on the screen. Also, make sure that Javascript is turned on in your browser, because Xitami uses it for the configuration.

Also note that Xitami saves its configuration data in regular ASCII text files that you can edit, if need be. Change to the directory that you installed Xitami in and look for files with a .cfg extension that have changed today. (Defaults.cfg should exist and have only the new stuff that you changed. Defaults.cfg actually contains the parameters that you have changed from Xitami's default setting.) Make sure that Xitami isn't running and edit them with notepad. Then restart Xitami. Don't modify Xitami.cfg. Instead, make all your changes in defaults.cfg. See the Xitami docs for more information.

Good. Now to try some real server based CGI.

Running your first CGI program

Change to the directory that Xitami uses to store the web pages. The default would be C:\xitami\webpages. Then use Notepad to create this version of the "Hello World" program.

#!/usr/bin/perl
print <<END_of_text;
Content-type:text/html

<html><head><title>Hello World</title></head>
<body bgcolor="ffffff">
<h1>Hello World</h1>
</body></html>

END_of_text

(Open notepad and copy and paste from this page.) Save it as hello2.pl

Now make sure that Xitami is running. Go to http://127.0.0.1 and see if you get the "Welcome to Xitami" page. Then go to 127.0.0.1/hello2.pl The browser should display "Hello World" If it does, then Yeah! because this means you ran a Perl program using CGI.

This not too dramatic demonstration is rather dramatic in its implications. What has happened is that the Xitami server detected that hello2.pl was an executable Perl script and rather than just "serving" it up as regular HTML page like it normally does, it passed control to the Perl interpreter. The Perl Interpreter "ran" the Hello2.pl program and passed its output (what the print statement printed) back to the web server which passed it back to your browser, which displayed it just like any other HTML page.

Neat, huh? What this means is that because Perl is a very powerful language, that we can start having our web pages do stuff. Interesting stuff.

A closer look at the hello2.pl program

It starts with #!/usr/bin/perl which is "magic" in that files that start with #! are classified (by the web server) as Perl scripts (that are to be passed to the Perl interpreter program to execute.) /usr/bin/perl is supposed to point to the directory where the Perl interpreter program is. This is the first anomaly, because your Perl interpreter isn't in this directory, but it worked anyway. (We are talking about the perl.exe program, not the perl scripts that you write.)

The explanation is that this "path" is used when this is run on a real Unix web server, but Xitami ignores this and just uses the regular MS/DOS path information to find the Perl program. This is good, because it lets you develop Perl programs that you will later upload onto a real Unix web server on your own Win 95 computer without having to hassle with the different pathing conventions. At least in this case.

The rest of the program is basically a print statement that prints out a complete HTML program. The only thing unexpected are the lines

Content-type:text/html

They are the http header that has to be passed to the server (and then to your browser). When you are doing regular HTML web pages the server automatically sends this information. But when you are passing the HTML as a CGI stream, then your program has to supply it. Luckily it is simple, Content-type:text/html followed by a blank line.

Then you can "print" out all the HTML you want, and the server will pass it to the browser where it will be displayed as a normal HTML web page.

Note the neat syntax of the Perl print statement:

print <<END_of_text;

Which means "remember the sequence of characters "END_of_text" and then continue printing until you see this sequence of characters again." Which is a handy way to sidestep having to worry about what characters are delimiters, like you do with more limited print statements. (Remember the kind of shenanigans you have to do to print " (double quote) characters with BASIC or "C" print statements?)

Further explanations

While you could access a program like hello2.pl from a web page by using a URL like

	<a href="hello2.pl">Hello World</a> 

and hello2.pl would run, this would be pointless because hello2.pl doesn't do anything more than generating a static HTML page. In most applications the program will do some processing based on something that the user chooses. This means that you need to have some kind of form to collect information.

HTML has this capability. This would be a real good time for you to go to your HTML reference and read about "forms." And follow along with the next example of how to use CGI and Perl to implement full text searching. The HTML form tags are used to allow the user to type in what they want to search for and are then used to invoke the Perl program that does the search by using the CGI conventions to interface between the HTML code and the Perl program.

Doing full text search

The best way to use Perl is to not write it at all. There are various archives of pre-written Perl scripts (programs) on the web. You can download Perl scripts that have documentation covering the modifications that you would likely have to make in order to use them.

Matt's Script Archive is really a good place to start. The scrounge.org text search function was implemented in just this fashion. We're using Matt's "Simple Search" script.

Here's how I got full text searching working on this site:

1. I looked at my web server hosting docs to find out how to implement CGI. I then uploaded a simple "Hello World" type program to see how I had to address the cgi scripts and verified that it worked and that there were no "issues."

2. I added an "alias" definition to Xitami to point to the actual directory on my hard disk that I develop the scrounge pages. I then can address this through Xitami as http://127.0.0.1/scrounge/ You can do this in the Xitami browser based configuration setup. If you don't want to mess with this now then just make a "search.htm" HTML file and put it in c:\xitami\webpages, which you would address as http://127.0.0.1/search.htm.

3. I downloaded the "Simple Search" script from Matt's Archive. Looked over the documentation and made the necessary changes so it worked on the scrounge page while running under Xitami on my computer. There are several variables in the Perl script that you have to change to correspond to the conditions of the computer that you will be running the script on. Mostly defining what directories to use, etc. Simple. I also made some cosmetic modifications.

4. After I was comfortable with how it was working on Xitami on my own computer, I uploaded the find.pl script to my "real" web server's cgi-bin directory and chmodded the access bits to 755 using WS_FTP Pro. Also uploaded the main scrounge page that contains the search function.

   For more on access bits, see this valuable article that Brian wrote for us because he felt that I glossed over too much important stuff.)

5. It took me several tries to get the directory variable definitions in the script correct. Shrug.

   I modified the search form for the main scrounge page. Here is the original version of the form as supplied by Matt and modified to work on the scrounge site.

   If you look at the HTML source, note that the <form action= tag that is the CGIish command that does the actual invoking of the find.pl program. The rest of the form is devoted to collecting information and sticking it into variables that are passed to find.pl using CGI conventions.

Wrapup

You will need more reference material to do anything meaningful. A few quick comments.

• I had you set up Xitami so it would run Perl scripts from any directory. This is handy for developing on your own computer, but is generally not the way that real production web servers are configured. This is for security reasons. Crackers love cgi programs, because they can abuse the capability in order to break in to a system.

  On a real web server that is exposed to the outside world, the administrator's try to make this possibility be as remote as is possible. And one way to do this is to restrict the conditions that CGI programs can run. Generally by being set up so that the CGI programs have to be in a directory that can't normally be read by a browser, typically some variant of cgi-bin.

  There is a lot more to learn about security. Please research this before you get too far into this. This is a much wider subject than I can cover here. But, you should be reasonably safe if you only use already written scripts, like Matt's, as long as you don't modify them too much until you learn more about security.

  You have to inspect the user documentation for whatever web server that you will be having your pages hosted on to see the conventions that you must follow. Then, hopefully, you can duplicate this with Xitami on your own computer so that you can develop on your own machine.

• Note that when Xitami is running that it is a real web server and anybody in the Internet can access whatever it is serving up if they know your IP address. Use the Windows Winipcfg program to find out what yours is. (Anybody who wants to can (usually) find out your IP address by checking email or usenet posting headers.) If you are using a dial-up connection, then, of course, the "world" can't connect to it unless you are actually connected. And dial up accounts generally give you a new IP address every time you connect anyway.

  But if you are using a cable modem, or have some other "always connected" kind of connection, then you have to think about this a bit. Running a "real" web server does open up a lot of security considerations. For starters, you might want to make sure that Xitami isn't configured to automatically start every time you boot.

• Hello2.pl isn't a very interesting program because all it does is generate a small web page that you could do by just writing a HTML file. But it gets you to the point where all the scaffolding is functional. You can go on from here, well just adding to the hello2.pl program. Always start with something that works.

• You should get some real books on Perl. One of the best is Learning Perl, 2nd Edition by Randal L. Schwartz & Tom Christiansen, published by O'Reilly. If you only get one book on Perl, then get this one. O'Reilly has a lot of other good books on Perl. The Perl Cookbook is good too.

• Most real web servers nowadays are either running under Windows NT, or (preferably) under some variant of Unix. You will probably have to learn a bit of Unix. Possibly a lot, depending on how involved your processing gets. This is way beyond the scope of this article. But you can do lot by just reading, processing, and writing, and that doesn't require that you need to learn too much Unix. You have already learned the bit about using forward slashes instead of backward slashes, right? Well, more stuff like that.

• When running a perl script (from the command prompt), instead of just typing

  myscript.pl

  you should execute

  perl -w myscript.pl

  Using the -w switch will enable debugging information which gives warnings in addition to errors. Frequently the warnings will reveal logic errors, unused vars, and etc...

  If you install ActiveState Perl to the default directory, you can open C:\perl\html\lib\pod in the browser and look at the Perl MAN pages. Of particular interest is the perlfunc.htm file. It has the majority of the functions you might use in Perl.

  SSI is not allowed on many systems for perceived security reasons. Note I say perceived. SSI is no more dangerous than normal CGI.

  This paragraph was contributed by Jeff J

  Several common pitfalls in first getting Perl scripts to run on a Unix based web server:

  1. Make sure that your FTP program is set to upload the Perl script in ASCII mode. Unix uses a different convention for ending lines and if it transfers in binary mode then all the line endings will be wrong and the Perl interpreter will choke.

  2. The Perl scripts have to have the execute access bits turned on. Typically this is done with the Unix chmod command. However, some Windows FTP programs allow you to do this. The scripts need to be "chmodded" as 755, which turns on the execute access bits. We have more information on this right here.

  3. If your Perl script will be updating a file (like a page counter), then this file has to have its write access bits turned on.

  4. Any error messages that the Perl interpreter generates are in the server's error log. Check to see if you have access to this log. You do, of course, have access to Xitami's error log. Change to the xitami\logs directory, stop Xitami from running, and examine the error log.

  5. #!/usr/bin/perl might need to be changed. It might need to be #!/usr/bin/local/perl, or some such. Check the user docs from your web hosting provider. People who administer web servers seem to delight in exhibiting a lot of creativity when setting this kind of stuff up, because they all seem to set it up differently. Ditto with how you access the cgi-bin directory.

  6. The notion of "current directory" might change when running on the real web server. You might have to get creative with using the ".." directory clause, and take special pains on whether directory references start (or end) with the "/" character.

  7. Some scripts that you find in an archive might not run on Xitami (or any other server) on your Windows computer because they depend on things that only exist in Unix. Hopefully this will be pointed out in the script's documentation. The web is a Unix world.

  8. See perl.com for comprehensive information on Perl. I left this reference for last, because it is easy to get lost following links and trying to figure what you really need to do to start doing this stuff. There are numerous links to other references at perl.com, activestate.com, cgi-resources.com and Matt's Script Archive. But I wanted to lead you into getting a few things actually working first. The rest of the web is well supplied with link farms.

• Consider not starting out by trying to do a page counter. This requires the further complication of learning about SSI (Server Side Includes) and Xitami is rather crude in supporting SSI. Besides, page counters are dorky. Do you ever see them on real professional web sites?

• Matt's FormMail script allows you to set up a HTML form and email what users type in the boxes. This is very handy and works well, but you can't run this script with Xitami, because it require a working sendmail program to actually send the email, and Xitami doesn't include any sendmail capability. But FormMail works fine on a Unix server.

• Debugging is tough because of all the layers of stuff going on. If you get really confused, it is sometimes helpful to try to run the script directly from the DOS prompt and see what happens. (And remember the server error logs.)

Back to the main scrounge page.