Introduction
Python
Webware Version
Installing Webware
Operating Systems
Web Servers
Apache
IIS
AOLserver
Architecture
Adapters
Permissions
CGI Adapter
1. Review and fix permissions
2. Set up WebKit.cgi
3. Launch the servers
4. Try it out
Stopping the App Server
Reloading Servlets
OneShot Adapter
FastCGI Adapter
Mod_python Adapter
AOLserver Adapter
Renaming Adapters
Configuration
Sessions
Activity Log
E-mail Errors
Contexts
AppServer Notes
ThreadedAppServer
AsyncThreadedAppServer
Monitoring
Other Notes
Future
Credit
WebKit provides an application server that runs on both UNIX and Windows in "CGI-" or "persistent-" mode, with several configuration options to change it's behavior.
This install guide takes you through the installation process while showing you what options are available and how you would use them. There are notes for specific operating systems and web servers preceded with tags such as [UNIX], [WinNT], [IIS], etc.
WebKit is pretty easy to install, but there are some important things to know. You will be doing yourself a favor by reading through this guide from top to bottom (skipping sections that don't apply to you).
We recommend that you get a very simple configuration working first, focusing most on a convenient environment for your development. After that, if you still want to play with more advanced set-ups, then go for it.
The term deploy means to install your web application/site in a well known location for access by your users. You first develop your web application and then deploy it. You will see deployment specific notes throughout the guide since the issues for these two situations can be different. In a nutshell, you want convenience during development, and security and performance during deployment.
You can always e-mail webware-discuss@lists.sourceforge.net to give feedback, discuss features and get help using WebKit.
Python
WebKit 0.4.1 was tested with both Python 1.5.2 and 2.0b2. In fact, this release was driven by fixes in support of Python 2.0b2 as well as improvements to this guide. If you have any problems whatsoever with Python 1.5.2 and up, please let us know so we can fix them (webware-discuss@lists.sourceforge.net).
Your installation of Python must be multi-threaded, even if you are using the OneShot adapter (which logically doesn't require threads). It's not uncommon for third party web host providers to leave this disabled in Python because they don't expect that it's needed and/or because the default installation of Python does not turn threads on. If your Python installation is not multi-threaded, you will have to reinstall it. If you're using a third party host provider in this situation, you may be able to install it yourself into your home directory via a telnet or ssh account. See the Python site for instructions.
To determine if threading is enabled, start the Python interpreter from the command line and enter import thread
. If you don't get an exception, all is well.
Webware Version
Unless you just downloaded Webware (and therefore WebKit), you should check to see if you have the latest version. This version you are looking at is 0.4. You can check for the latest version at http://webware.sourceforge.net.
If you're feeling adventurous, you can get the latest in-development source code from the public repository. Instructions are located at the Webware CVS page. You can find more information about CVS in general at http://www.cvshome.org.
Installing Webware
Webware's main directory contains an install.py script that should always be run first.
Note that the install program doesn't actually copy the Webware files to any separate directory. It copies files within the Webware directory tree, modifies permissions, generates documentation, etc.
Also, you can run the install program as many times as you like with no ill effect, so if you're not sure whether or not it has been run, feel free to do so again.
Operating Systems
WebKit actively supports UNIX, Windows and their various flavors.
Note that you don't have to develop and deploy on the same platform. One of the WebKit developers develops everything on Windows 98 2nd edition and deploys his web sites on BSD (http://www.bsd.org/).
What follows are some OS specific notes:
[UNIX] Nothing special to report. Both Linux and BSD have been used with WebKit.
[WinNT] If you are using IIS as your web server, please see the IIS notes below.
[Win9x] Note that AsyncThreadedAppServer.py often doesn't function correctly on Win9x. Since it's not the default app server (AppServer.bat points to ThreadedAppServer.py), you don't have to think hard about this unless you go digging.
[Win9x] Although a lot of the development of both WebKit and various web sites that use it has been done on Windows 9x, we don't recommend that you use this operating system to actually serve your users. UNIX and NT are generally more robust and secure and therefore, more appropriate for deployment.
[Win] Some Windows users are surprised that URLs passed to WebKit are case sensitive. While it's true that the various Windows file systems are case insensitive, the data structures that WebKit uses internally are not. This is on the to do list.
[Win] When editing path names in configuration files, keep in mind that you are editing a Python dictionary populated with Python strings. Therefore your paths must either use double backslashes (\\) (since the backslash is a special character), use forward slashes or use "r-strings". A bad example is 'C:\All\Web\'
. Good examples include:
'/All/Web'
'C:/All/Web'
'C:\\All\\Web'
r'C:\All\Web'
[WinMe] [Win2k] To our knowledge, as of this release, no one has used WebKit on Windows Me or 2000. If you do so, please report the results to webware-discuss. These probably work fine since these products are the successors to Windows 98 and Windows NT, respectfully, both of which have been tested.
Web Servers
WebKit has been tested with both Apache and IIS, so chances are your web server has already been tested. For the most part, WebKit doesn't care what web server you use. The least common denominator is standard CGI, so as long as your web server supports that, you should be fine.
Apache
Apache is the world's most popular web server, which means it's also well tested. The Apache home page is located at http://www.apache.org.
You can use WebKit CGI adapters with Apache or you can try out the new mod_python adapter described below.
IIS
Microsoft's Internet Information Server, or IIS, requires some special configuration due to it's default non-standard CGI behavior. You have to do two things:
1) Configure IIS to handle PATH_INFO and PATH_TRANSLATED according to the CGI specification, by running the following command:
cscript adsutil.vbs SET W3SVC/1/AllowPathInfoForScriptMappings 1
This is lightly documented in http://support.microsoft.com/support/kb/articles/Q184/3/20.ASP.
2) Use Gordon McMillan's Standalone utility, which is part of his Installer package, to convert WebKit.cgi into an executable program, WebKit.exe. IIS apparently will not handle requests with additional path components after the name of the script, like http://localhost/WebKit.cgi/foo/bar
where WebKit.cgi is a script, but will handle requests like http://localhost/WebKit.exe/foo/bar
where WebKit.exe is an executable. In any case, using the Standalone utility probably results in a speed increase, so this is probably a good thing to do anyway.
In order to convert WebKit.cgi into WebKit.exe using the Standalone utility, you first have to rename it to WebKit.py, and add import socket
to the top of the file (you could use a configuration file instead, but this is easier).
AOLserver
The PyWX project aims to meld Python and AOLserver and as part of that, they created an AOLserver "stay resident" WebKit adapter for tighter integration. More information can be found at http://pywx.idyll.org.
Architecture
The WebKit architecture involves three main entities at the top level:
The browser will be something like Microsoft Internet Explorer or Netscape Communicator. The web server will be something like Apache or IIS. And finally, the app server will be WebKit, e.g. a Python process running the WebKit modules and your custom servlets.
The chronological order of events goes from top to bottom and then back up to top. In other words, the browser makes a request to the web server which in turn makes a request to the app server. The response then goes from the app server to the web server to the browser.
The key to installing WebKit is to get the web server and app server talking to each other. For this purpose, you must use a WebKit adapter. See below.
Adapters
A WebKit adapter takes an HTTP request from a web server and, as quickly as possible, packs it up and ships it to the app server which subsequently sends the response back to the adapter for delivery to the web server and ultimately the web client. More concisely, an adapter is the go-between of the web server and the app server.
The current suite of adapters for WebKit are:
Note that on UNIX and NT, regardless of which adapter you use, you may need to set some permissions so that the app server can write files to the WebKit directories Cache, ErrorMsgs, Logs and Sessions.
The easiest way to do this is to make these directories writeable by all (on UNIX, that would be cd Webware/WebKit; chmod -R a+rX Cache ErrorMsgs Logs Sessions
).
The most secure way to do this, is to make sure the app server and it's adapter are executed by a specific user and that the specific user has write permissions to these directories (but no one else). This requires some savvy with both your operating system and your web server. You may wish to take the easy approach first in order to bootstrap yourself and then come back to this.
CGI Adapter
The "classic" WebKit set up uses the CGI adapter, known as WebKit.cgi. Almost all web servers support CGI so it was obvious that WebKit should, too.
In this configuration, the app server stays resident (similar to the web server) and is tapped for responses by a small CGI program that forwards the request to the app server.
Note that even though this approach uses CGI, such a WebKit application will run faster than the equivalent all-CGI solution. The reason is that that the app server (and possibly your servlets) can do quite a bit of caching, and the CGI adapter is fairly small. On the other hand, this is one of the slower adapters out of the bunch.
To set this up, do the following:
#!python
rather than #!/usr/bin/env python
. Make sure that python is in your search path as defined in AUTOEXEC.BAT. You can check this by typing python
at the command prompt to see if Python launches.
#!/usr/bin/env python
, works just fine. You can check this by typing /usr/bin/env python
at the command line to verify that Python launches. In the event that it does not, or if you wish to point to a different installation, you can change the line to something else, for example, /usr/local/bin/python
or /home/me/bin/python
.
WebKitDir = None
to point to WebKit. For example, WebKitDir = '/home/me/Webware/WebKit'
.
cd ~/Webware/WebKit
on UNIX and cd \Webware\WebKit
on Windows).
AppServer
and press return. You should see the start up messages for the app server. This works on both Windows and UNIX, via the AppServer.bat
and AppServer
start up scripts. You will see that they are very simple scripts and simply point to the recommended default app server.
http://localhost/WebKit.cgi/
. By default, the WebKit app server will display the Examples page. A sidebar provides navigation of the various examples which you can look through. Note that for any example, you can view it's source code by clicking the View source of... link in the sidebar.
Stopping the app server is accomplished by simply going to it's terminal/command window and hitting Control-C. The app server is designed to intercept this interruption and shut down gracefully. This includes saving session data to disk, if necessary.
[Windows] Control-C normally shuts the app server down gracefully, where as Control-Break does not. Keep that in mind and use Control-C, unless the server is unresponsive to it.
[UNIX] If you don't have access to the terminal window of the app server (perhaps because you used rlogin, telnet or ssh to remotely access the machine), you can use ps -ax | grep AppServer
to get the pid and kill -s 2 <pid>
to effect a Control-C.
Reloading Servlets
As you develop your web application, you will change the code of your various Python classes, including your servlets. The WebKit app server will detect a change in the timestamp of a servlet's source file and automatically reload it for you.
However, reloading fails in two areas. The first is that WebKit doesn't check ancestor classes of servlets for modifications. So if you modify an abstract class (for example, SitePage, AccountPage, etc.), it won't be reloaded. The second is that WebKit can't check non-servlet classes. So if you modify a utility class (for example, ShoppingCart, Story, etc.), it won't be reloaded.
You can deal with reloading problems by stopping the app server (Control-C in it's terminal/command window) and restarting it. If this gets too bothersome during development (and it probably will), use the OneShot adapter.
OneShot Adapter
The OneShot adapter is another CGI based adapter, but unlike WebKit.cgi, the app server launches, serves a response and shuts down for every single request via OneShot.cgi.
There are three major reasons to use the OneShot adapter:
Of course, the cost of these benefits is performance. OneShot is at least an order of magnitude slower than a persistent app server. For development, that's not a problem. But for a deployed site with multiple users it could be.
We generally recommend that you develop with OneShot and deploy with something else, like WebKit.cgi or FastCGI (see below).
To set up OneShot, follow the same instructions as for WebKit.cgi with two exceptions and one addition:
'SessionStore'
from 'Memory'
to 'File'
. See Sessions for more information.
The essence of the FastCGI protocol is to keep the CGI program resident. In our case, that program is a combination of the Python interpreter, several standard Python modules and the WebKit adapter code. By eliminating the constant reloading of these components, a substantial speed up can be obtained (2 X in informal tests).
Your web server will have to be FastCGI enabled, which you may be able to accomplish by downloading software at http://www.FastCGI.com where you can also learn more about FastCGI.
The top of FCGIAdapter.py contains a doc string explaining it's set up.
Note that to date, we have only gotten this to work on UNIX.
For development purposes, the FastCGI adapter doesn't offer anything substantial. However, for deploying your site for real use, the extra configuration effort could be well worth the performance boost.
Another adapter that provides the same type of benefit is the mod_python adapter, described below.
Mod_python Adapter
Mod_python is a module that embeds the Python interpreter into the Apache web server. This can make for a substantial speed up over WebKit.cgi which reloads the Python interpreter and various modules for each request.
mod_python itself and the adapter are both somewhat new, which makes this approach the least tested among the bunch.
The top of modpHandler.py contains a doc sting explaining the set up.
This has been tested on both UNIX and Windows.
More information about mod_python can be found at http://www.modpython.org.
Another adapter that provides the same type of benefit is the FastCGI adapter described above.
AOLserver Adapter
This adapter comes with the Python-enhanced AOLserver created by the PyWX project and is described there.
Renaming Adapters
Adapters such as WebKit.cgi and OneShot.cgi do not rely on their name. Consequently, when you deploy your web site, you can rename the adapter to something like serve.cgi
. This allows you to switch adapters later without affecting the URLs and therefore the bookmarks of your users (provided you're still using some form of CGI adapter).
Configuration
In this section, we'll briefly touch on some of the configuration options related to installing and running WebKit. A full reference to these options can be found in the User's Guide under Configuration.
The settings referenced below are found in the configuration file, Configs/Application.config.
Sessions
WebKit provides a Session utility class for storing data on the server side that relates to an individual user's session with your site. The SessionStore
setting determines where the data is stored and can currently be set to Memory
or File
.
Storing to memory is the fastest solution and works well with a persistent app server. Note that the session data will be saved to disk periodically (as a backup measure) and upon app server shutdown. It will then be automatically reloaded upon app server startup.
Storing to files is provided mainly in support of the OneShot adapter. It may also prove useful in the future in support of load balancing. In this scenario, each individual session is stored in it's own file, loaded for every request and saved when delivering the corresponding response.
All on-disk session information is located in WebKit/Sessions.
Also, the SessionTimeout
setting lets you set the number of minutes of inactivity before a user's session becomes invalid. The default is 60.
Activity Log
Three options let you control:
See Configuration in the User's Guide for more information.
E-mail Errors
EmailErrors
, ErrorEmailServer
and ErrorEmailHeaders
let you configure the app server so that uncaught exceptions land in your mailbox in real time. You should definitely set these options when deploying a web site.
See Configuration in the User's Guide for more information.
Contexts
WebKit divides the world into contexts, each of which is a directory with it's own files and servlets. WebKit will only serve files out of it's list of known contexts.
Some of the contexts you will find out of the box are Examples, Documentation and Admin. When viewing either an example or admin page, you will see a sidebar that links to all the contexts.
Another way to look at contexts is a means for "directory partitioning". If you have two distinct web applications (for example, PythonTutor and DayTrader), you will likely put each of these in their own context.
To add a new context, simply add a new one to the Contexts
dictionary of Application.config. The key is the name of the context as it appears in the URL and the value is the path (absolute or relative to the WebKit directory). Often the name of the context and the name of the directory will be the same:
'DayTrader': '/All/Web/Apps/DayTrader',
The URL to access DayTrader would then be something like:
http://localhost/WebKit.cgi/DayTrader/
The special name default
is reserved to specify what context is served when none is specified (as in http://localhost/WebKit.cgi/
). Upon installation, this is the Examples context, which is convenient during development since it provides links to all the other contexts.
Note that a context can contain an __init__.py
which will be executed when the context is loaded at app server start up. You can put any kind of initialization code you deem appropriate there.
AppServer Notes
ThreadedAppServer
WebKit uses a process called an AppServer to handle requests. The AppServer is responsible for receiving a request from the adapter, and then running it through the Application, and then sending the response back to the adapter. The default AppServer is called ThreadedAppServer. This AppServer uses a set number of threads to process requests. The number of threads used is configurable. As each request comes in, a thread from the thread pool will be handed that request and will be responsible for receiving the request from the adapter, running the request through the Application, and then sending the response back to the adapter.
AsyncThreadedAppServer
There is also a modified version of ThreadedAppServer, called AsyncThreadedAppServer. The modification is that the threads are only responsible for running a request through the Application and not for any network I/O which is handled by the Python asyncore module. The reason for this alternate form is that heavy network traffic could theoretically tie up threads if they are responsible for sending the response, thus leaving the potential for the AppServer to run out of threads to process requests.
[Linux] In testing on Linux systems, AsyncThreadedAppServer has shown a slight drop in performance for light loads, but has held up better than the threaded version under very heavy loads.
[WinNT] On Windows NT, testers have reported little difference between the two AppServers under heavy loads, and a slight slowdown when using the Async version under light loads.
[Win98] On Windows 98, this version of the app server has major problems simply serving a handful of simultaneous requests, so stick with the default threaded version.
You can change which AppServer is used by editing the file AppServer (on UNIX) or AppServer.bat (on Windows).
Monitoring
[UNIX] In order to provide additional stability and peace of mind, a monitor process has been developed. The monitor is responsible for starting the AppServer and monitoring it's health. If it detects that the AppServer is not responding, it will start a new AppServer and clean up the old one. In order to use the monitor, run the file wkMonitor.py. The wkMonitor process will handle the starting of the main AppServer. In order to shut down WebKit when using the monitor process, run stopMonitor.py.
This process will be available on Windows systems in the near future.
Other Notes
See Stopping the App Server and Reloading Servlets above in Adapters.
Future
This section documents future items for this install guide. To learn about future developments for WebKit, see WebKit's Future document.
Chuck Esterbrook (primary author)
Geoff Talvola (IIS notes)
Jay Love (AppServer notes)