B336 Lab Week 3

B336 Advanced Internet Computing

Laboratory Week 3

Learning Objective:

Install, configure and run the Apache web server.

Consider issues and carry out tasks in web server administration.

Required Reading:

Week 2 lectures.

Access to Software:

In the lab, we will be using the machine red.murdoch.edu.au, which is running Linux Red Hat 7.0. We will access the machine via a ssh or telnet session.

Students may use the puTTY program on Windows for a ssh or telnet session. Internal students have access to the program in lab SC2.31. External students may download the required software and install it on your own machine off campus. Internal students may do the same, but are encouraged to still attend labs for supervision when doing the exercises below.

Instructions:
For all instructions below, replace the number 12345678 with your own 8-digit student number, and the number 5678 with the last 4-digits of your own 8-digit student number.

In the examples, the words in teletype bold like this are commands you type in the ssh or telnet sessions. The words in teletype italics like this are comments and should not be typed in.

Logging on to red

Using a ssh or telnet session, connect to the address red.murdoch.edu.au, or 134.115.157.138. When prompted for a username, use the character 's' followed by your new 8-digit student number, without any spaces (eg. s12345678). The first time you log in, you will not be required to enter a password. Once you are in, use the passwd command to set a password of your own choosing. The following is an example session:

Red Hat Linux release 7.0 (Guinness) Kernel 2.2.16-22 on an i686 login: s12345678 Never logged in [12345678@red ~]$ passwd Changing password for s12345678 New UNIX pasword: blahblah Retype new UNIX password: blahblah [12345678@red ~]$

Installing the web server

You will compile and build the Apache distribution in your own area on red. There is already a copy of the appropriate Apache distribution in the test account. Copy it into your home directory, unzip and untar it, then configure the makefiles and make the installation. To do that, type in the following set of commands on red, in the order specified. Remember to replace 12345678 below with your own student number.

cd cp ~test/apache_1.3.19.tar.gz . (note the " . " at the end) gunzip apache_1.3.19.tar.gz tar xvf apache_1.3.19.tar cd apache_1.3.19./configure --prefix=/home/students/s12345678/apache make make install

The purpose of the putting your own directory in the "./configure ..." line is to ensure that the final server you build will operate from your home directory rather than the default /usr/local/apache.

Configuring the web server

You should now have a the web server ready to run. You now need change the httpd.conf (see lectures) so that your server is configured appropriately for this unit. The two major categories of changes that needs to be made are:

Change the port number from the standard 80 to one unique to you. You will use the port number starting with 1 followed by the last 4 digits of your student number. Eg. if your student number is 12345678, then your port number if 15678.

Change the process control directives so that it starts up a lot less processes than a normal web server. This is because we will have potentially 80 copies of the web server running at the same time. We need to rationalise processor and memory usage (see below on how to monitor everyone's server resource consumption on red).

To make the changes, you can use the editor pico. You may also use the more powerful vi or emacs if you are familiar with the those editors. Open the httpd.conf file by typing the following commands on red:

cd ~/apache/conf pico httpd.conf

You should see the contents of the httpd.conf file (refer to the lectures for a description of the file's contents). Most of them will be comments starting with #, but a few of the lines will be the actual directives. You need to change the following directives to the values below. In pico, you can search for a string using CTRL-W. Remember to replace 5678 with the last 4 digits of your student number. Also, for the ServerAdmin directive, put down your email address.

MinSpareServers 1
MaxSpareServers 3
StartServers 3
MaxClients 10
Port 15678
ServerAdmin <your email address>

Running the web server

To run the web server, you use the apachectl script in the bin directory of your installation. To start the server, type the command:

~/apache/bin/apachectl start

Accessing the web server and changing the web site

To use your running web server, start up any web browser, and access the URL http://red.murdoch.edu.au:15678/ - again replace 5678 with the last 4 digits of your student number. You will see a page with a message saying that your installation was successful. This is one of the index.html.* files in the htdocs directory of your installation (exactly which one depends on the language priority of your browser - probably en for English).

To change the contents of the web site served by your web server, just make changes to the htdocs directory. For example, if you put a index.html in there, that will become the default HTML file that any browser will get from the URL http://red.murdoch.edu.au:15678/. If you put a mypage.html in there, then the URL is http://red.murdoch.edu.au:15678/mypage.html. You may also put sub-directories in the htdocs directory.

You can change the file and directory contents of the htdocs directory by using an FTP client to access red, or directly using the command line on red.

As a note, there are ways of aliasing files and directories in Apache, so sometimes the available URLs do not completely conform to the directory structure of the htdocs directory. The default cgi-bin directory is a prime example of this. Refer to the Alias and ScriptAlias directives, or look at the httpd.conf for examples.

Looking at the server's resource usage
While the server is running, you will be able to see the status of processes doing the job and how much resources they are using. Two ways of doing so is by the ps command, or the top command.

For example, to look at the processes involved in the web server you started up, you can type the command

ps -l -u s12345678

You will see something like the following:
  F S   UID   PID  PPID  C PRI  NI ADDR    SZ WCHAN  TTY          TIME CMD
100 S 33683 21838 21837  0  64   0    -   669 rt_sig pts/0    00:00:00 csh
040 S 33683 21858     1  0  65   0    -   573 do_sel ?        00:00:00 httpd
040 S 33683 21859 21858  0  64   0    -   576 wait_f ?        00:00:00 httpd
040 S 33683 21860 21858  0  64   0    -   576 wait_f ?        00:00:00 httpd
040 S 33683 21861 21858  0  65   0    -   576 wait_f ?        00:00:00 httpd
000 R 33683 21872 21865  0  78   0    -   688 -      pts/1    00:00:00 ps
The httpd with PID of 21858 is the parent process, while the other 3 are the child processes waiting to serve requests. If the server has been running for a while, these child process IDs will change as they get killed off and new ones get started (see lecture notes on Apache's process model). Some useful fields to know for the "ps -l" display:

UID: user identification - every user has a number which is different from the username. Your username is of the form s12345678, but your UID is of the form 33xxx. You can find out your own UID by typing the "id" command on the command line.
PID: process identification - unique number for every process.
PPID: parent process ID - the parent process identification. In the example above, you can see that all child httpd processes are spawned by the parent process 21858.
WCHAN: name of the kernel handler the process is currently stopped at.
CMD: command typed by the user at the command line.

To look at all web servers running on the machine, use the command:

ps -l -A | grep httpd

A better (but much more processor intensive) way of looking at the processes is by using the top command:

top

The top few lines of the results are the overall and available system resources and the summary usage by all processes. Below that is a display of the processes sorted by highest resource usage in real time. By default it refreshes every 5 seconds. If you access you web server to get a HTML file, you will see your own process pop up to near the top of the screen for a few seconds before going back to idle.

To quit the display, type "q".

A good way to monitor your web server resource usage is by typing "u" after starting top and then type in your user name (eg. s12345678). Then the display will be restricted to your processes only. To illustrate what the display can show you, bring up a web browser window but keep the top monitor on the screen at the same time. Now access a web page on your server (preferably one which you haven't accessed before) and you will see the CPU and memory usage of one of the child httpd processes increase.

Some useful fields to know for the top monitor screen (besides the fields already listed for ps above):

USER: username - yours is of the form s12345678.
SIZE: total memory used in kb - physical and virtual memory.
%CPU: estimated average percentage of total CPU used since the last screen update.
%MEM: estimated average percentage of total physical memory used since the last screen update.
TIME: total amount of CPU time used in hour:minutes format - for idle httpd processes, this will stay 0:00 until it gets requests.

USE THE top COMMAND SPARINGLY SINCE IT CHEWS UP A LOT OF MEMORY AND CPU TIME! If you start top and you see many others already have theirs started up, it may be a good idea to stop yours and try again later.

There is a lot more options to ps and top. Look up the manual entries for them by typing:

man ps man top
Stopping and restarting the web server

YOU SHOULD STOP THE SERVER AT THE END OF EVERY LAB SESSION, since you are no longer using it. If you need to access it from home, you should start it up again using a ssh or telnet session when you want to use it.

To stop the server, use the apachectl script again:

~/apache/bin/apachectl stop

In doing your assignments, you will be making changes to httpd.conf and perhaps other configuration files. Remember that none of the changes will take effect until you restart the server. To restart the server (ie. to stop it and then start it), you can issue the two commands above, or just:

~/apache/bin/apachectl restart

Manuals

The standard online manuals is found in the directory htdocs/manual/ in your installation directory. This means you can access the pages using a web browser by accessing the URL http://red.murdoch.edu.au:15678/manual/. A gain remember about 5678.

Linux tutorial

If you have no prior experience with using the linux command line, here is a linux tutorial I prepared for another unit. It may contain extra material you do not need for now, but read it will give you a idea what you can do at the command line.

Hints

A few useful hints about linux and the default shell you are using on red:

Use the up and down arrow keys to bring back through the list of previous commands.

Press the tab key to automatically complete a partially typed command name or file name. This makes typing long file names easy.

Multiple options for commands can be usually be condensed. Eg. "ps -l -u s12345678" can be rewritten as "ps -lu s12345678".

A few useful symbols used in filenames:

~ means your home directory (so ~/.cshrc means the .cshrc file in your home directory)

~s12345678 means the home directory for user s12345678

. means the current directory (so ./configure means the configure script in the current directory)

.. means the directory above the current one
Exercises:

Start putting you own HTML files into the htdocs directory. Add sub-directories and put files in them as well. Access the files to make sure your server is doing its job properly. Understand the conversion rules your web server is using to convert between the URL string and the file system's pathnames.

For internals students, find out the port number of someone who has his/her server running in the lab, and access their web server and web site.

Look at the access_log and error_log files in the logs directory of your installation. You can use the pico editor to do so. Can you recognise the log entries corresponding to your accesses from the above exercises? Refer to the online manuals (especially the tutorials section) for more details of logging and the format of the log files.

Using the process watching mechanisms described above, look at the changes in system resource usage for your server. Start accessing different resources (HTML, images, etc) you put in your web site, and see how the process indicators change.

Questions to consider:

(To test your understanding of the concept of the web) Exercise 1 above is mainly to get you to think about the correspondence between a URL (a concept used on the public WWW), and the file's pathname (a concept only relevant on the operating system's file system). One of the web server's job is to convert between the two. What is the role of both the naming convention above? Can we get rid of one and just use the other? That is, can the web just make use of file systems' pathnames? Or can file systems just make use of URLs to refer to files?

Based on the default settings for logging (see Exercise 3), how much is a visitor's behaviour on a your web-site tracked? What potential benefits does this bring? What potential harm can this do? Should a web server be configured log such activities?

Look at the process performance of your server, and other servers running on red for the next few weeks. How capable is the machine with it's available resources at handling the load?

Experimentation:

The purpose of having each one of you have your own installation is so that you can try different things. DO NOT BE AFRAID TO TRY DIFFERENT SET-UPS AND CONFIGURATIONS,or to move and change files. In the worse case, you delete you whole apache directory and do the install again - it takes less than 5 minutes once you are familiar with the steps.

The only thing you should be wary of is starting off too many processes. If you think about what you're doing, read the manuals on the configurations you are changing, or consult your tutor, it shouldn not cause too many problems. Even if you mess up, then learn and move on. These lab exercises are worthless if all you do is following instructions as stated above but do nothing else.

If you are afraid of making mistakes or breaking machines, the unit coordinator will assist you in your considerations for enrolment-changes to English Literature.

Laboratory Week 3

Learning Objective:

Required Reading:

Access to Software:

Instructions:

Logging on to red

Installing the web server

Configuring the web server

Running the web server

Accessing the web server and changing the web site

Looking at the server's resource usage

Stopping and restarting the web server

Manuals

Linux tutorial

Hints

Exercises:

Questions to consider:

Experimentation: