Postfix will receive email and deliver it to the user's inbox, but additional software is required to read it with ease. There are two standards for retrieval of email from a host. The first is called POP (Post Office Protocol). POP3 is most commonly used. This is normally used to read email from the server, store it in a client application, and remove the email from the server. This is often used by ISPs (Internet Service Providers). The email is subsequently manipulated by the client application, for example Outlook Express or Mozilla Thunderbird.

The second protocol is called IMAP (Internet Message Access Protocol). The IMAP system is usually used when the email is to stay on the server. IMAP allows users to create folders for email, and to move or copy emails between the folders. The client application accesses the email on the server, but does not have to store it on the client machine. The email server must be able to store all of the email for all of its users, and the amount of data is expected to grow constantly - users rarely delete email. IMAP is therefor more frequently used in large organizations with centralized I.T. Facilities.

There are many POP3 and IMAP servers. Some perform only one of the tasks. The Courier suite of software contains both a POP3 and IMAP servers, and is covered in detail in this chapter.

Courier operates by accessing the maildir of the user. An overview of the operation is shown below:

(todo: nice diagram. Postfix putting email into maildir, courier accessing it, email client accessing it via POP3 and IMAP)

Downloading and Installing Courier

Courier is a suite of programs, and includes a fully-fledged MTA. This book assumes that Postfix is used. It is important that only the POP3 and IMAP components of Courier are installed and configured - an email system would be very unstable if there were two MTAs operating at once.

There are a couple of ways to install Courier. RPMs (Redhat Package Manager) of it are available for several different distributions of Linux. These will either be available from the manufacturer of the distribution, or may have been built by a third party, typically an enthusiast or developer of courier. If a package of Courier is not available in RPM, then it has to be built from source.

Installing Courier-IMAP from RPM

With RPMs, it is important to get one that matches the distribution in use. Using an RPM for another distribution may not work correctly, and may also make existing software unstable.

To locate an RPM of Courier-IMAP, first check if one is provided by the Linux distributer. If so, then download and use it. If the vendor does not provide a package, then it is possible that another may be provided. To check this, search the web. There is a database of RPMs available at www.rpmfind.net, and by searching for "courier", coupled with the name of the distribution, will locate any suitable packages. It is best to only use a package designed for a particular version of a distribution, for example a package for Mandrake Linux 8.0 should not be used for Mandrake Linux 8.1. If in doubt, it is best to install Courier-IMAP from source, as described in the next section.

To install Courier-IMAP from RPM, firstly download the RPM, and use a command prompt to change to the directory containing the file. As root, use the rpm command to install the RPM:

# rpm -ivh Courier-imap-mandrake-8.1.rpm

The RPM command may fail as prerequisite software may be required. In this case, the output will name the software required. The appropriate package can be downloaded and installed using the rpm command as above. Once all prerequisite software has been installed, then courier-imap can be installed using the rpm command shown above.

Due to the complexities of managing dependent packages, a graphical interface can be used to install Courier-IMAP. This may only work for Courier-IMAP if the RPM is provided by the distributer, but should be successful in installing prerequisite software.

If the rpm command was used to install Courier, then it can be used to uninstall it. The command will be similar to the following:

# rpm -e Courier-IMAP

Installing Courier-IMAP from source

Installing Courier-IMAP from source is not a difficult task on a modern Linux distribution. On older versions of Linux, and on other Unix platforms such as AIX, Solaris, and HP/UX, problems may arise, particularly if the rest of the system software is not up to date.

Prerequisites

The following are required to install Courier-IMAP:

·        A working C++ compiler. We recommend the Gnu C ++ Compiler, which is part of virtually every Linux distribution and is available free for most platforms. If an RPM or other package of gcc is available (and it almost certainly will) then it should be used in preference to building from source.

·        A make utility. We recommend the gnu make utility, which will be available for most platforms, or can be downloaded from www.gnu.org/software/gcc/gcc.html

·        The Gnu linker, available from www.gnu.org/software/binutils/.

·        Gnu libtool, available from www.gnu.org/software/libtool/.

·        Either the Berkeley DB library or the GDBM library. These are libraries that allow programs to make databases in files. Again, these should be available in packaged form, but can be downloaded from www.sleepycat.com and http://www.gnu.org/software/gdbm/gdbm.html respectively. One or both of these will almost certainly be installed already.

·        The Courier source code.

To successfully install Courier-IMAP, all the prerequisites must be installed first.

Installing Courier-IMAP versions below 4.0 from source

In January 2005, Courier-IMAP 4.0 was introduced. Generally, when a major version of an open-source package is released, distributions tend delay implementing it for a period of months or even years. However, the new package always offers major improvements and new features. We recommend that you install the latest version available, wherever possible. Often, only the latest version of a package is actively maintained, and bugs or security errors in a package may only be fixed in the latest version. For an application which listens for connections on an Internet connection, security vulnerabilities can be very serious.

However, there are often good reasons for using an older version - documentation is abundant and help is often readily available. An older version often has a "tried and tested" reputation that is appealing.

If you wish to install a version of Courier-IMAP below 4.0, then the instructions are similar to below, but there is no need to download and install the Courier Authentication library. Please skip the following section and proceed to "Building Courier-IMAP"

Building the Courier Authentication Library

There are two phases to installing Courier-IMAP. First of all, the Courier authentication library, called Courier-authlib, must be built. Once this is done, Courier-IMAP can be installed.

The Courier-authlib source should be downloaded from www.courier-mta.org/authlib/. As with many open-source packages, the Courier Authentication Library uses a configuration script to detect system capabilities, then uses the make command to build and install the software.

To build the Courier Authentication Library, enter the following commands. You should see responses similar to those below:

After the commands have executed successfully, the Courier Authentication Library will be installed. Before it can be started, some configuration is required.

Configuring the Courier Authentication Library

Several decisions need to be made once the authentication library is installed.

The Courier Authentication Library provides the system administrator with flexibility in how to authenticate users. Authentication is when a user proves their identity, typically by providing a valid username and corresponding password. The following options are available:

Authentication Method	Description
authshadow	Most Linux distributions hold user passwords in a system file called /etc/shadow by default. Using authshadow for authentication validates passwords against system accounts. This is suitable only when users have system accounts – i.e. they can log onto the machine using telnet or ssh.
authpwd	On older systems, passwords were stored in the /etc/passwd file. The authpwd module allows users to be authenticated against their system password. Again, users must have system accounts.
authuserdb	Unlike authshadow, where each user needs a system account, authuserdb stores user details seperately from the system accounts. This allows a ”virtual mailbox” facility, where users can be defined without having real accounts on the machine. A number of scripts are used to administer the database, which is held in two files.
Authmysql	This is similar to authuserdb, but uses a MySQL database instead of the files used in authuserdb. MySQL is a popular relational database provided by most Linux distributions, and offers both advantages and disadvantages over the
authpam	Authentication is provided by the programmable access method (PAM) library. PAM is a commonly used library, and should be provided by most Linux distributions. Pam is flexible, and can in turn authenticate users from a variety of sources, including the system password database (typically the /etc/passwd file).
authcustom	This allows the system administrator to develop their own, custom authentication method.

 

Choosing an authentication method can be a difficult decision. Here a re some guidelines:

·        If all users will have system accounts, then authshadow, authpwd or authpam can be used. If PAM is already installed and configured, then it should be used in preference.

·        If a virtual email system is required, then use either authdb, or authmysql. For small sites, there is little advantage in choosing authmysql over authdb.

In this book, only simple authentication with authshadow (or authpwd) is covered, although if PAM is installed and configured, then no additional configuration will be required. Authuserdb and authmysql require further configuration, which is described in the documentation for the authentication library.

The directory /usr/local/etc/courier/authlib contains the configuration files for the Courier authentication library. For security purposes, it's best to make the whole directory readable only by certain users. The default authdaemonrc file can be copied from the installation directory.

To complete the configuration, edit the file /usr/local/etc/courier/authlib/authdaemonrc and alter the following entries as appropriate:

In the line beginning authmodulelist, enter only the module(s) that you wish to use.

The daemons= line lists how many processes should wait to authenticate users. Unless there will be a very high number of users, a value of 3 to 5 should suffice. The bigger the number of daemons, the more memory will be used up by the authentication library, and there will be less available for other processes, which may affect overall system performance.

The authdaemonvar line lists where the courier authentication library places its run-time files, in particular the socket used to connect to it. The directory listed here (in this example, it is /var/lib/courier/authdaemon) should exist and be only readable by the root user. Use the following commands as root to create the directory:

For security purposes, it's best to make the authdaemonrc file readable only by certain users:

The authentication daemon needs to be started when the system boots. Typically, a script is placed in /etc/init.d/ to enable easy starting and stopping of a daemon. A sample script is included with the source of the authentication library, in ./courier-authlib.sysvinit.

This file should be placed in /etc/init.d:

The service can in future be started and stopped with the commands:

Initially, we should run the daemon directly from the command line. Any errors will then be displayed.

In the example above, the /usr/local/etc/authlib/authdaemonrc file was missing.

If the service is started correctly, then it can be stopped by passing it the parameter stop:

To get the service to automatically start as Linux boots, consult the documentation for the distribution. On RedHat systems, the service command can be used to configure a service to start automatically:

For other distributions, the chkconfig command might be used:

Resolving errors

Errors can be received at each phase of the build. Errors while running the configure script will probably relate to a missing dependency. Check the README and INSTALL files supplied with the software, and ensure that all dependencies are installed. If the problem is not obvious from the error message provided, then an Internet search for the exact error message may find a solution.

An error at build time is unusual, as most errors will be prevented by the configure script. Again, the error message should provide a good clue to the source of the error, and use of an Internet search engine may pay off.

Errors when running the software are generally due to erroneous configuration. There are few configuration options with the Courier Authentication Library, but errors can and do occur.

If an answer can't be found, there is a courier mailing list which can be approached for help. As always, firstly search list archives for your problem, and consult the FAQ. For Courier-IMAP, the mailing list is at http://lists.sourceforge.net/lists/listinfo/courier-imap, searchable list archives are available at http://sourceforge.net/mailarchive/forum.php?forum_id=7307, and the FAQ is available at http://www.courier-mta.org/FAQ.html.

Building Courier-IMAP

The Courier-imap source code is available in a tarball - a package of all the files, similar to a zip file. This should be downloaded from www.courier-mta.org/, but be careful to download the source for courier-imap and not for the Courier MTA.

To install Courier-IMAP, a few commands must be entered. As with much free software, a configuration script is run first. The configuration script checks the software installed on our machine and configures the software so that it will build correctly.

When Courier-IMAP is used as an IMAP server, it assumes that its clients are going to follow the IMAP standard exactly. Unfortunately, this is not the case, and if Courier-IMAP expects the clients to conform to the IMPA standard exactly, then the clients would not function correctly. The Courier-IMAP developers recognize this, and have built the capability to work with non-standard clients, by passing the –enable-workarounds-for-imap-client-bugs flag to the configure script.

Courier-IMAP also includes a check functionality when building it. Unfortunately, using --enable-workarounds-for-imap-client-bugs prevents the check from working successfully. As the check functionality is useful, we will build the software twice, firstly without the --enable-workarounds-for-imap-client-bugs, and then run the check, and then build again with the flag, and install the software.

To build Courierr-IMAP, enter the following commands. Choose a suitable directory to build the software, in this example we choose /tmp, and the software unpacked itself into the courier-imap-3.0.8 directory.

If the output appears similar to above, then Courier-IMAP will have been successfully installed. You may continue at the section "Configuring Courier-IMAP".

Handling errors

It is possible that the configure command will fail. Configuration attempts to detect existing software, and ensure that Courier-IMAP works with it, but it occasionally makes an error.

In this example, the configure command assumed that vpopmail was installed, and failed when it couldn't find parts of vpopmail. In reality, vpopmail was not installed, and should not have been detected, By consulting the INSTALL file, we read:

This suggests that the authvchkpw is used for vpopmail. Further up the INSTALL file, we read:

Upon checking the /etc/passwd file, we find that there is an account for vpopmail, which explains the detection. The lack of vpopmail files explains the failure of the configure script. In the INSTALL file, the parameters to the configure script are described:

The solution is therefor to use the --without-authvchkpw option:

Most problems can be solved in a similar way. It is best not to be put off by terms and names that aren't understood - just by searching for the term vpopmail (which was mentioned in the original error message), but without understanding anything about vpopmail, it is possible to resolve the error by reading the documentation.

If an answer can't be found, there is a courier mailing list which can be approached for help. Details are given in the section on the Courier Authentication Library.

Using POP3

As mentioned above, POP3 is typically used when email is to be stored on a client computer. It is most often used when there is an intermittent connection to the email server, for example when using a dial-up line to access an email account at an ISP. This approach has the advantage that the email is always available to the client, who can work when not connected to the email server. Emails can be read, and replies created ready for when the user is next on line.

The main disadvantage of using POP3 is that email is generally only available on the client PC. If the client PC fails, or is stolen, then the email is lost, unless a backup has been made.

POP3 clients can be configured to keep email on the POP3 server, for other clients to access. IMAP is more often used in this situation.

Configuring Courier for POP3

The configuration files are located in /usr/lib/courier-imap/etc/courier-imap/, if courier was built from source. If using a packaged distribution, they may be located in /etc/courier-imap. The file pop3d contains the settings for the pop3 server.

If you are using a packaged distribution of Courier, then the configuration files can be found with this command:

Edit the file and locate and alter the following settings:

Setting	Description
PIDFILE	The pop3d daemon keeps track of the process ID that it uses. Specify a valid path and a name which suggests the use of the file. Typically, this might be /var/run/pop3d.pid
MAXDAEMONS	This specifies the maximum number of pop3d process that can run at one time. The number of users that can connect at one time is limited by this number. A number higher than the expected number of users may be wasteful, but users attempting to connect are also included in this number. Set this to a number around the maximum that may connect at one time, or a little higher.
MAXPERIP	This specifies the maximum number of connections from each IP address. A low number prevents malicious acts such as denial-of-service attacks, where an attempt is made to use up all the connections on the mail server.
AUTHMODULES	This specifies the authentication method to be used. If the courier authentication daemon is used, as with Courier-IMAP v4.0 and later, then set this to “authdaemon”
POP3AUTH	If the courier authentication daemon is used, set this to blank, otherwise set it to indicate the type of login authentication performed. For versions prior to 4.0, this should probably be set to “LOGIN”.
PORT	This specifies the port that the daemon listens on. The standard port is 110, and a different one should only be chosen if all client software is configured to use the non-standard port.
ADDRESS	This specifies the IP address to listen on. If the machine has multiple network interfaces, then Courier can be configured to listen only on one of the addresses. A value of 0 indicates that all network interfaces should be used.
TCPDOPTS	These are options to be used. Typical ones used include -nodnslookup, which prevents the pop3 daemon from attempting to resolve the name of each connection, and -noidentlookup prevents it from attempting an ident query for the incoming connection. Specifying both of these settings can decrease the time taken to authenticate a user connection.
MAILDIRPATH	This is the path to a typical user's maildir. Specify the appropriate value for your system, for example “.maildir”

A samplepop3d configuration file is shown below.

Once the pop3 server has been configured, it is time to test it. If you are using a distribution-supplied version of Courier-IMAP, then use their startup script, probably called /etc/init.d/courier-imap. This will attempt to start imapd as well as pop3d, but as most of the configuration will have been done by the distributors, imap should start successfully.

To start the pop3 service for testing, run the following command:

# /usr/lib/courier-imap/libexec/pop3d.rc start

Once the pop3 and imap services are configured correctly, they can be started automatically when the machine is started. As this should be done once imap is configured, it is described at the end of that section. The instructions can be followed, even if IMAP is not required.

Testing the POP3 service

The easiest way to test a service like POP3 is by using the telnet utility and connecting to the appropriate port. This avoids any problems that there may be with network connectivity or client configuration. POP3 uses port 110, so telnet to port 110 on the local machine:

The POP3 protocol is based on text commands, and so it is easy to emulate a client with telnet. Initially, use the USER and PASS commands to authenticate a user. If the user is authenticated correctly, then the STAT command lists all emails and their combined size in bytes. LIST lists each email and its size. The RETR command retrieves (or lists) an email when the email number is specified with the command. The DELE command, which is not shown in the example, will delete an email from the server.

Now that POP3 appears to be working, it is time to configure an email client to collect email.

Retrieving email via POP3 with Outlook Express

Outlook Express is a popular email client, shipped with most versions of Windows. It includes POP ability. Follow the following steps to set up a windows client

These instructions are for Outlook Express 6, shipped with Windows XP. Other versions will be configured in a similar way.

Start Outlook Express by clicking on the icon on the desktop, or locate it in the Start menu hierarchy. Select the Tools->|Accounts menu.

 

Click on the Add button, and select Mail from the menu. The Internet Connection is displayed.

Enter a descriptive name for the server as the "Display Name", then press Next.

 

Enter your email address and press Next.

Enter the IP address of the email server in both the Incoming mail and the Outgoing Mail fields. Ensure that POP3 is selected in the list of server types at the top of the screen, then press Next.

Enter the account name. This is the system account on the email server. If desired, leave the "Remember Password" box checked and enter a password, otherwise leave the password field empty. Press Next one more time

Press Finish to complete the wizard.

The account has been added to the summary screen. Press Close, or select the account and press the Properties button if some of the details need to be changed. You can return to this screen and modify the account by selecting it in the list and pressing the Properties button.

To retrieve mail, select the Tools->Send and Receive->Send and Receive All menu. If you did not provide a password, then you will be prompted for the password.

Enter the password and press OK, and email will be downloaded.

Now that POP3 has been successfully configured, it's time to move on to IMAP.

Using IMAP

IMAP stands for Interactive Mail Access Protocol. It is another method of communicating with an email server. Generally, the mail is held on the server, and not on the client. This makes it ideal for organizations with a central administrative function, as it eases backups, and also allows users to change the client computers that they work at – as long as the email client is installed, they can access their email from any computer on the network, or even from home or another office.

In contrast to POP3, IMAP allows users to create folders on the server, so that related emails can be stored together.

Configuring Courier for IMAP

After Courier-IMAP has been installed, either from package or from source as described above, it needs to be configured before it can be used.

The configuration files are located in /usr/lib/courier-imap/etc/courier-imap/, if courier was built from source. If using a packaged distribution, they may be located in /etc/courier-imap. The file imapd contains the settings for the IMAP server.

If you are using a packaged distribution of Courier, then the configuration files can be found with this command:

Once the file has been located, it can be modified as appropriate. Here are the main configuration directives:

Setting	Description
PIDFILE	The imapd daemon keeps track of the process ID that it uses. Specify a valid path and a name which suggests the use of the file. Typically, this might be /var/run/imapd.pid
MAXDAEMONS	This specifies the maximum number of imapd process that can run at one time. The number of users that can connect at one time is limited by this number. A number higher than the expected number of users may be wasteful, but users attempting to connect are also included in this number. Set this to a number around the maximum that may connect at one time, or a little higher.
AUTHMODULES	This specifies the authentication method to be used. If the courier authentication daemon is used, as with Courier-IMAP v4.0 and later, then set this to “authdaemon”
POP3AUTH	If the courier authentication daemon is used, set this to blank, otherwise set it to indicate the type of login authentication performed. For versions prior to 4.0, this should probably be set to “LOGIN”.
PORT	This specifies the port that the daemon listens on. The standard port is 143, and a different one should only be chosen if all client software is configured to use the non-standard port.
ADDRESS	This specifies the IP address to listen on. If the machine has multiple network interfaces, then Courier can be configured to listen only on one of the addresses. A value of 0 indicates that all network interfaces should be used.
TCPDOPTS	These are options to be used. Typical ones used include -nodnslookup, which prevents the pop3 daemon from attempting to resolve the name of each connection, and -noidentlookup prevents it from attempting an ident query for the incoming connection. Specifying both of these settings can decrease the time taken to authenticate a user connection.
MAILDIRPATH	This is the path to a typical user's maildir. Specify the appropriate value for your system, for example “.maildir”
MAXPERIP	This specifies the maximum number of connections from each IP address. A low number prevents malicious acts such as denial-of-service attacks, where an attempt is made to use up all the connections on the mail server.
IMAP_CAPABILITY	This describes the IMAP capabilities that the server reports to clients. It should probably be left on the default setting.
IMAP_EMPTYTRASH	This specifies how long email messages should be kept in certain folders for. Messages older than the date specified are automatically deleted, either when the user logs in or logs out. This can be used to automatically delete email from the trash folder after a certain period. This works for all folders, so emails in the "Sent items" folder could be deleted after a longer period has expired. For example, IMAP_EMPTYTRASH=Trash:7,Sent:30 specifies that emails in the Trash folder are deleted after 7 days, and those in the Sent folder will be deleted after 30 days. If very large numbers of emails are present in the folders specified, then performance will suffer, as each file will be checked every time the user logs in or logs out of the IMAP server. In this case, it would be better to disable this setting and run a separate script to run once (or more often) a day to remove old files.
IMAP_IDLETIMEOUT	This is the length of time (in seconds) that a client can be idle for (not make any request to the server), before the connection is closed. Values lower than the default of 60 may results in client connections being terminated prematurely, but a well-written client will reconnect without notifying the user. Higher values should be used if users report particular problems.
IMAP_TRASHFOLDERNAME	This specifies the folder to be used when email is deleted.
SENDMAIL	This specifies the path to sendmail, for sending emails.

Here is a sample imapd configuration file:

Testing the IMAP service

To start the imap service for testing, run the following command:

The easiest way to test a service like IMAP is by using the telnet utility and connecting to the appropriate port. This avoids any problems that there may be with network connectivity or client configuration. IMAP uses port 143, so telnet to port 143 on the local machine:

Each command is prefixed with an identifier. Here we use unique numbers. The first command asks the IMAP server to list its capabilities. The second command is a user login, and includes the username and password. If this is successful, then the final namespace command shows that the server has accepted the login and the client can determine where in the folder hierarchy the user is placed.

That's enough to confirm that the user can log in and issue commands. The whole IMAP command set is quite large and complex, and does not lend itself to use by telnet.

Once the pop3 and imap services are configured correctly, they can be started automatically when the machine is started. If you installed from a package, then the distributer will probably have created a suitable startup script in /etc/init.d. To start this when the machine boots depends on the distribution. For RedHat Linux, the command might be

For other distributions, the chkconfig command might be used:

Now that IMAP is configured correctly, it is time to configure an email client.

Retrieving mail via IMAP with Mozilla Thunderbird

Mozilla Thunderbird is a popular open-source email client. It is available for download from http://www.mozilla.org/, for a variety of operating systems, including Windows and Linux.

Here are steps to configure it to connect to a courier-imap server:

From the main Thunderbird screen, select the Tools->Account Settings menu.

Click on the Add Account... button. On the next screen, choose "Email Account", and then press Next.

On the next screen, enter your name and email address, then press Next

On the Server Information screen, enter the name or the IP address of the servers for incoming and outgoing email. Then press the Next button.

On the next screen, enter the username. This will normally be the Linux account name. After checking this, press the Next button.

Finally, provide a useful tag for the email account, in case more than one account is defined in the future. Press Next.

On the next screen, the details are summarized. Press Finish to save the account details for use with Thunderbird.

Finally, the Account Settings screen is shown, listing the account that has just been defined. Press OK.

To retrieve messages, click on the File->Get New Messages for menu, and select the account that has just been created.

Thunderbird will prompt you for the password. Enter the correct password then press enter. Thunderbird will then connect to Courier-IMAP and retrieve details of all the emails. If you click on an email, then Thunderbird will retrieve it using the IMAP protocol.

Summary

In this chapter, we covered the following:

• The POP3 and IMAP protocols were introduced

• The best method of installing Courier-IMAP is to use a package provided with your Linux distribution, but it an be installed from source if desired.
• After installation, Courier-IMAP may require configuration. We saw how to configure and test both the POP3 and IMAP services.
• We included examples of how to configure popular email clients to use POP3 and IMAP