POP3, IMAP4, Telnet

POP3 and IMAP4

POP3 and IMAP4. Why do we need them ?
How does POP3 work ?
Using Telnet from within the CSA Computing Lab
An Example of a POP3 Session using Telnet
Further References

POP3 and IMAP4. Why do we need them ?

We know that it is not possible for all computers to be continuously connected to the internet. This is especially true for users who use dial-up connections to access the Internet. These computers cannot run their own SMTP (RFC 821) servers to manage incoming and outgoing mails because the server needs to be running and connected to the internet all the time. Imagine a situation where someone can only send mail to you while you are logged onto the Internet. This situation is certainly unacceptable and ways have been developed to remedy it. Read on.

One solution is to maintain a mailbox on a machine (the server) which is continuously connected to the Internet. This machine would accept all mails on our behalf. We (the clients) could periodically connect to this server via the Internet to view or download the mails in our mailbox using a agreed upon protocol. Some of the ways to access ones's mailbox are listed below:

POP3 or the Post Office Protocol - Version 3 (RFC 1939), is one simple protocol that allows users to access their mailbox on a remote machine. POP3 does not provide "extensive manipulation operations of mail on the server". It was designed so that clients could download the mails from the servers which would then delete them. One limitation of downloading the mails to a client is that a user cannot access his mails from multiple clients because the mail is no longer on the server (people using cyber cafes would not be very happy). The POP3 server can usually be instructed to keep a copy of the mail on the server but this method is not very versatile because of the way POP3 was designed. The IMAP4 protocol is a more advanced (and complex) protocol which solves the above problem.
IMAP4 or the Internet Message Access Protocol - Version 4 (RFC 1730) was designed as a more powerful way of accessing and manipulating mail messages on a server. It was designed with the idea that a user would like to keep all his mails on the server which would then be accessble from any client anywhere in the world. IMAP4 includes operations for "creating, deleting, and renaming mailboxes; checking for new messages; permanently removing messages; setting and clearing flags; RFC 822 and MIME parsing; searching; and selective fetching of message attributes, texts, and portions thereof."
Another option that has become very common nowadays is to provide a web-interface to the mailbox (also known as webmail). This is the kind of interface that websites such as hotmail, yahoo, indiatimes and countless others provide. This interface works over the HTTP protocol using a standard markup language such as HTML. The user does not need to have special mail clients to access his mail, a browser will do. The downside is that it is relatively slower that POP3 and IMAP4 access.

It is common for a mailbox to be accessible using multiple methods. For example, the mailbox provided to me by VSNL is accessible using both POP3 and a web-interface.

Since POP3 is a more commonly used protocol (and simpler too), we will discuss it in detail here.

How does POP3 work ?

(much of the stuff in this section is derived from RFC 1939)

Initially, the server host starts the POP3 service by listening on TCP port 110. When a client host wishes to make use of the service, it establishes a TCP connection with the server host. When the connection is established, the POP3 server sends a greeting. The client and POP3 server then exchange commands and responses (respectively) until the connection is closed or aborted.

Commands in the POP3 consist of a keyword, possibly followed by one or more arguments. All commands are terminated by a CRLF (Carriage Return and Line Feed) pair.

Responses in the POP3 consist of a status indicator and a keyword possibly followed by additional information. All responses are terminated by a CRLF pair. There are currently two status indicators: positive ("+OK") and negative ("-ERR"). Responses to certain commands are multi-line. In these cases, lines sent are terminated by a CRLF pair. When all lines of the response have been sent, a final line is sent, consisting of a "." and a CRLF pair. A server responds to an unrecognized, unimplemented, or syntactically invalid command with a negative status indicator.

There are two methods of authentication specified by the protocol. A system needs to implement at least one of them. (Details of the commands are given later)

Using a combination of the USER and PASS commands. The disadvantage of this approach is that the passwords are sent in the clear (without any encryption) over the network.
Using the APOP command. The APOP command computes a MD5 message digest of a string containing the password and a timstamp sent by the server in the inital greeting message. This method provides both origin authentication (using the password) and replay protection (using the timestamp).

A list of valid commands is listed below. Note that the implementation of some commands is optional. Such commands are denoted by "(Optional)" towards the end of the definition. Some of the arguments to a command may also be optional, such arguments are enclosed in square brackets.

USER name: To authenticate using the USER and PASS command combination, the client must first issue the USER command. If the POP3 server responds with a positive status indicator ("+OK"), then the client may issue either the PASS command to complete the authentication.
PASS string: When the client issues the PASS command, the POP3 server uses the argument pair from the USER and PASS commands to determine if the client should be given access to the appropriate mailbox. The password is sent in the clear on the network so this method is not good from a security viewpoint.
APOP name digest (Optional): The APOP command provides an alternate method of authentication, which does not involve sending a password in the clear over the network. A POP3 server which implements the APOP command will include a timestamp in its banner greeting. The POP3 client makes note of this timestamp, and then issues the APOP command. The `name' parameter has identical semantics to the `name' parameter of the USER command. The `digest' parameter is calculated by applying the MD5 algorithm (RFC 1321) to a string consisting of the timestamp followed by the password.
STAT: The response consists of the number of messages in the mailbox and the size of the mailbox in octets (bytes).
LIST [msg]: The response consists of the message-number of the message and the exact size of the message in octets. If the msg argument (optional) was given then the information is listed for that particular message number, else a multi-line listing containing information for all messages in the mailbox is returned.
RETR msg: The POP3 server sends the entire message corresponding to the given message-number.
DELE msg: The POP3 server marks the message as deleted. The POP3 server does not actually delete the message until the user gives the QUIT command.
NOOP: The POP3 server does nothing, it merely replies with a positive response.
RSET: If any messages have been marked as deleted by the POP3 server, they are unmarked.
QUIT: The POP3 server removes all messages marked as deleted from the mailbox. The server then closes the TCP connection.
TOP msg n (Optional): The server sends the headers of the message, the blank line separating the headers from the body, and then "n" lines of the indicated message's body
UIDL [msg] (Optional): The server response with a line containing the "unique-id listing" for a specified message or for all messages if no msg argument was specified. A unique-id listing consists of the message-number of the message, followed by an arbitrary server-determined string, consisting of one to 70 characters

Using Telnet from within the CSA Computing Lab

This section may seem a bit out of place, but we need to use Telnet in the next section so i thought it would be nice to give all of you a little introduction.

The telnet command is used to communicate with another host. It sets up a TCP connection with a specified host at a given port number and presents us with an interface so that we can type messages to be sent to the other host and view the replies sent by the host. Since the POP3 command is a text based protocol, we can see what actually goes on beneath all the layers of the mail client by directly interacting with the POP3 server usign Telnet.

The problem I encountered was that Telnet didn't seem to work from hosts within CL. After a lot of asking around, i finally got the answer on the CSA Newsgroups. There is a IP firewall installed on all CL linux hosts which filters out IP packets from all but a few ports. The details of the permitted connections can be obtained here. Telnet is on the banned list, but there is still a ray of hope.

The solaris machines in CL (kohinoor, bhrigu and osiris) have no firewall. So we can telnet remote hosts through these machines. :)

An Example of a POP3 Session using Telnet

I am going to show you samples from a conversation I had with the POP3 server of VSNL. The server's replies are prefixed with a "S:" and the lines displayed by the client are prefixed with a "C:".

$ telnet mail.vsnl.net 110
C: Trying 203.200.235.182 ...
C: Connected to mail.vsnl.net.
C: Escape character is '^]'.
S: +OK Messaging Multiplexor (iPlanet Messaging Server 5.2 HotFix 1.16 )
C: USER malhotra_g
S: +OK password required for user malhotra_g@vsnl.net
C: PASS ***********
S: +OK Maildrop ready
C: STAT
S: +OK 86 1228352
C: LIST
S: +OK scan listing follows
S: 1 1720
S: 2 65796
S: 3 54771
..................
S: 84 6962
S: 85 5336
S: 86 5036
S: .
C: UIDL 7
S: +OK 7 7-1053395393
C: TOP 86 5
S: +OK
S: Return-path: < managementstudies@indiatimes.com >
S: Received: from mx4.vsnl.net (mx1-c2.vsnl.net [172.16.28.149])
S: by pop2.vsnl.net (iPlanet Messaging Server 5.2 HotFix 1.16
S: with ESMTP id <0HMC0007BLFVBA@pop2.vsnl.net>; Mon,
S: 06 Oct 2003 23:46:44 +0530 (IST)
............................
S: Date: Mon, 06 Oct 2003 23:45:03 +0000
S: From: managementstudies@indiatimes.com
S: Subject: Management Entrance Preparatory SERIES... Mumbai
S: Reply-to: managementstudies@indiatimes.com
S: Message-id: <309.286955.724678@indiatimes.com>
S: MIME-version: 1.0
S: Content-type: text/html; charset=us-ascii
S:
S:
S: <html>
S: <head>
S: <title>Management CDROM</title>
S: <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
S: .
C: DELE 86
S: +OK message deleted
C: NOOP
S: +OK
C: STAT
S: +OK 85 1223316
C: QUIT
S: +OK
C: Connection closed by foreign host.

Further References

As always, here are some links for those thirsty for more

Back to Network Programming

Contents

POP3 and IMAP4. Why do we need them ?

How does POP3 work ?

Using Telnet from within the CSA Computing Lab

An Example of a POP3 Session using Telnet

Further References