Chapter 1: Introduction to SMTP

SMTP (Simple Mail Transfer Protocol) is an email communication protocol used to reliably transfer emails between two systems.

An important feature of SMTP is been platform-independent, which means in order to run SMTP, you don’t necessarily require any specific programming language or software to be pre-installed on your computer/server. Also, SMTP is capable of relaying mail across different transport services. Let’s learn how SMTP can be used with various transport services:

TCP Transport service

TCP stands for Transport Control Protocol and this is the most popular transport service for establishing an SMTP connection between the two systems. In case of TCP, an SMTP transmission channel is established between the sender and receiver over a port. The standard port assigned for establishing this connect is port 25. This method of TCP and SMTP based connection establishment is mostly used in ARPA Internet.

NCP Transport service

NCP stands for Network Control Program. In case of NCP, an SMTP transmission channel is established using the NCP between the sender and receiver over a port. The standard port assigned for establishing this connect is port 25. This method of NCP and SMTP based connection establishment is mostly used on the  ARPANET internet.

NITS Transport service

NITS stands for Network Independent Transport Service, In case of NITS, an SMTP transmission channel is established using the NITS between the sender and receiver. The sender executes the CONNECT primitive and the receiver executes the ACCEPT primitive for establishing an SMTP connection between the two systems.

Chapter 2: SMTP Data

By default, the SMTP standards (as mentioned in RFC 821) limits each line of mail to 1000 characters and each character should be 7 bits.

This means during the SMTP data transmission, you cannot have any of byte with the highest-order bit set to 1. If that would have been the scnerio, then you won’t have been able to send any of the media files like image, video or even text which are multi-lingual or contain Unicode characters. This is because in all such cases, you will find the 8th bit which is set to 1.

While RFC 821 was the good golden rule for sending textual messages between the two systems, but really failed to describe the support of multimedia messages that might include images, audio, video or what about the non-US users who want to use multi-lingual characters which are not part of US-ASCII.

This problem gave birth to Multipurpose Internet Mail Extensions (MIME) in RFC 2045, which allows the inclusion of non-US-ASCII characters both in the header and mail body. This includes the use of images, audio, video and other attachments files like zip, pdf, doc.

Chapter 3: SMTP Connection Architecture

The SMTP connection architecture was designed in a very simple layman term, where the conversation between the two systems; the sender and the receiver happens completely in a robotic way:

Step 1: Sender SMTP server sends a HELO command to the receiver SMTP server for establishing a two-way communication channel. The receiver SMTP server responds back with a confirmation message OK.

Step 2: Sender SMTP server sends a MAIL command indicating the sender of the mail. If the receiver SMTP server can accept email from the sender, then it responds back with an OK command.

Step 3: Once the sender SMTP receives OK, it sends another RCPT command indicating the receiver of the mail. If the receiver SMTP can accept mail for that particular recipient, then it will again respond back with an OK command.

If the receiver server doesn’t want to accept mail, then it can reply back with a rejection response. Receiving a rejection response doesn’t mean the termination of the SMTP connection between the two SMTP servers, instead the SMTP connection will remain alive and the sender can send a new request for the recipient.

Step 4: If the sender SMTP server receives the OK response on sending the RCPT command, this means the negotiation between the two SMTP server is completed and now the sender can transmit the complete mail data to the receiver SMTP server by adding a special sequence of characters at the end, indicating it as the EOF. 

Step 5: Once the recipient receives the EOF, it acknowledges the receipt of the complete data by responding back with an OK command. Thereafter the connection gets terminated.

This architecture is completely scalable when you have to just send a handful of emails, but if you want to send thousands or millions of emails, then you will end up into scale issue because after one SMTP connection can only send one email at a time.

In order to fix this problem, the concept of persistent SMTP connection came into the picture. As the name suggests, you have to just open one SMTP connection and using the persistent function you can keep the same SMTP connection alive. By doing this, you can send multiple emails from the same connection without getting disconnected. This is also known as SMTP Keep Alive.

One of the examples of using persistent SMTP connection in PHP:

phpMailer = New PHPMailer();

$phpMailer->isSMTP();

$phpMailer->SMTPKeepAlive = true;

for ( … ) {

// Send your emails right away

[ … ]

}

$phpMailer->SmtpClose();

Chapter 4: SMTP Commands

In this section, you will learn the different commands which are used to communicate between the sender and receiver SMTP server. All SMTP commands are character strings ending with <CRLF> and the command code itself are alpha characters ending with <SP>.

List of basic SMTP commands which are a must to initiate an SMTP based mail sending:

HELO
MAIL
RCPT
DATA
RSET
NOOP
QUIT

HELO

This command is used by the sender SMTP server to open up a dialogue connection (a two-way communication channel) with the receiver SMTP.

The receiver SMTP server comes to know about the sender SMTP server’s identify and responds back with its identity and an OK command. Once this happens successfully, this means both are servers are at their ideal state with no buffers to accept input.

MAIL

This command has an argument field containing the reverse path to reach the source sender. If there are multiple hosts, then those can be passed as a part of this argument. The first host in the series will be the one which has most recently relayed the mail. The primary use of this sender address is to route the non-delivery notices to the sender.

Here is an example of a sender trying to connect with receiver SMTP to deliver email:

R: 220 smtp.pepipost.com Simple Mail Transfer Service Ready
S: HELO mail.example.com
R: 250 smtp.pepipost.com
S: MAIL FROM:<mo@LBL-UNIX.ARPA>
R: 250 OK

RCPT

This command is used to share the actual email address of the recipient. In case there are multiple recipients, then this command has to be executed multiple times by the sender SMTP server.

This is actually the forward path, which indicates the actual destination of the mail. In case there are multiple hosts from where the mail should be routed, then the list of all those hosts can be optionally passed as an argument.

If the receiver SMTP server can relay the mail to the mentioned recipient email address, then it can confirm the sender SMTP server by sending an OK command else it can reject the request too. If the receiver SMTP server doesn’t have the relay to the recipient, then it can reply with a 550 error message saying, the unknown user.

S: RCPT TO:<geniusdibya08@gmail.com>
R: 250 OK

S: RCPT TO:<dibya1@pepipost.com>
R: 550 No such user here

S: RCPT TO:<dibya@pepipost.com>
R: 250 OK

DATA

On receiving a 250 OK on the RCPT request, the sender SMTP can make another DATA command call for transmitting the mail data to the receiving SMTP server.

On lines following the DATA command will be treated as the mail data.

The end of the mail data is determined by a line containing only a period. The end character sequence will be like “<CRLF>.<CRLF>”. SMTP is an open protocol with every aspect of it very well documented in RFC.

If the beginning of your mail text is a period, then in such cases an additional period has to be inserted at the beginning. This is because when the receiver SMTP receives the data, it checks for the beginning of the line. If the line is only composed of a single period then it considers it to be the end of mail. So, in case you have not appended one more period before that, then the receiver will actually receive a blank mail.

On the other hand, if the first character of the mail text is a period, but it is followed by a few more characters, then the first character that is the period will be auto-deleted. If you intentionally want a period at the start of the mail. Then in all such cases, a double period needs to be appended at the start.

R: 220 smtp.pepipost.com Simple Mail Transfer Service Ready
S: HELO mail.example.com
R: 250 BBN-UNIX.ARPA

S: MAIL FROM:<admin@mail.example.com>
R: 250 OK

S: RCPT TO:<mike@mail.example.com>>
R: 250 OK

S: DATA
R: 123 Start of the email; end with <CRLF>.<CRLF>
S: Blah blah blah…
S: Blah blah blah...
S: ...etc. etc. etc.
S: .
R: 250 OK
S: QUIT
R: 221 smtp.pepipost.com Service closing transmission channel

RSET

This is an SMTP connection RESET command, which you can use to abort the current mail transaction. Once the receiving SMTP server, receives this command it clears all buffers and state tables, it will also discard the stored sender, recipient and mail data and sends back an OK command to the sender.

NOOP

This is like a heartbeat check command. Once the sender SMTP sends this command to the receiving SMTP server, the job of receiving SMTP is to just reply with OK. NOOP command does not affect any of the previously entered commands neither the buffer or state tables.

QUIT

Once the sender SMTP server sends this command, this means the receiver SMTP has to reply with an OK message and close the SMTP transmission pipe.

There is a strict restriction on the order in which any of the SMTP commands can be executed between the sender and the receiver. For example, the first command in an SMTP transaction should be HELO. NOOP, HELP, EXPN and VRF commands can be used any time during the transaction session.

RSET can be used anytime during the MAIL, SEND, SOML or SAML commands begin a mail transaction.

Chapter 5: SMTP Ports

Special computer ports are assigned to perform the connection between the sender the receiver SMTP.

List of SMTP ports which are been globally used:

Port 25

This is the first and the official port designated for SMTP in 1982. Since then port 25 is been used as the default port to transmit emails from one computer to another across the internet.

Port 587

In 1998, RFC 2476 was submitted to introduce port 587 as a new port for transmitting mail over SMTP. Port 587 was assigned as a new message submission to make sure that the new policy and security requirements don’t interfere with the traditional mail transmission already happening over port 25.

This port, coupled with TLS encryption, will ensure that email is submitted securely and following the guidelines set out by the IETF.

Port 465

Port 465 was never been published as an official SMTP port for mail transmission by IETF, but was registered as an SMTP port by IANA (Internet Assigned Numbers Authority). This is the reason why this port is generally not accepted by every email service providers across the globe.

There are definitely huge merits of using this port over other SMTP ports. One of the biggest merits is, it is more secure as it uses SSL (Secure Sockets Layer) for encrypting mail communication over the internet.

Port 2525

This port was primarily introduced by Google. Gmail and Google Cloud exposed its SMTP for the outside world over port 2525. The other SMTP ports 25 and 587 are default blocked on Google Cloud. This was an intentional move, to protect the infrastructure from regular spamming happening over the default SMTP ports 25 and 587.

Having so many SMTP ports for mail transmission definitely started creating confusions among the starters. In the following chapter, you will learn how to select the right SMTP port for your use case.

Choosing the right SMTP Port

There are two important parameters, which can help you decide the right SMTP port:

  • Data Security
  • Connectivity

If data security is your top priority then port 587 (with SSL enabled) or 465 is the best option. As mentioned in the previous chapter, port 465 is not the official port and hence not accepted by every email service provider. So, you need to check with each provider to whom you want to connect over SMTP to deliver email.

As an anti-spam measure, most ISPs have now started blocking SMTP outgoing connection over port 25.

The other factor to choose the right SMTP port is “Connectivity”. When is the last time you experienced “connect to address A.B.C.D: Connection refused. Unable to connect to remote host”?

This is the most common problem when you’re using a shared hosting server. Almost all the hosting service provider like Godaddy, Rackspace blocks outgoing SMTP connection over port 25. There are many cases where the hosting provider also blocks port 587 and 465 too. This is intentionally done to protect the infrastructure from spammers.

Port 25 been the default port, is widely used among the spammer community.

Submission on mail to its destination is only to be done on port 25 or 587. As these are the only official globally acceptable port. But, in case you’re using an email gateway like an email service provider (ESP) and facing issues while connecting with them over SMTP port, then you can try connecting over port 2525 to overcome this connectivity problem.

Chapter 6: What is Email Encryption & its significance?

In the original email protocol (as described in RFC 821), the communication between two SMTP servers was proposed to be plain text. But, over a period of time, this increased the security risk with this channel. Over the year, many mechanisms were proposed to make it end-to-end encrypted or at least at the transport layer.

Like any other data, the email also requires encryption. The chances of having a vulnerability or stealing information are even higher when it comes to email. This is because, emails are prone to the disclosure of personal information like Name, Bank Balance, confidential conversations and a lot of other important documents.

All major mailbox providers like Gmail or Outlook by default encrypt the emails sent from their platform. But this is not true when it comes to some of the other providers. Most of the emails are currently transmitted in clear text without any encryption.

Email encryptions mostly rely on public key-based cryptography, in which the recipients can publish a public key that senders can use to encrypt mail messages to them. The secret private key will remain with the recipient which they can use later to decrypt the received encrypted mail message. The same key can be used by them to digitally encrypt/sign messages they send from their mailbox.

Thanks to our global group of innovators who helped to solve this problem of encryption both at the transport level and at the end-to-end level too. Out of these two, implementing transport-level encryption is fairly easy as compared to end-to-end encryption.

Transport-level encryption

STARTTLS is one of the most commonly used email encryption extension. STARTTLS is actually nothing but a TLS (SSL) layer over the plaintext mail communication.

In this model, both the sender and the receiver server should have the support for TLS encryption. Anyone between these servers can’t really be able to sniff the mail content.

In this encryption model, the encryption actually takes place between the two SMTP relays and not really between the sender and the receiver. One of the biggest pros of this type of encryption implementation is that the user doesn’t have to worry about making any changes to send emails with encryption. Encryption will happen automatically at the transport layer and in fact, the receiving server doesn’t have to be dependent on the user to decrypt the message.
As per Google, more than 90% of the GMail’s incoming and outgoing email is encrypted using STARTTLS.

While STARTTLS looks lucrative, but it has its cons too. In case, someone breaches the security of the email system, then they can easily read and edit the emails.

In order to avoid this case, end-end encryption is recommended.

End-to-End Encryption

This is the most secure encryption for emails. In the case of end-to-end encryption, the data is encrypted at the source and is not readable by anyone on the network neither by your mail service provider Gmail, Yahoo.

Here are a few protocols which help to enable end-to-end encryption:

  • Pretty Good Privacy (PGP)
  • GNU Privacy Guard (GPG)
  • Bitmessage
  • S/MIME

While doing end-to-end encryption and decryption seems to be more secure, but when it comes to the usability it fails. It is difficult to use this because the user has to set up public and private key and make the public key available for everyone.

While Gmail has recently become stricter for TLS encryption by implementing a red lock on each email which is received to your Gmail mailbox. In case the email is not TLS encrypted, the lock will be in open state with a message.

This encourages the sender to send emails with TLS encryption. While all those good, but it’s been three years and Gmail still doesn’t has any updates on when it can have end-to-end encryption for all of your incoming and outgoing emails.

Implementing end-to-end encryption requires effort from both sender and receiver, so both the parties have to mutually understand its importance and implement.

What is SMTP Relay?

SMTP Relay Service is a gateway for sending emails, where the service provider takes care of almost everything leaving you with an ample amount of time to focus on your core product and business.

Before we begin, let’s read this detailed blog on Emails, protocols and other important terms.

A brief discussion on Email SMTP.

SMTP stands for Simple Mail Transfer Protocol.

SMTP is definitely one of the oldest and reliable protocols for sending emails. And, almost every programming language has a default function to support SMTP based MIME assembling and email sending. But, it has its PROS and CONS. The biggest advantage of SMTP is that it’s Universal which means you can switch from one platform to another platform easily just by changing the SMTP settings. No real coding or software changes. Now, let’s try to understand the challenges while dealing with Email SMTP for sending emails. Check out the flow of emails when you send it via the SMTP server through an email service provider, in the below diagram.

email-smtp-relay

In-house SMTP Vs using a third party SMTP relay service

It’s definitely not a big task to set up an in-house Email SMTP relay server and write code to assemble the MIME header and send emails, but the major challenge comes;

  • when you want to scale- Here, the scale means sending more emails in the same time frame. On one hand, Email SMTP is an open protocol using which you can send an unlimited amount of emails but each mailbox providers like Gmail, Yahoo, Outlook who today dominates the entire mailbox industry has their own set of rules to accept emails over SMTP. So, sending more emails means hitting more such rules and thresholds.
  • Managing and blocking the spam complaints/unsubscribes
    The more the number of emails you will be sending, the more feedback loop integrations you have to do with the mailbox providers.
  • Analyzing the performance of your emails
    You should not be just sending emails, but you should be actually analyzing the performance of your emails, to know what is working and what’s not. The best way to analyze these is to track the email activities like Bounces, Opens, Clicks, Unsubscribes and Abuse complaints. Read on setting up bounce notifications.
  • Ensuring your emails are secure and comply with the latest standards, best practices, and trends in the email industry
    If you have not yet heard any of the terms like DKIM, SPF, DMARC, r-DNS, TLS then probably it’s time to stop and get these implemented.
    Also, TLS offers a great deal of email security.
  • Managing the reputation of your sender domain and IP address
    There are different global bodies who keep a track of your email sending practices and assigns a score to your domain and IP address. It’s important to analyze these scores and to keep yourself updated about it.

Most of the ISPs and mailbox providers like Gmail, Yahoo, Hotmail, etc, use these scores to decide whether to deliver an email or not.

Read this exclusive guide on Email Sender Reputation.

While there are open source libraries and tools which you can just install to get your Email SMTP server ready to send emails, but you will definitely agree that it’s not easy to manage all of the above. And, not to mention the cost of procuring servers and operations can’t be discounted.

This is exactly where you can decide to choose a good Third Party SMTP Relay Service, where you can just connect over SMTP ports like 25, 465 or 587 for your email delivery and rest assured about all of the above points.

Also, read this amazing article on the History of Electronic Mail by Tom Van Vleck.

When is SMTP Relay used?

SMTP Relay is used automatically. It does not require the involvement of the sender or the receiver. It is not used when the emails are to be sent within the same server. In cases where multiple servers are involved, SMTP Relay automatically comes into operation to speed up the process of sending emails beyond servers. Read our blog to dive deeper.

Also, WATCH this detailed video on sending emails via Email SMTP.

Pro Tip: You can also reduce SPAM with SMTP! Know more.

Tracking Emails sent via SMTP

When you send your emails with the help of an email service provider there is another layer of processes running through your email delivery process. It’s interesting that before Pepipost’s Email SMTP server sends your emails, our system tags it with link trackers in the body of your email. These pixels allow you to track and update you regarding emails open and clicked.

SMTP Integration

SMTP is the fastest way to get started with sending emails. Integrating Pepipost with your application is quite easy as it requires only the modifying of SMTP configuration.

You need to:

  • Change the SMTP username and the password to your Pepipost credentials.
  • Set the server hostname to smtp.pepipost.com.
  • You can connect via unencrypt
  • ed or TLS on ports 25, 2525, and 587.
    In SMTP, one mail is sent at a time. All you have to do is connect your mailing server, or servers (if you have multiple) to Pepipost’s server.

Other helpful resources:

Confused about what to choose between SMTP and Email API?

Pepi thinking

Start typing and press Enter to search

Thank you for your details!

Fill out your information below, and we will send you a PepiAlert, that will describe your domain’s email deliverability situation. Please note that your email address must match the domain, or the domain must be owned by the company matching the email address. We have the right to refuse the request, if we can’t verify the information.

*All fields are required



Pin It on Pinterest