Sending Emails from Olympus

Olympus supports a sophisticated email sending system geared towards sending large email batches. It does so by processing a lot of data on EPS servers and then passing on the actual emails to SendGrid (an email sending cloud-service, owned by Twilio).

There are a number of different ways to trigger sending emails in Olympus, such as using the “Email Batches” feature from the menu, triggering emails from within the issue edit form, or triggering emails from within the event edit form. The specific interfaces presented are slightly different (to support each scenario's exact needs), but all these scenarios really use exactly the same infrastructure (configured in slightly different ways).

Email Batches

Email operations within Olympus trigger the creation of batches. This happens in slightly different ways depending on the target audience, but it follows a general pattern:

  1. Retrieve a list for the appropriate “target audience”, by querying the appropriate people from our database, such as getting a list of active subscribers or a list of email newsletter subscribers.
  2. Ensure that the email addresses used by those people are considered “valid”. (Typically, email addresses get flagged as invalid if SendGrid reports them to us as invalid adter we attempt to send a batch).
  3. The system then checks whether the recipient should be excluded from the batch for some reason (such as already having received a new-issue notification or because they had already received an email of the same batch earlier).
  4. The system checks whether the recipient is on our internal “do not send” list, which would eliminate them from the batch.

Once this process completes, the system has a list of all the people that are included in this batch. (Note that this process can take a while… sometimes it is minutes, sometimes it is hours) Once the generation of the batch is complete, the system starts processing all those one-by-one by passing the actual email on to SendGrid for processing/sending.

All email generation interfaces feature the ability to estimate the number of recipients. This happens slightly different depending on the scenario. All the estimation algorithm have in common that they do not go through the complete process described above, due to the time it would take to go through all the steps. The estimation feature aims to procide a quick estimate. However, since the full process usually eliminates some of the recipients from the list, the actual number is almost always slightly lower than the estimated number.

Olympus features an “Email Batches” interface that shows the list of recent outbound email batches and information about how those batches have been processed, or - for batches that are still in progress - a percentage of completion. Note that email batches can be triggered from a number of places, and usually, those interfaces have individual lists of batches. However, the “Email Batches” interface always shows ALL batches, regardless of where they were originally generated from.

This interface shows a number of interesting columns:

Note that email batches are also grouped by month. This makes it easy to see what emails where sent in what month. There also is a sum of all emails that were passed on to the email processor (SendGrid) for processing. It thus provides a good idea about how close we are to going over our SendGrid limit.

Note that this is not the total number of emails we are processing through SendGrid. It is just the number of emails processed through the email batches system (as described in this topic). However, the system also sends other emails (such as password reminders, subscription confirmations, event signup confirmations,...) that count towards our SendGrid limit. However, the email batches make up the vast majority of emails we send, and therefore, this number provides a good yard stick.

Stalled Batches

Sometimes, large email batches can “stall”. This is likely due to an overburdening of the server infrastructure, and Azure terminating processing. Such batches can be resumed by clicking the Resume Interrupted Batches button in the email pane. When the user clicks this button, the server checks for stalled batches (these are batches that have not shown any activity for 10 minutes or more) and resumes them. Note that this can take a moment. Click Refresh a few seconds after clickig this button, in case no immediate change is apparent.

Copying the Email Body

Using the Copy Email Body button, it is possible to copy the email body text of the selected email batch. This is the raw version of the email, including the placeholders for blocks and fields. The text is copied to the clipboard and can then be pasted into new emails.

Email Newsletters

Email newsletters are the most generic kind of email the system can send. New email newsletter batches are triggered from the Email Batches interface (see above).

To send an email batch, one must first pick the target audience, which can be a combination of any of the following:

Note that our system currently has a somewhat odd combination of where opt-ins might be stored. We are considering a consolidation/simplification of that.

Picking these different options (or any combination thereof) results in a list of email recipients. The system knows how to combine the different options and avoid duplication of recipients. The Estimate Target Audience button in the toolbar allows estimating how many people that will be.

Note that the estimate almost always results in a larger number than will actually be sent. The estimate provides a quick estimate, but can't run a complete duplication check. It also can't include complete block-list checks. Also the system does NOT look at how many people were already in the same batch/campaign (see below). And it completely ignores the “max count” checkbox. Doing a complete count simply takes way too long due to the complexity involved. The estimate is meant to be an estimate that can be provided in a few seconds (or less than a minute at least), which can be somewhat slow even then. An accurate count would take minutes or sometimes hours.

The system also allows limiting the maximum number of recipients. This is useful if you want to send an email to a limited group (perhaps to test a campaign, or some similar reason). It is also useful if you want to split badges into multiple individual email batches (see below for batches/campaigns).

Then, there are some common email fields:

Finally, there is the actual email text, which is entered in markdown. (Note that this form remembers your most recent email text and opens up with the last text sent as the default text).

Email text can contain all kinds of markdown or HTML. However, it is recommended that you stick to relatively simple markdown, since it is very difficult to hand-craft a complex email that looks good on the wide variety of email clients and form factors (widnows, web, mobile, watches,...). Recommended markdown options can be seen in the toolbar (used by highlighting some text and hitting the toolbar button), and include:

In addition, it is possible to embedd special fields in the text. These fields can be accessed through the Insert Field and Blocks toolbar button:

A different type of “field” is a “marketing block”. These are standard text elements that can be set up globally and then be placed into emails. For more detail on marketing blocks, see below.

Note that when an email is just about ready to be sent, you can hit the Send Test Email button to send yourself a test email and see what it will look like in your email client. The system then asks for a recipient email for the test. (It remembers that setting on your computer, so it doesn't have to be re-entered every time).

Note that the preview of the email is somehwat imperfect. It is also recommended that you use the light theme in Olympus to preview emails, as that shows colors that are more likely to be close to what people will see in their clients. However, there are so many different email clients, with vastly different capabilities and formats. Therefore, one should always use the test-feature to see what the email will really look like.

New Issue Notification Emails

These are the emails sent to subscribers when a new issue of the magazine is available. New issue notifications are sent through the Issue Edit interface:

This interface shows the outgoing batches for the current issues. (These batches also show up in the Email Batches interface - see above. However, this is a list filtered for the current issue only. The interface also shows totals in the column headers.)

The interface for creating new issue emails is generally similar to the one for email newsletters (see above):

The main difference is that this is geared specifically towards announcing a new issue. The differences are:

Event Promotion Emails

Event promotion emails are largely similar to email newsletters (see above). They are triggered from the event edit form:

The main difference is in the additional placeholders that can be embedded in the email (all of which are supported in the preview):

Event Attendee Emails

A special type of event emails are emails sent to people who are already signed up for an event. This is typically done to inform attendees of online URLs, or to send them follow-up information, such as downloads.

These emails are similar to event promitional emails:

However, they are triggered from the list of signups. In addition to the standard event fields supported (see above), this type of email batch supports these additional fields:

Marketing Blocks

Marketing blocks are text elements that can be set up globally and can then be embedded into emails. This is typically used to generically set up marketing messages and then routinely place into outgoing emails. The big advantage is that marketing blocks can be maintained and updated independently of the main email body.

Marketing blocks are simple in nature. They have the following fields:

Marketing blocks can then be inserted into all emails by means of placeholders. All the email UIs provide a drop-down menu that shows all active marketing blocks to be inserted into an email:

Note the syntax for the block placeholder: ##MARKETINGBLOCK:Standard1##. The Standard1 part of this placeholder is the name/ID of the block.

Note that the block references by the placeholder must exist and be active. Otherwise, the preview will show an error message in red. If an actual email was sent out with a non-existing or inactive block, the error message is inserted in the email but with style="display: none;". This way, one can look at the source of the email to see what went wrong, but the customer will typically not see the error.

Previous
Next