Sendgrid-csharp: IP Warmup functionality integrated into mail helper

[mbernier]

This issue has the potential to save other developers quite a bit of time and frustration when buying IP addresses because it will help them warm up the IPs appropriately. Warming up is a process of slowly ramping up sending on an IP address to allow Email Recipient servers (e.g. gmail, hotmail, etc) to see that this IP is legit.

Goal

• Add the ability to automate warm up IPs that allows a customer to take a conservative or aggressive approach to how they warm up.
• This should work for API purchased IPs or for IPs that are not warmed up yet.
• It should pay attention to “warm up days” not calendar days
• It should be able to calculate the IP’s warmup schedule without the need for a database or excessive API calls.
• The functionality should use the names of the IP pools retrieved from the API rather than a database to track the IPs and where they are in the process (see below for more info).
• This functionality should be able to be used in the nodejs mail helper, with a function call to "turn it on" during sending, but the code for this can be physically located in a helper called "Warmup".

Definitions:

• poolName - This is the name of the pool, defined in the following way: warmup_warmupDay_lastSendDate. e.g. warmup_3_20170101. This naming convention allows the pool name to dictate the calculation of how many emails to send to this pool, without the need to track this information in a database. Day 1 warmup pools will be named warmup_1, because there is no lastSendDate.
• warmupDay - If a day 3 pool is currently being used to send emails on Thursday, but it hasn’t sent emails since Monday, the warmup function will re-send using the day 2 limits rather than using the day 3 limits.
• lastSendDate - The last calendar date that this pool was used on.
• currentDailyVolume - The top end volume for a single day’s worth of sending
• numIpsInThisPool - When retrieving pools, this is the number of IPs in this pool, as we will want to send the same warmup volume to each IP in the pool. As you send using a pool, SendGrid uses a round-robin to make sure the IPs are used evenly.
• totalDaysToWarmup - This is the number of warmupDays (not calendar days) you would like to have your IPs spend warming. Default: 15
• growthRate - This is the rate at which the warmupDay’s totalEmailsToSend is calculated, based on the other values that control warmup procedure
• totalEmailsToSend - The number of emails to send to this pool on this warmupDay
• resetDays - # of days allowed between today() and lastSendDate, default 1

Workflow

Purchased IP address

• Customer purchases the IP over the API
• The code, upon acknowledgement of the purchase adds the IP to a pool called warmup_1 [warmup_warmupDay] - note, there is no lastSendDate, because this pool has not been used to send to yet.
• If the pool does not exist, please add it.
• Go to Warmup Procedure, below

UI purchased IP Address

• Customer purchases IP address via the UI
• Customer manually adds the IP address to “warmup_1”
• If the pool does not exist, please add it.
• Go to Warmup Procedure, below

Warmup Procedure

• In order to initiate the IP warmup process, customer adds the IP to the warmup_1 pool
• Upon sending, pull all available pools, sorting them in descending order by warmupDay and lastSendDate
  
  
  
  • If the lastSendDate is not the calendar “Yesterday”, then the previous warmupDay’s sending volume will be repeated to avoid over-sending (this is days of warmup, not calendar days, which matters to ISPs) … e.g the pool is named warmup_5_20170101 and today’s date is 20170103 (2 days later), then based on the default resetDays value of 1 the day4 warmup will be performed and if completed, the date will be updated but not the warmupDay.
  
  
  • In the example above, if the customer configures resetDays to be 2 or greater then the warmup would continue as if the days were contiguous without repeating the previous warmupDay’s volume
  
  
• Based on the “day” that is in the pool name, add the “user_warmup_pools(true)” function to your mailsend helper to allow the code to calculate how many emails should be sent using any available warmup pools.
  
  
  
  • The available warmup pools should be pulled daily and updated as they are no longer available for today’s sending. This will prevent the server from needing to request pools over and over.
  
  
  • If no warmup pools are found, then [NEED LOGIC HERE]
  
  
  • If you have the environment variable set for your current sending volume, the function will calculate how many emails to send to each IP
  
  
  • If you do not have this, please pass that daily max volume to this function
  
  

• Sending emails from a pool

  • As emails are sent, the code will calculate and manage the correct amount of emails to send to the pool (# of IPs * warmup day’s sending limit per IP)
  • As each pool reaches it’s limit for the day, the warmup function will update the pool name and roll over to the next available pool based on the day and the date.
  • Pool name updates the warmupDay and the lastSendDate
    

    Calculation: Today’s Volume for this pool

• This calculation will look at all of the items that control the number of emails sent through each IP and set the number of emails to send through this pool. This will allow the pools to be rotated, as well as the IPs.

totalEmailsToSend = 50 * numIpsInThisPool

if warmupDay > 1
    growthRate = (currentDailyVolume / (totalEmailsToSend))^(1/totalDaysToWarmup)
    totalSend = (50 * growthRate) ^ (warmupDay - 1)

Examples:

Happy Path

Calendar Day 1 / WarmupDay 0 - 20170101

• 3 IPs are purchased and added to warmup_1
• Warmup is configured to max out at 50,000 emails per IP over 15 days

Calendar Day 2 / WarmupDay 1 - 20170102

• All pools are requested and warmup_1 is returned
• lastSendDate is not considered because warmup_1 is returned
• Upon sending, each IP sends 50 emails
• Once 150 emails are sent, the pool name is no longer added to mailsend and regular sending is resumed
• Pool name is updated to warmup_2_20170102

Calendar Day 3 / WarmupDay 2 - 20170103

• All pools are requested and warmup_2_20170102 is returned
• The lastSendDate is = 1 day old, follow warmupDay 2 rules
• Upon sending, each IP sends 100 emails
• Once 300 emails are sent, the pool name is no longer added to mailsend and regular sending is resumed
• Pool name is updated to warmup_3_20170103

Calendar Days 4-15 / warmupDays 3-14

• Repeat process, following growth curve
• The lastSendDate is = 1 day old, follow warmupDay ‘n’ rules

Calendar Day 16 / warmupDay 15 - 20170116

• All pools are requested and warmup_15_20170103 is returned
• The lastSendDate is = 1 day old, follow warmupDay 15 rules
• Upon sending, each IP sends 50,000 emails
• Once 150,000 emails are sent, the pool name is no longer added to mailsend and regular sending is resumed
• Pool warmup_15_20170103 is dissolved, as it is no longer needed.
• Customer builds rules to handle IP assignment accordingly.

Date Skipping Path

Calendar Day 1 / warmupDay 0 - 20170205

• 3 IPs are purchased and added to warmup_1
• Warmup is configured to max out at 50,000 emails per IP over 15 days

Calendar Day 2 / warmupDay 1 - 20170206

• All pools are requested and warmup_1 is returned
• lastSendDate is not considered because warmup_1 is returned
• Upon sending, each IP sends 50 emails
• Once 150 emails are sent, the pool name is no longer added to mailsend and regular sending is resumed
• Pool name is updated to warmup_2_20170106

Calendar Day 3 / warmupDay N/A - 20170207

• No Sending - Nothing happens, no updates are made, nothing changes

Calendary Day 4 / warmupDay 1 - 20170208

• All pools are requested and warmup_2_20170106 is returned
• The lastSendDate is > 1 days old, revert to warmup_1 rules (pool’s warmupDay minus 1 warmupDay)
• Upon sending, each IP sends 50 emails
• Once 150 emails are sent, the pool name is no longer added to mailsend and regular sending is resumed
• Pool name is updated to warmup_2_20170208

Calendar Day 5 / warmupDay 2 - 20170209

• All pools are requested and warmup_2_20170108 is returned
• The lastSendDate is = 1 day old, follow warmupDay 2 rules
• Upon sending, each IP sends 100 emails
• Once 300 emails are sent, the pool name is no longer added to mailsend and regular sending is resumed
• Pool name is updated to warmup_3_20170209

Day 15

• All pools are requested and warmup_1 is returned
• Upon sending, each IP sends 50,000 emails
• Once 150,000 emails are sent, the pool name is no longer added to mailsend and regular sending is resumed
• Pool is dissolved, as it is no longer needed.
• Customer builds rules to handle IP assignment accordingly.

Acceptance Criteria

• Tests should be generated to test the two paths taken above
• The calculations for number of emails should have tests
• The calculations for which day is "today" and which is "the day to be run" should have tests

[Jericho]

@mbernier ,

I was under the impression that SendGrid could take care of automatically warming up IP addresses based on the information I read here, here and here:

SendGrid can automatically warmup dedicated IP addresses by limiting the amount of mail that can be sent through them per hour. This section allows you to use the API to create IP pools, assign IP addresses to them, and enable IP warmup pools and the IPs within pools.

Am I wrong about this?

[mbernier]

The warmup that is available over the API provides no control to the user. The IPs go into warmup and they come out of warmup when they're done. This process also assumes a constant rate of sending each day.

This process would allow the user to customize the rate at which they warm up and it accounts for the fact that some users send in an inconsistent way (monday, wed, fri or "on the 15th"). Our compliance team has seen some issues for customers who send inconsistently and helped me write up this spec.

This process will allow users who send in/consistently or who want to raise/lower their daily rate at will.

[Jericho]

@mbernier what are your thoughts on keeping track of the number of emails sent throughout a given day before totalEmailsToSend is reached?

What you have proposed so far I think will work fine for scenarios where the number of emails in a single Mail.Send exceeds the number of warmup emails for a given day. But what if a customer sends only a few emails at a time? Let's use examples to illustrate what I'm talking about.

First example, a rather simple scenario:

• Let's say the max number of emails for a given day is 100
• and let's say the customer is attempting to send 150 emails in a single Mail.Send call.

In this scenario, our logic will automatically send 100 emails using the IP address being warmed up and the remaining 50 emails will be sent using the default IP.

Second example:

• Again, let's say the max number of emails for a given day is 100
• but let's say the customer is sending password reminders, one at a time with separate Mail.Send calls.

In this scenario, we want our logic to use the IP address being warmed up until the 100th password reminder is sent and to use the default IP for password reminders sent subsequently.

Since you want to avoid using a repository (such as a database for example) to track where we are in the process, how do you propose we keep track of the number of emails sent in a given day?

Would using the name of the pool to track the number of emails sent in a given day be reasonable? This would match your proposal to use the IP pool name to track the date when the warmup pool was last used but it would potentially involve a lot of updates in a given day.

[Jericho]

@mbernier An email with 4 TOs, 3 CCs and 2 BCCs counts as 9 emails right as far as counting the number of emails sent on an IP being warmed up is concerned, right?

[mbernier]

@Jericho yep, any email address that receives an email actually gets a single email request to the ISP.

In your example, the SMTP server would actually create 9 different requests, setting the primary recipient on each one and then sending it.

Great point, I didn't think about this when writing the spec...

[Jericho]

@mbernier I'm about to publish a new version of the StrongGrid library with a WarmupEngine which meets all your goals but I have departed a little bit from your design by not using the pool name to store data such as the warmupDay and the number of emails sent. I'm finishing up updating the readme and will publish shortly.

[Jericho]

StrongGrid version 0.34.0 was just released to nuget. The version includes a new WarmupEngine which achieves the following goals:

• Can be initialized with IP addresses that were added to account using SendGrid's UI or new IP addresses can be automatically added to account when initializing
• Custom warmup schedule can be specified or a recommended schedule can be tailored according to your estimated daily volume. This recommended schedule is based on SendGrid's own recommended schedule available here
• The engine keeps track of emails sent each day
• When the daily volume limit is attained, the engine sends subsequent emails using the default IP pool
• The email that causes the daily volume limit to be attained can potentially be split into two messages: one sent using the ip pool and one using the default pool (in order to ensure you don't exceed the daily volume limit)
• If too many days have elapsed since the last time you sent emails, we repeat the last day's volume schedule. This ensure that we don't go through your warmup schedule too quickly
• Your email frequency (also known as "reset days") defaults to "daily" but can be configured if you don't send emails every single day. This is actually quite important: if you send emails every other day, you can set this value to "2"; or you can set it to "7" if you send emails weekly, etc.
• When the process completes, the temporarily created ip pool is deleted and the warmed up addresses are returned to your default pool

More documentation available in the readme

[mbernier]

@Jericho ... this is amazing! I am sharing with our deliverability team right now, they are going to be super excited.