Email Tracking

Posted on 08 May 2018 by Marek 
Bar chart showing something very like quantised exponential decay.
Most emails are only opened once, but there is a very long tail of repeat interactions.

One of the features our CRM-as-a-service provides is the ability to track who has opened emails sent by that system. It is a very common feature in CRM and email marketing products, and our customers have told us how useful it is — especially when a customer of theirs disputes having ever received an invoice. It has been useful to marketing departments and content editors too, although mainly as aggregated statistics, to help understand audience segmentation.

Is email tracking (or link-click tracking) which can be correlated to an individual (therefore personally identifiable behaviour tracking) legal with GDPR? Is it even legal now? On May 11, 2017, Dr. Sonja Branskat of Germany's Federal Commissioner for Data Protection and Information Freedom cited the Working Party 29 Opinion 2/2006, and stated that:

[A user of email tracking] will have to get consent according to article 6, 7 and maybe 8, if children are concerned, of the GDPR.

This is an unambiguous statement. Additionally, we believe:

  • that having this data personally identifiable is — for the most part — a "nice to have" rather than of "legitimate interest" to our customers
  • that even if it is legal today (assuming all recipients have consented) the bar for consent is higher in GDPR
  • customers using this feature will therefore have to seek GDPR-level consent to track email opens and link clicks

Therefore we are taking the decision to completely anonymise the email-open and link-click tracking functionality in fulcrm, regardless of whether recipients have knowingly opted in to it or not. We believe this is the right decision to balance the privacy of our customers' customers, simplify our customers' GDPR compliance for email marketing, and still be able to produce useful data for our customers' reporting requirements.

In the next few days we will be pushing out a change to the production servers for fulcrm which will:

  • anonymise the existing data held in fulcrm
  • only generate email and link tracking URLs that have no personal unique identifiers
  • ensure any pre-GDPR email and link tracking URLs discard personal identifiers before storing statistics into the customer's fulcrm database
  • remove the UI elements in fulcrm which showed any personally identifiable link tracking data

We want to reassure customers that fulcrm will still track how many times a campaign has been opened (where possible) and how many times links within that email have been followed. But fulcrm will not track who interacted with the email.

Pros

  • customers using fulcrm will still be able to see relative popularity of newsletters
  • and will still be able to see which content in the newsletter had the most click-throughs
  • where customers using fulcrm already have a contractual relationship with their customers to provide information, there should be no need for a "GDPR re-opt-in" consent request

Cons

  • email engagement statistics will be even less accurate than before
  • audience figures will be have to apply statistical methods
  • customers will not be able to segment click-through statistics by e.g. recipient type

Addressing the Gap

Email engagement statistics always were inaccurate: many email clients do not show the user any content which could be used as a tracking "bug" and over time this fraction has been getting larger. We typically have advised that the "email opened" statistic is about 50% lower than it is reported — but that figure is getting harder to measure. I would argue that a well designed email campaign would have a "call to action" which is more measurable and useful to the customer than simply "did our customer open this email?"

As we will no longer be able to identify whether a user has clicked on a link in an email twice, or if they were two separate users clicking the link in their emails, we will need to perform a correction. To advise our customers about this correction, I have crunched some numbers.

I've used three of our largest email-sending customers, who have slightly different target audiences.

User Site 17
An organisation which sends marketing newsletters (across several thousand users) and transactional emails to their customers. Also has a paid membership group of around 10-20% of their total engaged users.
User Site 21
An organisation sending business support advice as newsletters, and a very small amount of transactional email (signup confirmations, etc) to their customers.
User Site 28
a membership organisation sending marketing newsletters (to their membership) and transactional emails to their customers. Over 90% of their engaged users and customers pay an annual membership fee. Customers print off transaction emails in the form of tickets, so may end up "viewing" an email more than once.

Sites 17 and 28 are similar, but 17 sends a lot more marketing information to a larger pool of people — not just members with whom it has a commercial relationship.

The table below shows the scaling factor by which the number of clicks should be divided to achieve the number of individuals in each category. That is to say, every 1880 email opens that User Site 17 sees corresponds to about 1000 individuals opening that campaign.

Mean number of opens/clicks per recipient per campaign
Dataset Email Track Link Click Overall
User Site 17 1.880 1.674 1.771
User Site 21 2.186 1.217 1.775
User Site 28 2.778 2.406 2.612
transactional 2.723 1.525 2.616
newsletter 2.061 1.679 1.870
Overall 2.065 1.679 1.873

As I expected, the repeat-engagement figures for Site 28 are higher than Site 17, mainly because those recipients print off tickets from the email which could cause a second page-load. And transactional emails have a higher repeat-engagement than newsletters, as they will mainly be for financial purposes. Organisations sending a high proportion of transactional emails (50% or more) will see figures closer to Site 28. Organisations whose email activity is sigificantly skewed towards marketing or information will see figures closer to Site 21.

Appendix

Should further statistical analysis be required on marketing data, I also extracted breakdowns for users who clicked a link (or opened an email), broken down by how many times this happened. This should help our customers in similar sectors to calculate error bounds as well as estimated numbers of individual users.

These statistics show, for example, that 64.85% of emails that are opened by recipients of User Site 17 were not tracked as having been opened a second time; a further 17.52% of emails were opened by a recipient twice; and so on till barely anybody opened the email more than 20 times.

Percentage of recipients of non-transactional emails
by number of opens/clicks
Opens/
Clicks
User Site 17
Marketing and Transactional
User Site 21
Mostly Informational Emails
User Site 28
Mostly Transactional Emails
Email Track Link Click Email Track Link Click Email Track Link Click
1 64.85 70.76 60.24 87.83 46.00 53.99
2 17.52 15.72 18.10 8.114 20.51 19.33
3 7.262 5.968 7.557 2.203 10.70 9.291
4 3.535 2.718 4.076 0.834 6.675 5.457
5 2.115 1.610 2.687 0.345 4.094 2.915
6 1.346 0.966 1.889 0.213 2.731 1.995
7 0.889 0.599 1.286 0.101 2.202 1.734
8 0.651 0.442 0.921 0.093 1.593 1.105
9 0.419 0.281 0.712 0.058 1.166 0.861
10 0.324 0.201 0.471 0.043 0.979 0.727
11 0.233 0.147 0.384 0.058 0.703 0.553
12 0.181 0.113 0.377 0.027 0.595 0.512
13 0.145 0.116 0.309 0.019 0.445 0.460
14 0.127 0.085 0.247 0.019 0.379 0.215
15 0.108 0.101 0.209 0.016 0.272 0.122
16 0.075 0.046 0.150 0.012 0.253 0.291
17 0.056 0.045 0.118 0.012 0.230 0.198
18 0.053 0.023 0.088 0.000 0.201 0.134
19 0.055 0.032 0.118 0.004 0.178 0.047
20 0.050 0.030 0.077 0.000 0.084 0.070