Setting up an automated Invoice Forwarder from Gmail to Datev

Datev is a German accounting software provider that offers a range of services for small and medium-sized businesses. It is among the most popular accounting software solutions in Germany and we, as many others, use it to manage our accounting tasks and to share data with our tax advisor.

However, one of our personal pain points with Datev is the fact that we have to manually forward / upload each invoice we receive for the company into the Datev system by hand.

At superluminar, we’re using Google Workspace to manage our employees emails, calendars, documents and more. We’re happy with this solution and see no reason to change to another provider. But while you can create filters in Gmail that act as forwarding rules, this doesn’t work for Datev specifically.

This is due to two reasons.

• Reason 1: Datev allows for invoices to be send to a dedicated email address. We’ll call this the “Datev Incoming Invoice Address” or DIIA for short, as this email address differs from the Outgoing Invoice Address. In order for the DIIA to receive emails and upload their attachments directly into the Datev system, the sender’s email address needs to be whitelisted in a simple two-step (“click link in confirmation email”) process. Any emails from non-whitelisted senders will be rejected.

It's also worthy to note that the DIIA is not a fully functional Email inbox: The users are not able to login into this email address, check the received emails or interact with them in any way. The DIIA is only a receiving endpoint for emails from whitelisted senders and has no other functionality.

• Reason 2: When creating a filter in Gmail, i.e. for the use case of forwarding all of our invoice emails into Datev, Gmail will by default “redirect” the emails to their destination. This default behaviour of Gmail cannot be changed or modified.

The difference between redirecting an email and forwarding it, is that forwarding changes the original sender’s email address to the address doing the forwarding. Redirecting keeps the original sender’s email address.

In our case, after whitelisting our Gmail address in Datev and creating the necessary filter, the redirected emails are rejected since their original sender’s are not whitelisted in Datev. Which is why a simple Gmail forwarding rule didn’t work for this specific case.

Logically, we can understand why Datev does this. However, for our administration it’s annoying that we are unable to automate this simple process, since our colleagues can spend their time in more meaningful ways than to manually forward each and every one of our invoices.

If you want to jump to the solution that worked for us, click here. Otherwise, read on for the research and thought process that led us to the solution.

Initial Research and our AWS Stack Solution

As AWS Cloud Consultants, while researching the problem and its possible solutions, our first idea was to implement something in AWS cdk.

After reading the Datev docs, we found a solution using AWS SES that looked promising. Quickly setting up an AWS stack with the Simple Email Service (SES), that stores the incoming emails in an S3 bucket and handles some forwarding logic with a Lambda function and SES shouldn’t, in theory, pose too much of a problem.

Extending our research to already existing components led to finding a suitable github repo that already offered the necessary functionality out of the box in a quick to set up manner.

We went ahead and created a dedicated AWS Account in our Organizational Unit to keep things neat and separate, registered a domain and set up the necessary components in a cdk+typescript App. After the initial administrative setup, waiting for the domain registration and for SES to be moved out of sandbox mode, we deployed our stack and were ready to test the solution…

…only to realize that Datev still rejected the forwarded Emails from SES :(

What went wrong?

On our search to debug the issue, we found out that we were not the first ones to run into this problem. The more we dug into the issue, the more bleak the situation appeared.

To summarize the two main reasons why our AWS SES stack didn’t work:

Firstly, Amazons Simple Email Service arbitrarily changes its senders email address when relaying the emails:

When checking the smtp.mailfrom field in the email headers that were returned to us from the Mailer-Daemon with a rejection message, we can see that the sender address differs each time. This is due to AWS SES giving each email a random ID and constructs its senders address from this ID and the region the stack was deployed to.

Also, Datev does not allow us to wildcard an email address, collectively allowing all emails from a specific domain. And since we cannot anticipate which ID SES will chose for its temporary sender address, we cannot whitelist them.

Additionally, as this user pointed out, AWS SES also only acts as a relayer and does not modify the sender’s email address. Meaning that we were back to square one and still facing the same issue as initially with Gmail.

That’s when we realized that maybe we had looked into the wrong direction. Maybe there was a simpler solution that didn’t require AWS at all. - And as it turns out, there was!

Microsoft Outlook to the Rescue!

The working solution was a simple Microsoft Outlook account.

In the Datev docs (Section 2.4 - “Defining Automated Functions”), there is a detailed section highlighting how to create a forwarding rule in Microsoft Outlook. Forwarding emails from Outlook to the DIIA should work out of the box, since Outlook has an option to forward emails instead of relaying them.

Since we only need the Outlook functionality of Microsoft 365 and no other products, we did not need to register for one of their paid plans and were able to set up the forwarding rule in the free tier Outlook Account.

Next, we created the forwarding rule in Outlook according to the info from the Datev docs. Then we adjusted our previously set up Gmail filter to forward all incoming invoice emails to our new Outlook address.

Lastly, we whitelisted the Outlook address in Datev to allow sending invoices to the DIIA and voilá, we had a working solution!

Bottom Line

Overall, I am surprised by how easy the working solution was in comparison to what we had initially envisioned. I am also a little annoyed at both Gmail and Datev:

• at Gmail for not offering an option to forward emails instead of relaying them. Personally, I have used Gmail for many years without knowing about this specification. But in this specific instance it felt very limiting, as I would assume that a giant like Google would provide this feature easily.
• at Datev for not working with Gmail Addresses. While I understand why it does not work, in my eyes it cannot be the answer for a popular service like Datev to just exclude this many individuals and businesses that use both Google Workspace and Datev and who would love to have access to an automated solution like this.

I’d like to add that the current setup is entirely free to use for us. This could change if we fill up the free storage in Outlook over time.

Additionally, we now have to store, save and maintain a new set of credentials for the Outlook Account, which adds an additional moving part into the mix. Though to be fair, this seems like a rather small caveat in the bigger picture.

Ideally, this blog post helped shed light into the issue and how to resolve it :)

Let us know if you have any questions or found another solution for this problem!

Valerie is a Junior Cloud Consultant at superluminar. She initially started her career in the blockchain and web3 space and joined superluminar after finding her passion for AWS. You can find on LinkedIn or check out a few of her hackathon projects at Devpost.