Spectral Web Services
Translating Ghost posts with DeepL
Make was annoying me. So I replaced it. Bonus: a story about accidentally post-bombing ActivityPub
I have a journalist client (who is on the edge of launch, so no link yet) whose community includes several non-English speaking populations. She wanted a way to provide content on her Ghost site to all these populations. Since she's pre-launch and not monetized yet, it needed to be inexpensive.
I'm a huge fan of human translations, and I hope that in the future she'll be able to offer them. Meanwhile, AI to the rescue.
My original plan was to use Make as the "glue" in between Ghost and DeepL (which has a machine translation API). Magnus over on the Ghost forum kindly shared this how-to and blueprint. It worked great. Mostly. But free Make only includes two scenarios, and my client has four languages. So then I got to play with Make, adding in an iterator to loop over the language list, and to tag posts with a tag corresponding to the language.
Make is super cool. You can set it to accept a webhook once, and you can watch the flow of data through the webhook. Super useful for debugging. Majorly cool.
However. (You knew there'd be a however, didn't you?) I spent a couple hours wrestling with Make, trying to get it to abort the flow if the post already had a #translated tag. This worked for a post with the #translated tag in the primary tag spot, but did not seem to work for multiple tags. I tried. I tried without the hash. I tried a variety of JSON parsing, to the point where I was using a fairly ridiculous number of operations. I never got it working.
Finally I threw my hands in the air and asked myself why I was doing this. I'm a perfectly capable catcher of webhooks. I do it all the time. Make at this point was not saving me any time, and the thing I was struggling with was two lines of JavaScript in a cloud function. And bonus, it took Make out of the stack and replaced it with Netlify, which will catch a whole lot more webhooks for free. (125,000/month for function invocations on Netlify vs 1,000 operations on Make, and the translation job was going to be probably 5-10 operations per webhook.)
I'm open-sourcing the code necessary to glue Ghost and DeepL together this way. Help yourself to the GitHub repo.
Ghost-DeepL glue (a Netlify function)
This package receives webhooks from Ghost, sends the post content to DeepL API for translation, then creates new posts in the target languages.
You can create a free DeepL account that provides 500,000 characters of translation per month. Because Netlify also has a generous free tier, this is potentially a free solution to translation for smaller/less prolific Ghost sites.
To use this function, fork this repo, make a Netlify account, and deploy it to Netlify. (A PR that provides detailed directions for this step or sets up a Deploy to Netlify button would be welcome.)
Use the example.env file to get environment variables set up in Netlify. (IMPORTANT: Never commit this file with your secrets. It's best to rename it to .env and make sure it is included in your .gitignore.) Here's how to get the values you need:
- Get your DeepL API key by signing up here: https://www.deepl.com/en/pro-api (Note that the API plans and other plans are not the same. You need an API plan.) Copy it into the.env file.
- Get your Ghost API key and url by going to /ghost > settings (gear icon) > integrations, clicking "custom", and adding a new integration. Copy these into the .env file.
- While you're on the integration page, click "Add webhook". Create a new webhook, named anything you like. The trigger is "Post published". The url is {whatever your Netlify site's url is, starting with https}/.netlify/functions/translator . Choose your own webhook secreit. Copy the webhook secret to the example.env file.
Go to your Netlify deploy, and import the .env (really copy-paste).
Deploy the site. (You have to do this any time you change environment variables.)
Also included is an example routes.yaml file that can be used to put each translation in its own part of Ghost.
Hints:
- The function looks for a tag called #translated, and aborts if it finds it. This stops newly published translations from triggering the translation function, which can otherwise cause a chain reaction. (Ask me how I know. Or don't.)
- Use the routes.yaml file (after editing it for your languages) to get basic collections in each language. Each translation gets a tag (like #ES) to denote its language.
Thanks & Support:
- If you'd like me to set this up for you, feel free to hire me for an hour: https://tidycal.com/catsarisky/60-minute-paid
- Tip Jar: https://www.spectralwebservices.com/tipjar/
- Sponsor on Github: https://github.com/sponsors/cathysarisky
Did you read all this way? Here's your reward (?): the story of how I accidentally blew through all my Make operations and bombed my ActivityPub followers...
Like this, but times 100...
A funny thing happened while I was working with Make
I thought I had everything worked out, so I was testing on my demo site. My demo site lives on Ghost Pro (because this makes a certain amount of warped sense), so I'd turned on ActivityPub to play with it when the beta first opened. It was still on.
I thought about that briefly when setting up my Make integration. I thought "well, I'm going to delete these tests, and I'm only going to publish a couple things, so not worth fussing with."
That's when I discovered that #translated wasn't correctly stopping the flow. So I published a post, with got translated to three languages, and those posts got published, which triggered three more webhooks, resulting in those three posts being translated to three more posts each, and 9 more webhooks, and then 27, and then... well, you get the idea. Fortunately, I hit a rate limit somewhere, which threw an error, which disabled the Make scenario, only a couple hundred posts in.
But so, back to the Ghost ActivityPub switch. All those posts went out to my followers on ActivityPub. Deleting them in Ghost apparently doesn't delete them in ActivityPub (at least, not using the bulk delete). So um... if you were following my demo account, I'm sorry. I don't think I can fix it at this point, either.
And yes, I know that Make has a 'run once' button, but it was all working perfectly that way until I switched from making drafts to making published posts, so... yeah.
Moral of story: 3x3x3x3x3.... is a really big number, and webhook responses that can webhooks are tricky! (Also: This wouldn't have happened in Netlify. Probably.)
Hey, before you go... If your finances allow you to keep this tea-drinking ghost and the freelancer behind her supplied with our hot beverage of choice, we'd both appreciate it!
