With the recent exodus from Twitter due to Elon being a deranged sociopath, many folks have found themselves moving over to Mastodon. I won't go into Mastodon except to say I've moved over there as well (@[email protected]) and have really enjoyed myself. It's a super nice community and I have a lot of hope for the ActivityPub model.
However when I got on Mastodon I found a lot of abandoned bot accounts. These accounts, for folks who don't know, tend to do things like scrape RSS feeds and pump that information into Twitter so you can have everything in one pane of glass. Finding a derelict Ars Technica bot, I figured why not take this opportunity to make a bot of my own. While this would be very easy to do with SQLite, I wanted it to be an AWS Lambda so it wouldn't rely on some raspberry pi being functional (or me remembering that it was running on some instance and then accidentally terminating it because I love to delete servers).
Criteria for the project
- Pretty idiot-proof
- Runs entirely within the free tier of AWS
- Set and forget
Step 1 - DynamoDB
I've never used dynamoDB before, so I figured this could be a fun challenge. I'm still not entirely sure I used it correctly. To be honest I ran into more problems than I was expecting given its reputation as an idiot-proof database.
You can see the simple table structure I made here.
Some things to keep in mind. Because of how DynamoDB stores numbers, the type of the number is Decimal, not int or float. This can cause some strange errors when attempting to store and retrieve ID values. You can read the conversation about it here. I ended up storing the ID as a string which is probably not optimal performance but did make the error go away.
When using DynamoDB, it is vital to not use scan. Query is what I ended up using for all my requests, since then I get to make lookups on my secondary tables with the key. The difference in speed during load testing when I generated a lot of fake URLs was pretty dramatic, 100s of milliseconds vs 10s of seconds.
Now that I've spent some time playing around with DynamoDB, I do see the appeal. It is a surprisingly generous free tier. I've allocated 5 provisioned Read and Write units but honestly we need a tiny fraction of that.
Step 2 - Write the Lambda
You can see my Python lambda here.
NOTE: This is not production-grade python. This is hobby-level python. Were this a work project I would have changed some things about its design. Before you ping me about collision, I calculate that with that big a range of random IDs to pull from it would take ~6 thousands of years of work in order to have a 1% probability of at least one collision. So please, for the love of all things holy, don't ping me. Wanna use UUIDs? Go for it.
For those who haven't deployed to AWS Lambda before, it's pretty easy.
- Make sure you have Python 3.9 installed (since AWS doesn't support 3.10)
- Copy that snippet to a directory and call it
- Change the
rss_feed =to be whatever feed you want to make a bot of.
python3.9 -m venv venv
- Then you need to install the dependencies:
pip install --target ./package feedparser
pip install --target ./package Mastodon.py
pip install --target ./package python-dotenv
- You'll want to
packagedirectory and then run
zip -r ../my-deployment-package.zip .to bundle the dependencies together.
- Finally take the actual python file you want to run and copy it into the zip directory.
zip my-deployment-package.zip lambda_function.py
You can also use
AWS SAM to do this all but I find the ZIP file is pretty idiot-proof. Then you just upload it through the AWS web interface, but hold off on doing that. Now that we have the Python environment setup we can generate the credentials.
Step 3 - Mastodon Credentials
Now we're back in the python virtual environment we made before in the directory.
- Start the Python 3.9 REPL
from mastodon import Mastodon
Mastodon.create_app('your-app-name', scopes=['read', 'write'], api_base_url="https://c.im")(note I'm using c.im but you can use any server you normally use)
- Follow the steps outlined here.
- You'll get back three values by the end. CLIENT_ID, CLIENT_SECRET from when you registered the bot with the server and then finally an ACCESS_TOKEN after you make an account for the bot and pass the email/password.
6. Copy these values to a .env file in the same directory as the
lambda_function.py file from before.
zip my-deployment-package.zip .env to copy the secret into the zip directory.
You can also store them as environmental variables in the Lambda but I prefer to manage them like this. Make sure it's not committed in your git repo.
Step 4 - Deploy
- Make a new AWS Lambda function with whatever name and ensure it has the ability to access our DynamoDB table. You can get instructions on how to do that here._
- Upload the ZIP by just uploading it through the web interface. It's 2 MB total so should be fine.
- Set up an EventBridge cron job to trigger the lambda by following the instructions here.
- Watch as your Lambda triggers on a regular interval.
Step 5 - Cleanup
- Inside of the Mastodon bot account there are a few things you'll want to check. First you want to make sure that the following two options are selected under "Profile"
2. You'll probably want to add an alert for failures under Cloudwatch Alarms. AWS has docs on how to do that here.
Hopefully this is a fun way of adding a simple bot to Mastodon. I've had a lot of fun interacting with the Mastodon.py library. You can see the bot I ended up making here.
If you run into problems please let me know: https://c.im/@matdevdug