I recently started working more with AWS Lambda functions, some of them with external dependencies, and I quickly ran into an issue where dependencies that are built on MacOS sometimes don’t play nicely with Amazon Linux on Lambda.
The solution, back in the early days of Lambda, was that you would have to spin up an EC2 box, and then build and zip up your dependencies there. Luckily, things are much easier now because Amazon offers a Docker image for Amazon Linux that you can use to build your dependencies.
I found a blog post by Quilt that discusses automating the process of creating your Lambda deployment package, but their solution stops at building the zip file you need. I wanted to take it a bit further and get the entire deployment process down to just one command, so I made some changes and additions.
First up, you have your Dockerfile. This is basically identical to Quilt’s, except that I’ve added the awscli package to be installed.
FROM amazonlinux:2017.03
RUN yum -y install git \
python36 \
python36-pip \
zip \
&& yum clean all
RUN python3 -m pip install --upgrade pip \
&& python3 -m pip install boto3 awscli
Then you need your package.sh file, this is the script that’ll get run inside your Docker container once it’s built. Note the $LAMBDA_FUNC environment variable. We’ll be setting that when we run the container via the Makefile.
Before we tie it all together with the Makefile, I should clarify that I wanted this automation to be set up so that I could use it to deploy multiple different Lamdba functions to different places, so the directory structure is pretty important. In my setup, the directory structure looks like this:
You’ll notice that a function may or may not have dependencies. Even without dependencies, this automation can still make developing in VSCode and deploying to Lambda slightly easier and faster (than copying, pasting, and saving). The main trick to this is that my folder names exactly match the name of the function in Lambda. So hash-function will deploy to hash-function in Lambda, and so on.
Note that we’re automatically grabbing the necessary AWS access keys from our local machine and injecting them into the docker container, so that it can deploy the package to Lambda for us.
Then to deploy to lambda it’s as simple as writing…
$ make deploy-lambda p=preview-function
That’s about it! Doing this has saved me a lot of time that would have otherwise been spent running a couple different commands to deploy my Lambda functions. Now it’s fast, easy, and worry-free. Again, kudos to Quilt and their blog post for doing more than half the work for me 😉
When I finished Life Saver it seemed obvious to host the downloads on S3, like the rest of the site. But I wanted to be able to track how many times it had been downloaded, and S3 doens’t have this functionality built in. Coming from a traditional web background (Apache, cPanel, etc) I wasn’t sure where to start, but luckily I saw a StackOverflow answer that pointed me toward S3stat.
What’s really great about S3stat is that it has a nice macOS utility to help setup all the complicated AWS stuff for you. One S3stat up and running on your bucket, you get a nice looking dashboard showing you tons of info. As an example, here’s the current daily stats for my bucket. (Note that I really only host two files there.)
You can drill down into individual files to see stats for that specific file:
One feature I like a lot is that it also shows you referrers, so I can see how people are getting to my file just in case any blogs or websites directly link to it (which hasn’t happened yet). Between this, GitHub’s traffic stats, and Google Analytics, I have a pretty complete view of where my traffic is coming from and how effective my site is.
The S3stat service is normally $10/mo, and as an open source developer with no commercial products under my belt, I can’t really bring myself to shell out even that amount of money for it. Luckily, the owner of S3stat is a generous soul and is willing to extend my free trial into perpetuity (or until I am rolling in open source riches and can pay for it myself). Many thanks to S3stat for giving me a way to gain some insight into how frequently my files are downloaded!
Since I was a kid I’ve been pretty intrigued by artificial life simulations. It started most predictably with Tamagotchi, but when I got a PC in the mid-90’s I learned that there were people out there interested in making their own, more elaborate, versions of artificial life. One that’s still available today is Gene Pool, and I remember one that consisted of a 3D modelled woman who wandered around a sparsely populated landscape (though I can’t remember what it was called).
My favorite was called Alife Tank HAKONIWA, which is supposedly still available on some download sites if you’re running Windows. It gave you a side-view of an extendible tank that housed a lot of randomly generated creatures that swam around, ate, and mated. Something about this one really appealed to me, though the creatures were often too small to identify easily and whether there was any actual decision making going on was pretty inscrutable.
About two years ago, I decided to tackle my own simple version as a learning project to strengthen my programming skills. I’d watched a tutorial and learned that SpriteKit was extremely easy to work with, so it seemed like a no brainer to use it. I spent about a week getting a basic version working, showed it off a bit to co-workers, then forgot about it until I decided to flesh out my GitHub profile with more projects. I decided to play with it a bit at that point, but the code was such a horrible mess that I got discouraged and forgot about it yet again.
Since then, it’s been forked twice, and starred twice, on GitHub. It’s not a lot of attention, but I figured if people were going to play with it, I might as well clean up the code using the skills I have now.
Wait, so… what is Aeon Garden?
I suppose trying to describe what Aeon Garden is in text is kind of silly, it’s probably easier to get what it is by seeing it in action, so here’s a video.
You can see the resemblance to Gene Pool almost certainly. With that out of the way, here’s some of what I’ve done to try to clean up the code base and why. If you don’t care about any of that, just go check out the project on GitHub!
Refactoring Creature Generation
In my first draft, creature generation was extremely sloppy with a lot of repeated code. Basically, a creature could be generated either “from scratch” or “from parents”, and both of these init methods have one key difference between them: in one case, the creature is generated from scratch, and in another it’s generated using attributes belonging to its parents.
When I originally wrote the code, I was in a rush so instead of taking the time to consider what aspects of the process are shared, and carefully creating reusable blocks of code to share between them, I hastily ended up writing two entirely separate, long-winded functions. You can see the original “create from scratch” method on line 93 through 352, and the original “create with parents” on line 354 through 692. Yup, that’s two functions totalling nearly 600 lines.
Not only are these functions simply way too long which makes understanding and maintaining them difficult, having two separate functions that do nearly the same thing means that when I wanted to make a change to how creatures were generated, I would likely have to make the change in two places. This means I’d always be at risk of forgetting to make the change in both places, introducing inconsistencies.
I decided that I would tackle limb generation first, since this was essentially the entirety of the `init` methods, and it didn’t make a lot of sense that I was doing the exact same thing, repeated four times for each limb. I honestly don’t know what I was thinking years ago, I mean, aside from “Who cares how bad this code is, no one will ever see it”. After thinking about what goes into each limb, I ended up with a very clean Limb class totaling ~70 lines. Yes, I replaced roughly 300 lines of repetitive code with just 70.
After replacing the limb generation with the classes, it was relatively simple to break the rest of the creature init methods into reusable methods. These pertain to placing the limbs on the body: setupLimbs() and creating the physics body: setupBodyPhysics(). At the end of my refactoring, _both_ creature generation methods now run from line 72 through 127, which is an improvement of ~550 lines!
Not only did this make the creature creation process easier to understand, but it makes changing aspects of limb generation far easier than before. Instead of changing one thing in several different places, I can make one change in one file and rest easy that I didn’t miss anything.
If I Only Had a Brain…
Another big problem I had was with the way my creatures made decisions. All of this was baked into one large think() method, which ran from line 694 through 827. Only 133 lines, but SwiftLint was very unhappy about the large number of if statements in it. It’s actually hard to count them, especially once you factor in the various for loops further nested inside of it, but there’s more than 10.
I figured if I was tackling this as an attempt to design the creatures in a more object-oriented way by separating the limbs from the rest of the body, I should probably refactor all the decision-making into a separate class that we can call a brain.
Creating the brain ended up being more involved, mostly because I had to make some decisions (which are still in flux) about how to divide up the responsibilities between the brain and the body. It made some sense to me that the body should be in charge of locomotion, health, and sensory data; while the brain should be in charge of analysis and decision making.
In practice, I ended up using the delegate pattern to set up a communication channel between the brain and the body, as the brain needs to be able to request sensory data from the body (like current health, and details of creatures or food around it), while also sending the details of decisions back to the body (like current food or creature target).
On top of that, I started to explore whether using a GKStateMachine would be more beneficial to use over my implementation of an extremely simple state machine. Long story short, at the moment there’s no particular benefit, but it was a fun exercise and did result in some better code organization. In the current generation my AeonCreatureBrain class contains all the methods pertaining to decision making, and every brain has a (largely unnecessary) state machine that uses the states listed in AeonCreatureStates.swift to decide which decision making methods need to be called.
In the end, this ends up being a larger number of lines total, but has made the code easier to maintain and understand. It’s now much easier to make changes to how creatures make decisions about their behavior, because all the logic is compartmentalized into separate methods instead of being jumbled together into one large series of if statements.
The Future?
While I feel like I have done a lot to make Aeon Garden easier to read and maintain, there’s still a long way to go. In some cases the creature’s body is still involved in some decision making, and there’s still code in there I have yet to try to clean up.
Cleaning up the code base will not only make it easier for me to explore ideas I have for making the creatures more interesting to watch, but it will make it easier to add the features necessary to turn Aeon Garden into a full-fledged app (complete with the ability to save a tank, manually modify tank attributes to engage in your own experiments, save and copy creatures between tanks, and so on).
I’ve already made many adjustments to how creatures think and interact. In the past I had a big problem with tanks becoming far too homogenous when left running overnight, but some of the changes I’ve made recently have brought about more interesting outcomes, with creatures of various colors managing to survive and forming into natural little groups.
Here are some screenshots showing how a tank can progress, with a little over an hour of run time separating them.
Allegedly someone named Emmert Wolf said or wrote, “A man is only as good as his tools”. Now, no one seems to know who Emmert Wolf is or where he might have said this, but he probably lived sometime before the last 5 years because if he were alive today he’d probably know better and say something like, “an individual is only as good as their tools,” instead.
While the statement might only be partially true—people have dug out of prison using only a spoon before, of course—when it comes to developers, the tools they use can greatly increase their efficiency at the very least. I used to program solely using Notepad++, without realizing that it totally held me back in some ways. With that in mind, I figured it would be good to make a recommended list of my favorite developer tools, especially the ones that I use on a daily basis.
Please note that I am primarily a Python and Swift developer at the moment, and these tools may be good for just one, or both. I also only develop on macOS, so some of these apps may be platform exclusive and not relevant to developers on other platforms.
Visual Studio Code
Visual Studio Code is just an all around great text editor that I use for everything unrelated to iOS development. I probably spend more time in VSCode than any other program on my machine. Right out of the box VSCode is great to use, but its extensive extension library pushes it right over the top for most people. I don’t use a lot of extensions personally, but VSCode’s customizability makes a big difference for a lot of developers (especially web developers).
Docker
Even if you’re not interested in using Docker as a tool for deploying scalable production code, Docker makes it very easy to set up a development environment that can be shared among people collaborating on your projects. For the Numu Tracker API, I use Docker so that it’s very easy for anyone who wants to contribute to bring up a local instance of the entire project without a struggle. It can’t be overstated how convienent it is to be able to pull down the repo from GitHub, and within minutes you have a full dev environment good to go.
Postico
If you work primarily with PostgreSQL and use macOS, I don’t think it gets much nicer than Postico. The app is very mac-like in appearance, and it’s just very easy to use. I love that all my server definitions are available in the right-click menu off the Dock, so getting into a database quickly is quite easy. The interface for browsing tables and table structures is just pleasant all around. This is probably my second most-used app on my machine.
Postico is worth paying for but the free trial has no time limit and the trial limitations likely won’t impact small-time or beginner developers for a long time.
If you use databases other than PostgreSQL, TablePlus looks to be the new hotness. I have some quibbles with some of their interface choices, so it hasn’t pulled me away from Postico, but it’s very nice and supports a mind-bogglingly large number of database types.
Paw
When you’re developing a web-based API, you need some method to test your endpoints with different sets of data to see how they’re working for you. I’ve used a couple different methods to do this, but they all pale in comparison to Paw. It has a very nice interface and makes it trivially easy for me to switch between different servers to test the same endpoints locally, on a remote test server, or on production. I even include the .paw file I use in the Numu Tracker API repo.
GitUp
My favorite Git client is GitUp. It has a very simple design with a novel graphing concept that’s just pleasing to look at; it reminds me quite a bit of the NYC subway map. I’ve tried several other Git clients, but GitUp is so fast, clean looking, and easy to use that I can’t get pulled away. The fact that it’s fully open source and free to use is just icing on the cake.
That said, I will give an honorable mention to Fork, whose merge conflict resolution tool is really nice and makes complicated rebases much easier to pull off. GitUp kind of fails here, though these days I’ll usually just handle my conflict resolution in VSCode instead of resorting to Fork.
GitHub
Why am I recommending GitHub? Doesn’t everyone already use GitHub? Maybe, but I wanted to highlight that their organizational tools are really useful. While bigger companies might want to use something like Clubhouse (which is great!), for solo developers you can’t really beat having your Kanban boards right in the same place your project is stored. I use their Projects feature extensively these days to keep track of ongoing development, and I’m learning to use Issues to keep feature requests and bug reports organized. Plus each repo has a Wiki built right in, great for developer documentation.
Lingo
Sometimes the hardest part of development is just keeping all your art assets organized and easily available. You could try to commit art assets into your repositories, but what if someone who isn’t a developer wants to grab or use your assets?
This is where Lingo comes in. Instead of stuffing assets away in a folder in your repo or elsewhere, Lingo provides you with a pretty visual interface for searching and browsing your art assets. Assets can range from images, to fonts, to colors, and more, so that you can guarantee consistency in your app’s branding.
Lingo isn’t currently priced in a way that makes it good for solo developers working on small projects by themselves, but for growing teams it can make life a lot easier. Please note that I currently work on Lingo professionally, but even if I didn’t I would still think it’s a great tool for keeping all your art in one easy-to-find place.
The Noun Project
Speaking of art assets, at some point every developer needs icons. Even if you’re just prototyping something and need a placeholder icon, you can’t go wrong searching through The Noun Project for what you need. A Pro subscription is only $40/year and lets you download as many icons as you want, and while you can download icons for free to use with attribution, it should fill you with a warm fuzzy feeling to support the artists who create all these great icons.
Please note that I am employed by The Noun Project, but I was a paying customer before I became an employee—nearly all the icons in Numu Tracker came from Noun Project.
Spectacle
Near the end of my last job I ended up using Windows 10 for a while. I quickly got addicted to its nice window management features, where you can quickly slot windows into either side of the screen. This is super helpful when you’re programming in one window but want to reference docs in the other, but don’t want to have to fiddle with manually sizing your windows all the time.
While macOS doesn’t have anything like this built in, Spectacle does nearly the same job. It’s not as slick as Windows 10’s Expose-like window chooser, but that doesn’t really matter to me.
The major keyboard shortcuts that I use are Alt + Cmd + (Arrow Key) for slotting a window into the left or right side of the screen—note that if you hit the shortcut multiple times it’ll switch between three sizes—and the modifiers for these shortcuts which allow you to slot apps into four quandrants instead of splitting up the screen, Ctrl + Cmd + (Arrow Key) for putting apps in the top left or right, and Shift + Ctrl + Cmd + (Arrow Key) for putting apps in the bottom left or right.
Black
This one is Python specific, but after I discovered Black—I’m so sorry—I don’t think I could ever go back. Black is an automatic code formatter for Python projects—basically it means that I no longer have to worry about whether I’m using single quotes or double quotes consistently, or that I have enough white space between my function definitions. It also ensures formatting consistency throughout an entire project no matter who is developing for it. It really makes a big difference to my development quality of life, and saves me a lot of time that would be otherwise spent fiddling with or deciding on formatting.
SwiftLint
While there isn’t a Black for iOS development just yet (though there are some similar projects), SwiftLint comes pretty close. While it doesn’t want to automatically format your code for you (it can correct some simple mistakes if you ask it to), it will help you avoid many common Swift anti-patterns and help you write cleaner code all around. It’s very easy to integrate with Xcode, so there’s no excuse to not add it to your pods and see what it recommends.
What do you think?
Do you have any development tools you can’t live without that you think I should know about? Reach out to me on Twitter @amiantos and let me know!
