Using GUI Automation to Create APIs for API-less Desktop Apps

Friday, May 26

16:05 - 16:35
Audience levelIntermediate
Elevator pitch

Learn how to control the mouse and keyboard from your Python scripts with PyAutoGUI. Using this library, you can also add automation to desktop apps that lack APIs. We do a case study of this with the Posterazor app.


The Unix philosophy is an approach to software development that puts the focus on minimal, modular software that can be plugged together to create functionality that is more than the sum of its parts. It lends itself well to automation. But desktop applications are often monoliths that may or may not play well with other software. What do you do when you want to automate software that lacks an API? GUI automation with the PyAutoGUI package to the rescue. This talk provides a case study of writing Python scripts that control the mouse and keyboard to add a programmatic interface to Posterazor, an application that produces large, printable posters from smaller images. We’ll cover the basics of PyAutoGUI’s mouse and keyboard features, screenshot image recognition, and how to detect and recover from GUI automation errors. It’s a messy process to bodge together an API for software that never intended to have it, but Python makes such quick and dirty hacks easy to create. Whether you’re a hobbyist motivated by curiosity or a professional developer on a deadline, this approach to automation can be an important tool in your toolkit.

speaker photo

Al Sweigart

Al Sweigart is an American software developer and author of “Automate the Boring Stuff with Python” and other programming books. All of his books are available for free under a Creative Commons license on his website