The topic of FileMaker tests is important, to me and to clients. I want to make sure my scripts work, and my clients don’t want bugs. For DevCon 2018, I put together a session about this very topic in the hopes of sparking the conversation. The recorded sessions from DevCon are now available online. Here is a link to my session on Building Testable Applications.
Introducing a new topic
In preparing for this talk, I did a fair bit of research on software testing. It became obvious early on that software testing has a lot of depth & breadth, so I wouldn’t have time to cover any other frameworks in the session in one hour. I was also grappling with the fact that the FileMaker community seems to have limited discussion on testing.
I decided to aim my session on talking about what FileMaker tests can do and prove testing’s worth with some live examples. My talk easily could have spanned two hours, so there are a few places where I had to gloss over key topics. I barely touched on modular scripting, the reason to use custom functions, and pre-conditions for testing. We will address these at some point–they play a vital role in building FileMaker tests. My goal was to pique interest in testing in the FileMaker community.
Why test?
The first few minutes were focused on why we should write tests for complex business applications. Any developer that has implemented changes on a complex system already knows the answer: we need to STOP BREAKING CODE!
Fair enough…on to the hard part. How?
Factors to Consider
As you design a testable custom app, you need to consider these factors and use these tools. At first, these ideas might seem foreign, but the more you work with them and engage with them, the easier it will be to work with these in the future.
Designing a system so it CAN be tested
Modular scripting is the vital first step to creating testable applications. If you have a piece of executable logic that should accomplish a specific function, you need to test that one function. However, if you have “spaghetti” code, functions are often times occurring in multiple locations, operating from a different context, or performing multiple discrete pieces of logic in one script. Instead, make your scripts modular and stay DRY. That’s the first step to enabling testing.
JSON In, JSON Out
Every discreet piece of logic should not only accept a script parameter that is JSON, but it should return JSON as well. Error objects come back if there was a problem and a result object comes back if everything worked. This means you have a modular script that can be given a payload and return results. And JSON is the way to go. Trying to build a complex payload using anything other than JSON introduces you to a world of hurt. You’ll be working with a lot of data and will have to find some way to keep it organized.
Custom Functions
Geist Interactive has released custom functions on Github. We have repositories for handling JSON validation and errors, and analyzing test results. Even if you aren’t adopting testing yet, go get the other custom functions now. It’s amazingly helpful to handle parameters in a meaningful way.
My examples
The following examples contain simple-to-advanced examples of what we mean by testing. Take a look at each one and pick out the above factors in each example.
The simple one
My first example was as simple as anything could be. I had a script that multiplied a number by 2. Amazing, right? This file isn’t about what it does, it’s all about how it does it. If you follow along in the video, you’ll get a good sample of how to separate scripts based on Interface scripts & Controller scripts. The controller script is testable, the interface is just what calls on the logic.
Purchase order sample
I released a purchase order example that is probably the best thing to take from my session. The file is a good dive into a simple solution that’s pretty intuitive. You just need to turn on script debugger and follow the code. You’ll learn in no time how we manage to create records using a controller.
Once you’ve got a grasp on that, open up the test file, and start writing your own tests. I dare say it is the best learning tool so far! I’d recommend imagining customer requests, changing the code, and writing tests. Then change the business logic, write more FileMaker tests, and you’ll get a feel for how testing can (or can’t) solve problems for you. As your solution gets more complex, you’ll need more tests, but it doesn’t really get more difficult. If you can implement a change to your code, and write tests to support it, you’re good!
We practice what we preach
Along with the custom functions and our work on modular FileMaker, we put this testing concept into practice.
Karbon Tests
As most people have heard by now, Geist Interactive released Karbon at DevCon. This is the most complex solution that is publicly available with tests integrated. If you’ve mastered the Purchase Order Example, feel free to give it a shot here as well.
Testing Generator
Writing tests can be monotonous. You are declaring a case, subject, payload, script, and Assert functions for each test. Where code is predictable, we don’t write it, we generate it. I released a generator file as well, that can generate a template for your testable scripts. Feedback welcome!
Long live FileMaker Tests
Since DevCon, I’m a few inquiries about testing. There are a few folks out there that are interested. I hope more folks adopt a testing model for their custom apps.
It might take some time before everyone starts using this, but I’m convinced our community will become more interested in testing over time. One issue is that testing has pre-conditions in your solution, so if scripting isn’t modular, you aren’t ready to start testing. I’ll keep telling myself that’s why I’m not getting questions. Everyone is just re-factoring code to be modular…that’s it!
What’s next
There are some interesting opportunities for test generation. We released Karbon_Generator, which is a developer tool to generate Controller Scripts. Because there are things known about your code, we should be able to produce scripts to test any other scripts. There’s a lot to iron out, but maybe by DevCon next year!