A Little Something . . .
We’ve got something special up our sleeves. It’s an awesome new version of a great product, and it features some great technology. . .
We’re pretty proud of it. Stay tuned!
FileMaker Drag and Drop: Handling Script Triggers and Validation
In a typical day, a FileMaker user is interested in storing information in a custom app and then reporting out on that data in many different ways. A record, at the very heart of FileMaker, accomplishes this need. Here, data is entered and stored, and from this object, information is retrieved. One way data is entered is through FileMaker Drag and Drop, and therein lies a problem.
The Life Cycle
A record has a life cycle. It is created, edited, and committed to the file. Sometimes it is deleted. This cycle happens over and over. A record may go through many edits and commits in its life.
A record’s life cycle
A primary job of FileMaker developers is to control that record life cycle. We set up strict controls about when a record can be created, edited, and deleted. We build logic to ensure a record has the correct type of information, and we work to ensure that the record is committed with integrity.
Like a vampire or a walker, however, there’s a user experience piece of FileMaker that is completely outside this life cycle and disrupts how a record is manipulated and then committed. That is the Drag and Drop.
FileMaker Drag and Drop: the Gotchas
FileMaker drag and drop is a great feature for fast data entry. This feature is commonly used for container fields and PDFs or pictures, but it can also be used to drag text data to a field. And that action is the disruption. When dragging data from another source into FileMaker, a user is almost completely subverting the record’s carefully structured life cycle, and thus the unexpected can happen.
Here’s a possible scenario: The user drags a text block from a website or email. She wants to save the text of an article or the current price of an item. The user simply highlights the text on the web page and clicks on it. Holding the mouse down, she drags the text over to the open empty and inactive field on the layout. Finally, she lets the mouse go and the text fills the field.
To the user, all is good, but under the hood, all sorts of unexpected things are happening:
The record is never opened. Or is it?
Normally when I type in a field, the record opens. FileMaker holds this record open for all edits. This doesn’t happen in a FileMaker drag and drop scenario. The data viewer, when watching the function Get(RecordOpenState), does not update from zero. Practically it may have opened, but the problem is that we cannot get to the open state of the record.
The record is never committed.
Well, I take that back a bit. Yes. It is committed because the data is stored in the field and record. Like the open state, we cannot get to the moment before the record is committed.
The modification count goes up
The result of the function Get ( RecordModificationCount) in the data viewer, increments by one each time.
Object script triggers do not run at all or properly
Mislav Kos and Wim Decorte at Soliant Consulting have a great rundown of all the script triggers and when they happen. The script trigger sequence, however it breaks down in a drag/drop scenario:
- onObjectModify does run, but it runs after the record has already been committed (or the record has been closed).
- The rest of the triggers do not run at all, no matter if they were meant to be a pre- or a post- trigger.
These facts present many problems for the developer. We are not able to control the life cycle of the record. We don’t get to capture when a record is open or before it is committed. Our lack of control over these means we cannot rely on script triggers to check field data for validation or for anything else we might do.
The Solutions
We can’t allow users to enter data without going through the proper business logic. So I can think of two solutions: either disallow drag and drop or highly control when a user can enter data into a record by any means they wish.
Disallow it
The first method is easier said than done. FileMaker Drag and Drop a preference in the app on each user’s machine. It is NOT specific to the file. So a developer or IT person would have to turn this off for every user. And of course, it can be turned on again in the preferences menu by a savvy user.
It would be awful to have to update this for each user
Disable it
The second method holds more promise: As a developer I might want to control when a user can enter data. I would have a separate data-entry layout and scripting to open a record and then check to make sure the record can be locked. When finished, the user would click a button that runs a script that checks various validations and allows the record to be committed or causes it to be reverted. Controlling the life cycle of the record means I could allow drag and drop.
Disallow it (Again)
Going back to method one, there is a way to disallow FileMaker Drag and Drop for all fields in a table without resorting to visiting each machine to turn off the app preference. The technique consists of a nondescript little field (text or number type) in the table where we want to prevent inadvertent drag and drop. I named my field “zz_ValidationField” so it sorts to the bottom of the field list. I’ve learned this from wise people: sort fields that do not contain data to the bottom.
On this field, I added validation in this manner:
- The “Allow user to override during data entry” is unchecked.
- The Validation can happen either “Always” or “Only during data entry”.
Data Validation for my zz_ValidationField
- Validation by calculation is used. Inside this is the function Get(RecordOpenState).
The field’s calculation
- Uncheck the “Validate only if field has been modified” checkbox at the bottom left.
Explanation
This setup simply prevents drag and drop from happening when the record is not previously open. This field’s validation will fail if the record is not open ( Get ( RecordOpenState) = 0 ) and thus will not let the data be saved. The custom message doesn’t come up since the record wasn’t open in the first place.
Conversely, if the user opens the record (Get (RecordOpenState) > 0 ) via scripting or clicking in any field and then drags and drops, the validation will allow the drag-drop to happen as normal.
As FileMaker developers, we have to control every aspect of our custom app so that we can ensure data integrity. The FileMaker Drag and Drop experience, dragging text blocks or PDFs gets around our control unless we take an extra step to ensure that users don’t get carte blanche in adding data to the file.
Download Demo File
Search FileMaker Script Parameters with FMPerception
Searching FileMaker Script Parameters
FMPerception, the fastest FileMaker analysis tool available, makes it easy to find everything you have passed as a FileMaker Script Parameter using the built-in Calculation Analysis. Watch the video for more info.
FMPerception Saved My Butt
It happens. You’re moving quickly, jammin’ on your favorite FileMaker custom app. Laying down some insane custom business logic in the form of one awesome, totally badass FileMaker Script. Then you notice that other script. The one you need to get rid of. No time like the present. So you pop over to it, hit “delete”, crush the popup confirmation before you have time to read it and… Whoops, something went very wrong!
NOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO!
You deleted the wrong script. Your awesome, badass script with the insane logic is nothing but a memory. There is no undo. It’s gone. It’s not in the backup because you were just working on it. It’s too new. You are totally f#%@ed! You weep, you rage, you even try mediation. Nothing works. The script is still gone.
Then you remember. “Wait I just made a fresh DDR for FMPerception“. A smile creeps across your face. Yeah, you’re gonna be OK. FMPerception is going to save your butt.
True Story
The above scenario happened to me this week. I deleted the wrong script. For a moment, it was bad. But then I remembered that FMPerception had the script. I could easily restore it, and reconnect it the other scripts that were using it. Just 2 minutes later I was done.
Phew! Watch the video above for exactly how to recover a deleted script using FMPerception.
Need For Speed
FMPerception is so fast that it doesn’t get in the way of your workflow. As a result, you’ll use it all the time. You will be much more likely to have a recent analysis than if you are using an analysis tool that forces you to wait minutes or hours. Most FMPerception users know all about that, but what they might not know is that your analysis is also a backup, and can be used to restore code that you deleted or broken. And that can save your butt!
FileMaker Testing, Part 1 – Your First Test
With the release of FileMaker 16 and native JSON functions, the ability to create a testable solution has been vastly improved. In this blog post, I will walk you through a simple example of what testing is, and how it can improve your solutions. Most developers have had situations where they need to alter some existing logic, and are concerned that they might alter something with unforeseen consequences. Wouldn’t it be nice if you could ensure that your changes didn’t break existing code? Welcome to FileMaker Testing.
Automated Software Testing
Software testing has a lot of different variations and buzzwords associated with it. You’ll hear all sorts of phrases and acronyms thrown about like TDD, BDD, Unit Testing, Regression Testing, etc. It’s worth a learning a little something about these at some point. But at the heart of all of them is a simple concept; write some code to automatically test some other code. If you do, your code will be less fragile, and easier to maintain.
What is FileMaker Testing?
A test is a FileMaker just script used to ensure that other scripts are working properly. Tests ensure that scripts are not only accomplishing their goal but also producing the desired result when they fail. While there are many ways to perform FileMaker testing, we test using the following assumptions:
- Each script to be tested accepts & returns JSON as a parameter
- We will be using custom functions for handling errors, testing JSON objects, and validating a JSON payload
Writing Testable Scripts
To prepare your script that needs to be tested, it should always return JSON to the script that called it. This means that if the script fails, it should return an error JSON object. If it succeeds, it should return an object representing the data that was changed. See the example script called “Create a Project” in FileToTest. You will notice that it returns verbose errors on all failures, and returns a JSON object when successful.
Writing The Test Script
To prepare your test script, you need to do three things in karbon_tester:
- Create a JSON Object to pass the script using the layout “Edit Test Data”
- Create a script in the Folder “Project Testing” (Copy the existing scripts…you’ll get the idea!)
- Add the script you created to the script “Test – Project”
Do We Test Every Script?
There is no reason to test each & every script. We only write tests for those scripts that play a critical role in the solution. Once you are comfortable with testing, it will become clear what you should test. It’s up to you.
Benefits of FileMaker Testing
The benefits of creating test scripts are compelling. First, it provides any solution with a level of stability that is difficult to achieve in a timely fashion. If multiple developers are collaborating on the same solution, the test file enables everyone to run common tests on core business logic, and see if they have produced any unforeseen fatal errors.
Second, it provides a level of clarity for you, the developer. Once you begin writing tests, you will take on an alternate persona, where you are actively poking holes in your own code. This may seem like a small difference, but we believe you’ll be surprised once you try it. You become much more critical, which improves your focus & clarity.
Third, the readability of your code will improve. One thing that makes JSON difficult to handle is that there is not a repository for JSON objects within a FileMaker solution. When a script is designed to handle records in multiple tables, karbon_tester serves as a clear, contextual example of how to create & pass JSON objects to work within a solution.
Is it Worth It?
The first question many people seem to ask when introduced to testing is “is this worth it?” Our answer would be that it depends on the solution. If you have a complex system, multiple developers, and a need for stable solutions, it’s not just worth it, it should be the standard.
Download
To start, download our sample files here. Open “FileToTest”, and create a project. Review the script “Create a Project” to see how the file works. Then open the file “Karbon_Tester” and run the existing test. See how easy that was? Now you should create your own scripts to test creating a status.
What About Editing Data & Managing Test Data?
We will be releasing part 2 in this series in the future. We will walk through options for loading test data, & managing records that are used only for testing. The testing in this sample is intentionally simple to grasp and does not confront some of the nuanced issues involved in testing.
What about Karbon?
We know some have you have been waiting for us to release some of internal tools and frameworks that we group under something we call “Karbon”. This is the piece piece of that puzzle. More coming soon.
Generator v2.0.6
One of Generator’s primary jobs is to remove the tedium of doing the same thing over and over again. Not only is it boring, but it is error prone. If something can be automated it should be automated. In v2.06, Generator makes it even easier to go from JSON to FileMaker database fields. We’ll cover that in this video.
If you have some JSON on your clipboard you can quickly create all the fields in a FileMaker table, and create a script that maps those fields to your JSON object. It takes about 30 seconds. Of course, if you already have fields in your table you can map to those too.
There is clearly more we can do here. We could add the table for example, or we could try to guess the field type before we create the field. We’ll get to that at some point.
It’s Free!
By the way, in case you didn’t know Generator is completely free an unlocked!
Happy Generating 🙂
Pause Bootleg
Thanks to Tim Neudecker for recording this.
Testing FileMaker Custom Functions
Back when we shipped the original JSONCustom Function Library, we adopted rigorous testing as part of our development process. Testing FileMaker custom functions proved incredibly valuable. Those functions went to become critical parts of fmQBO, Generator, and every custom project we have done since. We have a new library that we want to develop in the same way. In this video, we’ll see how it becomes such a critical part of an open source project.
Contributions
We love getting contributions. We are trying to create useful tools that everyone can use. But to do that we have to be able to collaborate effectively. We need to make sure that new bug fixes or features don’t break existing features. The best way to do that is with tests. We ask everyone who wants to contribute to help us by writing simple tests. Don’t worry they aren’t hard. Our Contributing Guide and this video show you how to do it. Remember it’s just FileMaker.
Moving Forward and Getting Smarter
When FileMaker decided to add native JSON functions to FileMaker they used the test file that we had made for our JSONCustomFunction library to make sure that the new functions behaved mostly like the ones we had created. That’s why the native functions are so similar to ours.
Why did they use our Test file? Why not create their own tests? Because our test file included a number of edge cases and tricky issues that needed to be solved in the real world. Every bug we encountered, every new feature we added had tests. Over time those tests got smarter and smarter and covered more and more of the use cases that we encountered in the wild. In short using the test file for our functions was the easiest way for FileMaker to ensure that the new functions were useful.
Testing FileMaker Custom Function is Easy
Testing isn’t hard. But it does take more work up front. But once you out the tests in place you will be glad you did. It will save you countless hours in the future. We’ll have more to say about testing FileMaker scripts and other parts of FileMaker in the future.
FileMaker Calculation Analysis with FMPerception
FileMaker Calculations are everywhere, not just in fields and script steps. They are all over your layouts, in your custom menus and even in your security settings. They might just be the heart and soul of your FileMaker system. But they are scattered all over the place, which makes them hard to track. FMPerception has a new feature that can help you find and fix problems related to calculations getting out of hand. It makes FileMaker Calculation Analysis a thing of beauty.
Heart and Soul
Most of the business logic of your FileMaker solution is either encoded in Calculations or passes through calculations in some way. You can use calcs to hide and/or format layout objects. You can use them to filter portals, display dynamic button labels, show tooltips, and populate placeholders. And that’s just layouts.
There are custom function calcs, custom menu calcs, and record level access calcs. There are of course calculations in scripts, and in fields. But hey, did you know that a field can have two calculations at a time, one for the auto-enter and one for validation calculation?
Think about this for a moment. A single field object on a layout can have the following calculations defined in or on it:
- Field Definition
- Auto-Enter
- Validation
- Layout Object
- Data Hide
- Tooltip
- Placeholder
- Several Script Trigger Parameters
- Dozens of Conditional Formatting Functions
- Button Parameter
That’s a lot of functions! Calc functions are everywhere and are used for lots of different types of logic. They are the heart and soul of any FileMaker Custom Application. Since they are so useful, they get copied and pasted around, they get broken, they get used out of context. Sometimes they just get huge and complex, or reference other huge and nested unstored calculations.
FileMaker Calculation Analysis
You have always been able to use FMPerception to help fix some of the problems we outlined above. But… It was scattered about. Since the FMPerception interface mirrors FileMaker’s structure we displayed the calcs embedded in each object. For example, if you searched for some text that appeared in every one of the calcs on the example field above, you would only get one result. Just the field. That’s it. Even though FMPerception found each instance of the string it had no way to show you all those results.
Enter the Calculations List view, located in the slow query section of the browser. The Calculations List view is a single list of every calculation in your custom application. You can also easily narrow the list down to just every calc in a single file, or script, or (my particular favorite) just one layout. Suddenly we have a much better way to look at and explore our calculations. But our story doesn’t end there.
The Devil is in the Details
Ok great. We have one list of calcs. Very helpful. But it’s the details we get with the list that makes it super interesting. There are about 2 dozen columns of information about each of those calcs, that you can sort, re-arrange and filter. You can easily find the biggest calculation in your system, or all the commented out calcs, or all the portal filters, or calcs that can’t possibly work, because of disconnected contexts or… or …etc. The list goes on and on. See the docs for more information or watch the videos on this post.
Now if you have a slow layout, you can just start with the list of calcs defined on that layout. Every calc on that layout is there. If you have a slow layout, it is almost certainly because of calcs that take a while to resolve. The culprit is probably on that list. Use the FileMaker Calculation Analysis information in the columns to help you locate it.
Problems Solved
Many of the problems come from calculations getting copied and pasted around in the form of conditional formatting, hide calcs, portal filters, button params etc. These calcs may have worked correctly at one time but now are pasted into a context that no longer makes sense. Sometimes the calcs are disabled when this happens if there is no valid path to the table occurrences it needs. But sometimes there is a valid path. It’s just the wrong path. And it could be slowing your system down or leading to unexpected behavior.
Selector Connector based systems are vulnerable to this because it intentionally creates a valid path to just about every part of your graph. One of the columns in the detail list is a list of TOs that are touched. Selector Connector users can use the TO List to make sure that layout calcs like formatting, hiding or portal filter calcs don’t pass through the Selector Connector TOs, which is a sign that something is amiss.
FMPerception can help you find and fix all those problems, plus much more that we haven’t even thought of yet. It keeps expanding our knowledge of how the heart and soul of our FileMaker custom applications work. FileMaker Calculation Analysis? Yeah. It’s a thing.
FileMaker Top Call Stats Analysis With FMPerception
If you turn on the Filemaker Top Call Stats log, FileMaker Server will log the longest running operations, during a log interval. It saves up to 25 calls per logging interval. This is a great resource to help find those areas of your database that might not be performing well. But it is difficult to read. First, there is a lot of data. You’ll need to import it into a FileMaker Database or a spreadsheet to be able to do any kind of sorting or filtering. Second, FileMaker Server doesn’t use the names of elements when it saves them in the log. It uses the internal IDs. This is good for security but it makes it difficult to tell what’s going on. FMPerception has a new feature that can help you make sense of your log. Read more