We are very excited to announce that our Karbon framework will be released at the FileMaker Developer Conference in Dallas next week.
Karbon is an open-source framework for building ambitious custom FileMaker applications for the modern era. By “ambitious” we mean complex business applications of significant value that will be around for a while. If you have a simple problem, Karbon will just frustrate you and get in your way. But if you believe, as we do, that a thriving business requires a flexible and robust custom application, then Karbon maybe what you re looking for. In this post, I am going to lay out why we built Karbon and why we are releasing the core as an open-source project.
Times Have Changed
A lot has changed in the over 20 years that we’ve been developing complex applications with FileMaker. The landscape into which we deploy our solutions bears little resemblance to what it looked like 20 years ago. Technology has changed. Competition has changed. What’s possible has changed.
FileMaker has done an incredible job adapting to changing technology demands. There are few, if any, platforms that can boast 25 plus years of continuous success. The FileMaker Platform is incredibly robust and flexible, and it provides a solid foundation upon which to build custom applications of significance and value.
But what does an application of value and significance look like today? What does it need to do that it didn’t before? What are its characteristics?
Here are some of the high-level issues that we encountered over and over as we built and rebuilt Karbon and worked to solve our customer’s needs over the last several years. A custom application with significant value is going to need to do most of the following.
- Integrate with other Apps and Handle Distributed Data
Critical data is now spread out over many apps and services. It is no longer reasonable or even possible to avoid integrating with other apps and services. Even FileMaker Go apps often have to work offline. The ability to exchange data with third party services or an offline FileMaker Go app is an important first step, but without understanding the implications of distributed data, you can quickly get into trouble.
- Work over the Internet
Even if you aren’t planning to deploy with WebDirect, your FileMaker application must be built to serve users in across the country or even the world.
- Allow for Collaborative Development and Maintenance
It is no longer reasonable to build a business on a custom application that that relies on a solo developer—the risk to the business is too high. Even with the very best application development platform, solo developers or small teams will find it challenging to innovate to remain relevant in today’s fast-paced tech world. Applications must be able to incorporate the efforts of other people effectively, either by employing larger teams of developers or relying on third modules, which can be incorporated quickly and with little effort.
- Automated Testing of Critical Business Logic
If you can’t verify that critical business logic works, you can’t make changes to your logic without the fear of breaking your application. If you can’t change your logic, you can’t change your business—if you can’t change your business… well… somebody will come and change it for you.
- Fail With Grace
Applications crash. They get disconnected from the server. People close their laptops. Somebody locks a record. The consequences of these kinds of failures on valuable data can be severe. Given the distributed nature of networks and applications, these types of problems are even more common today. A valuable application must be able to handle failures while safeguarding critical data and enforcing complex business rules.
- Solid, but Flexible
For an application to have lasting value, the data has to be dependable and lasting. If you build a data structure that can’t be easily changed or adapted, you may find yourself painted into a corner.
- Embrace Standards
It is no longer reasonable to invent solutions to problems that are already solved by the rest of the world. All this does is create confusion for new developers coming from other other platforms. Time spent doing it our own way is time we could have spent creating value.
- Continuous Improvement
A complex business application today is never complete. There is no finish line. There is no final feature to be added. It continuously evolves. The application and its development process must be able to thrive in such an environment.
We have tried to build Karbon to address or begin to address all of the issues raised above. We should be clear that it isn’t close to “done” (nor will it ever be), but we think it’s far enough along to share with the rest of the world. We’ll have more information and downloads available at DevCon, but for now, I wanted to lay out what we call our Karbon Values. These are the principles that we tried to use while making decisions on our development approach.
- First Correct, Then Fast Enough, Finally Pretty
A beautiful user experience that hides fundamental problems in the data model or logic is far worse than an ugly user interface that handles the data logic and data model well. That isn’t to say we don’t value nice user interface and user experience. We do. We just value data integrity more.
- Modular Design
We follow modular guidelines laid out a few years ago. Keeping things in their own box as much as we can within what FileMaker offers us helps us with organizing code especially at the sub-file level.
- Database Transactions
We consider any complex logic that updates more than one record broken if it doesn’t use database transactions. Once you learn to write transactional scripts, you are unlikely to want to return to any other method. It’s easy and safe and frankly has fewer moving parts.
- Separation of Concerns
This is not the same thing as the Separation Model, although it shares some of the ideas. Karbon has three core files: 1) a UI file, 2) a Controller file, and 3) a Data file. We don’t do this to avoid imports while migrating from development to production. We do this because this makes it easy to make consistent decisions on how to write good code. When we are writing a code in the Controller file we “know” that we are not dealing with user interface, or data modeling relationships. We can focus on scripting solid business logic. When we are in the UI file we can focus on building a good user interface, and not be concerned about building layouts or graphs that are concerned with enforcing business or database integrity rules, as those are handled by the controller scripts.
- APIs First
Karbon’s modular design allows building testable and transaction-safe API scripts based on JSON for all critical business logic.
- Automated Testing
We have integrated techniques into Karbon and into its supported files that make it easier to write testable code and write tests to validate rules. Tested logic is maintainable logic.
- Separate Dev and Production
Working on live files is a recipe for disaster. Separate development and production environments make testing, development and maintenance easier. Data migration issues have been mostly handled by new tools from FileMaker and other 3rd parties.
- Understand the Source of Truth
In modern systems, you likely will work with data that comes from third party systems; you need to know which app is considered correct. Sometimes this isn’t always clear. Take invoices as an example: If you are integrating with an accounting system—like Karbon does with QuickBooks Online (QBO)—then you are likely sending invoices to that system. Once the invoice gets there, it may be modified in QBO; which is one is now correct? Does FileMaker hold the truth, or does QBO? The answer, in this case, is provably QBO. Once an invoice lands there, it has impacts on accounting reports and taxes. And once an accounting period has closed, it can’t be updated again from FileMaker. So clearly QBO holds the truth. Understanding that drives your development approach. For example, Karbon (by default) let’s QBO assign new invoice numbers.
- Better Data Models Last
Data models like the Party Model take longer to implement, but once implemented they allow the app to scale and adapt and avoid disruptive changes.
- Spend the Time on The Right Stuff
User interfaces change frequently. Every year we get new and different widgets from FileMaker that might radically impact how easy it to build a type of layout. For example, FileMaker 17 shipped with a drop-dead simple way to build Master Detail layouts. We’re wasting our time if we spend our time on complex workarounds but should instead focus on investing in transactional scripts with tests—this code likely won’t need to be changed or to become obsolete by FileMaker’s next release.
- Sharing is Caring
Sharing is not just about “feeling good” or “giving back to the community,” although those are critical aspects of it. When you write code that is shareable, you are writing better code. It will have been worth it if the only person you share your code with is your future self.
- Simple is Better
Avoid building abstractions that don’t add value. Saving yourself some keystrokes is not value. Try to keep things as simple as you. Create abstractions that provide more value, without making things harder to read.
Documented code has a better chance of being successfully maintained then undocumented code. The standard we use also allows us to build code-generation tools.
- Write Code that Writes Code
If possible use, write FileMaker applications that can write other FileMaker apps. We proved this works well with Generator, our free utility for generating scripts to interact with JSON web APIs.
OK, But What Can You Do With It?
That’s a fair question. Karbon’s first release is focused on Customer Relationship Management. It handles contacts, activities, and some sales transactions. It has a fleshed-out Party Model. It has tested transactional scripts for critical business logic. It’s pre-integrated with several APIs like Smarty Streets and Quickbooks Online. It is a solid foundation to build a business on, and we have plan to continue extending the feature set in the future.
We think that some people who are at the beginning stage of a project might just choose to adopt Karbon in its entirety. Others who are already deep into a project will be able to grab large chunks of it and move them into their own system. Still, others may just tear it apart and try to learn everything they can from it. All of those outcomes are a win as far as we’re concerned.
Why Give it Away?
The core of Karbon is open source and will remain free. We think that a project like this has to be free to fully express the values listed above. But it is also a practical approach to a large and complex problem. As we encountered more of the issues outlined above, it became clear we needed a different approach to provide our clients with awesome value. We also needed this for our internal applications. We know that it’s a large and challenging task.
We want feedback and help from people who are interested. Good software is always widely used, and we’d like others to use Karbon and tell us where it doesn’t make sense, or what’s broken, or what it should do instead. Lastly, we want to help improve the state of the art in the community. We want people in the community who want to build—or already are building—these kinds of projects to have good examples of how to deal with the issues we discussed above. We hope that this will create a cycle of virtuous feedback that will benefit anyone who chooses to participate.
How does Geist Interactive Get Paid?
At the end of the day if we can’t afford to work on projects it doesn’t matter how cool they are. They can’t get done. So clearly, we think that this will pay off in some way.
- Karbon’s core will be free, but we will likely ship it with several optional modules, which we will charge for. Users can remove them or not use them at all, and not have to pay anything. But if they want to use them they will need to license those parts. For example, Karbon will ship pre-integrated with Quickbooks Online. The module that empowers that integration is called LegerLink and will cost $500/yr.
- We also have some other modules that will not ship with Karbon but will be built to work with Karbon. Those modules may carry a fee.
- We will also offer support, implementation and training for Karbon.