Form Tools 3: progress update

Hi all,

Hope your year is treating you well! Boy it went fast for me.

It’s been a couple of months since my last update so it’s time for another post. I’ve been using the Form Tools twitter account for announcing day-by-day news so if you want to stay more current, do check that out.

Form Tools 3

Version 3 is coming along well. It’s still in alpha (meaning not production ready) and will remain that way until we have compatible versions of all components (themes, modules, API) and the script itself is nice and stable [N.B. it’s actually really not bad right now]. As of today the themes are done and 2/3 of the modules have been updated. The alpha download package was getting a bit bloated with all the compatible modules so I’ve trimmed it down to include just the major ones.

The remaining modules that still need to be updated are:

Client Audit
Custom Fields
Form Backup
Form Builder
JS Error Logs
Submission Accounts
Submission History
Submission ID Manager
Submission Pre-parser

Plus the API. The API will be changing quite a lot (like the rest of Form Tools it’ll be written as a single API class) so I’ll provide an adapter for people upgrading to FT3 that’ll emulate the old API method interface.

The end is finally in sight!

After that, what’s next?

Boy Form Tools 3 has been a hell of slog. I’m sure I’ve said this before but working on library updates, code improvements, PHP7 compatibility and so on are all necessary, but frightfully tedious when done en masse. And that’s pretty much been my life for… how long? 2 years? Urgh. When do I get to the fun stuff?! New features! Improved UI! Performance updates!? 

Well, my original plan was to get FT3 out the door containing all the hum-drum work, then embark on a shiny new FT4 rewrite containing all the fun stuff. But having seen how long it takes to update a script the size of Form Tools I’ve decided this is the wrong approach. Instead I’m going to do incremental updates rather than embark on another massive rewrite that’ll take years to see the light of day.

So various features I had planned for Form Tools 4 will actually be released one by one in v3. I think this is a much more sensible approach: it’ll mean new features will be quicker to roll out and ensure Form Tools doesn’t ever get as out of date as Form Tools 2 did.

There will still be a FT4, but it’ll be a much more realistic goal.

And what about that upgrade script? What’s that coming back online?!

Gah, I’m so sorry about that. In short: that script is a project unto itself. When I moved the codebase to github I borked a number of things and it’s been a nightmare to bring back to life. Note to self: dependency management is NOT to be taken lightly (I should get that tattooed somewhere). The ultimate goal is to remove the upgrade script from the website and include it within Form Tools itself. But that’s longer term. In the short term: I’m working on it. 🙂 I actually think I have it beat – it’s now largely working locally, I just need a lot more testing before releasing it.


I’m taking off a couple of weeks in early December to go bird watching in Panama (it’s a hobby, what can I say) so don’t expect much activity during that time. But I think it’s pretty reasonable to expect the Beta to be release in January, and v3 to become the official build a month later. I’m actually relatively confident with the current alpha – I’ve had a few people report issues but by and large they haven’t been too daunting.

All the best!


Posted in Alpha / Beta Versions | 6 Comments

Form Tools 3 alpha available

Hi all,

Well it’s been a looong time in the works, but the first Form Tools 3 alpha is now available! This is a complete under-the-hood rewrite of Form Tools which now supports PHP 7. Any help testing it out to find bugs is much appreciated!

Important stuff:

– This is NOT production-ready code.
– You CANNOT upgrade your existing Form Tools 2 installations yet. If you do, your Form Tools installation will stop working, so don’t do it!
– At this stage only the Form Tools Core is available. The Core includes all necessary code to install the script and for basic usage.

You can download the alpha here:

Please report all bugs here:

You can find more information on Form Tools 3 at the repo, here:

I’m going to start updating the modules etc. for FT3 compatibility, and fix whatever issues come up as they get reported.



Posted in Alpha / Beta Versions | 3 Comments

Form Tools 3 – and 4

Hi all,

I’ve been working pretty continuously towards the 3.0 release these last few months, but I’m afraid it’s still a ways off yet. But my plans have now solidified about the foreseeable future of Form Tools, so a post was in order. Much of this has been posted before – now more than anything it has a concrete roadmap.
Form Tools 3

As I’ve written about before, Form Tools 3 is an under-the-hood rewrite of Form Tools to support PHP 7. There will be a few small changes here and there but the interface will look the same, the way you install modules the same, the API etc. etc. All pretty much the same.

Upgrading to 3.0 will be possible from 2.2.6 or 2.2.7 to 3.0, but no earlier. The upgrade code supports upgrading from waaaaay back in time and it’s time to clean house. So yes, there will be an upgrade path, but you have to upgrade to the latest versions of Form Tools 2 first (I’ll be updating the System Check module to add the one additional FT version check on the “FT3 compatibility” section).

Once 3.0 is released, along with all the modules, themes and API, I’ll be releasing bug fixes for 3.0 only – that major version is effectively finished. At that point I’ll be working on…
Form Tools 4

As I’ve been slowly refactoring the code for 3.0, I’ve had to hold myself back again and again from doing more aggressive changes. I now have a long list of things that I think the script needs, but avoided working now to prevent 3.0 being held up any further.

So 4.0 will be a much more aggressive change to Form Tools. These are the main goals:
User interface redesign & rewrite
Personally I don’t find Form Tools’s UI or UX too bad (okay, I’m biased) but it’s 2017 and it can be better. Better use of space, better look and feel & better IA (information architecture – the structure of how the pages tie together)  I want to do a complete re-write of the front-end portion of the script from the JS code to the user interface. Language-wise I’ll be using React – but it’ll probably remain a traditional page-by-page app and not a one-page web app. We’ll see.

This is an item that’s long overdue, but one that I’m the least sure about how to implement yet.

I want to completely revise the installation and upgrade code to remove the dependencies on the website. Adding modules should be as simple as selecting them via the interface and clicking a few buttons. No more manual FTP’ing of files to your server to install things! (well, other than the original installation). Right now I create a bundle of files via the website which generates a zipfile containing the Core, API and list of modules & themes. Bah! Instead, Form Tools should be a single identical zip for everyone who downloads it, containing ONLY the Core code. Then, during the installation phase they would select what pieces they want. Users shouldn’t have to concern themselves with manually downloading separate pieces of code and installing them separately.

This would result in a lot of complex code being dropped from the website making the project far more maintainable for me, and much simpler for people.

Form Tools is too slow in places! Time to benchmark and improve.

User Roles / Permissions
Form Tools has a very basic user account system at the moment. It has a sort of basic way to change configuration settings for multiple users at once, but no real concept of a “role” which you see in most large applications. The idea of roles is that you create a role that describes what a user can do, then set each user to a particular role. Pretty straightforward. This means re-writing the backbone of the current permissions code and remove the current admin/client separation.

Schema ALL the things!
This one’s is a bit technical, but the idea here is to make all configuration data stored within Form Tools adhere to schemas, i.e. make them “well-defined”. Form Tools changes and configs have always been very ad-hoc: each version comes out with new settings for your forms, client accounts, form builder forms and so on. A schema will provide an explicit description of a particular version of each piece of data, outlining the semantics for each and every part of the configuration. So why do this? Well it’ll allow for my ultimate goal of Form Tools, which is…

Form Tools market
I’m really proud of what Form Tools does. It’s crazily feature rich and lets you set up complex forms and let’s you fine-tune who sees what, what can be downloaded, edited, viewed etc. etc. But the pain of Form Tools lies in configuring it. It takes TIME setting up a form with all the right settings, form fields, users and so on. The goal of the Form Tools market is to allow users to share configurations so rather than manually setting up a form field by painstaking field, you just browse the market and click “Import!”. That imports the form configuration, views, email templates, form builder templates, etc. etc. Then you can go to town tweaking it.

This is eminently do-able, it just requires a lot of work and planning. But there’s one catch to this last one: I may have to put a small fee on this feature, like $20 a year or something… it’ll mean increasing the server load for the website a great deal, and Form Tools costs me a fair bit of money as it is to host.
Alrighty, there’s the update. I’m sorry development on 3.0 is taking so long. I’m still not at the point when I could estimate a release date for the alpha.

Posted in Alpha / Beta Versions | 2 Comments

Boring technical post about FT-next

Stem your excitement! Time for an extremely boring technical post about the next version of Form Tools. Caffeine pills on standby!

I’ve dubbed it FT-Next partly to sound cool, and partly because I’m no longer sure if it’s going to be a 2.3 or 3.0.

High-level stuff

So… I’ve been making a fair bit of progress the last couple of weeks. Currently I’m working on updating the core code. I have no estimates on how long it’s going to take, and the joy of not working for The Man is that I don’t have to provide one. Mwahah.

With FT-Next there will be two small functional improvements over previous versions:

  • All language files will be included with the core. I really don’t know why I didn’t always do this; not including the language files has no benefit and it just adds an extra chore for non-English speakers having to manually install the language files. This’ll also allow me to simplify the custom build page.
  • The install/ folder will no longer need to be manually deleted after installation. Again, there was no real technical reason the folder had to be deleted – it can be solved perfectly well via code, and that’s what FT-Next will do.

Technical stuff

Whenever I find myself revisiting old code I find I have to resist the temptation to rewrite everything at once. (“Well I just have to change that… and that…. and that….”). With a codebase the size of FT that’s just not possible so I’m trying to focus on the essentials. My #1 goal right now is PHP 7 support but I’m finding I still have to do more changes that I anticipated.

  • Smarty upgrade.
    In order to support PHP 7 I need to update Smarty. This is the change that concerns me the most: Smarty underpins the entire Form Tools script – it’s not just used for the rendering of the HTML, but a lot of the internals – so I’m really going to have to confirm everything’s backward compatible and find a way to patch the pieces that aren’t. Because Form Tools allows customization of the Smarty logic in various places this will brook a great deal more thought.
  • Composer.
    The next version of Form Tools will use Composer for dependency management. Initially it’ll probably just be used to include Smarty, but down the road I may even use it for including the Form Tools modules, themes and API too. But to be clear: there will be no extra steps for installations: you’ll continue to download the single Form Tools package from the site and install in one go.
  • Codebase being moved to OO.
    Years ago I made a judgement call to write the application functionally, not object-oriented. I think it was the right call at the time (simple = good) but today I find there are too many benefits to object-oriented code. PHP grew up and has pretty decent support for all sorts of OO features and making use of them will simplify a great deal. This is something I’ve had on the docket for years and now’s the time. It will be a gradual upgrade.
  • Backward compatibility.
    This is my primary concern for the next release. I want it to be as backward compatible as possible. But I am requiring PDO with the PDO-MySQL extension installed for all installations. This has come standard with PHP installations for years so it’ll only affect a small minority of users. The minimum PHP version will be 5.3.

Misc other stuff

  • I’m dumping global/library.php in favour of a global/code/Core.class.php file. That’ll contain all the various available application-wide settings like before, only no more globals. Now only the settings that can be overridden will actually be overridden. Sheesh, no more wild west…!
  • The config.php content will change; I’m leaning towards using define() for all the various DB settings, etc. just to clean things up. But again, backward-compatibility is the name of the game so your old config.php file content will continue to work.
  • Converting the database over to use PDO has gone pretty well so far. The one addition that needs to be tracked for the DB info is port. Seems like 3306 is the standard – so upgrading will give you that value by default. But depending on how things go – i.e. how extreme the changes are going to be – this may become a 3.0 release and I’ll release a pre-upgrade script to verify your system, verify settings etc. before you actually upgrade. Just to be safe.
  • I may convert all language files to JSON… haven’t decided yet… that may be in a later version.

Anyway, things are plodding on. More posts to come, no doubt.  🙂



Posted in News | 5 Comments

PHP 7 support

Hi all,

One week in, and the new site hasn’t completely melted yet! Hell, I’d consider that a triumph! 😉

Naturally I’ll continue to patch any bugs that come in, but I’ll soon be turning my attention to the actual Form Tools codebase, starting with….

PHP 7 support

By far the most important thing to address is getting Form Tools to run on PHP7. I’m sorry this has taken so long to address, but the site & moving to code to github had to come first. PHP7 dropped a few things, such as the database functions that Form Tools relies on, so I need to roll out updates to every module, the core and the API.

I’ll be starting with the System Check module (to let you know if you can safely upgrade) followed by the Form Tools core.

I’ll do a follow-up post (or maybe post on twitter) with how things are coming along.



Posted in News | 4 Comments

Website update

Hi all,

Well it finally happened! The new site is up and running. Over the next few days I’ll be making a few more fixes and updates –  but really just to get it to a point where everything’s once again working as expected.

This site update was a lot more than just a superficial style update, it’s actually the culmination of a mountain of work. I’ve posted about these changes before, but I think it’s worth bring up one more time.

  • All Form Tools code now publicly available on github.
    This is a massive change. Form Tools has always been open source but it was hidden away in an SVN repository, lacking visibility. And for a project of this size and scope that’s just not good enough. So, I’ve moved over all code to github where other developers can contribute to the codebase. This is absolutely vital for pushing the project along: it can’t always just be me writing the code.
  • All documentation is now editable.
    Previously the Form Tools documentation was in a closed database that only I could edit. Now on all documentation pages you’ll see an “EDIT” button that takes you to github to fix whatever typo I created, clarify the intent of what’s being said, or make whatever other change you think it needs. The entire documentation can also now be downloaded to run on your own computer, offline.
  • The website has been completely rewired.
    Any project that’s been around as long as Form Tools has a lot of accumulated chaos and noise going on behind the drapes. This new site throws out or rewrites a lot of old code, and now connects with the new github repos. In all honestly, it’s been a long, tedious process getting to this point, so despite its imperfections, it’s really great to see the new site out. 🙂

The upgrade and custom build scripts and currently offline. I’ll be working on getting those two back up tomorrow. Shouldn’t take too long – I’ll post about it on twitter.

Exciting times! Once I’ve done a little more housekeeping I’ll be making another post about what I’m planning for the project. Keep you posted.


Posted in News | 2 Comments

New site up and running

Let me introduce you to the new site! There are still a few wrinkles to iron out, so bear with me another day or so then I’ll make a full post about it.

Posted in News | Leave a comment

Last message on the old site!

Howdy folks,

I’ll be putting up the new version of the website soon, to be found at a temporary URL where I can iron out the various bugs before going live. Part of this means copying over and updating the Form Tools website database to make it compatible with the new site – and that means halting any changes here.

So, this’ll be the last piece of news posted on the old site, and Form Tools accounts have been taken offline until the new site is released. Feel free to continue posting in the forums: I’ll tackle that separately.

Once the new site is up and running, I’ll post on the twitter account where it can be found.

See you on the other side!


Posted in News | Leave a comment

Twitter account rename

Just a quick heads up that the Form Tools twitter account has been renamed to @formtools. I’ve been posting my various progress these last few weeks. It’s coming along! 🙂

Posted in News | 1 Comment

Yet another update

Quick note. The new site is coming along – I’ve updated the documentation to use Jekyll 3.3.1 and I’ve gotten it hooked into the new site design. I’ve pared back virtually everything I can think of and the priority is to the site out as fast as possible, then iterate to improve it. Things like searching the user doc won’t be in the first release, I’m afraid – and a few other things are going to be nixed. But it’ll be worth it!

Posted in News | 10 Comments