Text: A Love/Hate Relationship

Having just wrapped up development of MarkdownKit, my CommonMark compliant Markdown parser for Xojo, I wanted to write down a few thoughts I have on the current state of cross-platform text handling in Xojo.

If you are about to start a new project with Xojo and hope to reuse some of your logic code on both iOS and desktop/web/console then you’re going to have to make some compromises. As it currently stands, Xojo does not live up to its promise of being a cross-platform development tool when it comes to manipulating text. This is a big deal as, unsurprisingly, manipulating text is a pretty common task for an app to have to do.

The Xojo Text class is a “better” data type for working with text than the String data type. It’s better in many ways that have been written about before, principally it’s handling of multibyte characters and encodings. It works OK on macOS and its the only option on iOS. The issue is that it’s just too slow on Windows and Linux.

Let’s take MarkdownKit as an example. When I set out to write MarkdownKit, I decided to use Text so that it would run on iOS as this was an important use case for me with the module. It’s worth nothing that I developed it on a Mac. Once MarkdownKit was passing its test suite, I decided to write a demo app to showcase it. Afterwards, I started testing the demo app on Windows and Linux. To my horror, the demo app would either hang/crash completely if you pasted in a large document or was just generally far slower at parsing when it came to smaller documents. This was in stark contrast to the demo running on macOS where everything was lightning fast and buttery smooth. Some disparity between platforms is OK but we’re talking about a process taking a few milliseconds on a Mac and several minutes on Linux and Windows. This is completely unacceptable.

After digging around in the debugger and reading several forum posts about similar issues it became very apparent that the Text data type is just an order (or two) of magnitude slower than using String.

Xojo announced at XDC 2018 that big changes were coming to the Xojo framework with “API 2.0”. They have stated that Text is officially deprecated. Here we are in mid-2019 and API 2.0 is still not even in alpha. I probed the engineers a couple of months ago at XDC 2019 about the replacement for Text on iOS but I wasn’t really satisfied with their answers. String will be the replacement for Text on all platforms and apparently they will add new methods to this type to bring it up to parity with the current String type.

One of the difficulties of migrating away from Text to String for existing projects is the character offset. String is 1-based and `Text` is 0-based. This leads to all sort of pain-the-arse bugs as it’s very easy to forget to change an offset on a single line of code in a big project. There are other little gotchas too, like differences in the behaviour of Split() that can lead to hard to track down bugs. I know this from first hand experience as I had no choice but to port/rewrite MarkdownKit using the String type in order to get great performance on Linux and Windows. Not only does this add to my development burden (as I now need to maintain two similar but different modules - one for iOS and one for all the other platforms) but it adds a level of complexity and confusion to my users.

What do I want from Xojo then? To be honest, I just wish they would knuckle down and put some effort into iOS. It’s all well and good banging on about Android and Web 2.0 but their existing platforms are not being serviced adequately. It is currently impossible to take code that manipulates text and is high performance on Windows and get it to run on iOS without significant re-engineering. Frankly, that’s not good enough.

XDC 2019