Monthly Archives: April 2019

Android, Databases, Mongo DB, MySQL, PostgreSQL, Web Apps

The hardest task to do well in every frontend/backend system – paging

Leave a comment

What is the hardest code to write efficiently in pretty much every frontend and backend system? I think it is paging.

Nearly every system I’ve worked on as full-stack developer has had some sort of reporting whether it is searching and filtering or news-feed style item display, all of these depend on paging. Whilst simple paging is trivial to implement, making it work quickly at scale is a very tricky challenge.

A simple approach (based on SQL, but could be with any backend data store) would be something like:

items = SELECT * FROM <table> WHERE <query> ORDER BY <order> LIMIT <rows>,<offset>;
count = SELECT COUNT(*) FROM <table> WHERE <query>;

It looks simple and it works in small-scale development systems. But there a massive number of issues and inefficiencies with this approach:

  • Whilst you need to redo the items query for every page, the count query only needs to be done once for each <query>. Counting is a very expensive operation as it essentially has to fetch all the rows, unless there are some very well optimized keys on your table. This especially the case if you are running over a table larger than a few megabytes on a mobile device, so you need to cache this as much as possible.
  • Often times as the <order> logic has gotten more and more complex I’ve found myself in a situation where some rows are equal in ordering and so they can move around as you switch between pages or redo the query. Always ensure that the last item of ordering is something which is well ordered to avoid this, for example an integer primary key
  • What happens with a LIMIT/OFFSET type query if you get a row inserted before the OFFSET? All the rows shift forwards by one so when you scroll down on your infinite feed you get a duplicate, or the user goes to the next page and sees a repeat of the last entry of the previous page. What happens when a row prior to the OFFSET is deleted? A row between the pages is not seen by the user. In some databases it may be possible to use a long-lived cursor but often in stateless backends for webapps this is not possible. Wherever possible you should try to get a column on the <order> which is simply increasing or decreasing and redo the previous query with use a > against that to ensure you fetch all the rows from the previous point the user saw regardless of what changed above it. This is easy when sorting by primary key, difficult or even impossible otherwise.
  • How do you handle a laggy backend when the user presses the next button multiple times? Does it re-fire the request for the next page multiple times to the backend (consuming unnecessary resources), ignore any presses after the first (better), or does it proceed forward multiple pages, potentially running past the last page and onto blank pages? I’d usually opt for the second approach but limiting so that it always stops on the last page, but it may vary depending on usage case (would users typically want to skip multiple pages quickly, or always want to page through one-at-a-time), backend latency (ie is the data stored locally or remotely, what is the connectivity like)
  • Following on from the above, if you are firing multiple requests for pages, or for infinite scrolling how do you cope with the fact that responses can come back out-of-order? Many times I see it would simply display the content from the last page returned, but this is open to a number of bugs if you are talking about remote services over wireless
  • How do you efficiently abort in-fly calls and database queries if the user changes page – on a large system this could be a significant performance gain

There are quite a large number of optimizations that can be done with paging code, whether this is with a remote backend or locally, for example:

  • Prioritize getting the first page of data to display, say “Showing results 1-50 of …” while the count is loading
  • Track historical data on how long it takes to generate counts or paging, and especially if running single-threaded (eg on a mobile device), delay the count until the main queries have completed and there is some idle time
  • Is it possible to parallelize the two querys against the database so that the query takes half the real-time for the user? Can you use non-blocking IO to open two database connections, one for each query and return a response when both have completed? Can you run 2 separate queries one for count and one for items and update your pager and results separately? For a few rows in a test system this won’t make any difference, but over millions of rows, especially if you need an accurate count, this can save seconds of time
  • Do you actually need a totally accurate count, or can you approximate it, does your database backend support something like this for queries? This can save significant time and resources
  • Have you set up good indexes and verified that they are correctly being used at the scale that you will have in your live environment? Do you optimize well for the common use case?
  • You don’t always need to bother running a count query – if the items returned are less than the items requested you know that you have finished paging and you can automatically count the total from that. This is true whether on the first on the 100th page.
  • Fetch the number of rows that you want to display plus one, so that you know if there is a next page or not. If the number of items is less than you requested you know there won’t be a next page no matter what the total actually is
  • If running on mobile devices or html, limit the number of items that are displayed on an infinite scroller as this can be very memory intensive. Perhaps allow infinite scrolling over 500-1000 rows and then put a next button at the bottom of the page. Or you can do some trickery to replace the top items with a large blank box but this can be very complex and difficult to do well
  • Don’t display a spinner when infinite scrolling (you are starting to load more items before the user hits the bottom of the list right?) Only display the spinner after a short time period or when the user hits the very bottom of the list to give visual feedback that there are more items loading to display

If this looks like a lot to think about, I don’t disagree. Simple, buggy and inefficient paging is easy to do, but fully-optimized, highly responsive complex paging is very tricky to do well. But if you want to scale your app you need to focus on this.

ACE Editor, Javascript

Creating flexible highlight rules in Ace that change according to editor mode

Leave a comment

I’ve only recently started using the excellent Ace editor in projects, but I’m really enjoying it so far. It has very flexible and well designed custom highlighting rulesets that developers can extend to various different syntax. However it does not currently support in-editor spell-checking (I believe this functionality is planned for a release soon though – hopefully!).

One of the situations I’m using the editor in is site which has input from many different languages and character sets. As part of this I wanted to ensure that the language that a given article is claimed to be written in matches the character set of various content sections of the editor. For example if the article is in Thai, but you are using Latin script characters it should highlight them as errors. A big issue we have is that there is a lot of Cyrillic content, but as a number of characters render the same or very similarly in Cyrillic and Latin (for example Р and P), some of the users input mostly Cyrillic, but with the occasional character of Latin. This wouldn’t be a problem if we were just rendering the text, but we are also unidecoding it for searching – in this case Latin P goes to p, but the Cyrillic character Р encodes to r (as it is pronounced). This means that it throws off the searching.

I wrote a script which parses the CLDR data’s exemplarCharacters data to get the expected character sets for a language, adds a few in (as the CLDR is not totally complete unfortunately, especially for languages that use extended Cyrillic sets such as Karakalpak, lacks Traditional Mongolian entirely, and is incomplete for some ideographic languages such as Chinese). It then adds some general punctuation and other characters and generates a javascript regexp for matching characters that should not exist. For example for Armenian, the regexp is /[^\u0020-\u0022\u0025-\u0029\u002b-\u003b\u003f\u005b-\u005d\u00ab\u00b4\u00b8\u00bb\u0531-\u0556\u055a-\u055f\u0561-\u0587\u058a\u2030]/.

So far so good, but how to integrate this with Ace editor? Usually you only have a mode for Ace which specifies the language (eg PHP, HTML), but I don’t want to create a new mode for each language/script that this app wants to support.

Initially I tried basing some code on this code which adds spell checking outside of Ace, however there are a number of limitations with this approach namely it doesn’t integrate with the existing highlighting system so if you are mean to be editing eg HTML document containing only Thai characters it doesn’t know which parts are HTML and which parts should be checked, without redoing the whole highlighting run a second time. It is also needs to reprocess the entire document every change.

Digging around in the Ace source I found that you can actually pass an object into the setMode function which enables you to pass new parameters, such as the invalid-characters regexp for the current language. However the highlighting functions are usually static which means it is complex to update this on-the-fly, especially after normalizing it for something based on the Text Highlight Rules. My solution is as follows:

define('ace/mode/my_highlight_rules', ... {
    var MyHighlightRules = function(regex) {
        var rules = function () {
            this.$rules = {
                start: [ ... ],
                words: [ ... something with regex ... ],
                ...
            this.normalizeRules();
        };

        rules.metaData = { ... };

        oop.inherits(rules, TextHighlightRules);
        return rules;
    };

    exports.MyHighlightRules = MyHighlightRules;
});

define("ace/mode/my_mode", ... {
    var Mode = function (opts) {
        this.HighlightRules = MyHighlightRules(opts.regex);
        ...
    };
});

You can then just do .setMode({ path: 'ace/mode/my_mode', regex: /.../ }) changing the regex for each different language or character set that you wish to validate.