Released v5.3 of the WordPress plugin

A few days ago I released an update for the WordPress plugin. The updated introduces a couple bugfixes and a few enhancements. I hadn’t done an update for maybe a year, and in the meantime WordPress has continued to evolve. For example AJAX technology has been introduced to the installation / deinstallation process of plugins. Which means that you can install or deinstall plugins directly from the page of the plugins manager dynamically without a page reload or without going to a different page. However this also meant that plugins needed to change some of their internal code to deal with this, so an update had become necessary because otherwise the deinstallation process could have thrown an error, and it would have become impossible to deinstall without forcefully doing so from an ftp connection, which is not the best experience for a site admin. With the last updates of the plugin this has been fixed. String translations in Italian, Spanish, French and German have also been updated.

And a new function has been introduced, that gives a wider choice of fonts if anyone should want more specialized typographic layouts for their website / blog. In particular I have added an option for a Google Fonts API key. Upon activating the key, the BibleGet styling section in the WordPress Customizer will then have an updated list of the fonts that are available on the Google Fonts website. And not only: previews of the fonts will be downloaded so that you can preview them in the select list just in case you’re not yet sure which one to choose from.

I must say I am quite proud of the results. It wasn’t exactly an easy thing to implement. And I’m not even sure if anyone else has done this yet, because you have to be very careful about trying to load a lot of fonts on a page. It would easily have a negative impact on the loading performance of the WordPress Customizer. I wanted to attempt it by finding a way to keep the performance hit to a minimum. And I did so by having the plugin download a minimal version of these same fonts with select characters, instead of downloading the full version of each font with all the characters from A to Z. Specifically only the characters that make up the name of the font will be downloaded for that font.

Google introduced a while back the possiblity to request only the necessary characters for any given font so as to load a minimal version of the font on a page. And in fact we were already using this functionality in the last few versions of the plugin with a fixed list of about 100 Google Fonts to choose from. This minimal version of each Google Font can be obtained when requested via a “<link>” element, for example:

<link href="https://fonts.googleapis.com/css2?family={$familyname}&text={$familyname}" />

This will download a stylesheet from Google Fonts which loads the minimal version of the font from a special url, with this minimal version containing only the letters or characters that are contained in the name of the font itself. A sample stylesheet loaded for the font Aclonica:

@font-face {
  font-family: 'Aclonica';
  font-style: normal;
  font-weight: 400;
  src: url(https://fonts.gstatic.com/l/font?kit=K2FyfZJVlfNNSEBXGb7WEI3hZkmbCQXi&skey=a69e8e3b9073e390&v=v11) format('woff2');
}

Now however there would be some non trivial difficulties to overcome if we want to have the whole updated list of about 1000 fonts. In order to preview the fonts in the select list, the WordPress Customizer would have had to insert about 1000 <link> elements on the page and the browser would have had to make about 1000 requests all at once to the Google Fonts API every time you would open the Customizer. However that would have simply choked the page, because web pages are not able to handle that many requests and the browser would simply block them.

So we had to find a way to download the previews locally to the website itself. But that posed another probem: as long as a web page requests a Google Font from a css “<link>” element effectively loading a stylesheet from Google, it is possible to request the minimal version of that font. But if it’s the web SERVER that makes the request (e.g. via a cURL request) in order to directly download a copy of the font file and save it locally, there is no clear way of requesting the minimal version of the font. All requests to https://fonts.googleapis.com/css or https://fonts.googleapis.com/css2 will ever only produce a stylesheet, not a font file, and https://fonts.google.com/download?family=Aclonica will produce a zip file. Any attempt to save the actual font file locally on the web server would seem to obtain the full version of the font file with all the characters. And we’d still have the problem of page loading performance.

Well, I did succeed in finding a way to request the minimal version of each font from Google. Practically when the minimal version of the font is requested via the link method, Google produces a stylesheet and in that stylesheet there is a URL that points to a file with a reduced version of the font. So I just needed to first make the request and get the stylesheet, then parse that stylesheet and capture the URL in it and use that URL to download the minimal version of the font.

So basically as a first step, for each font name in the list of Google Fonts, we check if we already have a local copy of the stylesheet associated with the minimal version of the font, and if not we obtain the stylesheet via a curl request:

$ch = curl_init("https://fonts.googleapis.com/css2?family={$familyname}&text={$familyname}");
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, TRUE);
curl_setopt($ch, CURLOPT_SSL_VERIFYHOST, 2);
curl_setopt($ch, CURLOPT_SSLVERSION, CURL_SSLVERSION_TLSv1_2);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
$response = curl_exec($ch);

The very first time, this will perform about 1000 cURL requests! But once it’s done, the information is saved to local files so it will no longer be requested unless the file is missing. Well actually, we are doing one more cURL request for each one of these: instead of saving the actual stylesheet to a file, we are parsing the stylesheet to extract that special URL which will allow us to obtain the actual reduced version of the font, and then we are making ANOTHER cURL request to download the actual minimal version of the font. Something like this:

if (preg_match('/url\((.*?)\)/', $response, $match) == 1) {
	$thisfonturl = $match[1]; //this will look something like this: https://fonts.gstatic.com/l/font?kit=K2FyfZJVlfNNSEBXGb7WEI3hZkmbCQXi&skey=a69e8e3b9073e390&v=v11
	$ch2 = curl_init($thisfonturl);
}

And yet it’s not that easy (if that were easy). Google produces the reduced version of the font ONLY when it’s a modern web browser requesting the file (for example when the browser interprets the stylesheet with the URL that points to Google’s servers). And by that we mean a browser that can handle compressed fonts. Google can tell when a browser can handle compressed fonts and only in that case does it actually produce the reduced version of the font file! So a simple cURL request like this one will still not succeed in obtaining the minimal version of the font, because Google thinks it can’t handle compressed “woff” fonts.

Google in fact introduced a new compression for font files in an attempt to overcome the problems inherent with page loading performance when it comes to introducing more fonts on a page. These compressed fonts have a “woff” extension (“Web Open Font Format“) instead of a “ttf” extension (“True Type Font” or “Open Type Font“). But this technology is still very new and operating systems can not yet handle this kind of font, only the more modern browsers can because it’s more of a browser and web page performance problem. So if it’s a server that’s making the request (e.g. via a cURL request) and not a browser, Google will decide to send the full uncompressed version of the font file and not the reduced version that was originally requested!

At this point it might have seemed that this was an impossible task, and in fact I was almost going to give up on this quest (and I’ll bet there are a number of web programmers who probably would have discouraged me from persevering along this path because they are all afraid of the consequences in page loading performance). And yet, I did find a way to get meself the reduced version of the font by simply shouting “hey, I know I’m not a browser, but heck I can still handle compressed fonts! Trust me!”. And lo and behold, I found that by doing so Google gave in and sent me the minimal version of the font, with just the requested characters! The font file received was not however in WOFF format but rather in TTF format, which is actually not so much a problem because perhaps not all broswers are yet capable of dealing with the WOFF format.

curl_setopt($ch2, CURLOPT_HTTPHEADER, array("Accept: font/woff2", "Content-type: font/ttf"));

However there is another problem, this time with the capabilities of the web server. Making about 1000 cURL requests that obtain stylesheets, parsing those stylesheets to get the special URL, and then making another 1000 or so requests for each of those URLs in order to obtain the actual font files, will probably result in a server timeout. A server script usually has a limited amount of time to perform a task, and this task is starting to become long and intensive. I solved this by breaking the whole process into chunks, and processing about 300 stylesheets and font files at a time. So PHP is doing the actual cURL requests, but Javascript is orchestrating the process making about 4 consecutive AJAX requests to the PHP script.

var startIdx = 0;       //starting index for this batch run
var gfontsCount = gfontsBatch.job.gfontsWeblist.items.length;
var batchLimit = 300;   //general batch limit for each run, so that we don't block the server but yet we try to do a good number if we can
var lastBatchLimit = 0; //if we have a remainder from the full batches, this will be the batchLimit for that remainder
var numRuns = 0;        //we'll set this in a minute
var currentRun = 1;     //of course we start from 1, the first run
if(gfontsCount % batchLimit == 0){
    numRuns = (gfontsCount / batchLimit);
}
else if((gfontsCount % batchLimit) > 0){
    numRuns = Math.floor(gfontsCount / batchLimit) + 1;
    lastBatchLimit = (gfontsCount % batchLimit);
}
//then we have a function that will handle the ajax requests based on the current state of the batch requests: when one batch completes successfully it will issue the ajax request for the next batch
//we even have a progress bar that tracks the progress for each one of these batches, so the end user has some kind of visual indication that something is happening

The next obstacle that needed to be overcome: in order to load a preview of each font in the WordPress Customizer we would still have to load them through stylesheets. But if we have one stylesheet for each font, that would mean loading about 1000 stylesheets. Which even if they’re requesting local resources, is still just too much for a browser to handle. So one of the last steps was to simply combine all those stylesheets into one larger stylesheet to be a little bit kinder to our browsers.

//LAST STEP IS TO MINIFY ALL OF THE CSS FILES INTO ONE SINGLE FILE
$cssdirectory = WP_PLUGIN_DIR . "/bibleget-io/css/gfonts_preview";
if (!file_exists($cssdirectory . "/gfonts_preview.css")) {
	$cssfiles = array_diff(scandir($cssdirectory), array('..', '.'));

	$minifier = new MatthiasMullie\Minify\CSS($cssdirectory . "/" . (array_shift($cssfiles)));
	while (count($cssfiles) > 0) {
		$minifier->add($cssdirectory . "/" . (array_shift($cssfiles)));
	}
	$minifier->minify($cssdirectory . "/gfonts_preview.css");
}

But even that didn’t quite solve everything. A browser doesn’t actually load a font on a page until it really needs to, when it actually has to render it upon coming into view. But if the font previews are initially hidden inside of a select list, the browser is going to think “hey, I don’t need to load these right away, nobody is looking at them just yet, let’s hold off on this; if someone decides to open that select, well then I should start to load and render those font previews”. But guess what, that means the browser would have to load and render 1000 font previews at a moment’s notice when you do open that select list. That would be quite a processor intensive process that would pretty much block the page for a bit while until the process had finished. The select list would seem apparently empty when opening it, and maybe only a half-minute later would you actually see all the font previews. Well that’s not too cool, and if that were the only way it wouldn’t be worth it, it would be a pretty terrible experience and most people would complain about it. So one last touch to make this work was to try to twist the browser’s hand into preloading the fonts even though they’re not immediately on the screen. I really don’t know if all browsers do listen to that kind of preloading instruction however, so I cannot guarantee that everyone will have a smooth experience with this. With my own trial and error using the latest Chrome version 80.0.3987.163 (oops it’s updating as I’m writing this) and a fairly performant HP laptop, I found the experience to be quite acceptable. So I decided to go for it and add this option to the latest update. If however some of you do see the FOUC (“Flash of Unstyled Content”) and are not happy with this, I would suggest not to activate the Google Fonts API key. Other than that, go and delve yourselves into interminable fonts to your heart’s content!

In the next update that I’m planning, I would like to implement a Gutenberg block, seeing that the WordPress pages editor has been evolving quite a bit and has started implementing this style of building pages with blocks which effectively would allow to eliminate shortcodes completely. Using shortcodes takes a little bit of technical ability whereas the block editor has a more intuitive interface that pretty much anyone can learn to use. Gutenberg, here we come!

• ← LibreOffice extension v2.8 released
• Add-on for Google Docs republished in the GSuite Marketplace →