If you are pioneering, you will likely hit the wall. If you walk new paths, then your favorite search engine won’t answer your questions. Here is a small story about me trying new ways to develop web applications.
Modern front-end development is all about reactive JavaScript frameworks like React or Vue and HTTP APIs sending you JSON. Yes?
Let’s try something different!
HTML Fragments over the wire.
Instead of sending JSON to a JavaScript application running in the browser, this new method sends HTML to the client.
The simplest use-case for htmx is a page which contains three forms. If you submit one form, only this particular form should get updated. The other two forms should not be changed.
This could be implemented with the htmx attribute hx-post
:
<form hx-post="/update/foo">…</form>
The http endpoint /update/foo
will process the form data and then return a HTML fragment.
The html fragment would replace the <form>
tag, or a different tag, if the server uses a “out of band” update.
If you want to learn more about it, then you find everything you need at: htmx.org
But this article is not about htmx, it is about pioneering. Now I will present seven steps on how I hit the wall while leaving the autopilot and trying something new.
In October 2020 I heard the first time of this new way of developing web applications. After some googling and asking questions in developer groups I was suffering “analysis paralysis” because of too many options.
If you are pioneering there are no best practices. Tools like Google-trends or Stackoverflow-Tagtrend won’t help you since they only show the past, not the future.
There was no simple answer about how to ride this new wave. So I created a simple list of tools that could be used. You can find this list in this README at Github. I played around with some of these tools and gathered feedback via social network groups and forums. I chose htmx because it was simple and easy to understand.
After some weeks of meandering forward and backward, I have made a choice that I don’t regret.
Lessons learned:
I have chosen htmx in a hobby project which I developed for a friend. A Django-based application with PostgreSQL as a database, running on a cheap virtual private server.
I could talk for hours about the joy I feel developing a small project with this new paradigm. It is just easy and simple.
For 20 years I have developed Python-based web applications. Nevertheless, I use my favorite search engine several times a day because I want to find an answer soon (without reading the whole documentation).
Haha. Here it comes. This works great if someone already stumbled upon a similar problem before, but that is not the case if you are pioneering or building highly specialized solutions.
Everything worked fine during the first days. The docs were good, and the examples worked fine.
Then I got this error:
It was too late in the evening, and my gut feeling told me that I won’t find a solution today. I googled and searched in the issue tracker of the project: This htmx error message seems to be like a ghost. Nobody has seen it before.
In my case the HTML fragment which was created by the server and sent to the client looked roughly like this:
<tr>
<td>2</td><td>two</td>
</tr>
<div id="sum" hx-swap-oob="true">MAGIC</div>
I have used the htmx feature hx-swap-oob for some days. I had no clue why it failed here. A small intro to what this HTML fragment does: This HTML fragment should update a part of the current page. The <tr>
should get swapped at the place where the user just clicked. The <div>
from the html fragment should replace the corresponding element with id "sum"
of the page in the browser. "oob"
means out-of-band.
My next goal was to create a reproducible example so that I can share the issue.
Stackoverflow has a feature that is a bit like jsfiddle. You can enter HTML/CSS/JS code and execute it. This works fine for most use cases. In my use case, I need an HTTP response like above. How to create it?
The friendly guys of the htmx discord channel gave me a hint. There is a cool free service that lets you create HTML endpoints that create any HTTP responses you want: https://designer.mocky.io/. Mocky lets you create HTTP GET or POST responses. You can configure the content, the content type, and the HTTP headers. With Mocky and Stackoverflow’s simple jsfiddle feature I was able to create a reproducible example:
https://stackoverflow.com/questions/67289814/htmx-e-queryselectorall-is-not-a-function
Lessons learned:
Of course, nobody had an answer to my question. I expected this. So some days later I had time to dig into this a bit deeper. My example used the minified version of the library, which meant the code was not easy to understand. After switching to the non-minified library, I got a stack trace that was more comprehensible:
Uncaught TypeError:eltOrSelector.querySelectorAll is not a function
at findAll (htmx.js:295)
at handleOutOfBandSwaps (htmx.js:501)
at selectAndSwap (htmx.js:712)
at doSwap (htmx.js:2284)
at handleAjaxResponse (htmx.js:2358)
at XMLHttpRequest.xhr.onload (htmx.js:2163)
After some digging around I found that the method makeFragment()
of htmx uses parseFromString()
. This method works fine, but not for my input:
In the above screenshot of Chromium’s devtools, you see that <tr>…</tr>
was discarded.
Now we hit a very fundamental part of “html fragments over the wire”. The method DOMParser.parseFromString()
validates the HTML and my input is not valid HTML, since a <tr>
tag must be below a <table>
tag. Up to now HTMX looks at the first tag of the HTML fragment and wraps the whole fragment accordingly. This works for most cases, but fails if the out-of-band element is of a different type.
Since this is a hobby project, I mostly code on weekends. This has the benefit that you have several days between each coding session, which gives you a fresh look if you start again.
So the work-around is obvious. If I use the same type of element for the out-of-band data, then the wrapping which gets done by htmx works. This html-fragments is parsable by htmx:
<tr>
<td>2</td><td>two</td>
</tr>
<tr id=”sum” hx-swap-oob=”true”>
<td>MAGIC</td>
</tr>
I just need to use the same type of tag in the second tag.
Lessons learned:
I was happy that I found a work-around. New developers running into this issue will find an answer soon if they type the error message into their favorite search engine. This gives me a warm fuzzy feeling inside of having accomplished something which helps other people.
In the long run, it would be nice if htmx could parse any html fragment. Instead of the current implementation. Maybe wrapping the fragment in a <template>
tag might solve it. But that’s a different question which I will dig into later. See Issue #469.
Lessons learned: