If you are pioneering, you will likely hit the wall. If you walk new paths, then your favorite search engine won’t answer your questions. Here is a small story about me trying new ways to develop web applications.
Modern front-end development is all about reactive JavaScript frameworks like React or Vue and HTTP APIs sending you JSON. Yes?
Let’s try something different!
HTML Fragments over the wire.
Instead of sending JSON to a JavaScript application running in the browser, this new method sends HTML to the client.
The simplest use-case for htmx is a page which contains three forms. If you submit one form, only this particular form should get updated. The other two forms should not be changed.
This could be implemented with the htmx attribute hx-post:
<form hx-post="/update/foo">…</form>
The http endpoint /update/foo will process the form data and then return a HTML fragment.
The html fragment would replace the <form> tag, or a different tag, if the server uses a “out of band” update.
If you want to learn more about it, then you find everything you need at: htmx.org
But this article is not about htmx, it is about pioneering. Now I will present seven steps on how I hit the wall while leaving the autopilot and trying something new.
Step1: You choose the tool
In October 2020 I heard the first time of this new way of developing web applications. After some googling and asking questions in developer groups I was suffering “analysis paralysis” because of too many options.
If you are pioneering there are no best practices. Tools like Google-trends or Stackoverflow-Tagtrend won’t help you since they only show the past, not the future.
There was no simple answer about how to ride this new wave. So I created a simple list of tools that could be used. You can find this list in this README at Github. I played around with some of these tools and gathered feedback via social network groups and forums. I chose htmx because it was simple and easy to understand.
After some weeks of meandering forward and backward, I have made a choice that I don’t regret.
Lessons learned:
- Use social networks to get new ideas.
- Be patient, and create a list or a spreadsheet to get an overview.
Step2: You use the new tool
I have chosen htmx in a hobby project which I developed for a friend. A Django-based application with PostgreSQL as a database, running on a cheap virtual private server.
I could talk for hours about the joy I feel developing a small project with this new paradigm. It is just easy and simple.
Step3: You hit the wall
For 20 years I have developed Python-based web applications. Nevertheless, I use my favorite search engine several times a day because I want to find an answer soon (without reading the whole documentation).
Haha. Here it comes. This works great if someone already stumbled upon a similar problem before, but that is not the case if you are pioneering or building highly specialized solutions.
Everything worked fine during the first days. The docs were good, and the examples worked fine.
Then I got this error:
It was too late in the evening, and my gut feeling told me that I won’t find a solution today. I googled and searched in the issue tracker of the project: This htmx error message seems to be like a ghost. Nobody has seen it before.
Step4: You reproduce it
In my case the HTML fragment which was created by the server and sent to the client looked roughly like this:
<tr>
<td>2</td><td>two</td>
</tr>
<div id="sum" hx-swap-oob="true">MAGIC</div>
I have used the htmx feature hx-swap-oob for some days. I had no clue why it failed here. A small intro to what this HTML fragment does: This HTML fragment should update a part of the current page. The <tr> should get swapped at the place where the user just clicked. The <div> from the html fragment should replace the corresponding element with id "sum" of the page in the browser. "oob" means out-of-band.
My next goal was to create a reproducible example so that I can share the issue.
Stackoverflow has a feature that is a bit like jsfiddle. You can enter HTML/CSS/JS code and execute it. This works fine for most use cases. In my use case, I need an HTTP response like above. How to create it?
The friendly guys of the htmx discord channel gave me a hint. There is a cool free service that lets you create HTML endpoints that create any HTTP responses you want: https://designer.mocky.io/. Mocky lets you create HTTP GET or POST responses. You can configure the content, the content type, and the HTTP headers. With Mocky and Stackoverflow’s simple jsfiddle feature I was able to create a reproducible example:
https://stackoverflow.com/questions/67289814/htmx-e-queryselectorall-is-not-a-function
Lessons learned:
- Reproducible examples are very helpful and you can create them for free with Stackoverflow and mocky.io
Step5: You dig into it
Of course, nobody had an answer to my question. I expected this. So some days later I had time to dig into this a bit deeper. My example used the minified version of the library, which meant the code was not easy to understand. After switching to the non-minified library, I got a stack trace that was more comprehensible:
Uncaught TypeError:eltOrSelector.querySelectorAll is not a function
at findAll (htmx.js:295)
at handleOutOfBandSwaps (htmx.js:501)
at selectAndSwap (htmx.js:712)
at doSwap (htmx.js:2284)
at handleAjaxResponse (htmx.js:2358)
at XMLHttpRequest.xhr.onload (htmx.js:2163)
After some digging around I found that the method makeFragment() of htmx uses parseFromString(). This method works fine, but not for my input:
In the above screenshot of Chromium’s devtools, you see that <tr>…</tr> was discarded.
Now we hit a very fundamental part of “html fragments over the wire”. The method DOMParser.parseFromString() validates the HTML and my input is not valid HTML, since a <tr> tag must be below a <table> tag. Up to now HTMX looks at the first tag of the HTML fragment and wraps the whole fragment accordingly. This works for most cases, but fails if the out-of-band element is of a different type.
Step6: You solve it
Since this is a hobby project, I mostly code on weekends. This has the benefit that you have several days between each coding session, which gives you a fresh look if you start again.
So the work-around is obvious. If I use the same type of element for the out-of-band data, then the wrapping which gets done by htmx works. This html-fragments is parsable by htmx:
<tr>
<td>2</td><td>two</td>
</tr>
<tr id=”sum” hx-swap-oob=”true”>
<td>MAGIC</td>
</tr>
I just need to use the same type of tag in the second tag.
Lessons learned:
- devtools are great. Even if you dislike JavaScript, it is easy to use. You can set breakpoints for the debugger and evaluate snippets on the console. It is like an IDE integrated into your browser.
- Taking breaks helps.
- “HTML fragments over the wire” is cool, but you need to ensure that your HTML fragments are parsable by your library.
Step7: You proceed
I was happy that I found a work-around. New developers running into this issue will find an answer soon if they type the error message into their favorite search engine. This gives me a warm fuzzy feeling inside of having accomplished something which helps other people.
In the long run, it would be nice if htmx could parse any html fragment. Instead of the current implementation. Maybe wrapping the fragment in a <template> tag might solve it. But that’s a different question which I will dig into later. See Issue #469.
Lessons learned:
- Pioneering is fun, but you should learn to help yourself if you get strange errors.
- Starting from scratch feels like flying, even if you hit the wall from time to time.
- htmx is cool, and I will use it in new projects.
- Like the skateboard in the above image: If you hit the wall, have fun and ride it.
