The new blog engine

GitHub started supporting custom code for generating static pages after I started my blog in 2015, so now I have more options for a blog engine. I initially set up a Zola generator, which worked for the standard posts. Zola doesn't have any plugin support though, and I wanted to add a bookmarks page, which would use a different page generator. The links would be written down in a compact IDM document and then a JavaScript-enhanced HTML bookmark page would be generated from it. Any code dealing with IDM needs to be a custom thing written by me. The Zola workflow couldn't support this, so I ended up writing a clumsy two-phase GitHub action that generated both the Zola site and my own bespoke bookmark page.

The bookmark generator turned out to be a surprisingly simple affair using IDM deserialization, std::convert::From conversion and Askama templating. I started thinking about ditching Zola and generating the entire site from IDM using my own code.

        idm::from_str    convert::From         Template::render
 ┌──────────┐ |  ┌──────────┐ |  ┌──────────────────┐ |  ┌─────────────┐
 │ IDM text ├────► IDM data ├────►  template data   ├─╭──► static HTML │
 └──────────┘    └──────────┘    └──────────────────┘ │  └─────────────┘
                                             ┌────────┴─────────┐
                                             │ Askama templates │
                                             └──────────────────┘

An insight that helped with this was that I could print out an entire directory as an outline with file and directory names as headlines and file contents indented under the file name headline. Fixed file or directory names in this scheme correspond to struct field names in the site datatype when the whole directory is printed out as an outline and deserialized with IDM.

Using the generic directory-to-outline routine, printing the following directory tree,

site/posts/one-post.md:
  Lorem ipsum dolor sit amet,
  consectetur adipiscing elit,
site/posts/another-post.md:
  Ut enim ad minim veniam,
  quis nostrud exercitation ullamco
site/links.idm:
  https://example.com/
    :title Example bookmark

produces the following IDM outline,

posts
  one-post
    Lorem ipsum dolor sit amet,
    consectetur adipiscing elit,
  another-post
    Ut enim ad minim veniam,
    quis nostrud exercitation ullamco
links
  https://example.com/
    :title Example bookmark

which deserializes into the following Rust type (assuming suitable Blogpost and Link types),

struct Site {
  posts: IndexMap<String, Blogpost>,
  links: IndexMap<String, Link>,
}

The site directory deserializes to an initial input struct that matches the structure of the IDM data exactly. This needs to be converted into an output struct that contains derived information like lists of posts by tag and can drive Askama templates. The conversion is straightforward and is implemented with Rust's standard From trait. The HTML and Atom feed pages for the static site are then generated by rendering the Askama templates associated with the output types. The entire site generator runs mostly on type definitions and involves little actual code.

Blog posts are written in Markdown, so how does everything being in IDM work with this? IDM's indent based structure provides a convenient multi-line string escape syntax, where an entire indented region can be treated as having a different syntax up until the next dedentation. The blog post files have metadata fields in the colon-prefixed header block and the rest of the content is interpreted as Markdown text. When the deserialized input structs are converted to output structs, the Markdown text is converted to HTML with the pulldown-cmark crate.

The whole site is now being generated using this system which was first developed for just the bookmarks page. The blog generation works fine, and any new features can be easily added by extending the IDM site schema and the From-conversion methods. For example, for the project outlines post, I wanted to be able to publish a raw outline instead of a Markdown post, so I added a format field to the post metadata struct, set it to default to Markdown so none of the existing posts needed to be touched, and set it to Outline for the new post. Then I wrote a new HTML converter by hand for the outline format and I had a brand new style option for my blogposts.

The source code for the blog engine can be found in the repository of my blog.