Specifying/understanding the rendering order for content

1

I’m trying to figure out the right way to construct a new project. I’ll ask a general question and get to the specifics a bit later.

The general question: How do I know the order in which pages are rendered and is there a way that I can ensure that certain pages are rendered before others?

The details. I have about 100 Markdown files in _posts that are rendered by a custom layout that pulls in some externally-generated text. The lunr.js plugin seems to run prior to these pages are completely rendered, so the search index does not have the external text. This text does appear in the final site. Is there a way to specify that lunr.js runs after the other pages are fully rendered so that I can get this text in the search index?

Background. I’m not a Ruby/JS pro, but I’ve been building Jekyll sites since the early 2010’s for work and fun. Everything the last 5+ years is done with a fork of minimal-mistakes that I maintain and I merge upstream changes. I use the local-theme plugin for developing with the theme as a git submodule in _themes. For production, I have the typical github build/deploy action with the remote-theme plugin pointing to my fork.

I know I can go to a more powerful search engine, use google, algolia, etc. The Algolia plugin runs post-build and that’s nice, but it’s not going to be an option for the contract that I am working on. There isn’t a great deal of text, so lunr will be fine. Also, it’s pretty obvious that Algolia isn’t supporting the tools for jekyll and their community support is pretty much gone now. Google may eventually be an option, but currently the site is behind a firewall and may or may not see the light of day for a public scan of the content. I’m open to other lightweight search options that don’t involve an enterprise contract.

Thanks!

2

I believe this specific to minimal-mistakes, so my apologies for asking here. MM is a popular theme, so I hope you’ll forgive me for asking.

I’ve been looking through the output with --verbose and it seems the lunr code renders after my posts are rendered. When lunr-store.js runs, it outputs {{ doc.content }}. I’m guessing this is the literal content of the .md file after the front matter and excerpt, rather than its rendered output. I’ll look to see if there’s another value that has the rendered content.

I can change the external process that puts this searchable text in side-car files into the actual .md, but that’s more messy than I would prefer. I preferred to have a custom page layout that has an {% include_relative foo.txt %}, but I can deal with having the search text as part of the .md. These pages will get updated at most once or twice per year.

Looks like this is where I need to go down the rabbit hole of plugins and pre-render hooks. This isn’t going on Github pages so I think I can use a plugin, but will need to be built using GH actions and deployed to AWS as a static site.

3

I’m going to mark this as solved, but I’ll summarize. It’s not a general-case scenario, so not something for the core theme, but might be useful for the next person wanting to index content for a page that isn’t in the original markdown.

  • I have a yaml value in some .md files in _posts that point to a sidecar file. These sidecar files are located in /project/_includes.

  • In /project/_themes/minimal-mistakes/assets/js/lunr/lunr-store.js, I added a test for my yaml value at the statement that generates the excerpt: json value (on, or about line 19). It goes something like this…

"excerpt": 
  {%- if doc.search_text -%}
    {% capture search_text %}{% include {{ doc.search_text }} %}{% endcapture %}
    {{ search_text | strip_newlines | jsonify }},
  {%- else -%}
            {%- if site.search_full_content == true -%}
              <!-- existing code -->
            {%- endif -%}
  {%- endif -%}

The rendered output file does not contain the text, which is fine. If I wanted to include the text on the rendered page, I’d use that same include mechanism in the layout generator.

As an FYI, the use case is government documentation originally in paper form. The pages link to PDFs in a cloud repository that were scanned as part of another contract. There’s a requirement that these documents be available for public view for a certain number of years. The site I’m building is a search/browse front-end to the documents along with follow-up and remedial actions taken by external contractors. I don’t need to include 40GB of PDFs in my project just to build a pleasing front-end for searching and browsing.