13 seconds build time per page

Help
1

Hey!

I am complete beginner to Jekyll and I have been tasked to create documentation for our big C# Framework. We decided to go for Just The Docs (GitHub - just-the-docs/just-the-docs: A modern, high customizable, responsive Jekyll theme for documentation with built-in search.) theme as built-in search was a must and it looked very minimalistic and nice.

I went to writing articles and everything was working fine - until I wrote a script that generates markdown files from our library. I now have over 2000 markdown files of a long documentation with a lot of nesting and references to each other. (Think Enterprise size library). I checked how long does it take for one page to load and it’s about 13.5 seconds. If you do the math it will take me few hours to build the whole documentation.

Our old solution using DocFX built time as a whole was around 7 minutes. There has to be a way to make it at least come a bit close to that I suppose.

So I have few questions:

  1. Is there a way to decrease this build time?
  2. Is there a way to cache built site so the next built just builds the changed files?
  3. What information should I provide you with regarding my project?
2

I checked how long does it take for one page to load and it’s about 13.5 seconds.
To ensure I fully understand the issue, you are not saying it takes 13.5 seconds per page to load but 13.5 seconds to build, correct?

I’ll assume that is what you mean for the time being.

There are many reasons it takes a long time for your site to build, but as you say, this does seem quite lengthy. There are thousands of themes for Jekyll. I do not know Just The Docs, but I have heard of others using it.

:link: Share your repo, if possible

The best and easiest way for any of us to help you here is to share the repo if it is public so we can take a look at it, maybe clone it and see what build issues there may be.

:stop_sign: Temporarily disable search functionality

The first thing I might do is look at your _config.yml file. I looked at the Just The Docs file, which contains many configurations. While I am not saying “do not use this feature,” you might be able to debug at least what is causing the slowness. To start, I would change search_enabled: true to search_enabled: false and rebuild the site to see if there is a noticeable impact.

:person_raising_hand: Questions

  • What version of Jekyll are you using?
  • What version of Ruby are you using?
  • Are you using any plugins?
  • Are you using incremental builds?
3

you could try doing jekyll b --profile and it should show you some info on what is taking so long.

There is also an incremental command to build just the pages that changed but it is still experimental and may not work properly in all cases.

4

Hi,
13-seconds per page is definitely appalling. Are you by any chance using Jekyll 3.9 or older? If yes, could you try with Jekyll 4?
However, if you’re using Jekyll 4.3.1, try updating to v4.3.2. If it’s still this slow with v4.3.2, I’d definitely like to clone your repository and see what’s the reason.

A

5

Thank you all for answers.

I am actually being serious when I say ~13s per page (post? / markdown file). So the build would be in hours.

Jekyll version: 4.2.2
Ruby version: 3.1.2p20
Plugins: jekyll-feed
Incremental builds: I have not been using incremental build - will start to do that from now.

I have tried turning off the searching options but there was no difference that I could notice.

I am building my project with: bundle exec jekyll serve --verbose --livereload --incremental
Reading of files happens fast but rendering is what is slowing the process down.
When running the above mentioned command below lines last around ~13s, then it moved to next file.

Rendering: docs/(FRAMEWORK)-api/Generated/(REDACTED).(REDACTED_OTHER_FRAMEWORK).BLEServices/ASHAService/ASHAAudioStatus.md
Pre-Render Hooks: docs/(FRAMEWORK)-api/Generated/(REDACTED).(REDACTED_OTHER_FRAMEWORK).BLEServices/ASHAService/ASHAAudioStatus.md
Rendering Markup: docs/(FRAMEWORK)-api/Generated/(REDACTED).(REDACTED_OTHER_FRAMEWORK).BLEServices/ASHAService/ASHAAudioStatus.md
Post-Convert Hooks: docs/(FRAMEWORK)-api/Generated/(REDACTED).(REDACTED_OTHER_FRAMEWORK).BLEServices/ASHAService/ASHAAudioStatus.md
Rendering Layout: docs/(FRAMEWORK)-api/Generated/(REDACTED).(REDACTED_OTHER_FRAMEWORK).BLEServices/ASHAService/ASHAAudioStatus.md

I am providing you a repository.
Natromitus/jekyll-doc-tsl2 (github.com)

I had to do few things so I won’t get into trouble:

  1. I had to do some patchwork censoring so it isn’t plainly obvious what the frameworks are.
  2. I had to delete a lot of files from the repository - resulting now in much faster build.
    • articles disclosing internal documentation
    • around 2000 markdown files generated from Framework used for development
    • images

I noticed that after deleting a lot of pages a render time per markdown file went down by half. Less markdown files there are faster the time is per markdown file.

6

Here you can see the issue: it iterates over all your content in all your pages. It probably does that also for the search, but I did not look into that.

{% include nav.html pages=collection key=collection_key %}

The include nav.html is used in the sidebar.

The code is not written with performance in mind, as you can see above. Some very poor choices have been made in building this theme. Having frontmatter like this is terrible from a maintenance POV. The performance of this theme could be fixed by having an autocollapsing menu that gets its active state from javascript. In that case it can be generated once instead of 2000 times (the amount of pages you have).

---
layout: default
title: (REDACTED).(REDACTED_OTHER_FRAMEWORK)
grand_parent: 
parent: (REDACTED_FRAMEWORK) API
has_children: True
permalink: /(REDACTED_FRAMEWORK)-api/ReSoundSETTestwareTestCaseActionsRatatoskAssembly
---

Also, I would not use Jekyll, but Hugo for this purpose (generating a big documentation website). If you want me to port it for you, contact me on joost@vdschee.nl.

2 Likes
7

Thank you for providing a link to a public repository, @Natro. However, I was unable to find a file with ASHAService in its relative_path. I’m guessing that it probably got thrown out during the censorship.

Since it has been pointed out that there are bits of non-performant bits in the theme, you’re either better off creating a custom theme from scratch or jump the ship and switch to another static-site-generator such as Hugo which is known for its fast build times.

1 Like
8

Yes most likely got disposed during censorship, sorry about it :frowning:

Ah, that’s a shame I really liked the theme and the documentation was almost done. I guess I am jumping ship.

Thank you very much guys.

9

@Natro thanks for the link to the cut-down repository. You wrote:

I checked how long does it take for one page to load and it’s about 13.5 seconds.

I noticed that after deleting a lot of pages a render time per markdown file went down by half.

I’ve profiled the provided repo (running on a MacBook Air M2):

bundle install
bundle exec jekyll build --profile

It took 420 seconds. The repo has 559 markdown files with front matter, so that was under one second per page. Perhaps you were running on a significantly slower platform?

Each page includes the entire navigation hierarchy, so when you double the number of pages, the size of the HTML for the navigation on each page also doubles. I guess your original 2500+ pages would take at least an hour on my platform, which is indeed unacceptably slow.

However, I noticed that your Gemfile.lock includes the line:

    just-the-docs (0.3.3)

That version of the theme was released in October 2020 (see the theme repo at GitHub - just-the-docs/just-the-docs: A modern, high customizable, responsive Jekyll theme for documentation with built-in search.). A significantly improved version (0.4.0) was released on Feb 6. That was before your initial post – but you’re using the theme gem, so you don’t get new releases unless you run bundle update (or delete Gemfile.lock).[1]

bundle update
bundle exec jekyll build --profile

Now building your 559 pages takes 42 seconds on my platform: a 10x speedup! Perhaps I’d build your original 2500+ pages in about 10 minutes.

The code shown by @jhvanderschee is from v0.4.0 of the theme. It was written with performance somewhat more in mind than the corresponding code in v0.3.3 was. However, to provide acceptable speed for building sites with thousands of pages, the navigation would surely need to be generated just once, as @jhvanderschee suggested.

BTW, I don’t understand @jhvanderschee’s remark about front matter (which is presumably referring to @Natro’s repo, not to the theme code). It doesn’t seem at all relevant to the slowness of the build.

I’m surprised to see one of the main Jekyll developers suggesting to create a custom theme, or switch to a competing SSG, for building a site with a few thousand pages! Isn’t Jekyll normally used to build much larger sites with the existing themes?

  1. Using remote_theme: just-the-docs/just-the-docs instead of theme: just-the-docs ensures that you always get the latest theme updates without updating the bundle. ↩︎

1 Like
10

Jekyll can be used to generate a large site performantly, especially with Jekyll 4.x, but it cannot be as fast as an SSG written using Golang or Python 3, etc Moreover, performance in Jekyll is subject to how Liquid code is written.

11

The front matter has parent and grand parent and children variables. What if you want to move something, or a add a child? Then you need to update quite a lot.

12

Presumably the front matter of all those files is generated too. If you move something, you simply regenerate the files, so there’s never any need to update the front matter manually.

Just the Docs is currently limited to 3-level navigation, but we’re aiming to support unlimited depth, specifying only the parent of each node. The current need to specify also the grandparent and children variables can indeed make maintenance tedious.

13

I am not sure if that is the right assumption. I am no expert in this market (segment). You are probably right.