We are looking for volunteers you to test some branches of our latest optimizations experiments.
If you have a large jekyll website (1000+ pages) , help us ship Jekyll v3.8.0 by testing the latest version in your Gemfile:
gem 'jekyll", "3.8.0.pre.rc2"
To get an average build time, you can run a few builds with a simple script like:
#! /bin/bash
set -ex
bundle exec jekyll clean
for i in {1..5}
do
bundle exec jekyll build
sleep 2
done
Thank you for posting the benchmark data for your site @mmistakes!
I had no idea Jekyll would take ~130s to build a site with 1006 posts and 52 documents on a macOS…!!
(I dread to wonder how long it would’ve taken on a Windows platform… )
Couple of questions:
Are these regular build times or build times with the --verbose switch ( or verbose: true)? (--verbose switch or verbose: true in v3.8.0 adds absolute_paths to be written to the disk as well… so that’ll add a few seconds to the overall build time as well…)
How large are your static files…? That’s one area where there hasn’t been much of an optimizing because its fairly just copy-paste-to-dest-verbatim
I’m not using the --verbose switch or verbose: true, just vanilla bundle exec jekyll build. The build times on Windows are pretty close to macOS… 10-20 seconds slower.
A big part of these ~140s times is due to the Jekyll Paginate v2 plugin. It spends about 70s generating 118 tag archive pages that are all paginated. Disabling it cuts the build time in half.
I’ve also done quite a bit of testing/experimenting to lower my build times.These numbers are from awhile ago but most of it is still valid. Yes I have a fairly large amount of static image files (~1GB). As you mentioned above it’s IO intensive for Jekyll to move these from the source to _site.
What I ended up doing was reducing Jekyll to “just creating HTML” files. Everything else I use Gulp to do, which is much faster at. My build process goes something like this now
Gulp builds CSS, JS assets, puts them in a temp folder that Jekyll can’t see. This has the benefit of being able to edit them without triggering a build when developing locally.
Gulp copies only the files needed to build the actual HTML to a temp folder (_layouts, _includes, .md documents, etc.)
Jekyll builds this temp folder in HTML.
Gulp moves the built HTML to dist folder where production CSS/JS and images are.
I go into this with greater detail on my site if it’s of interest:
Thanks for the informative response.
So, the crux of the performance sink lies within a 3rd-party plugin which somehow got worse with my attempts at optimizing Jekyll.
Guess, I’ll have to dig into Jekyll’s master and see what is adding those extra 20secs (in comparison to v3.7.3)
Are you able to discern the difference (as to which file(s)) in the --profile output…?
So I ran a build with 3.7.3 and master using --profile and here’s what I got. Biggest change I notice is the post.html layout took longer with master.
The other strange thing is jekyll-paginate-v2 paginates 118 tag pages with 3.7.3, but the master branch version of Jekyll has it doing 120. I wonder if something there is adding the extra time as it seems to be creating more pages.
its more than just the pagination plugin… If you look at the time used by _includes/related.html, you’ll see that it has increased by ~12s (meaning jekyll:master probably sees more related posts, /cc @parkr )
But yes, _layouts/autopage_tags.html with jekyll:master has been used 348 times in comparison to 346 times with jekyll-3.7.3, which is probably related to https://github.com/jekyll/jekyll/commit/f8a66ca76c1dcc0ce79aed2655f5660527aa7d60 because of which two documents ignored earlier are now available, and the remaining increased build time is probably related to these two documents.
Also I’m using this “related posts” plugin…
This is an interesting development…
The following PR added related_posts to the SiteDrop so that a document’s “related-posts” are calculated only if and when its asked for in the Liquid Templates. (That’s how Liquid::Drop and its subclasses, of which is SiteDrop, works…) https://github.com/jekyll/jekyll/pull/6730/files#diff-e910c275a646d396259a5a70bd2d4552R44
While the above PR also memoized the built-in method (the use of @related_posts ||=…), making it more efficient by having to run just once for a given document irrespective of how many times the method is called…, the plugin’s method run the full code for each call to the method.
So, @toshimaru will now have to memoize his method definition for the plugin’s users to efficiently use Jekyll 3.8.0
You can get the ball rolling on this by opening an issue ticket at the plugin’s repo informing the author about the developments upstream.
Thanks to the people that took the time to share their build times.
3.8.0 has been published but feel free to continue to share benkmarks, especially if you have a very large website, and if your repo is public, share it, as we might use it for our tests.
For those who want to follow closely with the latest addition on the performance front, we suggest you to subscribe to opened PRs with optimization label.
We’re still working on improving Jekyll’s performance.
For people who suffer from slow builds, we’d be interested in testing:
refactor your includes with jekyll-include-cache when possible
use liquid-c gem to speed up Liquid parsing
use jekyll-commonmark to speed up Markdown rendering
test current jekyll master (make sure to exclude .jekyll-cache folder from Git)
Your Gemfile should therefore contain:
gem "jekyll", github: "jekyll/jekyll"
gem "liquid-c"
group :jekyll_plugins do
gem "jekyll-include-cache"
gem "jekyll-commonmark"
end
Then publish you site on a Netlify branch or any CI where you can use these gems (you can’t on GitHub Pages) and share the result.
I advised those to a friend of mine and its build went from 931s to 59s. The build outuput is exactly the same.
I tested on Kong Documentation and build shrinked from 54s to 8.3s. Didn’t test the output.
This might conflict with existing plugins, if you encounter errors, we’re interested to know which plugins would benefit a fix to use our new cache API.
Here’s some documentation on what has already changed for plugins authors:
Jekyll community,
)
<3 <3 <3