Help us benchmark Jekyll


#1

:wave: Jekyll community,

We are looking for volunteers you to test some branches of our latest optimizations experiments.
If you have a large jekyll website (1000+ pages) , help us ship Jekyll v3.8.0 by testing the latest version in your Gemfile:

gem 'jekyll", "3.8.0.pre.rc2"

To get an average build time, you can run a few builds with a simple script like:

#! /bin/bash
set -ex
bundle exec jekyll clean

for i in {1..5}
do
   bundle exec jekyll build
   sleep 2
done

Please share your results, for instance like:

Build times

3.7.3 3.8.0.pre.rc-2
13.2 13.2
12.711 11.042
14.993 11.022
15.474 11.144
13.51 11.54

Built on:

  • macOS 10.13.4
  • ruby 2.6.0preview1

Bonus: If you can generate a flamegraph with rbspy, it’s even better!

rbspy record -- bundle exec jekyll build
(on macOS run the command with sudo)

Thanks for the feedback you can provide. :back:


#2
  • Repo: https://github.com/mmistakes/made-mistakes-jekyll
  • Posts: 1006
  • Collections: 4 / Documents: 52
  • Plugins: jekyll-sitemap, jemoji, jekyll-seo-tag, jekyll-paginate-v2, jekyll-typogrify, jekyll-figure, jekyll-algolia, jekyll-tagging-related_posts

Built on:

  • macOS 10.13.3
  • ruby 2.4.1p111 (2017-03-22 revision 58053) [x86_64-darwin17]

3.7.3 master 3.8.0-alpha-2 optima/benchmark
129.07 140.442 143.011 146.495
129.117 142.169 145.258 143.583
130.617 141.097 143.603 142.024
127.591 141.658 146.271 144.834

#3

Thank you for posting the benchmark data for your site @mmistakes!
I had no idea Jekyll would take ~130s to build a site with 1006 posts and 52 documents on a macOS…!!
(I dread to wonder how long it would’ve taken on a Windows platform… :sweat:)

Couple of questions:

  • Are these regular build times or build times with the --verbose switch ( or verbose: true)? (--verbose switch or verbose: true in v3.8.0 adds absolute_paths to be written to the disk as well… so that’ll add a few seconds to the overall build time as well…)
  • How large are your static files…? That’s one area where there hasn’t been much of an optimizing because its fairly just copy-paste-to-dest-verbatim

#4

No problem.

I’m not using the --verbose switch or verbose: true, just vanilla bundle exec jekyll build. The build times on Windows are pretty close to macOS… 10-20 seconds slower.

A big part of these ~140s times is due to the Jekyll Paginate v2 plugin. It spends about 70s generating 118 tag archive pages that are all paginated. Disabling it cuts the build time in half.

I’ve also done quite a bit of testing/experimenting to lower my build times.These numbers are from awhile ago but most of it is still valid. Yes I have a fairly large amount of static image files (~1GB). As you mentioned above it’s IO intensive for Jekyll to move these from the source to _site.

What I ended up doing was reducing Jekyll to “just creating HTML” files. Everything else I use Gulp to do, which is much faster at. My build process goes something like this now

  1. Gulp builds CSS, JS assets, puts them in a temp folder that Jekyll can’t see. This has the benefit of being able to edit them without triggering a build when developing locally.
  2. Gulp copies only the files needed to build the actual HTML to a temp folder (_layouts, _includes, .md documents, etc.)
  3. Jekyll builds this temp folder in HTML.
  4. Gulp moves the built HTML to dist folder where production CSS/JS and images are.

I go into this with greater detail on my site if it’s of interest:


#5

Thanks for the informative response.
So, the crux of the performance sink lies within a 3rd-party plugin which somehow got worse with my attempts at optimizing Jekyll.

Guess, I’ll have to dig into Jekyll’s master and see what is adding those extra 20secs (in comparison to v3.7.3)

Are you able to discern the difference (as to which file(s)) in the --profile output…?


#6

So I ran a build with 3.7.3 and master using --profile and here’s what I got. Biggest change I notice is the post.html layout took longer with master.

The other strange thing is jekyll-paginate-v2 paginates 118 tag pages with 3.7.3, but the master branch version of Jekyll has it doing 120. I wonder if something there is adding the extra time as it seems to be creating more pages.

3.7.3

Pagination: Complete, processed 118 pagination page(s)

Filename                                                     | Count |     Bytes |   Time
-------------------------------------------------------------+-------+-----------+-------
_layouts/default.html                                        |  1379 | 51611.20K | 27.773
_layouts/autopage_tags.html                                  |   346 |  6937.28K | 19.775
_includes/head.html                                          |  1379 | 28950.18K | 19.648
_includes/posts-paginated.html                               |   356 |  6563.74K | 18.542
_includes/entry.html                                         |   353 |  6373.72K | 17.973
_layouts/post.html                                           |   997 |  6830.92K | 17.214
_includes/head-seo.html                                      |  1379 |  2796.52K |  9.315
_includes/page-intro.html                                    |  1378 |  1887.64K |  9.153
_includes/pager.html                                         |   997 |   800.04K |  3.107
_includes/favicons.html                                      |  1379 |  1082.73K |  2.882
_includes/contact-list.html                                  |  1379 |  1451.72K |  2.331
_includes/related.html                                       |   997 |   448.23K |  2.261
_includes/comment.html                                       |    46 |  1226.87K |  1.672
_includes/navigation.html                                    |  1379 |  1537.91K |  1.528
_includes/footer.html                                        |  1379 |  1151.41K |  1.420
_includes/comments.html                                      |    53 |   965.58K |  1.338
_layouts/archive.html                                        |    10 |   538.51K |  1.309
_includes/scripts.html                                       |  1379 |  2042.91K |  1.172
sitemap.xml                                                  |     1 |   149.69K |  1.096
_layouts/atom_feed.html                                      |     6 |  1800.47K |  1.015
_includes/author                                             |   998 |     1.95K |  1.010
_includes/masthead.html                                      |  1379 |   480.77K |  0.929
_includes/popular-tags.html                                  |    11 |    65.95K |  0.926
_includes/breadcrumbs.html                                   |  1378 |   259.83K |  0.797
_includes/gallery.html                                       |  1027 |    46.32K |  0.520
_includes/read-time.html                                     |   218 |    14.66K |  0.360
_includes/algolia-search-scripts.html                        |  1379 |  1956.73K |  0.295
_layouts/page.html                                           |    24 |   185.17K |  0.149
_layouts/json_feed.html                                      |     1 |   502.45K |  0.137
_includes/critical-post.css                                  |  1021 |  9750.35K |  0.108
_layouts/archive_work.html                                   |     1 |    12.78K |  0.085
_posts/mastering-paper/2015-06-24-pencil.md                  |     1 |    40.97K |  0.081
_pages/sitemap.md                                            |     1 |     9.40K |  0.075
_posts/mastering-paper/2014-04-21-drawing-faces.md           |     1 |    29.11K |  0.071
_includes/post-list.html                                     |     1 |     8.02K |  0.060
_includes/skip-links.html                                    |  1379 |   478.07K |  0.058
_includes/search-form.html                                   |  1379 |   153.52K |  0.044
_posts/mastering-paper/2015-01-02-drawing-outer-space.md     |     1 |    25.02K |  0.042
_includes/comment-form.html                                  |    53 |    98.06K |  0.042
_posts/mastering-paper/2013-09-05-drawing-clouds.md          |     1 |    12.24K |  0.037
_posts/articles/2016-02-17-using-jekyll-2016.md              |     1 |    45.72K |  0.037
_includes/critical-archive.css                               |   357 |  3134.21K |  0.035
_posts/mastering-paper/2014-02-09-basics.md                  |     1 |    19.02K |  0.035
_posts/articles/2014-08-29-paperfaces-retrospective.md       |     1 |    12.97K |  0.033
_work/paperfaces.md                                          |     1 |   306.07K |  0.030
_pages/style-guide.md                                        |     1 |    59.45K |  0.027
_posts/mastering-paper/2013-11-25-drawing-textures.md        |     1 |    17.05K |  0.026
_posts/mastering-paper/2013-11-08-moleskine-book.md          |     1 |    13.29K |  0.023
_posts/mastering-paper/2013-07-31-introduction-tool-guide.md |     1 |    17.01K |  0.021
_posts/mastering-paper/2014-11-28-pencil-53-review.md        |     1 |    13.45K |  0.021

done in 131.325 seconds.

master

Pagination: Complete, processed 120 pagination page(s)

Filename                                                       | Count |     Bytes |   Time
---------------------------------------------------------------+-------+-----------+-------
_layouts/post.html                                             |   997 |  6830.92K | 43.007
_layouts/default.html                                          |  1383 | 51729.46K | 27.360
_layouts/autopage_tags.html                                    |   348 |  6940.36K | 20.142
_includes/head.html                                            |  1383 | 29032.10K | 19.514
_includes/posts-paginated.html                                 |   358 |  6563.95K | 18.893
_includes/entry.html                                           |   353 |  6376.35K | 18.323
_includes/related.html                                         |   997 |   448.23K | 14.142
_includes/head-seo.html                                        |  1383 |  2803.32K |  9.148
_includes/page-intro.html                                      |  1382 |  1890.55K |  8.974
_includes/pager.html                                           |   997 |   800.04K |  3.154
_includes/favicons.html                                        |  1383 |  1085.87K |  2.860
_includes/contact-list.html                                    |  1383 |  1455.93K |  2.228
_includes/comment.html                                         |    46 |  1226.87K |  1.693
_includes/navigation.html                                      |  1383 |  1542.37K |  1.547
_includes/footer.html                                          |  1383 |  1154.75K |  1.427
_includes/comments.html                                        |    53 |   965.58K |  1.342
_layouts/archive.html                                          |    10 |   538.51K |  1.228
_includes/scripts.html                                         |  1383 |  2048.84K |  1.153
sitemap.xml                                                    |     1 |   150.03K |  1.070
_layouts/atom_feed.html                                        |     6 |  1800.47K |  1.047
_includes/author                                               |   998 |     1.95K |  1.003
_includes/masthead.html                                        |  1383 |   482.16K |  0.950
_includes/popular-tags.html                                    |    11 |    65.95K |  0.853
_includes/breadcrumbs.html                                     |  1382 |   260.30K |  0.799
_includes/gallery.html                                         |  1029 |    46.32K |  0.515
_includes/read-time.html                                       |   218 |    14.66K |  0.381
_includes/algolia-search-scripts.html                          |  1383 |  1962.40K |  0.297
_layouts/page.html                                             |    26 |   192.17K |  0.157
_layouts/json_feed.html                                        |     1 |   502.45K |  0.132
_includes/critical-post.css                                    |  1023 |  9769.45K |  0.107
_layouts/archive_work.html                                     |     1 |    15.44K |  0.088
_pages/sitemap.md                                              |     1 |     9.51K |  0.082
_posts/mastering-paper/2015-06-24-pencil.md                    |     1 |    40.97K |  0.071
_includes/post-list.html                                       |     1 |     8.12K |  0.065
_posts/mastering-paper/2014-04-21-drawing-faces.md             |     1 |    29.11K |  0.056
_includes/skip-links.html                                      |  1383 |   479.46K |  0.053
_includes/search-form.html                                     |  1383 |   153.97K |  0.045
_posts/mastering-paper/2015-01-02-drawing-outer-space.md       |     1 |    25.02K |  0.042
_includes/comment-form.html                                    |    53 |    98.06K |  0.041
_posts/articles/2016-02-17-using-jekyll-2016.md                |     1 |    45.72K |  0.040
_includes/critical-archive.css                                 |   359 |  3151.77K |  0.036
_posts/mastering-paper/2014-02-09-basics.md                    |     1 |    19.02K |  0.035
_posts/articles/2014-08-29-paperfaces-retrospective.md         |     1 |    12.97K |  0.031
_work/paperfaces.md                                            |     1 |   306.07K |  0.028
_posts/mastering-paper/2013-11-25-drawing-textures.md          |     1 |    17.05K |  0.028
_posts/mastering-paper/2013-11-08-moleskine-book.md            |     1 |    13.29K |  0.027
_posts/mastering-paper/2013-09-29-drawing-water.md             |     1 |    10.77K |  0.026
_pages/style-guide.md                                          |     1 |    59.45K |  0.026
_posts/articles/2012-08-31-paperfaces-ipad-portrait-project.md |     1 |    14.65K |  0.025
_posts/mastering-paper/2014-12-18-color-picker.md              |     1 |    10.29K |  0.021

done in 142.253 seconds.

#7

its more than just the pagination plugin… If you look at the time used by _includes/related.html, you’ll see that it has increased by ~12s (meaning jekyll:master probably sees more related posts, /cc @parkr )

But yes, _layouts/autopage_tags.html with jekyll:master has been used 348 times in comparison to 346 times with jekyll-3.7.3, which is probably related to https://github.com/jekyll/jekyll/commit/f8a66ca76c1dcc0ce79aed2655f5660527aa7d60 because of which two documents ignored earlier are now available, and the remaining increased build time is probably related to these two documents.


#8

Also I’m using this “related posts” plugin…


#9

Also I’m using this “related posts” plugin…

This is an interesting development…
The following PR added related_posts to the SiteDrop so that a document’s “related-posts” are calculated only if and when its asked for in the Liquid Templates. (That’s how Liquid::Drop and its subclasses, of which is SiteDrop, works…)

While the above PR also memoized the built-in method (the use of @related_posts ||=…), making it more efficient by having to run just once for a given document irrespective of how many times the method is called…, the plugin’s method run the full code for each call to the method.

So, @toshimaru will now have to memoize his method definition for the plugin’s users to efficiently use Jekyll 3.8.0

You can get the ball rolling on this by opening an issue ticket at the plugin’s repo informing the author about the developments upstream.


#12

#14

Hello Jekyll community! Here are some more results.

Details

Build Times

3.7.3 master 3.8.0-alpha-2 optima/benchmark
14.775 12.299 13.887 12.97
13.578 12.592 13.126 12.982
13.811 12.814 15.154 11.939
13.797 12.462 12.858 11.86
14.584 12.627 13.131 12.518

Build on

  • fc27.x86_64
  • ruby 2.4.1p111

#15

Thanks to the people that took the time to share their build times.

3.8.0 has been published but feel free to continue to share benkmarks, especially if you have a very large website, and if your repo is public, share it, as we might use it for our tests.


#16

For those who want to follow closely with the latest addition on the performance front, we suggest you to subscribe to opened PRs with optimization label.

We added a performance check in PR’s, you can see for instance that optimizing rendering of Liquid templates cuts build time by more than 20% on average.


#17

:wave: Jekyllers,

We’re still working on improving Jekyll’s performance.
For people who suffer from slow builds, we’d be interested in testing:

  • refactor your includes with jekyll-include-cache when possible
  • use liquid-c gem to speed up Liquid parsing
  • use jekyll-commonmark to speed up Markdown rendering
  • test current jekyll master (make sure to exclude .jekyll-cache folder from Git)

Your Gemfile should therefore contain:

gem "jekyll", github: "jekyll/jekyll" 
gem "liquid-c"

group :jekyll_plugins do
  gem "jekyll-include-cache"
  gem "jekyll-commonmark"
end

Then publish you site on a Netlify branch or any CI where you can use these gems (you can’t on GitHub Pages) and share the result.

I advised those to a friend of mine and its build went from 931s to 59s. The build outuput is exactly the same.
I tested on Kong Documentation and build shrinked from 54s to 8.3s. Didn’t test the output.

This might conflict with existing plugins, if you encounter errors, we’re interested to know which plugins would benefit a fix to use our new cache API.

Here’s some documentation on what has already changed for plugins authors:

Thanks for helping making Jekyll faster.
The most notable gain should come from current master where we cache a lot


Benchmarking Jekyll
Jekyll 4 Speed on Build
#18