The first time I tried to build my website with --incremental flag, it took about 1000 seconds. Then immediately I tried to build again and it took 14 seconds and it is really a good result. However I restarted my pc and then tried rebuilding again. This time it took about 400 seconds to build while nothing was changed in my website. Although its still better than the initial 1000 second to build the website from scratch but its still too much while nothing is changed. I cant understand this behaviour.
I added --verbose and --profile flags to find the bottleneck however I couldnt find anything
bundle exec jekyll build -d public --incremental --profile --verbose Logging at level: debug Jekyll Version: 4.0.0.pre.alpha1 Configuration file: /home/hosna/Storage/repositories/zoom/_config.yml Requiring: /home/hosna/Storage/repositories/zoom/_plugins/metareader.rb Requiring: /home/hosna/Storage/repositories/zoom/_plugins/ext.rb Requiring: /home/hosna/Storage/repositories/zoom/_plugins/jdate.rb Requiring: /home/hosna/Storage/repositories/zoom/_plugins/include_absolute.rb Requiring: /home/hosna/Storage/repositories/zoom/_plugins/jekyll-postfiles.rb Requiring: /home/hosna/Storage/repositories/zoom/_plugins/file_exists.rb Requiring: /home/hosna/Storage/repositories/zoom/_plugins/onebox_filter.rb Requiring: jekyll-sitemap Requiring: jekyll-paginate-v2 Requiring: jekyll-redirect-from Source: /home/hosna/Storage/repositories/zoom Destination: /home/hosna/Storage/repositories/zoom/public Incremental build: enabled Generating... EntryFilter: excluded /Gemfile EntryFilter: excluded /Gemfile.lock EntryFilter: excluded /commit.sh Reading: _posts/time/2018-03-15-time.markdown EntryFilter: excluded /common/commit.sh Reading: _daddys-home-collection/index.md Reading: _daddys-home-collection/daddys-home-i/index.md Reading: _daddys-home-collection/daddys-home-i/with-music/the-co-dads-battle-ii/index.md Reading: _daddys-home-collection/daddys-home-i/with-music/index.md Reading: _daddys-home-collection/daddys-home-i/with-music/co-dads-make-peace/index.md ////......deleted some bunch of reading log here Reading: _thrive/part-02/index.md Generating: ReadMetaFiles::Generator finished in 9.736257167 seconds. Generating: Jekyll::PostFileGenerator finished in 0.002440468 seconds. Generating: JekyllRedirectFrom::Generator finished in 2.191940422 seconds. Generating: Jekyll::JekyllSitemap finished in 0.013961898 seconds. Pagination: Starting Pagination: Is enabled, but I couldn't find any pagination page. Skipping pagination. Pages must have 'paginate: enabled: true' in their front-matter for pagination to work. Generating: Jekyll::PaginateV2::Generator::PaginationGenerator finished in 0.020414395 seconds. Rendering: redirect.html Pre-Render Hooks: redirect.html Rendering Markup: redirect.html Rendering Layout: redirect.html Layout source: site Rendering: redirect.html Pre-Render Hooks: redirect.html Rendering Markup: redirect.html Rendering Layout: redirect.html Layout source: site Rendering: redirect.html Pre-Render Hooks: redirect.html Rendering Markup: redirect.html Rendering Layout: redirect.html Layout source: site Rendering: redirect.html Pre-Render Hooks: redirect.html Rendering Markup: redirect.html Rendering Layout: redirect.html Layout source: site Rendering: redirect.html Pre-Render Hooks: redirect.html Rendering Markup: redirect.html Rendering Layout: redirect.html Layout source: site Rendering: redirect.html ///.... deleted some bunch of rendering logs here Pre-Render Hooks: robots.txt Rendering Liquid: robots.txt Rendering Markup: robots.txt Rendering Layout: robots.txt Writing: /home/hosna/Storage/repositories/zoom/public/p/part/2066/index.html Writing: /home/hosna/Storage/repositories/zoom/public/p/part/2059/index.html ///....deleted some bunch of other writing here Writing: /home/hosna/Storage/repositories/zoom/public/p/part/2076/index.html Writing: /home/hosna/Storage/repositories/zoom/public/redirects.json Writing: /home/hosna/Storage/repositories/zoom/public/sitemap.xml Writing: /home/hosna/Storage/repositories/zoom/public/robots.txt Writing Metadata: .jekyll-metadata Filename | Count | Bytes | Time -----------------------+-------+----------+------ sitemap.xml | 1 | 310.97K | 0.849 _layouts/redirect.html | 2095 | 1555.54K | 0.066 robots.txt | 1 | 0.04K | 0.000 done in 471.909 seconds. Auto-regeneration: disabled. Use --watch to enable.
I just relized that there is a great time gap between the line
EntryFilter: excluded /common/commit.sh
I dont know what is being done there.
In my experience, the first run (e.g. “cold start”) of Jekyll after a reboot is always slower.
I’ve noticed this with tools like Gulp as well. I think it has more to do with the environment getting setup and run that first time, than anything related to Jekyll or it’s incremental mode.
I don’t think it would make sense to be slower as much as 400 seconds.
You’d be surprised.
Every time I start up Jekyll after my computer was restarted, it takes awhile for it to get going. Each build after is about 50% faster, and I don’t use Jekyll’s incremental mode.
I don’t know the internals of Ruby, but I’m assuming there’s some stuff held in memory that speeds up tasks after you’ve done that initial run. Restarting your device clears all that.
do you know why the bottleneck isn’t logged when I’m adding --profile flag?
It’s not a Jekyll bottleneck. It’s your Ruby environment getting started from a fresh reboot. The delay is with that… and before Jekyll even runs.
No. The delay is here:
I just don’t know what is being done there.
Sure, but is that gap a
600s one? I’m willing to bet a bulk of the delay is before Jekyll even starts after you restart.
The other delays are normal content related things as Jekyll builds your site. I don’t think the profile flag will surface that as it just shows you the times of layouts and includes, not reading files… which is where those
EntryFilter lines happen.
Also it’s my understanding incremental builds have quirks to them. Which would explain why your results aren’t consistent.
the build after restart takes 471 seconds and I think that delay I’m talking about is taking about 450 seconds.
to add more information, in this line
Reading: _daddys-home-collection/index.md it seems jekyll is starting to read my collection files. I have too many collections. Somthing about 2k documents. And all of them have huge files like video and audio. Maybe jekyll is reading all of those static files and it takes so much time?
It most certainly is reading those static files and/or transferring them to
If you have that many static assets I would suggest excluding them in
_config.yml and use a task runner like Gulp or Grunt to move those assets to your
_site folder instead.
You can also set
keep_files: [DIR, FILE, ...] in your config to prevent those files from being flushed from
_site so they can persist between builds.
I do this for my personal site that has over a 1GB of images. It’s an I/O intensive thing to have Jekyll try to read and copy those files ever build so I took them out of the equation.
I used --incremental flag cause I thought its handling such situation. I mean with this flag, shouldn’t jekyll try not to transfer unchanged static files to _site?
Maybe, but I think incremental in this case is related to what core Jekyll does… transforms your Markdown into HTML files.
That’s what it’s incrementally building and caching, not your static assets.