i have simple site with default theme. how much time it will take to generate 500000 pages
That’s too broad a question. There are factors like your computer speed. Are you running a Pentium from 1993 or a 10th gen i7? What’s the site structure? Do you have 15 includes in every document or are you passing the page content direct to the default layout?
You might find the article, How I reduced my Jekyll build time by 61%, interesting.
My best guess is, using Jekyll v4 and a decent processer on an *nix system, it’ll take you about 60-70 seconds. This supposes you aren’t running a boatload of plugins and have made some effort to optimize the site.
1 Like
60 seconds seems a bit optimistic. Below is a naive script that creates and builds a default-theme Jekyll 4 site with 1000 pages (dups of About page). On an i7 iMac (~5 years old), it builds in ~30 seconds.
If the speed scales linearly (O(n)), then 500K pages would take ~4 hours. Unfortunately, it appears to scale quadratically (O(n^2)), so it may take 80+ days!
Of course, this is an extremely naive approach on an older machine. A faster system and various optimizations might achieve linear speed with a few doublings – maybe it could be reduce to ~1hr?
Ultimately, rather then try to force Jekyll to build a 500K site on its own, it would be better to split the site into pieces, so Jekyll can be run in parallel. Then a 256-core system could probably build 500 1K page sites in a minute or two.
#!/bin/bash
set -Eeuxo pipefail
: CPU and Version Info
[[ -d /proc ]] && cat /proc/cpuinfo | grep 'model name' | uniq || sysctl -a | grep brand_string
ruby --version
jekyll --version
: Create a new Jekyll site with default theme
TEST=/tmp/jtest
rm -rf $TEST && jekyll new $TEST && cd $TEST
: Only list About page in menu
printf "header_pages:\n - about.markdown\n" >> _config.yml
: Create copies of About page without permalink
sed 's/permalink:.*//' about.markdown | tee page{1..1000}.md
: Build
time bundle exec jekyll build
1 Like
Just a quick anecdote, I have a 90k page Jekyll site that builds in 40-60 minutes.
It’s not optimized for build speed at all…lots of includes, bunch of plugins, multiple collections, etc.
I develop locally with a subset of the pages (~100) which builds in seconds. When I push it to production a CI/CD service takes over for the actual build, so build times don’t really matter. YMMV.
Hope that helps.
2 Likes
Wow! That’s a HUGE site. Is that site open-source via a public GitHub repository?
If it is I’d like to fork and run profilers on it…
Yup, here’s the repo: GitHub - grantmakers/profiles: Grantmakers.io Profiles - Summary profiles for all US-based foundations who have filed electronic IRS Form 990-PF.
There’s a smaller sample dataset in the repo, but if you (or anyone else) want to run profilers with the full dataset PM me and I’ll send you a link. I had to take the full dataset out of source control as I kept blowing past Github soft limits for repo size.
And @tejas456sawant, though my use of collections might potentially be useful as you explore how to pull data into your project, please note my project is basically a lesson in how NOT to optimize Jekyll code for build speed 
2 Likes
Hey everyone — just wanted to contribute an additional data point to this discussion.
I’ve been dealing with a very large Jekyll-based site myself, with tens of thousands of includes and over 50,000 pages. At a certain point Jekyll simply couldn’t keep up, even with aggressive caching and incremental builds. Build times were becoming impractical, and CI/CD pipelines would frequently time out.
Because of that, I ended up building an alternative static site generator in Rust called Rustyll, designed to be compatible with most Jekyll sites but optimized for extremely large content sets. Rust’s performance and memory safety allowed me to process large collections and deep include trees far more efficiently than Ruby could handle in my use case.
If anyone here has similar performance challenges or works with very large Jekyll projects, I’d love feedback or contributions — especially as I’m working toward finalizing the 1.0 release. The goal isn’t to replace Jekyll, but to offer a high-performance alternative for edge cases where scale becomes a bottleneck.
Happy to answer questions about migration, compatibility, or performance if it helps others dealing with large Jekyll sites.
Thanks for the suggestion. I like “zero dependencies”. However, it looks like Rustyll is not available via Homebrew nor cargo at this time?
`brew install rustyll` → No available formula with the name “rustyll” …
`cargo install rustyll` → could not find `rustyll` in registry…
This seems to be an AI’s “aspirational confabulation” (a less charitable description is “AI Slop”).
https://github.com/betterwebinit/rustyll/commit/4281158e77db9c6111348a1dcf3bd32e52c9fb36
Thank you! Oh my, I am still too naive and think people check what they’re writing, better: what they are having written for them 
. So I’ll look at installing via the GitHub code.