# Long time between watcher starting and server running **URL:** \ **Category:** Help\ **Created:** [April 4, 2019, 9:14pm UTC](https://talk.jekyllrb.com/t/long-time-between-watcher-starting-and-server-running/2799 "2019-04-04T21:14:29Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![mvogelsang](https://avatars.discourse-cdn.com/v4/letter/m/9de0a6/32.png) [@mvogelsang](https://talk.jekyllrb.com/u/mvogelsang)\ **Post date:** [April 4, 2019, 9:14pm UTC](https://talk.jekyllrb.com/t/long-time-between-watcher-starting-and-server-running/2799/1 "2019-04-04T21:14:29Z")
Hi everyone! I am new to Jekyll and started rebuilding our work group’s old website using version 3.8.5 (according to the lock file). This includes a lot of pdfs, videos and other static content, some of which is lecture slides with interactive material. This content sits in a directory called “downloads” next to the source directory since the static files are not necessary for site generation. Inside \_site I have a symbolic link to “downloads” which I also keep and exclude using “keep\_files” and “exclude” in my \_config file. There is no “downloads” in the directory that is being built into the target. If I remove the symbolic link the page builds in around 9 to 12 seconds and starts the server immediately afterwards. Making changes takes around 7 seconds until they are built in. Once I put in the link (or just the whole “downloads” folder containing 6GB of small files) the build time stays around the same, but between Jekyll displaying build time and the server actually running around 8 minutes pass. My guess here is that the files in “downloads” in \_site are still being watched which is a thing I entirely do not want. If I then make a change in a single file Jekyll needs around 4 minutes more to actually tell me it made changes and finish the build. After this first hickup everything behaves quick and as usual. My second guess there is that Jekyll needs to construct some index structure once for the first update which lists all the files that are being watched in \_site. Both of these things are guesses, I have no idea how Jekyll actually works. All of this won’t be a problem for actual deployment. I plan to let a hook start rsyncing “downloads” into the \_site directory while I deploy the whole thing to the actual web server directory. It is however very cumbersome for testing and previewing the site offline. There are [other questions](https://talk.jekyllrb.com/t/exclude-from-watch/661) already asking how to avoid excluded files and directories being watched. As far as I could tell these did not get to a conclusion but instead found a workaround. The core problem was a bit different, too. My current workaround is using a python SimpleHTTPServer to view the site and rebuilding manually every time I make changes. I’d like to avoid this on behalf of other people working on the site later on. If you have any guesses what causes the problem and how to avoid it I would gladly appreciate those! Thanks for reading and in advance for helping! ---
**Author:** ![ashmaroli](https://avatars.discourse-cdn.com/v4/letter/a/df705f/32.png) [@ashmaroli](https://talk.jekyllrb.com/u/ashmaroli)\ **Post date:** [April 5, 2019, 11:24am UTC](https://talk.jekyllrb.com/t/long-time-between-watcher-starting-and-server-running/2799/2 "2019-04-05T11:24:23Z")
Any entry in the `exclude` list defined in your config file is automatically ignored by the watcher. You could use simple filenames or file paths or entire directories. As of `v3.8.5`, while excluding a custom file or directory, you need to list paths that are excluded by default (`Gemfile`, `Gemfile.lock`, `node_modules/`, `vendor/`, etc) in addition to your custom paths. **In `v4.0`** , you need to just list _your custom file paths_. The paths excluded by default are **always ignored** internally. The point here being, if you don’t want Jekyll to handle files in `downloads/`, just `exclude` the entire directory and delegate their handling to a Gulp task or a Python script or similar routes (all of which need to be implemented by yourselves). Moreover, if you’re building / developing on Windows add `gem 'wdm'` to your Gemfile to improve the watcher’s response times. ---
**Author:** ![mvogelsang](https://avatars.discourse-cdn.com/v4/letter/m/9de0a6/32.png) [@mvogelsang](https://talk.jekyllrb.com/u/mvogelsang)\ **Post date:** [April 5, 2019, 12:00pm UTC](https://talk.jekyllrb.com/t/long-time-between-watcher-starting-and-server-running/2799/3 "2019-04-05T12:00:29Z")
Thanks for your reply! `downloads` is already being ignored as a directory per exclude and not even located in the source directory. In the target directory merely a symbolic link to it is present. Using or not using this one link in `_site` makes all the difference in serve times. I’m developing under Ubuntu 18.04. bionic by the way. ---
**Author:** ![ashmaroli](https://avatars.discourse-cdn.com/v4/letter/a/df705f/32.png) [@ashmaroli](https://talk.jekyllrb.com/u/ashmaroli)\ **Post date:** [April 5, 2019, 12:22pm UTC](https://talk.jekyllrb.com/t/long-time-between-watcher-starting-and-server-running/2799/4 "2019-04-05T12:22:30Z")
> [@mvogelsang](#): > > not even located in the source directory. In the target directory merely a symbolic link to it is present. Ah yes! You had mentioned this clearly in the opening post of this thread. I misread it. This is a strange behavior and totally unexpected because the watcher only watches the source directory. Will you be able set up a public test repository with the same file-folder-tree and config file as your production site but without the actual files in the `downloads/`…? (so that I can clone it locally and experiment with it…) ---
**Author:** ![mvogelsang](https://avatars.discourse-cdn.com/v4/letter/m/9de0a6/32.png) [@mvogelsang](https://talk.jekyllrb.com/u/mvogelsang)\ **Post date:** [April 5, 2019, 12:41pm UTC](https://talk.jekyllrb.com/t/long-time-between-watcher-starting-and-server-running/2799/5 "2019-04-05T12:41:02Z")
I’m afraid I can’t give you the actual files I’m working on but I can set up a dummy page after work using the same structure with similar linking to a theoretical downloads directory. Would that be of any help to you? A colleague mentioned it could be the Jekyll server managing URLs internally. To my knowledge Jekyll does things like setting the URL of files like `people/doe-john.md` to `people/doe-john/` without an index.html file being present instead of setting it to `people/doe-john.html`. I guess this needs some internal indexing as well. Maybe this process is eating up a lot of time if many files are in `_site`? ---
**Author:** ![ashmaroli](https://avatars.discourse-cdn.com/v4/letter/a/df705f/32.png) [@ashmaroli](https://talk.jekyllrb.com/u/ashmaroli)\ **Post date:** [April 5, 2019, 12:56pm UTC](https://talk.jekyllrb.com/t/long-time-between-watcher-starting-and-server-running/2799/6 "2019-04-05T12:56:05Z")
A dummy site would be perfect! Jekyll does manage URLs internally but doesn’t consider files inside `_site` for this. Will have to test if symlinks play a nefarious role here though… ---
**Author:** ![mvogelsang](https://avatars.discourse-cdn.com/v4/letter/m/9de0a6/32.png) [@mvogelsang](https://talk.jekyllrb.com/u/mvogelsang)\ **Post date:** [April 5, 2019, 5:59pm UTC](https://talk.jekyllrb.com/t/long-time-between-watcher-starting-and-server-running/2799/7 "2019-04-05T17:59:05Z")
Ok, got around to setting it up. Just clone > **[wmvogelsang/dummy\_site](https://github.com/wmvogelsang/dummy_site)** > > Dummy Jekyll site for debugging purposes. Contribute to wmvogelsang/dummy\_site development by creating an account on GitHub. You’d be surprised how little it takes to reproduce that behaviour, at least on my end. Your setup could look somewhat like this: . `├── dummy_top` `| ├── dummy_site.git` `| | ├── _site` `| | | ├── downloads->../../downloads` `| | | └── index.md` `| | ├── index.md` `| | └── _config.yml` `| └── downloads` (6GB folder containing lots of small files) If you don’t get very long server loading times please tell me. This setup has the problems on my computer. If you clone it, put another downloads next to it and don’t get problems it has something to do with my setup. ---
**Author:** ![ashmaroli](https://avatars.discourse-cdn.com/v4/letter/a/df705f/32.png) [@ashmaroli](https://talk.jekyllrb.com/u/ashmaroli)\ **Post date:** [April 6, 2019, 3:10am UTC](https://talk.jekyllrb.com/t/long-time-between-watcher-starting-and-server-running/2799/8 "2019-04-06T03:10:13Z")
I was not able to reproduce this issue with your setup. Due to technical constraints, I couldn’t test this on my local system. So booted up a Linux virtual machine with Ruby, Git and Jekyll installed. I then cloned your `dummy_site` to `trials/dummy_site` and programmatically generated a `trials/downloads` folder with 10000 files each amounting to `95kb` Running `du -h downloads` produces: ```auto 899M downloads ``` I then added a Gemfile to ensure that I’ll be building with just Jeyll 3.8.5 ```ruby # frozen_string_literal: true source "https://rubygems.org" gem "jekyll", "3.8.5" ``` I then ran the server by running `bundle exec jekyll serve` **Observation:** `done in 0.016 seconds.` In a new terminal window to the VM, i made some changes to `trials/dummy_site/index.md` (added front matter so that it is not static, edited HTML markup). **Obsevation:** `...done in 0.002979388 seconds.` **Inference:** Symlink inside `_site` to a 899MB directory outside `` directory has no visible effect on Jekyll regeneration. ---
**Author:** ![mvogelsang](https://avatars.discourse-cdn.com/v4/letter/m/9de0a6/32.png) [@mvogelsang](https://talk.jekyllrb.com/u/mvogelsang)\ **Post date:** [April 6, 2019, 3:20pm UTC](https://talk.jekyllrb.com/t/long-time-between-watcher-starting-and-server-running/2799/9 "2019-04-06T15:20:27Z")
Thanks and sorry for your effort! But sadly, build time is not the issue. My dummy builds in 0.088 seconds. The time between Jekyll reporting `Generating...` ` _____________ done in 0.088 seconds.` and it telling me `Auto-regeneration: enabled for '/home/mvogelsang/jekyll/dummy'` `__Server address: http://127.0.0.1:4000` `_Server running... press ctrl-c to stop.` is what is taking all the time. You don’t even need to create data to see what I mean. Instead of a symbolic “downloads” I just made a normal directory “downloads” and put symbolic links to some arbitrary 1.2GB large directory on my system I had lying around. The difference between having that link and not having it is 11 seconds between “`done in 0.0xx seconds.`” and “`Auto-regeneration: enabled for '/home/mvogelsang/jekyll/dummy'`” So with your setup, does it make a difference in time until you can access `localhost:4000` between having and not having the link? ---
**Author:** ![ashmaroli](https://avatars.discourse-cdn.com/v4/letter/a/df705f/32.png) [@ashmaroli](https://talk.jekyllrb.com/u/ashmaroli)\ **Post date:** [April 6, 2019, 5:55pm UTC](https://talk.jekyllrb.com/t/long-time-between-watcher-starting-and-server-running/2799/10 "2019-04-06T17:55:43Z")
The reason I booted a Linux VM is because my local system is on Windows. Either ways, I tried to reproduce this issue with both a symbolic `downloads/` and a normal directory with symlinked file, both cases pointing to a `1.2GB` sized directory. In both cases, the time between `done in 0.0xxx seconds.` and `Auto-regeneration: enabled for 'xxxxxx'` was negligible. Therefore, until someone else is able to reproduce this, it looks like an issue specific to your system. Sorry. ---
**Author:** ![mvogelsang](https://avatars.discourse-cdn.com/v4/letter/m/9de0a6/32.png) [@mvogelsang](https://talk.jekyllrb.com/u/mvogelsang)\ **Post date:** [April 8, 2019, 8:55am UTC](https://talk.jekyllrb.com/t/long-time-between-watcher-starting-and-server-running/2799/11 "2019-04-08T08:55:36Z")
Too bad, but thanks for your help! Hopefully it really is a problem specific to me and noone else will get the same issues.