I have a bash script that is building my web sites and pushing it to the right directories via rsync. When running this command from the terminal everything is working. When starting the script from a background script it is starting but jekyll is only reading the configuration but not building. I have set the path varialbe within the bash script like the path that is used on the terminal session.
Any ideas how to debug this?
Regards
Hagen
You might start by comparing the output of the env command in both environments. In addition to PATH, there might be Ruby related variables involved (GEM_PATH, etc).
When Bash is run in the background (via cron, etc), it is initialized in a different way. For example, it doesn’t load .bash_profile, which might explain the difference.
so this are the differnces between background job and terminal session. I removed the lines that are the same.
Background
SHLVL=1
BLOCKSIZE=K
Terminal Session
GROUP=wheel
MACHTYPE=x86_64
EDITOR=vi
OSTYPE=FreeBSD
TERM=xterm-256color
HOST=static-sites
SHLVL=2
PAGER=less
VENDOR=amd
MAIL=/var/mail/root
HOSTTYPE=FreeBSD
I added all of the variables from Terminal Session to the script but no difference 
When you say “background” are you launching the bash script filename from the terminal by appending & after the bash script name? EG my_script.sh works fine but my_script.sh & which launches in background does not work?
Or are you are using cron or anacron which are two different beasts than normal background jobs? Or are you using something even more different for background processing?
Additionally have you checked journalctl -xe or something similar for error messages from the background script running?
Indeed check the log of your background job. You may find it is running as the wrong user and you need to be explicit on which user directory to run in. And also as pointed out above, it may be path issues so you should load your shell config or set PATH in the cron configuration
If you use cron, you will get a notification on a new shell session that you have 1 unread mail on the host about the cron log.
I am running a background script via & that is launching the build script at specific events (mqtt topics). Path and shell variables are set equally to the command line which is working
Is it possible that standard-error output is being lost when the script runs in the background? Usually Jekyll outputs some sort of error message when it runs into trouble. You could try logging errors to a file, like:
bundle exec jekyll build 2> error.log
I would image command & inherits your current environment just as if you ran command directly in your terminal. When you use command & to run a job in the background you can still attach to it’s screen output and view it but, I haven’t personally done that before.
I am starting the build with trace piping into a logfile
/usr/local/bin/jekyll build --trace | /usr/bin/tee -a jekyll.log
This is the output when started from background script (started as root).
GROUP=wheel
MACHTYPE=x86_64
EDITOR=vi
PWD=/root/site-source/www-hb
LOGNAME=root
HOME=/root
LANG=C.UTF-8
OSTYPE=FreeBSD
TERM=xterm-256color
HOST=static-sites
USER=root
SHLVL=2
PAGER=less
MM_CHARSET=UTF-8
LC_ALL=C.UTF-8
VENDOR=amd
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/root/bin
BLOCKSIZE=K
MAIL=/var/mail/root
HOSTTYPE=FreeBSD
OLDPWD=/root/site-source/www-hb
_=/usr/bin/env
====
Tue Dec 7 07:36:51 CET 2021
Configuration file: /root/site-source/www-hb/_config.yml
sending incremental file list
====
and this is the output when started from commandline
GROUP=wheel
MACHTYPE=x86_64
EDITOR=vi
PWD=/root/site-source/www-hb
LOGNAME=root
HOME=/root
LANG=C.UTF-8
OSTYPE=FreeBSD
TERM=xterm-256color
HOST=static-sites
USER=root
SHLVL=2
PAGER=less
MM_CHARSET=UTF-8
LC_ALL=C.UTF-8
VENDOR=amd
PATH=/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin:/root/bin
BLOCKSIZE=K
MAIL=/var/mail/root
HOSTTYPE=FreeBSD
OLDPWD=/root/site-source/www-hb
_=/usr/bin/env
====
Tue Dec 7 07:39:23 CET 2021
Configuration file: /root/site-source/www-hb/_config.yml
Source: /root/site-source/www-hb
Destination: /root/site-source/www-hb/_site
Incremental build: disabled. Enable with --incremental
Generating...
done in 32.119 seconds.
Auto-regeneration: disabled. Use --watch to enable.
sending incremental file list
atom.xml
social-feeds/github.xml
social-feeds/twitter.xml
Your static Jekyll site is turning out to be rather dynamic. If you have slow build times, a huge amount of incoming data, and your data is not appearing on the frontend in realtime, you can consider going for a different architecture.
You might find that there’s a better fit to build the site once and then pull data in dynamically another way.
Such as fetching data on the frontend from API or something which has all the data and cares about the queue.
You can also use a Lambda or Netlify Functions which can connect to API or database or whatever to get data and then return JSON data (or HTML data) on the frontend.
Both of those could still be with Jekyll.
Or use a server in Node, Python, or PHP, which doesn’t have a static step and can render pages based on the latest content. In fact you wouldn’t even have the server listen to the event queue. When a user hits the page, the server will return the latest HTML page with data in it, and that will always be up to date. You can add a cache layer on top if performance is more important than accuracy.
Another option would be to move your building from locally on a shell script and rsync, to rather building with GH Actions or Netlify. And then you trigger a rebuild by using GitHub API. In Netlify, all you have to do is hit an endpoint with a token and your site will rebuild. So maybe you get your event queue to hit the API.
curl -d '' https://api.netlify.com/build_hooks/abcdef123
I run this on a schedule using GH Actions here
But your case would not be a schedule.
Thank you for this advise but I am currently happy with the architecture and will not change it.
I just want to continue with it and need to solve the problem that its not building.
You could try redirecting the error output to jekyll.log by adding 2>&1 to the Jekyll command like this:
/usr/local/bin/jekyll build --trace 2>&1 | /usr/bin/tee -a jekyll.log
That will send any error messages to the log, which might help locate the problem.
unfortunately there seems to be no error. I changed it to
/usr/local/bin/jekyll build --trace 2>&1 | /usr/bin/tee -a jekyll.log
but its still ony this jekyll related message. (sending incremental file list" is a message from rsync which is coming after jekyll build)
/root/site-source/www-hb
Configuration file: /root/site-source/www-hb/_config.yml
sending incremental file list
Hmm, I’m running out of ideas. At this point I’d do an error sanity check by introducing an error to observer the error message. For example, the following should show a fatal error in the log (note foo instead of build):
/usr/local/bin/jekyll foo --trace 2>&1 | /usr/bin/tee -a jekyll.log
me too if it comes to ideas. The sanity check is “positiv”. If I change the build command
/usr/local/bin/jekyll bld --trace 2>&1 | /usr/bin/tee -a jekyll.log
I can see an error in the logfile
fatal: 'jekyll bld' could not be found. You may need to install the jekyll-bld gem or a related gem to be able to use this subcommand.
Anybody else any idea?
Have you tried using bundler to launch Jekyll?
/path/to/bundle/bundle exec jekyll build
no, I want to reduce the amount of software and till today I didnt not see any reason for bundler. Why do you believe this would make a difference?
In the log above, after reading the _config.yml file seems like it doesn’t understand where the Source is, something is broken with the next step… Bundler usually makes things better with that kind of problem.
If you don’t want to install Bundler maybe you should try to hardcode with the full path the :collections_dir and other path-related variables…
I looks like your hint was the right one 
After i changed the jekyll command in my batch file to
/usr/local/bin/jekyll build -s /root/site-source/myrepo --trace 2>&1 | /usr/bin/tee -a $LOG
its compiling my blog. It looks like the default source directory “./” is ignored in some situations. (The script is at the right directory when jekyll build is started)
I will need to do some additional testing but I am very confident that its solve.