Error: invalid byte sequence in UTF-8

clemetre · January 22, 2018, 5:41pm

Hi There!

I am relatively new to Jekyll but I have been working on a website relatively smoothly until now.
I could create the pages I needed and got the site and made it live.
While working on some additions the other day, out of the blue it started to give an error:

Invalid byte sequence in UTF-8.

I have started to look around and saw several topics on the github page, but nothing that worked so far for me.
I reverted everything I had been doing between the last time it built successfully and this error, and I am not able to get back to a working version. I removed all the images that I had added, a font that I had added and the 3 different pages I was working on at the time (removed them from the working directory). Still the same error.

THis is what I get:

$ bundle exec jekyll serve --watch
Configuration file: /Users/clemetre/Documents/Work/DFDwebsite/_config.yml
Configuration file: /Users/clemetre/Documents/Work/DFDwebsite/_config.yml
            Source: /Users/clemetre/Documents/Work/DFDwebsite
       Destination: /Users/clemetre/Documents/Work/DFDwebsite/_site
 Incremental build: disabled. Enable with --incremental
      Generating...
jekyll 3.4.5 | Error:  invalid byte sequence in UTF-8

$ bundle exec jekyll doctor
Configuration file: /Users/clemetre/Documents/Work/DFDwebsite/_config.yml
jekyll 3.4.5 | Error:  invalid byte sequence in UTF-8

I added the encoding utf-8 in the _config.yml and it didn’t solve anything.

I ran a new test site jekyll and this works fine.

Does anybody have any idea to help me troubleshoot this? Or have an advice on how to solve this?

Thanks a lot!
Christophe

clemetre · January 22, 2018, 5:51pm

I forgot to mention that I am on MacOS10

rdyar · January 22, 2018, 7:01pm

been a while since I had trouble with this, but back then I think had to resave all the files in my editor and make sure the editor was saving them as utf-8.

clemetre · January 22, 2018, 8:19pm

Thanks so much for your response.
I am unsure why that would be the case as I was only working with VIM on my console not using an editor. But I am willing to try anything at that point.

Thanks!

Frank · January 22, 2018, 8:26pm

FWIW we fixed a few UTF-8 encoding related bugs in 3.6.2, does it get better if you update to latest Jekyll?

pabloduo · January 23, 2018, 2:18am

Hmm, I can take a look at the file itself… Here is an empty Gist, feel free to update it with the content of your _config.yml copy/pasting it as is (so we can track down the offending character).

gist.github.com

https://gist.github.com/ptsurbeleu/335c12559886a7f0c67c0a01ae1c66a7

_config.yml

# could you please create a new revision to this Gist copy/pasting your _config.yml as-is?

clemetre · January 23, 2018, 3:25pm

Hi Pabloduo,

Thanks for the reply. I just pasted the _config.yml.
What really troubles me is that it started kind of out of nowhere… The config.yml was not touched.

Thanks a lot
Christophe

clemetre · January 23, 2018, 5:40pm

I got the latest version 3.7, but still get the same error:

$ bundle exec jekyll serve --watch
Configuration file: /Users/clemetre/Documents/Work/DFDwebsite/_config.yml
            Source: /Users/clemetre/Documents/Work/DFDwebsite
       Destination: /Users/clemetre/Documents/Work/DFDwebsite/_site
 Incremental build: disabled. Enable with --incremental
      Generating...
jekyll 3.7.0 | Error:  invalid byte sequence in UTF-8


$ bundle exec jekyll doctor
Configuration file: /Users/clemetre/Documents/Work/DFDwebsite/_config.yml
jekyll 3.7.0 | Error:  invalid byte sequence in UTF-8

rdyar · January 23, 2018, 6:52pm

I don’t think it has to be the config file that has the problem, I think it could be any file…

Try running it with --verbose and maybe it will give you more specific info. Seems like there is another argument that gives you additional error detail but I can’t think of it at the moment.

clemetre · January 23, 2018, 7:52pm

THanks a lot Ron,

Here is my output with the --verbose option:

$ bundle exec jekyll serve --watch --verbose
  Logging at level: debug
Configuration file: /Users/clemetre/Documents/Work/DFDwebsite/_config.yml
  Logging at level: debug
         Requiring: /Users/clemetre/Documents/Work/DFDwebsite/_plugins/generator_scss.rb
         Requiring: jekyll-sitemap
         Requiring: kramdown
            Source: /Users/clemetre/Documents/Work/DFDwebsite
       Destination: /Users/clemetre/Documents/Work/DFDwebsite/_site
 Incremental build: disabled. Enable with --incremental
      Generating...
       EntryFilter: excluded /LICENSE.txt
           Reading: _posts/2016-10-20-PNAS-Urban-Park-Soil.markdown
           Reading: _posts/2017-09-15-PNAS-Australia.markdown
           Reading: _posts/2017-11-29-We-re-back.markdown
           Reading: _posts/2017-12-06-Recruiting-Teachers.markdown
           Reading: _posts/2018-01-16-Wired-Magazine-Article.markdown
       EntryFilter: excluded /automation/images
       EntryFilter: excluded /automation/minifying
       EntryFilter: excluded /illustrator/bg.png
       EntryFilter: excluded /illustrator/bg_1.png
       EntryFilter: excluded /illustrator/bg_2.png
       EntryFilter: excluded /illustrator/BG_image-01.png
       EntryFilter: excluded /illustrator/BG_image.ai
       EntryFilter: excluded /illustrator/chosen-sprite.png
       EntryFilter: excluded /illustrator/chosen-sprite@2x.png
       EntryFilter: excluded /illustrator/headerRU.gif
       EntryFilter: excluded /illustrator/out copy.png
       EntryFilter: excluded /illustrator/out.png
       EntryFilter: excluded /illustrator/Whats-this-about.ai
jekyll 3.7.0 | Error:  invalid byte sequence in UTF-8

mmistakes · January 23, 2018, 10:09pm

What does this plugin do? Have you tried deactivating all plugins to see if maybe one of them is the offender?

pabloduo · January 23, 2018, 10:35pm

Hi Cristophe,

No worries, it is always my greatest pleasure to help folks! Sorry, it is my fault I wasn’t clear from the beginning - could you either create a new Gist (on Github) or share your _config.yml as is (Dropbox, Git or upload here as an attachment to the thread)?

Apologies for that extra roundtrip.

On the same note, if you can share your entire Jekyll folder that’s even better - I can try to debug it in & out (that’s what I usually do) to help you pinpoint the issue and find a cure to it.

rdyar · January 24, 2018, 5:56am

you know I think I once had that error with sass cause it didn’t like a comment I had in one of the files.

Maybe try removing all the sass contents and see if you still get it.

clemetre · January 24, 2018, 5:40pm

Hey all,

Thanks a lot for your help! I finally realized what happened. In the data files that I have to generate automatically the individual pages, I had geographic locations. When my colleagues passed on to me the updated version with new samples in the data file, I didn’t realize that she had written some of these locations with spanish characters (e.g .México, Tza Itzá, Yucatán…). I just realized this and changed all the “á” into “a” and “é” into “e”, and now all is back to normal!

Thank you all again for all your valuable and appreciated input!

Christophe

pabloduo · January 25, 2018, 5:33am

@clemetre, glad you figured it out at the end!

3CatMax · August 18, 2018, 9:27pm

I am new at Jekyll and was making a post file in Win10 Notepad. This was exactly my problem. Thanks!

adiazdurana · July 12, 2019, 9:45am

Hi clemetre,

your explanation is the closest to the situation I am having since the text we write is in Spanish and the Spanish characters are everywhere.

My doubt is still where should I replace the characters?

The “á” into “a” and “é” into “e” should be done in the data files, you wrote.

I’m converting Jupyter notebooks into markdown files. Where should I do the change?

xplosionmind · January 27, 2022, 4:17pm

I am experiencing this same issue, for a Markdown file. If I convert it to HTML, the error does not appear. How could I understand specifically where the invalid sequence is?

Thanks a lot!

PuritySyrup · July 19, 2022, 1:00pm

I am having this issue with image files and jekyll 3.9.2 on macOS 11.6.7 (Big Sur). I saved the image file as a PNG, and when I typed bundle exec jekyll serve, I got this:

Error: could not read file /LocalHome/blog/blog.github.io/_posts/2022-07-18-postname/filename.png: invalid byte sequence in UTF-8

So I tried a different graphic editor: loaded the file and resaved it. Same error.

So I tried saving the file as a GIF and a JPG. I just saved the files to my drive; that’s all. I didn’t reference them anywhere. And when I typed bundle exec jekyll serve a second time, I got this:

             Error: could not read file /LocalHome/blog/blog.github.io/_posts/2022-07-18-postname/filename.gif: invalid byte sequence in UTF-8
             Error: could not read file /LocalHome/blog/blog.github.io/_posts/2022-07-18-postname/filename.jpg: invalid byte sequence in UTF-8
             Error: could not read file /LocalHome/blog/blog.github.io/_posts/2022-07-18-postname/filename.png: invalid byte sequence in UTF-8

What’s interesting is that the same image file stored in the directory /LocalHome/blog/blog.github.io/_posts/images/filename.png throws no error.

rdyar · July 19, 2022, 3:40pm

I don’t think putting images in the _posts folder is supported very well. If you put the image in assets/images or something like that you have no issues?

Maybe the invalid sequence is cause it is looking for a text file of some sort and getting an image file.

I think there are some other threads in here about people wanting to have the images in with the post and different ways to do that.