CSV Filled Table (Via JS) Not Loading In Browser Until Manual Refresh

Help
1

Hi All,
I basically want to render a CSV as a table on my Jekyll site - I have this working with the below script.

Only problem is it never seems to load the table first time, I need to click refresh in the browser for it to appear - the code seems to work fine standalone so am thinking it may have something to do with how its embedded with Jekyll? Does anyone have any ideas as to what could be the cause of this?

You can see the behavior on the page here around 2/3s down - A350 Flights With GH Actions – Clint Bird

Thanks for any hints!

<script>
    fetch('https://raw.githubusercontent.com/clintjb/A350-Tracking/main/flight_data_a350.csv'
    ).then((response) => {
        return response.text();
    }).then((text) => {
        document.getElementById('A350').innerHTML = tbl(text);
    })

    function tbl(csv) {
        return csv.split('\n')
            .map(function (tr, i) {
                return '<tr><td>' +
                    tr.replace(/,/g, '</td><td>') +
                    '</td></tr>';
            })
            .join('\n');
    }

</script>

<table border="0" style='font-size:50%' id="A350"></table>
2

The reason could be because the table is dynamically rendered as a result of resolving a JS Fetch promise.
There’s a possibility that there’s some delay in having the promise resolved. But rest of the page continues to be rendered without waiting for the table being rendered first.

This has nothing to do with Jekyll and is purely in the domain of JS.

Edit:

@clintjb You could add some console.log statements to your JS depending on whether the promise status is pending, or fulfilled.

<script>
  console.log('Fetching CSV data..');
  fetch('https://raw.githubusercontent.com/clintjb/A350-Tracking/main/flight_data_a350.csv'
  ).then((response) => {
      console.log('Fetch Successful.');
      return response.text();
  }).then((text) => {
      console.log('Rendering table..');
      document.getElementById('A350').innerHTML = tbl(text);
      console.log('Table rendered successfully.');
  })

  function tbl(csv) {
      return csv.split('\n')
          .map((tr, i) => '<tr><td>' + tr.replace(/,/g, '</td><td>') + '</td></tr>')
          .join('\n');
  }
</script>
1 Like
3

@clintjb is the issue on Jekyll as a local dev server? Because maybe something is going with livereloading or other assets on the page which are affecting it.

It should not be an issue on a deploy site or plain HTML, as Jekyll doesn’t have a role after the site is built.

Loading issues

I’d say check your dev tools when the page loads.

I’m seeing no gets from document.getElementById when I load that page.

Ordering

You can also make sure that your script only loads only after the body is rendered, so that the document.getElementById('A350') bit has an actual element to find.

You left out an important bit and that is when element is rendered. I found this on your site

<script type="8b0732c79ba4c7d352b62ab6-text/javascript">
  console.log('Fetching CSV data..');
  fetch('https://raw.githubusercontent.com/clintjb/A350-Tracking/main/flight_data_a350.csv'
  // ...
</script>
<table border="0" style="font-size:50%" id="A350"></table>

You may be getting inconsistent results because sometimes the CSV request is slow and then the body is already loaded, other times the request is after and then the JS has no element to attach to and errors out.

Solutions

Reorder:

<table border="0" style="font-size:50%" id="A350"></table>
<script>
  console.log('Fetching CSV data..');
  fetch('https://raw.githubusercontent.com/clintjb/A350-Tracking/main/flight_data_a350.csv'
  // ...
</script>

Or put your script tag high up on the page in the head and have it run after the body has loaded.

<head>
<script>
  window.onload = function () {
    console.log('Fetching CSV data..');
    fetch('https://raw.githubusercontent.com/clintjb/A350-Tracking/main/flight_data_a350.csv'
    // ...
  }
</script>
</head>
<body>
    <table border="0" style="font-size:50%" id="A350"></table>
</body>
4

Advice

Some other improvements to suggest

CDN

Turn your data site in to a static site.

Go to GitHub - clintjb/A350-Tracking: An experiment to see how the new actions feature works combined with an API pull of flight data on the A350 , add .nojekyll as an empty file in the root (for faster deploys) and enable GitHub Pages

You’ll be able to load the CSV as:

"https://clintjb.github.io/A350-Tracking/flight_data_a350.csv"

I did some tests using my CSV of a few lines.

The timing for the two was close in this case when I used curl - of 0.04 seconds or 0.4 seconds without hitting cache.

  • https://michaelcurrin.github.io/network-graphs/assets/data/sample.csv
  • https://raw.githubusercontent.com/MichaelCurrin/network-graphs/master/assets/data/sample.csv

I did notice from the headers that the GitHub.io one has cache length of 600 seconds and the other was 300 seconds, so in that regard it is better.

Fetch async-await

If you are tired of dealing with chained .then promises, use the async await syntax.

Also note that fetch will not throw an error on 4XX or 5XX response, so you should handle that as below, based on the docs.

function tbl(csv) {
  // ..
}

// You have to wrap your main logic in `async` for it to work.
(async function() {
  const URL = 'https://raw.githubusercontent.com/clintjb/A350-Tracking/main/flight_data_a350.csv'
  const TARGET_ID = 'A350'

  const resp = await fetch(URL);
  if (!resp.ok) {
    throw new Error(`${URL} HTTP error: ${resp.status} - ${resp.statusText}`);
  }
  const text = await resp.text();
  const tableHtml = tbl(text)
 
  const el = document.getElementById(TARGET_ID)
  el.innerHtml = tableHtml
})()

Edit: I needed () at the every end so it does something

See my fetch guide

5

BTW Looking at the workflow on the data site

You can make one workflow if you prefer, and have it run in two ways.

on:
  workflow_dispatch:

  schedule:
    - cron: "0 */8 * * *"

jobs:
  # ...
6

Hi @MichaelCurrin
Dude! Your reply is nothing short of brilliant - firstly, went through your points and learnt a few new things in there…
That CDN topic itself was interesting to understand - strangely (it may have something to do with my HTTPS routing via Cloudflare?) but basically after making the swap over, I could directly link to the file e.g.

"https://clintjb.github.io/A350-Tracking/flight_data_a350.csv"

But when I call it in the Jekyll site nothing shows at all :man_shrugging: - nevertheless interesting to understand the advantages and potential performance benefits of converting it directly to static.

More importantly though was that your solutions you proposed unfortunately didn’t work in my case. I tried both ordering solutions but unfortunately they didn’t pan out - I wanted this to be a quick and dirty experiment and in the end this damn table rendering issue is taking more time than anything else…
Sorry bud, with the amount of effort you went to in your feedback I was hoping as much as you this would’ve provided me the solution…

As said though, appreciate the efforts and if its any minor consolation Ive definitely learnt a few things from your insights anyway!

1 Like
7

I’m that you learnt something. Web development is complex and often you don’t know what is there is out there, so you have some direction to explore more and be better equipped.

Yeah it is frustrating when it doesn’t work. Take a break from it or step back and try a different angle, to save getting stuck and feeling defeated.

8

Also I’d recommend watching a YouTube video on dev tools for Chrome or Firefox.

Being able to understand the JS log and network tab and the DOM helps me for daily work and my projects. It tell you what is or isn’t running and clue as to why (such as the error message in the console section)

9

Something else to add as another approach.

If you are tired of writing JS to fetch data on the frontend, consider a backend solution

If you have the CSV in the same repo as the site like in _data/my_file.csv, you can access the data when the site builds as

{{ site.data.my_csv }}

And whenever you change the CSV with a commit, the site will rebuild with latest data.

I have a tutorial I added to Jekyll docs on rendering CSV with Jekyll

If you want to keep the data in a separate repo, you can easily add a command to run add build time

The simplest is to use a curl command in the shell and have the file ignored by Git. i.e. file is not stored in the site repo GitHub but only stored on the server that is building

e.g.

bundle install
curl https://... > _data/my_data.csv
bundle exec jekyll serve --trace

That will work locally and on Netlify. Or GH Pages with GH Actions.

You’ll just have to rebuild the site with a manual trigger whenever the CSV data changes. Or build on a schedule like nighty or weekly, for frequently changing data.

Or you can write a Ruby plugin that fetches the data and handles error codes like 400 properly.

but the one line curl approach is elegant and short and worth doing still

1 Like
10

@clintjb Your idea and script works fine if one accesses the webpage directly via the permalink (the link you provided in the opening post). It also works as expected if one were to refresh the webpage (successful render on every refresh).

The only time it doesn’t work is if I were to navigate to the page from https://clintbird.com/blog/.

I’m hypothesizing that the result should be the same if you were to navigate to the A350 page from say, /business/ or /stuff/, etc.

If the hypothesis is valid, then the bug is in the theme’s global script.

Try building your site with a theme that doesn’t need JavaScript to render a webpage’s content sections.

1 Like
11

It looks like you are getting some great responses here. I’m not sure if you are aware of this, but Jekyll supports data, so you can render it without JS.

Here is a video that specifically covers using custom data files in Jekyll:

There are more videos there if you are interested, but with this one I specifically cover csv, JSON, and other file types. Jekyll also supports TSV (tab separated variables)

1 Like
12

I am seeing the table render fine as well.

But as mentioned before - it is fragile because of the timing of loading the page and the element not being ready. I would suggest one improvement as before and this just making sure the script tag for the table is put after the div it is targeting.

https://github.com/clintjb/clintjb.github.io/blob/master/_includes/a350_csv.html

And on what Ash is saying - if you disable JavaScript for browser, then whole page is just about blank. Which is not typical for a Jekyll site and means it is not so much a static site anymore (maybe more like SinglePage App such as with React). Which means the table and also the rest of the site is not visible to some users and also not to seach engines (which have limited ability to run JS - they sometimes can but limit total processing time of how much they do).

https://clintbird.com/blog/ghactions-a350-flights-post

Screen Shot 2021-11-25 at 7.52.13 am
Screen Shot 2021-11-25 at 7.52.13 am2180Ă—758 22.9 KB

13

So this is the option I went for in the end - created another action to push them across to the main site and used your Tabualte tutorial…

Genuinely mean it when I say thanks @MichaelCurrin so much for the solution (as well as the time you spent highlighting an ENORMOUS number of other minor learning’s, fixes, best practices, code contributions etc) Honestly mate you went so far above and beyond - learnt a lot these past days - thanks!

4 Likes