Loop through data file with "uniq" array filter


Hello, I'm trying to loop through a data file, and filter the output so that duplicate values of a field are removed. Any help? Many thanks in advance. Below is what I'm working with.

Data file:

- title: Heading 1
  source: Source 1

- title: Heading 1
  source: Source 2

- title: Heading 2
  source: Source 3

- title: Heading 2
  source: Source 4

HTML & Liquid:

    {% for item in site.data.elements %} {% assign my_filter = item.title | split: ' ' | uniq %}
  • {{ my_filter }}
  • {% endfor %}

Expected output:

  • Heading 1
  • Heading 2


I think you need to assign the array before you loop through the items. So maybe something like this?

<!-- get an array of the titles without dupes -->
{% capture filtered_titles_array %}{% for item in site.data.elements %}{{ item.title }}|{% endfor %}{% endcapture %}

<!-- start loop -->
{% for item in site.data.elements %}

    <!-- create a filter from the title -->
    {% assign filtered_title = item.title | append: '|' %}

    <!-- check the filter is in the title array -->
    {% if filtered_titles_array contains filtered_title %}
        <h2>{{ item.title }}</h2>
        <p>{{ item.source }}</p>
    {% endif %}

    <!-- remove title filter from the original array -->
    {% assign filtered_titles_array = filtered_titles_array | remove: filtered_title %}
{% endfor %}

This seems quite hacky, and I’m not even sure it’ll work. Let me know if it does. Anyone else who sees this could they check my math? Thanks

Edit: Turns out you don’t need to filter the array, the remove filter removes any instance of the string which is handy


Thanks very much, David! It works on my end. I don’t know enough to know whether it’s hacky. Let’s see if anyone else chimes in.


Another way to get this done is as follows:

{% assign titles = site.data.elements | map: 'title' | uniq %}

  {% for item in titles %}
    <li>{{ item }}</li>
  {% endfor %}


This is far more elegant, thanks! I need to look into actually how map works :laughing:, I’ve used it in es6 code but not sure what it’s doing


map simply creates a new array of values of the named property (in the above example, title)


Ah I see, but surely this only filters out the duplicate titles? I think in @andrewlobo’s case they want to compare titles of the items and strip out the items which have duplicate titles. This would only loop through the titles


True, from Andrew’s “HTML-Liquid” Example and “Expected Output”, it looks like this is all he wants…


Yep, filtering out duplicate titles was all I was trying to do. Sorry, @DavidDarnes , I wonder if adding a "source" field in the original post was overcomplicating the example. @ashmaroli, thanks very much for adding your method. I'd never be able to figure that out myself. Very grateful to both of you.


No problem! Might keep that example up just in case someone wants to achieve specifically that. Thanks for explaining @ashmaroli :+1: