# Loop through data file with "uniq" array filter

**URL:** <https://talk.jekyllrb.com/t/loop-through-data-file-with-uniq-array-filter/425>\
**Category:** Help\
**Created:** [April 12, 2017, 1:05am UTC](https://talk.jekyllrb.com/t/loop-through-data-file-with-uniq-array-filter/425 "2017-04-12T01:05:26Z")\
**Posts on this page:** 15\
**Page:** 1

<div class="post-metadata">

**Author:** ![andrewlobo](https://avatars.discourse-cdn.com/v4/letter/a/9d8465/32.png) [@andrewlobo](https://talk.jekyllrb.com/u/andrewlobo)\
**Post date:** [April 12, 2017, 1:05am UTC](https://talk.jekyllrb.com/t/loop-through-data-file-with-uniq-array-filter/425/1 "2017-04-12T01:05:26Z")

</div>

Hello, I'm trying to loop through a data file, and filter the output so that duplicate values of a field are removed. Any help? Many thanks in advance. Below is what I'm working with.

### Data file:

```
- title: Heading 1
  source: Source 1

- title: Heading 1
  source: Source 2

- title: Heading 2
  source: Source 3

- title: Heading 2
  source: Source 4
```

### HTML & Liquid:

```

```

 {% for item in site.data.elements %} {% assign my\_filter = item.title | split: ' ' | uniq %} - {{ my\_filter }}
 {% endfor %}

### Expected output:

```

```

- Heading 1
- Heading 2

---

<div class="post-metadata">

**Author:** ![DavidDarnes](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.jekyllrb.com/daviddarnes/32/1349_2.png) [@DavidDarnes](https://talk.jekyllrb.com/u/DavidDarnes)\
**Post date:** [April 12, 2017, 8:21am UTC](https://talk.jekyllrb.com/t/loop-through-data-file-with-uniq-array-filter/425/2 "2017-04-12T08:21:39Z")

</div>

I think you need to assign the array before you loop through the items. So maybe something like this?

```auto
<!-- get an array of the titles without dupes -->
{% capture filtered_titles_array %}{% for item in site.data.elements %}{{ item.title }}|{% endfor %}{% endcapture %}

<!-- start loop -->
{% for item in site.data.elements %}

    <!-- create a filter from the title -->
    {% assign filtered_title = item.title | append: '|' %}

    <!-- check the filter is in the title array -->
    {% if filtered_titles_array contains filtered_title %}
        <h2>{{ item.title }}</h2>
        <p>{{ item.source }}</p>
    {% endif %}

    <!-- remove title filter from the original array -->
    {% assign filtered_titles_array = filtered_titles_array | remove: filtered_title %}
{% endfor %}

```

This seems quite hacky, and I’m not even sure it’ll work. Let me know if it does. Anyone else who sees this could they check my math? Thanks

Edit: Turns out you don’t need to filter the array, the `remove` filter removes any instance of the string which is handy

---

<div class="post-metadata">

**Author:** ![andrewlobo](https://avatars.discourse-cdn.com/v4/letter/a/9d8465/32.png) [@andrewlobo](https://talk.jekyllrb.com/u/andrewlobo)\
**Post date:** [April 12, 2017, 2:04pm UTC](https://talk.jekyllrb.com/t/loop-through-data-file-with-uniq-array-filter/425/3 "2017-04-12T14:04:49Z")

</div>

Thanks very much, David! It works on my end. I don’t know enough to know whether it’s hacky. Let’s see if anyone else chimes in.

---

<div class="post-metadata">

**Author:** ![ashmaroli](https://avatars.discourse-cdn.com/v4/letter/a/df705f/32.png) [@ashmaroli](https://talk.jekyllrb.com/u/ashmaroli)\
**Post date:** [April 13, 2017, 2:58pm UTC](https://talk.jekyllrb.com/t/loop-through-data-file-with-uniq-array-filter/425/4 "2017-04-13T14:58:28Z")

</div>

Another way to get this done is as follows:

```auto
{% assign titles = site.data.elements | map: 'title' | uniq %}

<ul>
  {% for item in titles %}
    <li>{{ item }}</li>
  {% endfor %}
</ul>

```

---

<div class="post-metadata">

**Author:** ![DavidDarnes](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.jekyllrb.com/daviddarnes/32/1349_2.png) [@DavidDarnes](https://talk.jekyllrb.com/u/DavidDarnes)\
**Post date:** [April 13, 2017, 3:40pm UTC](https://talk.jekyllrb.com/t/loop-through-data-file-with-uniq-array-filter/425/5 "2017-04-13T15:40:22Z")

</div>

This is far more elegant, thanks! I need to look into actually how map works 😆, I’ve used it in es6 code but not sure what it’s doing

---

<div class="post-metadata">

**Author:** ![ashmaroli](https://avatars.discourse-cdn.com/v4/letter/a/df705f/32.png) [@ashmaroli](https://talk.jekyllrb.com/u/ashmaroli)\
**Post date:** [April 13, 2017, 5:38pm UTC](https://talk.jekyllrb.com/t/loop-through-data-file-with-uniq-array-filter/425/6 "2017-04-13T17:38:54Z")

</div>

`map` simply creates a new array of values of the named property (in the above example, `title`)

---

<div class="post-metadata">

**Author:** ![DavidDarnes](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.jekyllrb.com/daviddarnes/32/1349_2.png) [@DavidDarnes](https://talk.jekyllrb.com/u/DavidDarnes)\
**Post date:** [April 13, 2017, 5:53pm UTC](https://talk.jekyllrb.com/t/loop-through-data-file-with-uniq-array-filter/425/7 "2017-04-13T17:53:16Z")

</div>

Ah I see, but surely this only filters out the duplicate titles? I think in @andrewlobo’s case they want to compare titles of the items and strip out the items which have duplicate titles. This would only loop through the titles

---

<div class="post-metadata">

**Author:** ![ashmaroli](https://avatars.discourse-cdn.com/v4/letter/a/df705f/32.png) [@ashmaroli](https://talk.jekyllrb.com/u/ashmaroli)\
**Post date:** [April 14, 2017, 2:59pm UTC](https://talk.jekyllrb.com/t/loop-through-data-file-with-uniq-array-filter/425/8 "2017-04-14T14:59:11Z")

</div>

> [@DavidDarnes](#):
>
> This would only loop through the titles

True, from Andrew’s “HTML-Liquid” Example and “Expected Output”, it looks like this is all he wants…

---

<div class="post-metadata">

**Author:** ![andrewlobo](https://avatars.discourse-cdn.com/v4/letter/a/9d8465/32.png) [@andrewlobo](https://talk.jekyllrb.com/u/andrewlobo)\
**Post date:** [April 14, 2017, 3:33pm UTC](https://talk.jekyllrb.com/t/loop-through-data-file-with-uniq-array-filter/425/9 "2017-04-14T15:33:07Z")

</div>

Yep, filtering out duplicate titles was all I was trying to do. Sorry, @DavidDarnes , I wonder if adding a "source" field in the original post was overcomplicating the example. @ashmaroli, thanks very much for adding your method. I'd never be able to figure that out myself. Very grateful to both of you.

---

<div class="post-metadata">

**Author:** ![DavidDarnes](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.jekyllrb.com/daviddarnes/32/1349_2.png) [@DavidDarnes](https://talk.jekyllrb.com/u/DavidDarnes)\
**Post date:** [April 14, 2017, 7:10pm UTC](https://talk.jekyllrb.com/t/loop-through-data-file-with-uniq-array-filter/425/10 "2017-04-14T19:10:13Z")

</div>

No problem! Might keep that example up just in case someone wants to achieve specifically that. Thanks for explaining @ashmaroli 👍

---

<div class="post-metadata">

**Author:** ![typo3ua](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.jekyllrb.com/typo3ua/32/2696_2.png) [@typo3ua](https://talk.jekyllrb.com/u/typo3ua)\
**Post date:** [July 24, 2020, 5:46pm UTC](https://talk.jekyllrb.com/t/loop-through-data-file-with-uniq-array-filter/425/11 "2020-07-24T17:46:24Z")

</div>

Hello! Somebody found resolved?

I have array in JSON format

```auto
[
  {
      "id": 0,
      "type": "Type Zero",
      "date": "2020-02-22",
      "phone": 380001000000,
      "images": ""
  },
  {
      "id": 1,
       "type": "Type One",
      "date": "2020-04-24",
      "phone": 380001000000,
      "images": ""
  },
  {
      "id": 2,
      "type": "Type Two",
      "date": "2020-07-08",
      "phone": 380002000000,
      "images": ""
  }
]

```

I only need to leave unique records. How to do it? for example by the field “phone” is it possible?

---

<div class="post-metadata">

**Author:** ![andrewlobo](https://avatars.discourse-cdn.com/v4/letter/a/9d8465/32.png) [@andrewlobo](https://talk.jekyllrb.com/u/andrewlobo)\
**Post date:** [July 25, 2020, 3:24pm UTC](https://talk.jekyllrb.com/t/loop-through-data-file-with-uniq-array-filter/425/12 "2020-07-25T15:24:11Z")

</div>

ashmaroli’s answer is correct:

> [@Loop through data file with "uniq" array filter](https://talk.jekyllrb.com/t/loop-through-data-file-with-uniq-array-filter/425/4):
>
> Another way to get this done is as follows: {% assign titles = site.data.elements | map: 'title' | uniq %} \<ul\> {% for item in titles %} \<li\>{{ item }}\</li\> {% endfor %} \</ul\>

For your array, I’m guessing it would look something like this (replace `name-of-array` with the name of your JSON file):

```auto
{% assign records = site.data.name-of-array | map: 'phone' | uniq %}

{% for item in records %}
  <div>{{ item }}</div>
{% endfor %}

```

---

<div class="post-metadata">

**Author:** ![typo3ua](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.jekyllrb.com/typo3ua/32/2696_2.png) [@typo3ua](https://talk.jekyllrb.com/u/typo3ua)\
**Post date:** [July 25, 2020, 4:00pm UTC](https://talk.jekyllrb.com/t/loop-through-data-file-with-uniq-array-filter/425/13 "2020-07-25T16:00:02Z")

</div>

Thank you for the reply I know the way I saw him, but I do not receive that what I need…

I have the array in JSON format:

```auto
[
  {
      "id": 0,
      "type": "Type Zero",
      "date": "2020-02-22",
      "phone": 380001000000,
      "images": ""
  },
  {
      "id": 1,
       "type": "Type One",
      "date": "2020-04-24",
      "phone": 380001000000,
      "images": ""
  },
  {
      "id": 2,
      "type": "Type Two",
      "date": "2020-07-08",
      "phone": 380002000000,
      "images": ""
  }
]

```

`{%- assign model = site.data.models | map: "phone" | uniq -%}`

I get: `Liquid Exception: no implicit conversion of String into Integer in`… hints at “phone”, but is not problem i changed it to…

```auto
[
  {
      "id": 0,
      "type": "Type Zero",
      "date": "2020-02-22",
      "phone": "380001000000",
      "images": ""
  },
  {
      "id": 1,
       "type": "Type One",
      "date": "2020-04-24",
      "phone": "380001000000",
      "images": ""
  },
  {
      "id": 2,
      "type": "Type Two",
      "date": "2020-07-08",
      "phone": "380002000000",
      "images": ""
  }
]

```

`{%- assign model = site.data.models | map: "phone" | uniq -%}`

```auto
<ul>
  {%- for too in model -%}
  <li>{{ too.id }} / {{ too.type }} / {{ too.data }} / {{ too.phone }} / {{ too.images }}</li>
  {%- endfor -%}
</ul>

```

…in the array `model` i expected get

```auto
[
  {
      "id": 0,
      "type": "Type Zero",
      "date": "2020-02-22",
      "phone": "380001000000",
      "images": ""
  },
  {
      "id": 2,
      "type": "Type Two",
      "date": "2020-07-08",
      "phone": "380002000000",
      "images": ""
  }
]

```

```auto
<ul>
  <li> 0 / Type Zero / 2020-02-22 / 380001000000 / </li>
  <li> 1 / Type Two / 2020-07-08 / 380002000000 / </li>
</ul>

```

but in the array `model` i get nothing if exactly I see

Now I see only one field “phone” i do not understand where other fields…?

```auto
<ul>
  <li> / / / 380001000000 / </li>
  <li> / / / 380002000000 / </li>
</ul>

```

Why? Did I miss something?

---

<div class="post-metadata">

**Author:** ![ashmaroli](https://avatars.discourse-cdn.com/v4/letter/a/df705f/32.png) [@ashmaroli](https://talk.jekyllrb.com/u/ashmaroli)\
**Post date:** [July 28, 2020, 6:35pm UTC](https://talk.jekyllrb.com/t/loop-through-data-file-with-uniq-array-filter/425/14 "2020-07-28T18:35:19Z")

</div>

@typo3ua The answer I gave years ago was tailored for the question asked at the time. Your case is different and there isn’t a simple solution for it.

When the `map` filter is used, the result is a new array with just the values of the given attribute.  
Therefore `{{ site.data.models | map: "phone" }}` will just result in the array `[380001000000, 380001000000, 380002000000]`. When this is passed onto the `uniq` filter, the resulting array would just be `[380001000000, 380002000000]`.

…

---

<div class="post-metadata">

**Author:** ![typo3ua](https://yyz2.discourse-cdn.com/flex030/user_avatar/talk.jekyllrb.com/typo3ua/32/2696_2.png) [@typo3ua](https://talk.jekyllrb.com/u/typo3ua)\
**Post date:** [July 28, 2020, 7:04pm UTC](https://talk.jekyllrb.com/t/loop-through-data-file-with-uniq-array-filter/425/15 "2020-07-28T19:04:57Z")

</div>

@ashmaroli thank you for the reply.  
Yes! I am already understood. I used the example by @DavidDarnes and it works for me… I had hope for the bast solution, but at the moment his none…