Benchmark: Remove duplicate objects by a property

HTML Preparation code:

<script src='https://cdnjs.cloudflare.com/ajax/libs/lodash.js/4.17.5/lodash.min.js'></script>

Script Preparation code:

const DATA = {}

for (let keyIndex = 1; keyIndex <= 10; keyIndex++) {
    const key = `key${keyIndex}`; // Dynamically generate key names
    const objectsArray = [];

    for (let n = 1; n <= 10000; n++) {
      objectsArray.push({ id: n % 35 }); // Push objects with id: n % 35
    }

    DATA[key] = objectsArray; // Assign the array to the key
  }

Tests:

Filter and FindIndex
const groupedWithoutDuplicates = {} Object.keys(DATA).forEach((key) => { groupedWithoutDuplicates[key] = DATA[key].filter( (item, index, self) => index === self.findIndex((t) => t.id === item.id) ) })
Filter and IndexOf
const groupedWithoutDuplicates = {} Object.entries(DATA).forEach(([key, value]) => { const ids = value.map((item) => item.id) groupedWithoutDuplicates[key] = value.filter((item, index) => index === ids.indexOf(item.id)) })
Lodash
const groupedWithoutDuplicates = {} Object.entries(DATA).forEach(([key, value]) => { groupedWithoutDuplicates[key] = _.uniqBy(value, 'id') })

Rendered benchmark preparation results:

Suite status: <idle, ready to run>

Previous results

Test case name	Result
Filter and FindIndex
Filter and IndexOf
Lodash

Fastest: N/A

Slowest: N/A

Latest run results:

Run details: (Test run date: one year ago)

User agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:133.0) Gecko/20100101 Firefox/133.0

Browser/OS: Firefox 133 on Mac OS X 10.15

View result in a separate tab

Test name	Executions per second
Filter and FindIndex	368.5 Ops/sec
Filter and IndexOf	396.7 Ops/sec
Lodash	860.2 Ops/sec

Autogenerated LLM Summary (model gpt-4o-mini, generated one year ago):

LLMs can make mistakes. Check important info.

The benchmark provided focuses on comparing various methods for removing duplicate objects from an array of objects based on a specific property (in this case, the `id` property). The benchmark tests three different approaches for accomplishing this task:

### Approaches Tested

1. **Filter and FindIndex**
   - **Method**: This approach utilizes the `filter` method in conjunction with `findIndex`. It goes through each object in the array and keeps only the first occurrence of each object's `id`.
   - **Pros**:
     - Straightforward, easy to understand, and leverages built-in JavaScript array methods.
   - **Cons**:
     - The use of `findIndex` inside the `filter` method leads to repeated scanning of the array, which can make this approach less efficient for larger datasets. The overall complexity is approximately O(n^2).

2. **Filter and IndexOf**
   - **Method**: In this approach, after mapping the `id`s of the objects to a new array, it uses `indexOf` to determine if the current object's `id` is the first occurrence.
   - **Pros**:
     - Conceptually simple and also uses basic JavaScript functionalities effectively.
   - **Cons**:
     - Similar inefficiencies as the first method. The `indexOf` method scans the array repeatedly, leading to a time complexity of O(n^2) as well.

3. **Lodash: `uniqBy`**
   - **Method**: This approach utilizes the `uniqBy` function from the Lodash library to remove duplicates based on the `id` property.
   - **Pros**:
     - More efficient than the native JavaScript options as Lodash functions are optimized for performance. The complexity is approximately O(n), which is a significant improvement for larger datasets.
     - Using a dedicated utility library like Lodash can lead to cleaner code with less boilerplate.
   - **Cons**:
     - Introduces a dependency on an external library, which might not be desirable in some projects where minimizing dependencies is a goal.
     - Some developers may prefer to avoid external libraries for simple tasks to keep projects lightweight.

### Summary of Benchmark Results

- The benchmark results indicate that the use of Lodash's `uniqBy` is the fastest method, achieving approximately 860 executions per second.
- The second fastest was the "Filter and IndexOf" method, which clocked around 396 executions per second.
- The slowest was "Filter and FindIndex," performing at about 368 executions per second.

### Other Considerations

- **Alternative Approaches**: 
  - One could implement the removal of duplicates using a `Set`, which automatically ensures uniqueness and would offer a cleaner solution with O(n) complexity.
  - Another alternative is to use a simple `reduce` approach combined with a helper object (or `Map`) to track existing `id`s, which could also yield O(n) performance.

- **When to Use Each Method**: 
  - For small data sets or situations where readability is prioritized, the simpler native methods might be sufficient.
  - For performance-critical applications or larger datasets, using Lodash or a custom O(n) solution is recommended.

In conclusion, this benchmark effectively highlights the trade-offs between different methodologies for removing duplicates in JavaScript, weighing the benefits of performance versus dependencies and complexity.

Related benchmarks:

Remove duplicate objects by a property (version: 1)

Comparing performance of: Filter and FindIndex vs Filter and IndexOf vs Lodash

Created: one year ago by: Guest

Jump to the latest result

Filter and FindIndex

Filter and IndexOf

Lodash

Suite status: <idle, ready to run>

Fastest: N/A

Slowest: N/A

Autogenerated LLM Summary (model gpt-4o-mini, generated one year ago):