Benchmark: Dedupe Obj by prop (150 items)

Script Preparation code:

var test = [
  { id: '649c69eaa50f1d609949c25f' },
  { id: '649b925d6c544a0fc4bee0ec' },
  { id: '649c81097035ecbfdafd6c26' },
  { id: '649c94eee84a19628209d172' },
  { id: '649bf4b554c98dff97659ea8' },
  { id: '649c7a75837b6102d2190846' },
  { id: '649b34c0d95e01cfffcd9eb8' },
  { id: '649b60e4cf83d21960f598c8' },
  { id: '649c84f1a08aae92538e635c' },
  { id: '649cb032da90ba90bd735383' },
  { id: '649c9ca83180cf2a8366783e' },
  { id: '649c48249e21e4bdcba37975' },
  { id: '649c7a75837b6102d2190846' },
  { id: '649c94eee84a19628209d172' },
  { id: '649bf6f970f6380b5758fbad' },
  { id: '649cb032da90ba90bd735383' },
  { id: '649c81369e21e4bdcba39b06' },
  { id: '649b60e4cf83d21960f598c8' },
  { id: '649b4e94f269eeeb85c56a94' },
  { id: '649bf4b554c98dff97659ea8' },
  { id: '649cb032da90ba90bd735383' },
  { id: '649c9ca83180cf2a8366783e' },
  { id: '649c48249e21e4bdcba37972' },
  { id: '649cb032da90ba90bd735386' },
  { id: '649c9ca83180cf2a8366782e' },
];

var array = new Array(6).fill(test).flatMap(value => [...value])

Tests:

filter
function dedupeByProperty(items, prop) { const merged = []; return items.filter((c) => { const reference = c[prop]; const isNotInArray = !merged.includes(reference); if (isNotInArray) merged.push(reference); return isNotInArray; }); }; dedupeByProperty(array, 'id');
hash
function dedupeByProperty(items, prop) { const hash = {}; const merged = []; const len = items.length - 1; for (let i = 0; i <= len; i += 1) { const ref = items[i][prop]; if (!hash[ref]) { hash[ref] = true; merged.push(items[i]); } } return merged; }; dedupeByProperty(array, 'id');
set
function dedupeByProperty(items, prop) { const merged = new Set(); return items.filter((c) => { const reference = c[prop]; const isNotInSet = !merged.has(reference); if (isNotInSet) merged.add(reference); return isNotInSet; }); }; dedupeByProperty(array, 'id');

Rendered benchmark preparation results:

Suite status: <idle, ready to run>

Previous results

Test case name	Result
filter
hash
set

Fastest: N/A

Slowest: N/A

Latest run results:

No previous run results

This benchmark does not have any results yet. Be the first one to run it!

Autogenerated LLM Summary (model llama3.2:3b, generated one year ago):

LLMs can make mistakes. Check important info.

Let's break down the benchmark and explain what's being tested.

**Benchmark Purpose:**
The purpose of this benchmark is to compare three different algorithms for deduplication (removing duplicates) from an array of objects, based on a specific property (in this case, "id").

**Algorithms Being Compared:**

1. **Filter Method**: This method uses the `filter()` method to create a new array with only unique elements. It works by iterating over each element in the original array and checking if it already exists in the merged array.
2. **Hash Method**: This method uses an object (`hash`) to keep track of seen values, and pushes each value onto the merged array only if it's not already present.
3. **Set Method**: This method uses a `Set` data structure to keep track of unique values.

**Pros and Cons:**

1. **Filter Method**:
	* Pros: Simple, efficient for small arrays, but may be slower for large arrays due to the overhead of filtering.
	* Cons: May have poor performance if the array is not sorted or if the property being used is not hashable.
2. **Hash Method**:
	* Pros: Fast and efficient, especially for large arrays, as it uses a data structure that can quickly look up values.
	* Cons: Requires extra memory to store the `hash` object, which may be an issue for very large datasets.
3. **Set Method**:
	* Pros: Efficient and fast, with good performance even for large arrays, as it uses a data structure optimized for membership testing.
	* Cons: May require additional dependencies (e.g., `Set` is not built-in to all JavaScript environments) or may be slower than the hash method due to the overhead of creating and managing a set.

**Other Considerations:**

* **Memory Usage**: The filter method may have higher memory usage than the other two methods, as it creates an entirely new array with unique elements.
* **Property Complexity**: If the property being used for deduplication is complex or has specific requirements (e.g., case-insensitive), one of the methods may be more suitable than another.

**Library Usage:**
The `Set` method uses a built-in JavaScript data structure, but if you're using an older browser or environment that doesn't support `Set`, you'll need to use a polyfill or alternative implementation.

There are other alternatives for deduplication algorithms, such as:

* **Map Method**: Using a map (e.g., `new Map()`) to keep track of unique values.
* **Array Set Methods**: Using built-in array methods like `reduce()` and `filter()` in combination with other data structures.

Keep in mind that the choice of algorithm ultimately depends on your specific requirements, performance constraints, and available resources.

Related benchmarks:

Dedupe Obj by prop (150 items) (version: 0)

Comparing performance of: filter vs hash vs set

Created: 2 years ago by: Guest

Jump to the latest result

filter

hash

set

Suite status: <idle, ready to run>

Fastest: N/A

Slowest: N/A

No previous run results

Autogenerated LLM Summary (model llama3.2:3b, generated one year ago):