Lodash remove duplicates from an array

javascript

lodash

array

performance

byNikita Barsukov·Jan 20, 2025

Get right to it with Lodash using _.uniq for easy tasks and _.uniqBy for a tad bit more complexity:

Primitives (e.g., numbers):

const nums = [1, 1, 2];
const unique = _.uniq(nums); // [1, 2] 🗡️ One fell swoop removes duplicates.

Objects, by a key (e.g., 'id'):

const items = [{id: 1}, {id: 2}, {id: 1}];
const uniqueItems = _.uniqBy(items, 'id'); // [{id: 1}, {id: 2}] 🧙‍♂️ Like magic, 'id' key helps us find the unique just like that.

For complex objects, that need a deep comparison, cast the _.isEqual spell in unison with _.uniqWith:

const items = [{id: 1, name: 'Apple'}, {id: 2, name: 'Orange'}, {id: 1, name: 'Apple'}];
const uniqueItems = _.uniqWith(items, _.isEqual); // [{id: 1, name: 'Apple'}, {id: 2, name: 'Orange'}] 🪄 Voila! A deep comparison does the trick.

Scaling large datasets

In case of large datasets, the right tool to minimize processing time key is choosing the efficient _.uniqBy function. It knows how to single out patterns in the sea of objects.

Always be mindful of the Lodash version you're using, you know how these breakup changes get.

Filtering on your terms

Taming the custom function

_.uniqBy can take a function that sets your rules for uniqueness:

const users = [{ name: 'John', age: 24 }, { name: 'Jane', age: 24 }, { name: 'John', age: 30 }];
const uniqueUsers = _.uniqBy(users, user => user.name);
// Results in: [{ name: 'John', age: 24 }, { name: 'Jane', age: 24 }] 🧠 Outsmart duplicates with custom rules.

Rally with _.union

_.union can be your war-cry to bring together multiple arrays and send off the duplicates:

const arr1 = [1, 2];
const arr2 = [3, 2];
const uniqueCombined = _.union(arr1, arr2); // [1, 2, 3] ⚔️ To war, send off the extras.

Maximizing performance

Consider these for performance optimization:

Benchmark Lodash methods to find the fastest.
Use native JavaScript when Lodash's additional abracadabra isn't needed.
Break down and process arrays in parallel. Remember, divide and conquer.

explain-codes / Javascript / Lodash remove duplicates from an array

Linked

Remove duplicate values from JS array



How to get distinct values from an array of objects in JavaScript?



How to remove all duplicates from an array of objects?



How can I find and update values in an array of objects?



How to get unique values in an array



Get JavaScript object from array of objects by value of property



Array.push() if does not exist?



Scaling large datasets Filtering on your terms