Explain Codes LogoExplain Codes Logo

How to remove all duplicates from an array of objects?

javascript
prompt-engineering
functions
callbacks
Alex KataevbyAlex KataevΒ·Oct 10, 2024
⚑TLDR

Zero in on duplicates in an array of objects using filter() together with findIndex(). This one-liner retains only the first unique copy of an object based on a designated key. In the following example, we keep or discard objects based on their id:

const items = [{ id: 1 }, { id: 2 }, { id: 1 }]; // Here be the magic πŸ§™β€β™‚οΈ const unique = items.filter((obj, idx, arr) => idx === arr.findIndex((t) => t.id === obj.id)); console.log(unique); // Output: [{ id: 1 }, { id: 2 }] //You just fought off the clones πŸ‘―β€β™‚οΈ

Deep object comparison

Let's crank up the heat a bit. Complex objects may call for JSON.stringify(), for a more deep-seated comparison. Be mindful, performance nerds, large datasets might suffer a screeching slowdown:

const unique = items.filter((obj, idx, arr) => arr.findIndex(t => JSON.stringify(t) === JSON.stringify(obj)) === idx);

Caution: Property order matters when using JSON.stringify(). If you've got fluctuating property orders, better saddle up with more intricate comparison function.

Building helpful utilities

For detecting duplicates, bag together your logic using isPropValuesEqual, a utility function that makes checking certain properties across objects as easy as Sunday morning:

function isPropValuesEqual(obj1, obj2, props) { // Now we're cooking with gas πŸ”₯ return props.every(prop => obj1[prop] === obj2[prop]); } const unique = items.filter((obj, idx, arr) => idx === arr.findIndex(t => isPropValuesEqual(t, obj, ['name', 'place']))); // You can now spot a doppelgΓ€nger from a mile away πŸ‘€

Likewise, embrace the might of getUniqueItemsByProperties, designed to dynamically weed out duplicates according to multiple properties:

function getUniqueItemsByProperties(items, props) { return items.filter((item, index, self) => index === self.findIndex(t => isPropValuesEqual(t, item, props))); } // You're the life of the party now 🍾πŸ₯‚

Taking advantage of ES6+ and Map

The release of ES6 introduced Map, great friends for those in dire need of unique key-value pairs:

// Brace yourselves, magic stuff coming upπŸ¦„ const uniqueItems = [...new Map(items.map(item => [JSON.stringify(item), item])).values()];

By populating new map with key-value pairs (object key = 'stringified' object), then turning the .values() iterator back into an array, you cast an effective spell to annihilate duplicates.

Complex keys and speedy performance

When unique identifiers go M.I.A., create compound keys by knocking together property values with function like objToId:

function objToId(obj, props) { //Jamming properties together to simulate complex keysπŸ‘·β€β™€οΈ return props.map(prop => obj[prop]).join('_'); } const unique = [...new Map(items.map(item => [objToId(item, ['name', 'place']), item])).values()]; // You turned into a master key maker πŸ”

Note of caution to those upholding original element order - Map-based methods might leave you high and dry. Avoid JSON.stringify() for large data sets due to performance cost. Check out external libraries equipped for complex object structures to avoid splitting hairs.

Bonus: Modern techniques for uniqueness

Remember to use Set although it doesn't play well with unique object references. Here's how you can make it work with array spreading and comparator functions:

// Getting innovative with `Set` and `map()` πŸ•΅οΈβ€β™€οΈ const unique = Array.from(new Set(items.map(item => JSON.stringify(item)))).map(item => JSON.parse(item));

For state-of-the-art multi-property uniqueness, leverage every() within filter(). Preserving last instance of duplicates? Swap findIndex with a tailor-made findLastIndex function.