How to remove items from a list while iterating?

python

list-manipulation

performance-optimization

best-practices

byNikita Barsukov·Nov 12, 2024

Efficiently remove items from a list while iterating using a list comprehension:

filtered_list = [item for item in original_list if not to_remove(item)]

In this snippet, filtered_list will hold the desired list, without the elements to be removed.

Or you can modify the list in place with this loop and slicing approach, while avoiding list resizing pitfalls:

for i in range(len(original_list) - 1, -1, -1):
    if to_remove(original_list[i]):
        del original_list[i]   # Goodbye, unwanted friend!

Advanced techniques for list item removal

If you're playing the performance tuning game, or wrestling with large lists, consider these techniques:

Reverse iteration to delete, keeps you from skipping items:

for i in reversed(range(len(original_list))):
    if to_remove(original_list[i]):
        del original_list[i]  # By going backwards we avoid any conundrums!

In Python 3, use filterfalse from itertools, a memory-efficient filter:

from itertools import filterfalse
original_list[:] = list(filterfalse(to_remove, original_list))  # Work smarter, not harder!

In Python 2, use the memory-efficient xrange instead of range:

for i in xrange(len(original_list) - 1, -1, -1):
    if to_remove(original_list[i]):
        del original_list[i]  # Back to the past. Keep memory lean and mean!

Creating a new list instead of deleting during iteration, beneficial when a majority of elements are to be removed:

new_list = []
for item in original_list:
    if not to_remove(item):
        new_list.append(item)  # A fresh start. A clean slate!

Performance considerations when manipulating lists

Understanding data structures and the cost of operations improves your code performance:

List Comprehension: Python's secret weapon for speed!
.remove(item): Full list scans can slow your processes; use sparingly.
del some_list[i]: Quick removal by index, slight overhead if elements need shifting.
Appending to a new list: Saves removing items, might save the day when most elements are to be discarded.
Linked lists: No direct index access, but a champ when it comes to element removal!

Keep code legible, maintainable, and magic tricks at bay

Code readability and maintainability go a long way. Choose techniques that balance performance, simplicity and utility:

In-place modifications: Using slicing notation somelist[:] = [new_elements] keeps list reference.
External libraries: Worth the trouble only for complicated list manipulations.
Comments and documentation: Explain the "why" behind the "how" in your code. It’s like leaving no trace in the wilderness – but in reverse.

explain-codes / Python / How to remove items from a list while iterating?

Linked

Traverse a list in reverse order in Python



How to delete an item in a list if it exists?



Compute list difference



Is there a simple way to delete a list element by value?



Removing duplicates in lists

