Remove duplicate items in List
Python List - Remove Duplicate Items
We know that Python List can contain values of any data type. This also means that a list can contain duplicate items. And during some scenarios, we may not want our list to have duplicate items. So, in this tutorial, we will learn how to remove duplicate items from a Python List.
1. Remove duplicate items from List using Membership Operator
The syntax of membership operator is
x in collection
x not in collection
where x is element, in is Python keyword and collection is any Python collection.
x in collection
returns True if x is present in collection, False otherwise.
x not in collection
returns True if x is not present in collection, False otherwise.
Following is a step by step process to remove duplicate items from a Python List using membership operator.
- Take a list of items. We should create a new list with duplicate elements removed from this list.
- Create a new empty list.
- For each item in the given original list, check if item is present in the new list.
- If the item is not present in the new list, add the item to the new list.
In this example, we will take a list of numbers, with some of them being duplicates. Then, we will apply the above step by step process to discard the duplicates and include only unique items.
Python Program
list1 = [2, 3, 7, 3, 6, 2, 8, 8]
list2 = []
for item in list1:
if item not in list2:
list2.append(item)
print(list2)
Output
[2, 3, 7, 6, 8]
2. Remove duplicate items from List in-place
In the above example, we copied unique elements to a new list. But, what if we would like to remove duplicate items from the original list.
Follow these steps to remove duplicate items from list inplace.
- Take a list of elements. We are going to remove duplicates present in this list.
- Initialize index with 1. Since, the first element is going to be unique, we are checking from second position instead of first.
- Start a while loop with condition that index should be less than the length of given list.
- If element at index is present in the sub list, part of the list that is present before this index position, remove the element at index. Otherwise, increment the index and continue with while loop.
- After the loop is done, we are left with unique elements in the list.
To the find the sub-list present before the index during each while loop iteration, we use Python Slicing technique.
Python Program
list1 = [2, 3, 7, 3, 6, 2, 8, 8]
index = 1
while index < len(list1):
if list1[index] in list1[ : index]:
list1.pop(index)
else:
index += 1
print(list1)
Output
[2, 3, 7, 6, 8]
Summary
In this tutorial of Python Examples, we learned how to remove duplicate items from a Python List.