How to find the duplicate records ??

Question

I have input file like csv or ext file. I need to find out & eliminate the duplicate records

Answer 1

There can be various approaches to that. You can parse the data and save it to a dictionary, where the possibly duplicated entry is the key (thus avoiding duplication automatically).
You can read it into a list and make it a set, disposing of duplicates, too. It depends on the structure of the data to process, mostly.