How to read 2+gb csv file on python?

Question

Looking to read,write a 2+gb csv file on python.

Answer 1

Hello, Kinjal Patel!!

I'm with Simon Sauter, but if you want an example then here it is

https://code.sololearn.com/c6r4tVNqNlvD/?ref=app

for more or extra information you can go and check the offical documentation of pandas

Happy Learning!

Answer 2

You can use the pandas.read_csv() method with the chunksize argument.

Answer 3

You can import the csv module. Then use it to read line by line or in chunks.Don't read in an entire file of that size into a variable as memory will be an issue.

import csv

Answer 4

Kinjal Patel ,

what kind of operations are you trying to perform after reading the data. do you need to:

- filter
- modify / update
- sort
- delete (data, columns, rows)
- insert (data, columns, rows)
...

of the data?