Eficient way to write to a file

Question

I have a 1-3 GB json file in which I am adding data. the problem is that as the file grow it takes more time to open the file, write to it and close. I add data to the file every 5-10 seconds.
Right now I am using 'with open()' in a function and I was wondering if I store the file in a class will help to improve the performance?
Is there a way to add to the file without storing all the data into memory?

Answer 1

Mirielle[ Exams ] I know I can read the file in chunks like this:

def read_in_chunks(file_object, chunk_size=1024):
while True:
data = file_object.read(chunk_size)
if not data:
break
yield data

with open('file.json') as f:
for piece in read_in_chunks(f):
pass

but the file is a json and I don't really know how to read only the last element.

Answer 2

https://gist.github.com/iyvinjose/e6c1cb2821abd5f01fd1b9065cbc759d