eid's - is there an optimised way to get a list of all eid's in the tdb?

Phuket2

Is there a optimised way to get a list of all the eid's (in the sequence they appear in the tdb). I am asking because it would be a nice way to implement a virtual table. Just have a list of eids and only retrieving the record when table row needs to be drawn using the eid. Would be very fast.

What would been even easier and better if tdb/table was scriptable which I am pretty sure it's not, I also get why it's not.

I realise I could build the list of eid's myself. But I imagine it would be self defeating if the table was getting large.

If what I am asking for is not available, maybe you have a work around in mind.
Regards

msiemens

Hey @Phuket2, there's currently no built-in way to get all EIDs. As a workaround you could use TinyDB's internal API to get the data dictionary and then get all keys which represent EIDs like this:

eids = list(db._read().keys())

Cheers!
Markus

Phuket2

msiemens
Thanks, this seems quick...
Not sure how to add code here, but i will try. I did the below test. Runs very fast, inc the creation, printing, etc...

from random import choice
try:
	from tinydb import TinyDB, Query
except :
	print('tinydb is not installed, exiting.')
	raise ImportError

from faker import Faker
fake = Faker()

def create_fake_db(filename, num_records):
	lst = []
	for i in range(num_records):
		d = dict(first=fake.first_name(),
				last=fake.last_name(),
				age=fake.random_int(min=18, max=89) 
				)
		lst.append(d)
		
	with TinyDB(filename) as db:
		db.insert_multiple(lst)
		eids = list(db._read().keys())
	return eids
		
if __name__ == '__main__':
	filename='eid_test.json'
	eid_list = create_fake_db(filename,500)
	print(eid_list)
	
	# pick a random record to display and get a record count
	with TinyDB(filename) as db:
		item = db.get(eid=choice(eid_list))
		num_records = len(db)
		
	print('Records in database={}'.format(num_records))
	print(item)

answerquest

Thanks! This helped also in my use case where I wanted just the counts of each of my tables and didn't want to extract the full data unnecessarily. Sharing a code snippet:

for tablename in db.tables():
	feedDb = db.table(tablename)
	count = len( feedDb._read().keys() )
	# use count value for some operations

Just for the curious, the .count() method needs a query argument and I had no way to make a single valid query that would select all the elements in all the different tables as I'm looping through them. So I couldn't go the usual way. I guess it's better.. the query route would probably have lower performance anyway.

answerquest

answerquest Yikes I just found the len(db) command works for tables too. #headsmack

Phuket2

answerquest , I am not sure if I am doing something wrong or not. But len(db) does not appear to return a count of all the documents in a database as per the docs. From what I can see, len(db) returns the count of the 'default' table or the table pointed to by setting TinyDB.DEFAULT_TABLE. My simple test below. In my test, there are no entries in the 'default' table, however a table called 'foo' exists with 1 doc. I get the reason why its the way it is, but the documentation gives me the impression that I would get a sum of all the documents in all the tables, even though it would not be that useful.

from tinydb import TinyDB

fn = 'db_len_test.json'
d1 = {'name':'Fred Jones'}

db = TinyDB(fn)

tbl = db.table('foo')
tbl.insert(d1)
print('len of table foo = ', len(tbl))
print('len of database = ', len(db))

db.close()

msiemens

Phuket2 You're right, len(db) returns the same result as len(db.all()) which runs on the default table. Could you open an issue on GitHub for the documentation to be improved in this regard?

Phuket2

msiemens , no problems I will do