How to use matches() with int values?

nicr

I'm trying to use matches() to determine if the database contains any one of a set of int values but am getting an exception thrown.

The following is some sample code that illustrates the issue (the exception information follows the code).

from tinydb import TinyDB, Query
db = TinyDB('testdb.json')
db.purge()

db.insert({'type': 'apple', 'ID': 1, 'ID2': '1', 'name': 'a'})
db.insert({'type': 'apple', 'ID': 2, 'ID2': '2', 'name': 'b'})
db.insert({'type': 'apple', 'ID': 3, 'ID2': '3', 'name': 'c'})
db.insert({'type': 'apple', 'ID': 4, 'ID2': '4', 'name': 'd'})

_attr = Query()

db.search(_attr['type'] == 'apple')

db.search((_attr['type'] == 'apple') & (_attr['name'].matches('[ab]')))

db.search((_attr['type'] == 'apple') & (_attr['ID2'].matches('[12]')))
db.search((_attr['type'] == 'apple') & (_attr['ID2'].matches('(1|2)')))

db.search((_attr['type'] == 'apple') & (_attr['ID'].matches('[12]')))
db.search((_attr['type'] == 'apple') & (_attr['ID'].matches('(1|2)')))
db.search((_attr['type'] == 'apple') & (str(_attr['ID']).matches('(1|2)')))

The exception is:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\nicholasr\Documents\FME\Plugins\Python\tinydb\database.py", line 478, in search
    docs = [doc for doc in self.all() if cond(doc)]
  File "C:\Users\nicholasr\Documents\FME\Plugins\Python\tinydb\database.py", line 478, in <listcomp>
    docs = [doc for doc in self.all() if cond(doc)]
  File "C:\Users\nicholasr\Documents\FME\Plugins\Python\tinydb\queries.py", line 45, in __call__
    return self.test(value)
  File "C:\Users\nicholasr\Documents\FME\Plugins\Python\tinydb\queries.py", line 61, in <lambda>
    return QueryImpl(lambda value: self(value) and other(value),
  File "C:\Users\nicholasr\Documents\FME\Plugins\Python\tinydb\queries.py", line 45, in __call__
    return self.test(value)
  File "C:\Users\nicholasr\Documents\FME\Plugins\Python\tinydb\queries.py", line 136, in impl
    return test(value)
  File "C:\Users\nicholasr\Documents\FME\Plugins\Python\tinydb\queries.py", line 243, in <lambda>
    return self._generate_test(lambda value: re.match(regex, value),
  File "C:\Users\nicholasr\AppData\Local\Programs\Python\Python36-32\lib\re.py", line 172, in match
    return _compile(pattern, flags).match(string)
TypeError: expected string or bytes-like object

thanks,
Nic

msiemens

Hey nicr ,

that's an interesting corner case. So regular expressions only work on string values. There might be an argument for matches queries to always convert the field to a string, but I think it might be really unintuitive as booleans and other types would get converted to strings too.

The other thing here is, that right now TinyDB has a way to check if a document field with a list of values contains a given field, but not the other way around like you mention. Basically, we only can check, if a field called foo – which is a list – contains a value like 5 (by calling User.foo.any(5)), but not if a field called foo is contained itself in a list of possible values. As a workaround, I think the easiest way here is to use a custom query test:

db.insert({'type': 'apple', 'ID': 1, 'ID2': '1', 'name': 'a'})
db.insert({'type': 'apple', 'ID': 2, 'ID2': '2', 'name': 'b'})
db.insert({'type': 'apple', 'ID': 3, 'ID2': '3', 'name': 'c'})
db.insert({'type': 'apple', 'ID': 4, 'ID2': '4', 'name': 'd'})

assert db.contains(_attr['ID'].test(lambda id: id in [1, 2, 3, 4]))
assert not db.contains(_attr['ID'].test(lambda id: id in [5]))

I will think though if there's a nice way we could add this in a better way to TinyDB itself.

Cheers!

nicr

Hi msiemens ,
thanks once again for the prompt reply.
I had looked at the any/all functions but, as you have noted, they don't work for my use case.
It would be great if support could be added to TinyDB for checking if an int exists in a list of values. For now I'm sure I can find a solution with your suggested "assert" approach.
thanks,
Nic

msiemens

FYI: I've openend issue #164 so I don't forget it ?