Advanced SQLite Usage in Python

Following the SQLite3 series, this post is about some advanced topics when we are working with the SQLite3 module. If you missed the first part, you can find it here.

Using SQLite's date and datetime Types

Sometimes we need to insert and retrieve some

date

and

datetime

types
in our SQLite3 database. When you execute the insert query with a date or datetime object, the

sqlite3

module
calls the default adapter and converts them to an ISO format. When you execute a query in order to retrieve those values, the

sqlite3

module
is going to return a string object:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18

>>> import sqlite3 >>> from datetime import date, datetime >>> >>> db = sqlite3.connect(':memory:') >>> c = db.cursor() >>> c.execute('''CREATE TABLE example(id INTEGER PRIMARY KEY, created_at DATE)''') >>> >>> # Insert a date object into the database >>> today = date.today() >>> c.execute('''INSERT INTO example(created_at) VALUES(?)''', (today,)) >>> db.commit() >>> >>> # Retrieve the inserted object >>> c.execute('''SELECT created_at FROM example''') >>> row = c.fetchone() >>> print('The date is {0} and the datatype is {1}'.format(row[0], type(row[0]))) # The date is 2013-04-14 and the datatype is <class 'str'> >>> db.close()

The problem is that if you inserted a date object in the database, most of the time you are expecting a date object when you retrieve it, not a string object. This problem can be solved passing

PARSE_DECLTYPES

and

PARSE_COLNAMES

to
the

connect

method:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

>>> import sqlite3 >>> from datetime import date, datetime >>> >>> db = sqlite3.connect(':memory:', detect_types=sqlite3.PARSE_DECLTYPES|sqlite3.PARSE_COLNAMES) >>> c = db.cursor() >>> c.execute('''CREATE TABLE example(id INTEGER PRIMARY KEY, created_at DATE)''') >>> # Insert a date object into the database >>> today = date.today() >>> c.execute('''INSERT INTO example(created_at) VALUES(?)''', (today,)) >>> db.commit() >>> >>> # Retrieve the inserted object >>> c.execute('''SELECT created_at FROM example''') >>> row = c.fetchone() >>> print('The date is {0} and the datatype is {1}'.format(row[0], type(row[0]))) # The date is 2013-04-14 and the datatype is <class 'datetime.date'> >>> db.close()

Changing the connect method, the database now is returning a date object. The

sqlite3

module uses the column's
type to return the correct type of object. So, if we need to work with a

datetime

object, we must declare the
column in the table as a

timestamp

type:

1 2 3 4 5 6 7 8 9 10 11

>>> c.execute('''CREATE TABLE example(id INTEGER PRIMARY KEY, created_at timestamp)''') >>> # Insert a datetime object >>> now = datetime.now() >>> c.execute('''INSERT INTO example(created_at) VALUES(?)''', (now,)) >>> db.commit() >>> >>> # Retrieve the inserted object >>> c.execute('''SELECT created_at FROM example''') >>> row = c.fetchone() >>> print('The date is {0} and the datatype is {1}'.format(row[0], type(row[0]))) # The date is 2013-04-14 16:29:11.666274 and the datatype is <class 'datetime.datetime'>

In case you have declared a column type as

DATE

, but you need to work with a

datetime

object,
it is necessary to modify your query in order to parse the object correctly:

1 2 3 4 5 6 7 8

c.execute('''CREATE TABLE example(id INTEGER PRIMARY KEY, created_at DATE)''') # We are going to insert a datetime object into a DATE column now = datetime.now() c.execute('''INSERT INTO example(created_at) VALUES(?)''', (now,)) db.commit() # Retrieve the inserted object c.execute('''SELECT created_at as "created_at [timestamp]" FROM example''')

Using

as "created_at [timestamp]"

in the SQL query will make the adapter to parse the object correctly.

Insert Multiple Rows with SQLite's executemany

Sometimes we need to insert a sequence of objects in the database, the

sqlite3

module provides the

executemany

method
to execute a SQL query against a sequence.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21

# Import the SQLite3 module import sqlite3 db = sqlite3.connect(':memory:') c = db.cursor() c.execute('''CREATE TABLE users(id INTEGER PRIMARY KEY, name TEXT, phone TEXT)''') users = [ ('John', '5557241'), ('Adam', '5547874'), ('Jack', '5484522'), ('Monthy',' 6656565') ] c.executemany('''INSERT INTO users(name, phone) VALUES(?,?)''', users) db.commit() # Print the users c.execute('''SELECT * FROM users''') for row in c: print(row) db.close()

Please note that each element of the sequence must be a tuple.

Execute SQL File with SQLite's executescript

The

execute

method only allows you to execute a single SQL sentence. If you need to execute several different
SQL sentences you should use

executescript

method:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

# Import the SQLite3 module import sqlite3 db = sqlite3.connect(':memory:') c = db.cursor() script = '''CREATE TABLE users(id INTEGER PRIMARY KEY, name TEXT, phone TEXT); CREATE TABLE accounts(id INTEGER PRIMARY KEY, description TEXT); INSERT INTO users(name, phone) VALUES ('John', '5557241'), ('Adam', '5547874'), ('Jack', '5484522');''' c.executescript(script) # Print the results c.execute('''SELECT * FROM users''') for row in c: print(row) db.close()

If you need to read the script from a file:

1 2 3 4

fd = open('myscript.sql', 'r') script = fd.read() c.executescript(script) fd.close()

Please remember that it is a good idea to surround your code with a

try/except/else

clause in order to catch
the exceptions. To learn more about the

try/except/else

keywords, checkout the Catching
Python Exceptions – The try/except/else keywords article.

Defining SQLite SQL Functions

Sometimes we need to use our own functions in a statement, specially when we are inserting data in order to accomplish some specific task. A good example of this is when we are storing passwords in the database and we need to encrypt those passwords:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

import sqlite3 #Import the SQLite3 module import hashlib def encrypt_password(password): # Do not use this algorithm in a real environment encrypted_pass = hashlib.sha1(password.encode('utf-8')).hexdigest() return encrypted_pass db = sqlite3.connect(':memory:') # Register the function db.create_function('encrypt', 1, encrypt_password) c = db.cursor() c.execute('''CREATE TABLE users(id INTEGER PRIMARY KEY, email TEXT, password TEXT)''') user = ('johndoe@example.com', '12345678') c.execute('''INSERT INTO users(email, password) VALUES (?,encrypt(?))''', user)

The

create_function

takes 3 parameters:

name

(the
name used to call the function inside the statement), the number of parameters the function expects (1 parameter in this case) and a callable object (the function itself). To use our registered function, we called it using

encrypt()

in
the statement.

Finally, PLEASE use a true encryption algorithm when you are storing passwords!

via:http://www.pythoncentral.io/advanced-sqlite-usage-in-python/