High Performance
SQLite
Introduction
Introduction to this course
Introduction to SQLite
SQLite is safe
It's just a file
Running SQLite locally
Good uses for SQLite
SQLite's limitations
The documentation
SQLite Internals
SQLite's structure
SQLite's file format
Dot commands
Pragmas
Virtual tables
Schema
Flexible types
Types
Strict types
Dates
Booleans
Floating point
Rowid tables
Auto increments
Without rowid
Generated columns
Optimizing SQLite
Locking
Rollback mode
WAL mode
WAL vs Journal benchmarks
Busy timeout
Transaction modes
Vacuum
Analyze & Optimize
Suggested pragmas
Faster inserts
Indexes
Introduction to indexes
B+ trees
Primary, secondary, and clustered indexes
Without rowid tables
Benchmarking without rowid tables
Imposter tables
Primary key data types
Where to add indexes
Index selectivity
Composite indexes
Composite ordering
Covering indexes
Partial indexes
Indexes on expressions
Automatic indexes
Duplicate indexes
Indexing joins
Advanced SQL
Explain
Index obfuscation
Joins
Subqueries
Unions
CTEs
Recursive CTEs
Window functions
Dealing with NULLs
Row value syntax
Indexed sorting
Upserts
Returning
Aggregates
Triggers
Operating SQLite
Altering schema
Altering schema with tools
Multi-database
Multi-tenancy
Backups
Exports
How to corrupt SQLite
Advanced JSON
Intro to JSON
JSON vs. JSONB
Argument types: path, value, JSON
JSON5
Valid JSON
Creating JSON objects + arrays
JSON Extraction
Updating JSON
JSON aggregates
JSON table functions
Indexing JSON
Full Text Search
Creating FTS tables
Performing searches
Ranking results
Highlighting matches
Bonus Interviews
DHH discusses SQLite in Ruby on Rails
Distributed SQLite with Litestream and LiteFS
Offline-first, multiplayer SQLite
Production SQLite with Turso and libSQL
Migrating from Postgres to SQLite with Kent C. Dodds
Ruby on Rails with SQLite
Moving from Redis to SQLite with Mike Buckbee
Locked video

Please purchase the course to watch this video.

Video thumbnail
Optimizing SQLite
Transaction modes

Full Course

$
179
$229
USD, one-time fee
I went from near 0 to 100 in less than two weeks with this course. If you want to be a SQLite wizard, and just want to up-level your SQL skills in general, this is definitely the course for you.
Anthony Bullard
Anthony Bullard

SQLite for Production

Build with and integrate SQLite into your production applications.

Test Data

I've made my test data available for you to use and follow along.

Download
or
Use on Turso

Summary

I explain the different transaction modes in SQLite and emphasize why it's crucial to use the "begin immediate" transaction, especially in wall mode, to avoid issues. Using "begin immediate" helps ensure a 5-second grace period when upgrading to an exclusive write lock, preventing immediate errors if the lock is already taken. I suggest configuring your application's database settings, like in Rails or Laravel, to use immediate transactions for smoother operation.

Video Transcript

In the last video, I briefly introduced begin exclusive transaction. There are 3 transaction modes deferred, exclusive and immediate. In wall mode, which you want to be in wall mode, intermediate and exclusive are the same thing. Deferred is the default so you never have to write it. And immediate and exclusive are the same in wall mode so you can write either one.

Now what is the issue here? Let me show you the issue or the the potential problem that you'll need to, be very cognizant of on the application side. Over here, begin immediate, same as exclusive, begin immediate transaction. So we begin immediate transaction over here. Let's check the pragma pragma busy time out over here.

Let's go ahead and set that to 5,000. And then we're going to begin a deferred. But because deferred is the default, we don't have to write it. Begin deferred transaction. So this transaction is open.

It has not declared its intent to write to the database. This one is holding the exclusive write lock. So over here, we can do select ID name from users limit 1. That's fine. Whoops.

That's not fine. Select ID first name limit users. That's fine. Get out of here. Give me the box mode.

Alright. That's fine like I was trying to say. So we can go ahead and select. We're inside of a transaction. Try to write, SQLite is going to try to upgrade our lock into exclusive.

Now we have shown in the past video that that should wait 5 seconds because this has the exclusive lock on it and we've set the busy time out to 5 seconds. Right? But, what is actually gonna happen? So, if we update users set first name equal to I don't remember what letter we're on, so we're gonna go with a where id equals 1. This should wait 5 seconds, but it doesn't.

So when you're inside of a deferred transaction and you try to upgrade to an exclusive lock and the lock is already taken, you don't get your 5 second wait time. It just immediately throws SQLite busy. So let's get out of here and clear this. And if we were to begin an immediate, same as exclusive, begin an immediate transaction over here, from the very beginning we are declaring our intent to write to the database and so we do get that 5 second grace period where it says, okay, I'm gonna try, but alas, the database was locked. This is what you want.

This is definitely what you want because you don't want to be in the middle of a transaction and think, okay, well I know there's only 1 rider allowed but I have up to 5 seconds to do my rights. That's fine. And you try to upgrade to an exclusive lock and it immediately fails and you get a single light busy and your sentry starts blowing up and your users start complaining. So on the application side, whenever you connect to the database, you're going to want to set your transaction mode to immediate. Now, this depends and this varies based on your application side, whether that's rails or Laravel or Next JS or Django or something like that.

You'll need to look up for your specific framework how to connect to the database and how to make sure that all of the rights are wrapped in a begin immediate transaction. Most web frameworks wrap the rights in a transaction regardless, and so you're looking for the thing where you can set, alright, we want that to be an immediate transaction. I think the rails adapter for SQLite, I think that does that by default. I know in PHP, you can set a p d o attribute to set it to immediate. So you'll have to do a little bit of digging.

Sorry, I can't speak to all of them directly. You'll have to do a little bit of digging, but you want all of your rights to begin immediate transactions so that you do have that 5 second grace period and you don't get hosed trying to update a transaction in the middle of that transaction.