High Performance
SQLite
Introduction
Introduction to this course
Introduction to SQLite
SQLite is safe
It's just a file
Running SQLite locally
Good uses for SQLite
SQLite's limitations
The documentation
SQLite Internals
SQLite's structure
SQLite's file format
Dot commands
Pragmas
Virtual tables
Schema
Flexible types
Types
Strict types
Dates
Booleans
Floating point
Rowid tables
Auto increments
Without rowid
Generated columns
Optimizing SQLite
Locking
Rollback mode
WAL mode
WAL vs Journal benchmarks
Busy timeout
Transaction modes
Vacuum
Analyze & Optimize
Suggested pragmas
Faster inserts
Indexes
Introduction to indexes
B+ trees
Primary, secondary, and clustered indexes
Without rowid tables
Benchmarking without rowid tables
Imposter tables
Primary key data types
Where to add indexes
Index selectivity
Composite indexes
Composite ordering
Covering indexes
Partial indexes
Indexes on expressions
Automatic indexes
Duplicate indexes
Indexing joins
Advanced SQL
Explain
Index obfuscation
Joins
Subqueries
Unions
CTEs
Recursive CTEs
Window functions
Dealing with NULLs
Row value syntax
Indexed sorting
Upserts
Returning
Aggregates
Triggers
Operating SQLite
Altering schema
Altering schema with tools
Multi-database
Multi-tenancy
Backups
Exports
How to corrupt SQLite
Advanced JSON
Intro to JSON
JSON vs. JSONB
Argument types: path, value, JSON
JSON5
Valid JSON
Creating JSON objects + arrays
JSON Extraction
Updating JSON
JSON aggregates
JSON table functions
Indexing JSON
Full Text Search
Creating FTS tables
Performing searches
Ranking results
Highlighting matches
Bonus Interviews
DHH discusses SQLite in Ruby on Rails
Distributed SQLite with Litestream and LiteFS
Offline-first, multiplayer SQLite
Production SQLite with Turso and libSQL
Migrating from Postgres to SQLite with Kent C. Dodds
Ruby on Rails with SQLite
Moving from Redis to SQLite with Mike Buckbee
Locked video

Please purchase the course to watch this video.

Video thumbnail
Indexes
Automatic indexes

Full Course

$
179
$229
USD, one-time fee
Fantastic! So much packed into one course. Full-text search was a cherry on top of what's possible with SQLite. The course opened my mind to completely new ideas, and gave me perspective on databases in general.
Przemyslaw Figura
Przemyslaw Figura

SQLite for Production

Build with and integrate SQLite into your production applications.

Test Data

I've made my test data available for you to use and follow along.

Download
or
Use on Turso

Summary

I learned that SQLite can create automatic indexes during query time to improve performance, but these are temporary and not as efficient as manually creating indexes. We explored an example with a bookmarking application where SQLite added an automatic index to a query, highlighting the importance of creating our own indexes for frequently run queries. Although automatic indexes are helpful for one-off analyses, I should create permanent indexes if I see them being used often in my application.

Video Transcript

Wanna show you another cool feature about SQLite and that is automatic query time indexes. This is different than internally created indexes. Like, when you declare a column unique, SQLite is gonna put a unique index on that. This is different. Automatic indexes are, created at run time and then destroyed.

Now, you wanna have your own indexes. This automatic indexing thing is a fail safe to protect you in certain scenarios, but it's always going to be worse than creating the correct indexes. I have created a new table here, select star from bookmarks, and we limit it down to 5. We see we've got user ID and a URL. Apparently, the application that we're creating is just a bookmarking application.

It's good enough. You can find all of this data down in the description below and you can spin it up on Terso really easily if you want to follow along. So clearing out, let's take a look at indexes. We don't have any. And tables, we've got the bookmarks and the users.

And this migrations table, select star from migrations, it's nothing. It's just creating some tables and putting stuff in a batch. It's just this is, that's what I use to actually create the database and so we can ignore that altogether. Now, what I want to show you is this, the automatic, the automatic indexing being created. So let's do I think we're already in mode box.

Let's do EQP probably is already on. So let's do select star. Actually, let's do select ID and count from users left join. Now we haven't talked about joins and sub queries and all that, we'll do that later, but I just want to construct a query that will cause SQLite to add an automatic index. So if we left join, select, count star, and user ID from bookmarks, limit, no, group by user ID.

And then out here we'll say join all of that on users dot id equals user id, and then we'll do yeah, 10 is fine. So if we run this, it's gonna take a long time. We did not alias we did not alias the really messed that up. As count and then we'll find our there we go. This is gonna take a really long time and you see we do get the ID and the count of their bookmarks.

So the query did work, but we see using automatic covering index. So what happened here is SQLite looked at this and said, I'm gonna have to scan that table multiple times because they didn't put an index on it. In fact, it would be faster for me if I went ahead and and ate the penalty of creating an index just to throw it away again. But during the query I can use that index to look up all those values instead of scanning the table multiple times. So if you ever see using automatic covering index or using automatic index, that's SQLite saying, oof, rough query.

I'm gonna do my best to make it better. I wonder I wonder if we turn expert on. I think maybe it's a no. We just say expert. And then, we've gotta run all of this again.

We'll see what it says. Create index there you go. Create index with random name on bookmarks user ID. So the automatic index that is being created by SQLite is also the same index that expert mode suggests that you create. While it is cool that they have automatic indexes, this is a sign that if you're going to run this query a lot more than once, you should create your own index on it.

This is fine if you're just doing 1 off data analysis and you're like, I'm never gonna run this query again. Great. Thanks for the help. But if you run this as a part of your application and you see automatic index, that's a red flag. I should go in and create an index myself so it doesn't have to create it every query, simply to throw it away.