Aaron has this fantastic way of breaking down complex topics to bits that are understandable. I finally understand how to tune my PRAGMA values instead of just copy/pasting from StackOverflow.Mathias Hansen
Build with and integrate SQLite into your production applications.
I've made my test data available for you to use and follow along.
I wanna give you 2 parting tips for inserting huge amounts of data. So maybe you're doing some sort of import or you're seeding the database locally or whatever it is. If you're inserting huge amounts of data, 2 things can make it faster. First thing, if you are risk tolerant, you can set pragma synchronous to off and that's gonna go a lot faster. If you have a power outage in the middle, you might run into some trouble.
The second thing that doesn't require risk taking but will speed up your inserts is bunching a bunch of inserts into a single transaction. So instead of running a 100,000 individual inserts, group them into transactions of, I don't know, 500, a 1000, whatever, Group them into transactions because then the transaction commit will be amortized across all of those inserts and it can speed it up. It can make it a lot faster instead of having to begin a transaction insert and commit every time. This makes a much bigger difference on spinning Rust versus a solid state drive. So depending on what drive you have, grouping all of those inserts up and wrapping them in a transaction, chunking them out into transactions can really really speed up your insert time.
So between pragma synchronous off and grouping a bunch of inserts into a transaction, those can be ways to load data a lot faster into Equal light.