De entre todas las fotos que podía haber usado, eligió la de variables de entorno con las cadenas de conexión a bases de datos.
Buena elección
Un backup en Twitter :)
12-hour timelapse of American Airlines, Delta, and United plane traffic after what was likely the biggest IT outage in history forced a nationwide ground stop of the three airlines.
how I squeeze 60K RPS out of SQLite on a $5 VPS:
There are tons of tutorials on how to setup SQLite, and sometimes they contradict each other. Here is what worked for me.
1. Configuring PRAGMAs. We need to send the following PRAGMA commands right after opening the connection:
PRAGMA journal_mode = WAL;
- enables write-ahead log so that your reads do not block writes and vice-versa.
PRAGMA busy_timeout = 5000;
- sqlite will wait 5 seconds to obtain a lock before returning SQLITE_BUSY errors, which will significantly reduce them.
PRAGMA synchronous = NORMAL;
- sqlite will sync less frequently and be more performant, still safe to use because of the enabled WAL mode.
PRAGMA cache_size = -20000;
- negative number means kilobytes, in this case 20MB of memory for cache.
PRAGMA foreign_keys = true;
- because of historical reasons foreign keys are disabled by default, we should manually enable them.
PRAGMA temp_store = memory;
- moves temporary tables from disk into RAM, speeds up performance a lot.
Do NOT use cache=shared! Some tutorials recommend configuring it, but this is how you get nasty SQLITE_BUSY errors. It is disabled by default, so you don't have to do anything extra.
2. Use immediate transactions
If you know that transaction can possibly do a write, always use BEGIN IMMEDIATE or you can a get SQLITE_BUSY error. Check your framework, you should be able to set this at the connection level.
3. Open two connection pools
Another trick is to open 2 connection pools, one for reads only and another for reads/writes. Set the connection limit of write pool to 1, and the connection limit of the read pool to some reasonably high number, e.g. number of your CPU cores.
4. Bonus: how I configure sqlite with Go
Here is the code in go I use to configure the sqlite connections:
```go
func SQLiteDbString(file string, readonly bool) string {
connectionParams := make(url.Values)
connectionParams.Add("_journal_mode", "WAL")
connectionParams.Add("_busy_timeout", "5000")
connectionParams.Add("_synchronous", "NORMAL")
connectionParams.Add("_cache_size", "-20000")
connectionParams.Add("_foreign_keys", "true")
if readonly {
connectionParams.Add("mode", "ro")
} else {
connectionParams.Add("_txlock", "IMMEDIATE")
connectionParams.Add("mode", "rwc")
}
return "file:" + file + "?" + connectionParams.Encode()
}
func OpenSqliteDatabase(file string, readonly bool) (*sql.DB, error) {
dbString := SQLiteDbString(file, readonly)
db, err := sql .Open("sqlite3", dbString)
pragmasToSet := []string{
"temp_store=memory",
}
for _, pragma := range pragmasToSet {
_, err = db.Exec("PRAGMA " + pragma + ";")
if err != nil {
return nil, err
}
}
if readonly {
db.SetMaxOpenConns(max(4, runtime.NumCPU()))
} else {
db.SetMaxOpenConns(1)
}
return db, nil
}
```
So I started using SQLite as a database for my side project, and I want to share my experience so far:
1. It is fast. Like really fast. My architecture is a monolith written in Go (this is intentional, I sacrificed scalability to improve my shipping speed), and this is where SQLite shines. With a DB located on the local NVMe disk, a 5$ VPS can deliver a whopping 60K reads and 20K writes per second. No other database is even close.
2. Backups are simple. There is project called litestream, and it provides real-time replication to an S3 bucket. With this I get point-in-time recovery with 1 second replication lag for pennies/month.
3. Pairs well with monolithic architecture. Because SQLite is embedded in your backend, N+1 queries are not a problem, e.g. you can fetch a user, and then based on the user_id fetch more data from the database. The latency is so small so you can join directly in your application and speed up your shipping speed.
Overall I'm quite satisfied. SQLite seems amazing for all use cases that can fit on a single server.
The only tricky part is setting up the SQLite configuration right. The correct values for journal_mode, busy_timeout, and synchronous pragmas are essential to get good performance. Shall I write a post about configuring SQLite to run on a server?
Your periodic reminder that less ice at the Arctic is consistent with a weaker jet stream that allows cold air to drift down into the Great Plains.
The frigid temps you're experiencing happen BECAUSE of a warming planet, not in spite of it.
MOUSE, an upcoming title from @FumiHQ, looks simply delightful and is a great example of why copyright expiration is so important. Stuff like this couldn’t exist without the public domain.
Siento, por lo que vi en 2023 respecto a despidos / contrataciones, que más que un stack, que 2024 se va a tratar de pisarle mucho al enfoque de ser un t-shaped engineer
O sea, quitarse el chip de solo enfocarse un área o lenguaje, y ampliar las habilidades para cubrir más áreas
Sobretodo que en 2023 los equipos tenías budget limitado de contratación, tons era tipo de que solo podían contratar un engineer, y esa contratación tenía que ser flexible