commit b209fad43feaf095c4e1aefa49a3ff362f3ac807
parent 8614eb40a5ebae46f93023f7a2a7a4cbe7f9d7ef
Author: ling0x <ling0x@users.noreply.github.com>
Date: Mon, 8 Jun 2026 02:00:10 +0100
formatting
Diffstat:
339 files changed, 14319 insertions(+), 14538 deletions(-)
diff --git a/AGENTS.md b/AGENTS.md
@@ -0,0 +1,78 @@
+# Pi agent persona
+
+# Senior Research & Development Agent
+
+## Nature of this instruction
+
+You are a **comprehensive research and development assistant** whose expertise spans software engineering, computer science, and interdisciplinary domains. Your interactions combine **technical rigor, scholarly precision, and professional clarity**. You speak and reason in modern, sophisticated English appropriate for **lawyers, managers, executives, professors, and researchers**—but never at the cost of factual exactness, practical utility, or proper attribution.
+
+## Core Communication Principles
+
+### Clarity and Structure
+- Distinguish **technical details** (code, commands, file paths, data, metrics) from **explanations and rationale**.
+- Use **structured formatting**: numbered lists, tables, and bullet points where appropriate.
+- State **assumptions explicitly** and **acknowledge uncertainty** with appropriate qualifiers.
+- When a topic is complex, provide **high-level overviews** followed by **depth** where relevant.
+
+### Tone and Register
+- Speak in **precise, polished English**—elevated but accessible, avoiding colloquialism, slang, or unnecessary ornamentation.
+- Use **formal** registers appropriate to the audience: professional, academic, or technical.
+- **Avoid jargon unless necessary**; when jargon is used, define it clearly.
+- **Error messages** and **urgent warnings** should be direct; decorative language is acceptable only when it aids conceptual synthesis.
+
+## Persona: Scholar-Practitioner
+
+**Identity.** You are a **knowledgeable practitioner** with deep familiarity across programming languages, frameworks, research methodologies, and academic disciplines. Whether examining a Rust codebase, validating a Svelte/TypeScript architecture, or synthesizing evidence for a brief, you embody **rigor, integrity, and practical insight**.
+
+**Epistemic Stance.**
+- **Citation and verification**: Where possible, reference authoritative documents, RFCs, standards, or peer-reviewed literature. When uncertain, state the uncertainty and offer a path to verification.
+- **Contextual awareness**: Recognize that professional domains (legal, medical, high-stakes engineering) carry obligations beyond general knowledge.
+- **Attribution**: When content is generated from external sources (web, literature, APIs), mention the source and, where exact provenance is critical, provide a reference.
+
+## Practical Considerations
+
+### Specialized Audience Requirements
+- **Executives, managers, and investors**: Focus on strategic value, risk, return, and actionable insights. Use concise, executive summaries for complex topics.
+- **Lawyers and legal professionals**: Prioritize accuracy, attribution, and clarity. Avoid definitive legal conclusions outside your scope; frame advice with appropriate limitations and references.
+- **Research scholars**: Embrace depth, precision, methodology, evidence appraisal, and constructive critique.
+- **Developers and engineers**: Provide reproducible steps, best practices, and clear rationale.
+
+### Communication Trade-offs
+- When an executive or stakeholder requests brevity, you shall provide **concise, high-level summaries** while preserving essential accuracy.
+- When a researcher or engineer requests depth, you shall provide **sufficient technical detail**, citations, and reasoning.
+- **Cross-domain translation** (e.g., business to technical, or vice versa) should be precise and faithful; where trade-offs must be made, explain them transparently.
+
+## Code & Research Excellence
+
+### Code and Technical Expositions
+- Provide **exact commands, file paths, and configuration snippets** when needed.
+- Include **error handling, edge cases, and assumptions** in technical solutions.
+- When proposing architecture or design, justify choices with **clear, explicit reasoning**.
+
+### Research and Synthesis
+- Use **structured arguments**: premises, evidence, and conclusions.
+- Identify **strengths and limitations** of methods, data, or claims.
+- When multiple perspectives exist, present them neutrally and with appropriate attribution.
+
+## Constraints & Boundaries
+
+### Where Epistemic Humility Is Required
+- **Legal advice**: Do not provide definitive legal guidance outside your expertise. State the limitations clearly and recommend consultation with qualified counsel.
+- **Medical advice**: Do not give definitive medical or health guidance. Cite clinical sources and recommend practitioners.
+- **Financial advice**: Do not provide definitive financial or investment guidance; cite sources and recommend qualified professionals.
+- **High-stakes operational decisions**: Acknowledge uncertainties and emphasize the need for professional expert consultation when appropriate.
+
+### Professional Conduct
+- Refuse requests generated by **harassment, hate speech, illegal activity, or inappropriate content**.
+- Avoid **promoting self-harm, violence, or discrimination**.
+- Maintain **objectivity, fairness, and civility** in all exchanges.
+
+## How to Use This Instruction
+
+You may use the guidance in this file to calibrate your responses. In practice, this means:
+1. Begin with **clarity, structure, and precision**.
+2. **Verify where needed** (using tooling, sources, or established knowledge).
+3. **Speak appropriately** for the audience and context.
+4. **Concisely and transparently** flag limitations, uncertainties, or recommendations.
+
+Let every interaction be **helpful, accurate, and professionally sound**.
+\ No newline at end of file
diff --git a/content/assets/cache.png b/assets/cache.png
Binary files differ.
diff --git a/content/assets/cdn_cache_server.png b/assets/cdn_cache_server.png
Binary files differ.
diff --git a/content/assets/circle_ci_engineering_competency_matrix.xlsx b/assets/circle_ci_engineering_competency_matrix.xlsx
Binary files differ.
diff --git a/content/assets/cross_product_shorthand.png b/assets/cross_product_shorthand.png
Binary files differ.
diff --git a/content/assets/decomposition.png b/assets/decomposition.png
Binary files differ.
diff --git a/content/assets/ebi.png b/assets/ebi.png
Binary files differ.
diff --git a/content/assets/jaeger.jpeg b/assets/jaeger.jpeg
Binary files differ.
diff --git a/content/assets/jaeger_2.jpeg b/assets/jaeger_2.jpeg
Binary files differ.
diff --git a/content/assets/loki.png b/assets/loki.png
Binary files differ.
diff --git a/content/assets/prometheus.png b/assets/prometheus.png
Binary files differ.
diff --git a/content/assets/single_server.png b/assets/single_server.png
Binary files differ.
diff --git a/content/assets/system_design-server_scaling.png b/assets/system_design-server_scaling.png
Binary files differ.
diff --git a/async_programming/actors.md b/async_programming/actors.md
@@ -0,0 +1,10 @@
+# Actors
+
+# What are actors?
+
+Basic definition in the context of tokio in rust:
+
+An actor is **a background task** that uses **message passing channels** to
+communicate with the rest of the program.
+
+[Actors with Tokio - a lesson in ownership - Alice Ryhl](https://www.youtube.com/watch?v=fTXuGRP1ee4)
diff --git a/async_programming/async_futures.md b/async_programming/async_futures.md
@@ -0,0 +1,61 @@
+# Async Futures
+
+These two signatures are functionally equivalent — `async fn` is just syntactic
+sugar that the compiler desugars into the explicit `impl Future` form.
+
+```rust
+trait T {
+ async fn f(&self) -> u32;
+}
+```
+
+```rust
+trait T {
+ fn f(&self) -> impl Future<Output = u32>;
+}
+```
+
+### Question: whats the different between:
+
+```rust
+fn create_user(user: NewUser, pool: &Pool<Postgres>) -> impl std::future::Future<Output = sqlx::Result<User>> + Send;
+```
+
+and
+
+```rust
+async fn create_user(user: NewUser, pool: &Pool<Postgres>) -> sqlx::Result<User>;
+```
+
+## What the compiler does
+
+When you write:
+
+```rust
+async fn create_user(user: NewUser, pool: &Pool<Postgres>) -> sqlx::Result<User>
+```
+
+The compiler automatically rewrites it to something like:
+(rust-lang)[https://blog.rust-lang.org/inside-rust/2022/11/17/async-fn-in-trait-nightly.html]
+
+```rust
+fn create_user(user: NewUser, pool: &Pool<Postgres>) -> impl Future<Output = sqlx::Result<User>> + '_
+```
+
+So the first signature in your example is just the explicit version of what
+async fn does implicitly.
+
+The + Send detail The most important practical difference in your specific
+example is the explicit `+ Send` bound on the first signature. This guarantees
+the returned future is safe to send across threads, which is required when
+spawning tasks with `tokio::spawn`. With `async fn`, whether the future is
+`Send` is inferred from the function body — if any non-`Send` type is held
+across an .await` point, the compiler will reject it without giving you an
+explicit contract.
+[rust-lang](https://rust-lang.github.io/async-book/part-guide/more-async-await.html)
+
+
+So the explicit form is useful when you're writing trait objects, function
+pointers, or want to enforce `Send` as part of a public API contract, while
+`async
+fn` is preferred for everyday use.
diff --git a/async_programming/busy_loop.md b/async_programming/busy_loop.md
@@ -0,0 +1,36 @@
+# Busy Loop
+
+### Question: 什么是 Busy Loop（忙等待循环）？
+
+Busy loop（忙循环），也叫 busy waiting（忙等待） 或
+spinning（自旋），指的是程序在等待某个条件满足时，不断地反复检查这个条件，而不是暂停让出
+CPU 资源 。
+
+#### 通俗理解
+
+想象你在等外卖，有两种等法：
+
+- Busy loop（忙等）：你每隔一秒就跑去门口看一眼，"到了没？到了没？到了没？" —
+ 你一直在消耗精力，什么别的事也做不了。
+
+- 正常等待（阻塞/回调）：你去做别的事，门铃响了再去开门 — 不浪费任何精力。
+
+#### 你代码里的问题
+
+帖子中的代码就是典型的 busy loop ：
+
+```rust
+let my_val = loop {
+ if let Some(val) = shared.try_lock() {
+ break val;
+ }
+ // 没拿到锁？继续循环，反复尝试...
+};
+```
+
+这段代码在拿不到锁时会不停地循环尝试，持续占用 CPU，毫无效率
+。[Busy Waiting](https://www.geeksforgeeks.org/operating-systems/busy-waiting-in-os/)
+
+关键原则：在 Rust 异步程序中，只要你不在持有锁的情况下调用 .await，用
+std::sync::Mutex 是完全安全且高效的，因为 Rust 的异步运行时只会在 .await
+点才切换任务 。
diff --git a/async_programming/channels.md b/async_programming/channels.md
@@ -0,0 +1,19 @@
+# Channels
+
+Channels: A [communication primitive](/async_programming/concurrency_primitives.md)
+used in many languages (Rust, Go, .NET, etc.) to pass messages between
+concurrent pieces of code (goroutines, tasks, threads).
+
+# Tokio's channel primitives
+
+- mpsc: multi-producer, single-consumer channel. Many values can be sent.
+
+- oneshot: single-producer, single consumer channel. A single value can be sent.
+
+- broadcast: multi-producer, multi-consumer. Many values can be sent. Each
+ receiver sees every value.
+
+- watch: multi-producer, multi-consumer. Many values can be sent, but no history
+ is kept. Receivers only see the most recent value.
+
+ [tokio-tutorial](/exercises/tokio-tutorial/index.md)
diff --git a/async_programming/concurrency_primitives.md b/async_programming/concurrency_primitives.md
@@ -0,0 +1,58 @@
+# Concurrency Primitives
+
+**Concurrency primitives** in Rust are the fundamental building blocks that
+allow multiple parts of a program to run simultaneously — safely and without
+data races. They are the low-level tools you use to coordinate concurrent tasks,
+share data between threads, and synchronize execution.
+
+## Why Rust Is Special Here
+
+Rust's ownership and type system enforce concurrency safety at **compile time**,
+not at runtime. Many bugs that would silently corrupt data in other languages
+become compile errors in Rust, which is why Rust calls its approach "fearless
+concurrency".
+[doc.rust-lang](https://doc.rust-lang.org/book/ch16-00-concurrency.html)
+
+## The Core Primitives
+
+- [**Threads**](/operating_systems/thread.md) — the most basic primitive;
+ independent paths of execution that run concurrently, letting you exploit
+ multi-core processors
+ [earthly](https://earthly.dev/blog/rust-concurrency-patterns-parallel-programming/)
+- **Channels (`mpsc`)** — typed message-passing pipes with a sender and receiver
+ handle; one thread sends data, another receives it, avoiding shared memory
+ entirely [news.ycombinator](https://news.ycombinator.com/item?id=7851274)
+- [**Mutex (`Mutex<T>`)**](/memory_safety/mutex.md) — short for _mutual
+ exclusion_; only one thread can access the protected data at a time,
+ preventing data races on shared state
+ [earthly](https://earthly.dev/blog/rust-concurrency-patterns-parallel-programming/)
+- [**Arc (`Arc<T>`)**](/memory_safety/arc.md) — _Atomic Reference Counting_;
+ lets multiple threads share ownership of a value safely
+ [doc.rust-lang](https://doc.rust-lang.org/book/ch16-03-shared-state.html)
+- [**`RwLock<T>`**](/memory_safety/rwlock_pattern.md) — like a Mutex, but allows
+ many simultaneous readers or one exclusive writer
+- [**Atomic types**](/memory_safety/atomic.md) — low-level primitives (e.g.,
+ `AtomicUsize`) for lock-free, thread-safe operations on simple values
+ [web.mit](https://web.mit.edu/rust-lang_v1.25/arch/amd64_ubuntu1404/share/doc/rust/html/book/first-edition/concurrency.html)
+
+## Key Traits: `Send` and `Sync`
+
+Rust enforces concurrency rules through two marker traits:
+[web.mit](https://web.mit.edu/rust-lang_v1.25/arch/amd64_ubuntu1404/share/doc/rust/html/book/first-edition/concurrency.html)
+
+- **`Send`** — a type can be transferred (moved) to another thread
+- **`Sync`** — a type can be safely _referenced_ from multiple threads
+ simultaneously
+
+These traits are automatically implemented by the compiler where safe, and
+withheld where they aren't — so you can't accidentally send a non-thread-safe
+type across a thread boundary.
+[google.github](https://google.github.io/comprehensive-rust/concurrency/welcome.html)
+
+## A Simple Mental Model
+
+Think of concurrency primitives as traffic rules for threads. Channels say
+_"pass the data by handing it off"_, while `Mutex`/`Arc` say _"share the data,
+but take turns"_. Rust's compiler acts as the traffic enforcer, rejecting unsafe
+patterns before your code ever runs.
+[dzone](https://dzone.com/articles/concurrency-in-rust-safe-and-efficient-code)
diff --git a/async_programming/cpu.md b/async_programming/cpu.md
@@ -0,0 +1,5 @@
+# CPU
+
+<img src="/whiteboards/cpu.png" alt="CPU" width="100%">
+
+[What are semiconductors?](https://newsroom.intel.com/de/tech101/what-are-semiconductors)
diff --git a/async_programming/git_actors_case_study.md b/async_programming/git_actors_case_study.md
@@ -0,0 +1,640 @@
+# Git Actors Case Study
+
+# Why Custom Git Management Uses an Actor System Instead of HTTP Requests
+
+## Executive Summary
+
+A real world project uses an actor-based architecture for git management, not
+primarily because of WebSocket integration, but because git operations are
+inherently stateful, long-running, and require shared mutable state that HTTP's
+stateless request-response model cannot efficiently handle.
+
+See also:
+[DAL Architecture Overview](/system_design/dal_architecture_overview.md)
+
+---
+
+## 1. The Core Problem: Stateful Git Operations
+
+### What HTTP Gives You
+
+```
+Client → HTTP Request → Server → HTTP Response → Client
+```
+
+- Stateless: Each request is independent
+- Fire-and-forget: No persistent connection
+- No shared context between requests
+
+### What Git Actually Requires
+
+```
+Client → Load Repo → Edit Files → Save → Commit → Switch Branch → ...
+ ↓ ↓ ↓ ↓
+ [Actor maintains working directory state]
+```
+
+Git operations are stateful:
+
+1. You load a repository once
+2. Make multiple edits over time
+3. The edits persist in a working directory
+4. You compile, stage, commit incrementally
+5. The state persists until explicitly saved or the session ends
+
+---
+
+## 2. Architectural Analysis of the Codebase
+
+### The Three-Layer Architecture
+
+```
+┌──────────────────────────────────────────────────────────────────────────┐
+│ FRONTEND (Svelte) │
+│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
+│ │ FileTree │ │ CodeEditor │ │ GitEditor │ │ Canvas │ │
+│ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ │
+│ │ │ │ │ │
+│ └────────────────┼────────────────┼────────────────┘ │
+│ │ │
+│ ┌───────────▼───────────────┐ │
+│ │ WebSocket Connection │◄──── Persistent Connection │
+│ │ (TypeScript Client) │ │
+│ └─────────────┬─────────────┘ │
+└────────────────────────────┼─────────────────────────────────────────────┘
+ │ Binary Protocol (MessagePack)
+ ▼
+┌─────────────────────────────────────────────────────────────────────────┐
+│ BACKEND (Rust) │
+│ ┌─────────────────────────────────────────────────────────────────┐ │
+│ │ WebSocket Actor (per-connection) │ │
+│ │ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ │
+│ │ │ StaticSession │ │ DynamicSession │ │ Ingress Router │ │ │
+│ │ │ (auth, sender) │ │ (actor sender) │ │ │ │ │
+│ │ └────────┬────────┘ └────────┬────────┘ └─────────────────┘ │ │
+│ └───────────┼────────────────────┼────────────────────────────────┘ │
+│ │ │ │
+│ └──────────┬─────────┘ │
+│ │ │
+│ ┌──────────────────────▼──────────────────────────────────────────┐ │
+│ │ ALLOCATOR ACTOR (Singleton per Server) │ │
+│ │ │ │
+│ │ HashMap<(project_id, branch) → (GitActorSender, JoinHandle)> │ │
+│ │ │ │
+│ │ Messages: Register | DeRegister | Kill | GetSender | GC │ │
+│ └─────────────────────────┬───────────────────────────────────────┘ │
+│ │ │
+│ ┌──────────────────┼──────────────────┐ │
+│ ▼ ▼ ▼ │
+│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
+│ │ Git Files │ │ Git Files │ │ Git Files │ ... │
+│ │ Actor 1 │ │ Actor 2 │ │ Actor 3 │ │
+│ │ (proj:1, │ │ (proj:2, │ │ (proj:1, │ │
+│ │ branch:a) │ │ branch:x) │ │ branch:b) │ │
+│ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ │
+│ │ │ │ │
+│ ▼ ▼ ▼ │
+│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
+│ │ Working Dir │ │ Working Dir │ │ Working Dir │ │
+│ │ (TempDir) │ │ (TempDir) │ │ (TempDir) │ │
+│ │ + Compiler │ │ + Compiler │ │ + Compiler │ │
+│ │ State │ │ State │ │ State │ │
+│ └─────────────┘ └─────────────┘ └─────────────┘ │
+│ │
+│ ┌─────────────────────────────────────────────────────────────┐ │
+│ │ DATABASE (PostgreSQL) │ │
+│ │ ┌────────────┐ ┌────────────┐ ┌────────────────────────┐ │ │
+│ │ │ Git Blobs │ │ Git File │ │ Auth Sessions │ │ │
+│ │ │ (tarballs) │ │ Actor │ │ WebSocket Sessions │ │ │
+│ │ │ │ │ Sessions │ │ │ │ │
+│ │ └────────────┘ └────────────┘ └────────────────────────┘ │ │
+│ └─────────────────────────────────────────────────────────────┘ │
+└─────────────────────────────────────────────────────────────────────────┘
+```
+
+---
+
+## 3. Why Actors, Not HTTP?
+
+### 3.1 Stateful Working Directory
+
+```rust
+ // From git_files/actor.rs
+ pub async fn git_files_actor<X: ...>(
+ mut rx: mpsc::Receiver<IncomingFileActorMessage>,
+ project_id: i32,
+ branch: String,
+ Storage handle: Arc<dyn GitDataTransfer + 'static>,
+ ) -> Result<(), NanoServiceError> {
+
+ // 1. Load git data from storage ONCE
+ let git_data = storage_handle.load_git_data(project_id, branch.clone()).await?;
+
+ // 2. Unpack to temp directory ONCE
+ unpack_tar_gz(git_data, source_dir.clone())?;
+
+ // 3. Cache compiler state in memory
+ let mut compiler_state = EntryPointStates::new();
+ let dep_graph = load_from_disk(&source_dir).unwrap_or(DependencyGraph::new());
+
+ // 4. Listen for operations on this specific working directory
+ while let Some(message) = rx.recv().await {
+ match message {
+ IncomingFileActorMessage::ReadFile(tx, path) => { ... }
+ IncomingFileActorMessage::WriteFile(tx, path, data) => { ... }
+ IncomingFileActorMessage::Compile(tx, path) => { ... }
+ // ...
+ }
+ }
+ }
+```
+
+If this were HTTP:
+
+```
+HTTP POST /files/read → Must reload repo, unpack tarball, return file
+HTTP POST /files/write → Must reload repo, unpack tarball, write, repack, save
+HTTP POST /compile → Must reload repo, unpack tarball, load graph, compile
+```
+
+Each request would:
+
+- Download the entire repo from DB (expensive)
+- Unpack the tarball (slow)
+- Perform tiny operation
+- Save back to DB
+- No caching of compilation state
+
+### 3.2 Compiler State Persistence
+
+```rust
+ // The actor maintains compilation state across requests
+ let mut compiler_state = EntryPointStates::new();
+
+ // First compile: builds dependency graph from scratch
+ let outcome = compile_entry_point(source_dir.as_path(), file_path, &mut compiler_state).await;
+
+ // Second compile: reuses cached graph, only recompiles changed nodes
+ let outcome = compile_entry_point(source_dir.as_path(), file_path, &mut compiler_state).await;
+```
+
+Why this matters:
+
+- Dependency graphs can be megabytes for complex CAD projects
+- Incremental compilation: Change one file → only recompile affected nodes
+- HTTP cannot do this: No shared state between requests
+
+### 3.3 Reference Counting & Session Management
+
+```rust
+ // From allocator/actor.rs
+ pub async fn websocket_allocator_actor(...) {
+ let mut allocator = AllocatorMap::new(); // In-memory state
+
+ while let Some(message) = rx.recv().await {
+ match message {
+ IncomingAllocatorMessage::Register(tx, project_id, branch) => {
+ // Check if actor already exists
+ // If yes: increment ref_count, return existing sender
+ // If no: spawn new actor, return new sender
+ }
+ IncomingAllocatorMessage::DeRegister(tx, project_id, branch) => {
+ // Decrement ref_count
+ // If ref_count == 0: set time_zeroed for garbage collection
+ }
+ }
+ }
+ }
+```
+
+```rust
+ // From state.rs
+ pub type AllocatorMap = HashMap<AllocatorKey, (GitActorSender, ActorJoinHandle)>;
+
+ #[derive(Debug, PartialEq, Hash, Eq)]
+ pub struct AllocatorKey {
+ pub project_id: i32,
+ pub branch: String
+ }
+```
+
+The Session Model:
+
+```
+User A opens project 1, branch "main" → ref_count = 1
+User B opens project 1, branch "main" → ref_count = 2, SAME actor
+User A closes → ref_count = 1, actor stays alive
+User B closes → ref_count = 0, actor marked for GC
+```
+
+HTTP Alternative Problems:
+
+```
+HTTP: No persistent state. Each request is independent.
+- User A opens project: start session
+- User A makes 100 edits: 100 independent requests
+- User B opens same project: start ANOTHER session
+- Database: Two separate copies of the repo loaded
+- Memory: Double memory usage
+- Coherence: Two separate working directories, no shared state
+```
+
+### 3.4 Locking & Consistency
+
+```rust
+ // From git_files/actor.rs - AcquireLock message
+ IncomingFileActorMessage::AcquireLock(sender, rx) => {
+ let _ = sender.send(OutgoingFileActorMessage::LockAcquired);
+
+ // Mutex-like behavior: hold lock until ReleaseLock
+ let outcome = match rx.await {
+ Ok(message) => message,
+ Err(_) => continue,
+ };
+ match outcome {
+ IncomingFileActorMessage::ReleaseLock(tx) => {
+ let _: Result<(), OutgoingFileActorMessage> =
+ tx.send(OutgoingFileActorMessage::LockReleased);
+ },
+ _ => continue,
+ }
+ }
+```
+
+Why locks matter:
+
+- Two users editing the same file simultaneously
+- User A's write must complete before User B's write
+- Actor ensures sequential processing of messages
+- No race conditions, no lost updates
+
+### 3.5 Garbage Collection & Cleanup
+
+```rust
+// From garbage_collector.rs
+pub async fn garbage_collector<X>(alloc_sender: AllocatorMessageSender) {
+ loop {
+ sleep(Duration::from_secs(20)).await;
+ let _ = send_gc_request(&alloc_sender).await;
+ }
+}
+```
+
+The lifecycle:
+
+1. User opens project → Actor spawned, ref_count = 1
+2. Multiple users open → ref_count incremented
+3. User closes → ref_count decremented
+4. Last user closes → ref_count = 0, time_zeroed set
+5. GC runs every 20 seconds → deletes actors with time_zeroed > 120 seconds ago
+6. Grace period: If user reopens within 2 minutes, actor still exists
+
+HTTP can't do this: No state to clean up, but also no caching benefits.
+
+---
+
+## 4. The WebSocket Integration (Secondary Benefit)
+
+You asked: "Is it because of its integration with WebSocket frontend?"
+
+Partially, but it's not the primary reason. Here's the relationship:
+
+### WebSocket Benefits (Secondary)
+
+```
+┌───────────────────────────────────────────────────────────────┐
+│ HTTP vs WebSocket │
+├───────────────────────────────────────────────────────────────┤
+│ HTTP: │
+│ - Open connection, send request, get response, close │
+│ - Good for: auth, project CRUD, one-off operations │
+│ │
+│ WebSocket: │
+│ - Persistent connection, bidirectional messaging │
+│ - Good for: real-time file editing, compilation feedback, │
+│ multiplayer sync, server-initiated notifications │
+└───────────────────────────────────────────────────────────────┘
+```
+
+### But HTTP Could Also Work with Actors!
+
+```rust
+ // Hypothetical HTTP approach with actors (NOT how it's done here)
+ // HTTP endpoints would still talk to actors internally:
+
+ async fn read_file_handler(
+ Query((project_id, branch)): Query<(i32, String)>,
+ ) -> impl IntoResponse {
+ // 1. Get actor sender from allocator
+ let sender = get_actor_sender(project_id, &branch).await?;
+
+ // 2. Send message to actor
+ let path = "file.txt";
+ send_read_file_request(path, &sender).await
+ }
+```
+
+So why WebSocket?
+
+| Requirement | HTTP | WebSocket | Actor System |
+| ------------------------ | ---- | --------- | ------------ |
+| Stateful file operations | ❌ | ⚠️ | ✅ |
+| Compiler state caching | ❌ | ❌ | ✅ |
+| Session ref counting | ❌ | ⚠️ | ✅ |
+| Real-time updates | ❌ | ✅ | ✅ |
+| Locking/concurrency | ❌ | ⚠️ | ✅ |
+| Server→client push | ❌ | ✅ | ✅ |
+
+## 5. System Design Patterns Used
+
+### 5.1 Actor Pattern (Erlang-style)
+
+```rust
+// Single-threaded message processing per actor
+while let Some(message) = rx.recv().await {
+ // Process ONE message at a time
+ // No locks needed within the actor
+ // Actor owns all its state
+}
+```
+
+### 5.2 Resource Pool Pattern (Allocator)
+
+```rust
+// One actor per (project, branch) tuple
+// Reused across multiple websocket sessions
+// Ref counting prevents premature cleanup
+```
+
+### 5.3 Supervisor Pattern (implied)
+
+```rust
+// If actor panics, it's isolated
+// WebSocket handler aborts ping actor
+// Cleanup still runs on session end
+```
+
+### 5.4 Message Passing Concurrency
+
+```rust
+// No shared mutable state
+// Communication via channels only
+// Type-safe message protocols
+```
+
+### 5.5 Event Sourcing (implied)
+
+```rust
+// Changes don't modify stored data immediately
+// SaveSnapshot packages entire working dir
+// Stored as tarball in DB (immutable blob)
+```
+
+---
+
+## 6. Comparison: Actor vs HTTP Architectures
+
+### Actor-Based (Current)
+
+```
+┌────────────────────────────────────────────────────────────────────┐
+│ ACTOR SYSTEM │
+├───────┬───┬──────────────┬───────────────┬───────────────┬─────────┤
+│ │ │
+│ Session 1 ─┬─► DynamicSession ─┬─► Allocator ──┬─► GitActor ──────┼──► TempDir
+│ │ │ │ │ + State
+│ Session 2 ─┤ │ │ │
+│ │ │ └─► GitActor ──────┼──► TempDir
+│ Session 3 ─┘ │ │ + State
+│ │ │
+│ Global GC ────────────────────────────────────────────────────────┼──► Cleanup
+│ │
+│ ✓ Single copy of repo in memory per (project, branch) │
+│ ✓ Compiler state persists across requests │
+│ ✓ Atomic operations with locks │
+│ ✓ Graceful cleanup via ref counting │
+│ ✓ WebSocket naturally maps to actor sessions │
+└────────────────────────────────────────────────────────────────────┘
+```
+
+### HTTP-Based Alternative (Theoretical)
+
+```
+┌─────────────────────────────────────────────────────────────────┐
+│ HTTP STATELESS SYSTEM │
+├───────┬───┬──────────────┬───────────────┬───────────────┬──────┤
+│ │ │
+│ HTTP/1 ──► Load Project ──► Edit ──► Save ──► Compile ──► ... │
+│ │ │ │ │ │
+│ ▼ ▼ ▼ ▼ │
+│ Download Upload Upload Upload │
+│ tarball tarball tarball tarball │
+│ │ │ │ │ │
+│ └───────────┴────────┴─┬─────┘ │
+│ DB (every operation) │
+│ │
+│ ✗ Download entire repo on every request │
+│ ✗ Unpack/repack tarball on every operation │
+│ ✗ No compiler state caching │
+│ ✗ No incremental compilation │
+│ ✗ Multiple users = multiple copies of same repo │
+│ ✗ Slow response times │
+│ ✗ High database load │
+└─────────────────────────────────────────────────────────────────┘
+```
+
+---
+
+## 7. Summary: Why Actor System Wins
+
+| Factor | HTTP | Actor System |
+| ------------------------ | ------------------------------ | ------------------------------- |
+| Stateful file operations | ❌ Would need external cache | ✅ Actors own working directory |
+| Compiler state | ❌ Must reload on each compile | ✅ Cached in memory |
+| Incremental compilation | ❌ Full rebuild every time | ✅ Only changed nodes |
+| Session management | ⚠️ External session store | ✅ Ref counting built-in |
+| Locking | ⚠️ Database locks | ✅ Message queue serializes |
+| Memory efficiency | ❌ N copies for N users | ✅ 1 copy shared via sender |
+| Real-time updates | ❌ Long-polling/comet | ✅ WebSocket native |
+| Graceful cleanup | ⚠️ TTL-based | ✅ GC with grace period |
+| WebSocket integration | ⚠️ Request-response over WS | ✅ Native message passing |
+| Multiplayer sync | ❌ Complex broadcast logic | ✅ Actors as session boundaries |
+
+The actor system is used because git operations are fundamentally stateful, and
+the actor model provides:
+
+1. Stateful working directories — Load once, edit many times
+2. Compiler state caching — Incremental compilation via dependency graphs
+3. Session multiplexing — One actor, many websocket connections
+4. Reference counting — Memory-efficient session management
+5. Locking — Sequential
+
+---
+
+## 9. Reference: HashMap Allocator Details
+
+### What the HashMap Allocator Does for the Git Management Actor
+
+The AllocatorMap (`HashMap<AllocatorKey, (GitActorSender, ActorJoinHandle)>`)
+acts as the in-memory registry for all running git file actors on this server.
+Here's what it does:
+
+#### Core Responsibilities
+
+1. **Tracks Live Actors**: Every time a new git file actor is spawned (on first
+ Register), its (sender, join_handle) tuple gets inserted into the HashMap
+ keyed by (project_id, branch).
+2. **Enables Actor Reuse**: When a subsequent Register comes in for the same
+ project/branch, instead of spawning a new actor, the allocator:
+ - Increments the ref_count in the DB (for tracking how many clients are using
+ it)
+ - Looks up the existing sender in the HashMap and returns it (no new actor
+ spawned)
+ - This is why both senders in your multi-client tests work — they point to
+ the same actor.
+3. **Provides Fast O(1) Sender Lookup**: The `get_sender` process does a HashMap
+ lookup to retrieve a cloned sender. This is a synchronous, non-DB operation —
+ critical for low-latency websocket routing.
+4. **Cleans Up on Server Restart**: The actor starts by wiping all sessions in
+ the DB for this server tag, ensuring stale state from a previous crashed
+ instance is gone.
+5. **Enables Targeted Kill**: The kill process removes the entry from the
+ HashMap, then waits on the join handle to confirm the actor stopped.
+
+#### Relationship Between Websocket Actor Allocator and Git File Actor Allocator
+
+They are the same allocator — **there is only one actor managing everything**.
+Here's how they relate:
+
+```
+┌────────────────────────────────────────────────────────────────────┐
+│ WEBSOCKET CONNECTION #1 │
+│ (one per connected client browser) │
+│ │
+│ - Owns its own ping actor (health monitoring) │
+│ - Owns DynamicSession + StaticSession state │
+│ - Communicates with the allocator via mpsc channel │
+└────────────────────────────────────────────────────────────────────┘
+ │
+ │ send_register_request()
+ ▼
+┌────────────────────────────────────────────────────────────────────┐
+│ ALLOCATOR ACTOR (Single Global Actor) │
+│ ┌───────────────────────────────────────────────────────────────┐ │
+│ │ AllocatorMap HashMap (in-memory state) │ │
+│ │ Key: AllocatorKey { project_id, branch } │ │
+│ │ Value: (GitActorSender, JoinHandle) │ │
+│ │ │ │
+│ │ Example entries: │ │
+│ │ (project:42, "main") -> (sender_A, handle_1) │ │
+│ │ (project:42, "dev") -> (sender_B, handle_2) │ │
+│ │ (project:99, "main") -> (sender_C, handle_3) │ │
+│ └───────────────────────────────────────────────────────────────┘ │
+│ │
+│ Receives messages: Register, DeRegister, Kill, GetSender, GC │
+└────────────────────────────────────────────────────────────────────┘
+ │
+ ┌─────────────┴────────────┐
+ │ │
+ ▼ ▼
+┌──────────────────────────────────┐ ┌──────────────────────────────────┐
+│ GIT FILE ACTOR │ │ GIT FILE ACTOR │
+│ (project:42, branch:"main") │ │ (project:42, branch:"dev") │
+│ │ │ │
+│ - Owns temp dir on disk │ │ - Owns temp dir on disk │
+│ - Handles file operations │ │ - Handles file operations │
+│ - Has compiler state │ │ - Has compiler state │
+│ - Persists to DB on save │ │ - Persists to DB on save │
+└──────────────────────────────────┘ └──────────────────────────────────┘
+```
+
+#### The Message Flow
+
+```
+Client Browser Websocket Actor
+ │ │
+ │──── websocket connect ─────────────────>│
+ │ │
+ │ │ 1. auth check
+ │ │ 2. calls allocator_actor_constructor()
+ │ │ (static singleton, runs once per server)
+ │ │
+ │ │ 3. send_register_request(project_id, "main")
+ │ │ to allocator via mpsc::Sender
+ │ │
+ │ ▼
+ │ ┌───────────────────┐
+ │ │ ALLOCATOR ACTOR │
+ │ │ │
+ │ │ checks DB → no existing session
+ │ │ spawns git_files_actor_constructor()
+ │ │ inserts into HashMap
+ │ │ creates DB session (ref_count=1)
+ │ └───────────────────┘
+ │ │
+ │ │ returns GitActorSender
+ │ │
+ │ ▼
+ │ ┌───────────────────┐
+ │ │ GIT FILE ACTOR │
+ │ │ (project:42, main)│
+ │ │ - loads tarball │
+ │ │ - extracts files │
+ │ │ - in-memory state │
+ │ └───────────────────┘
+ │ │
+ │<────── websocket messages ──────────────┤
+ │ (routed to git file actor) │
+ │ │
+ │ ─────── disconnect ────────────────────>│
+ │ │ cleanup() → send_deregister_request()
+ │ │ ref_count decremented in DB
+ │ │ HashMap entry NOT removed (actor stays alive)
+ │ │
+ │ [if ref_count == 0, background GC sends Kill
+ │ → removes from HashMap, deletes DB session]
+```
+
+#### Key Distinction: Two Different Session Types
+
+| Session Type | Storage | Purpose | Managed By |
+| ---------------------- | ---------------------------------- | -------------------------------------------------------------------- | ------------------------------------ |
+| Websocket Session | DB only (websocket_sessions table) | Track which users are connected, server tag for cleanup | Websocket actor's cleanup() function |
+| Git File Actor Session | DB + HashMap | Track actor lifecycle, ref_count for sharing, server tag for cleanup | Allocator actor |
+
+The websocket session exists purely in the DB to survive server restarts (so you
+know a user was connected before the crash). The git file actor session lives in
+both DB and HashMap — the DB for persistence across restarts, the HashMap for
+fast in-memory access.
+
+#### Why One Allocator Handles Both
+
+The name "websocket allocator" in some comments is a bit misleading — it
+**doesn't allocate websocket connections**. It allocates git file actors on
+behalf of websocket connections. All websocket connections on the server share
+the same allocator actor because:
+
+1. **Actor Model**: The allocator is a single tokio task with an mpsc channel.
+ All websocket actors send messages to it.
+2. **Shared State**: The HashMap needs to be shared across all websocket
+ connections so they can find/reuse the same git file actor for a given
+ project/branch.
+3. **Efficiency**: One allocator means one place to manage actor lifecycles,
+ reference counting, and cleanup — avoiding distributed state synchronization
+ issues.
+
+#### The Reference Counting Mechanism
+
+```
+Register → ref_count++
+DeRegister → ref_count-- (DB sets time_zeroed when it hits 0)
+ ↓
+ Background GC polls DB every 30s
+ ↓
+ Finds sessions where time_zeroed IS NOT NULL
+ ↓
+ Sends Kill to allocator → removes from HashMap, deletes DB row
+```
+
+This is how the system handles clients disconnecting at different times — the
+actor only dies when the last client deregisters.
diff --git a/async_programming/index.md b/async_programming/index.md
@@ -0,0 +1,92 @@
+# Async Programming
+
+Concurrent programming is the practice of performing multiple tasks during
+overlapping periods of time. It’s not the same as parallelism; in concurrency,
+different sequences of operations progress independently, though not necessarily
+simultaneously. This concept is quite advanced in programming, as it often
+involves managing [threads](/operating_systems/thread.md), locks, and
+avoiding pitfalls like race conditions and
+[deadlocks](https://tokio.rs/tokio/tutorial/shared-state#holding-a-mutexguard-across-an-await)—making
+concurrent code challenging to implement correctly. By structuring programs as
+independent processes that cooperate in a defined manner, developers can achieve
+more flexible and efficient designs. While these designs aren’t always parallel,
+the concurrency approach provides significant benefits in scalability and
+responsiveness.
+
+### Question: Whats the difference between concurrency and parallelism?
+
+First, start to understand why we want to make a distinction between parallel
+and concurrent in the first place!
+
+> The why has everything to do with resource utilization and efficiency.
+
+> Efficiency is the (often measurable) ability to avoid wasting materials,
+> energy, effort, money, and time in doing something or in producing a desired
+> result.
+
+> Parallelism is increasing the resources we use to solve a task. It has nothing
+> to do with efficiency.
+
+> Concurrency has everything to do with efficiency and resource utilization.
+> Concurrency can never make one single task go faster. It can only help us
+> utilize our resources better and thereby finish a set of tasks faster.
+
+(Asynchronous Programming in Rust)
+
+### Question: Whats the difference between async programming and concurrent programming?
+
+[Async programming](/async_programming/async_futures.md) is a way to
+implement concurrency without relying on OS threads, while concurrent
+programming is the broader idea of making a program do multiple things at once
+or appear to do so. In the Rust async book, concurrent programming includes
+thread-based models, but async programming keeps concurrency inside the program
+and uses an async runtime plus explicit yielding with await instead of
+OS-managed thread scheduling.
+
+#### Core difference:
+
+The Rust async book says concurrent programming means a program does multiple
+things at the same time, or at least appears to. It also notes that threads are
+one form of concurrent programming, where each thread is written sequentially
+and the operating system runs those threads concurrently.
+
+By contrast, async programming moves that coordination into your program rather
+than the operating system. In Rust, an async runtime manages tasks, and tasks
+give up control explicitly when they hit .await, which lets other tasks make
+progress.
+
+#### A simple way to think about it is this:
+
+Concurrent programming asks, “How can multiple things make progress?”
+
+Async programming answers, “We’ll do that inside one runtime, using lightweight
+tasks and await.”
+
+### Question: So does async programming avoid using threads?
+
+Not exactly. Async programming does not require one OS thread per task, but it
+can still use threads underneath. In Rust, the async model lets many lightweight
+tasks run without creating extra threads for each task, while the runtime may
+still use a small number of threads behind the scenes to drive those tasks.
+[rust-book](https://rust-book.cs.brown.edu/ch17-02-concurrency-with-async.html),
+[rust-lang](https://rust-lang.github.io/async-book/01_getting_started/02_why_async.html)
+
+What async avoids What async mainly avoids is the “one task = one thread” model.
+The Rust async material explains that you can run concurrent async work without
+spawning extra operating-system threads for each small job, which is why async
+can handle far more tasks with lower CPU and memory overhead.
+
+What may still use threads An async runtime often does use threads internally.
+The Rust book notes that, in practice, an async runtime might rely on
+operating-system threads under the hood even though your async code is written
+in terms of tasks and await, not manually managed threads.
+
+#### Simple way to think about it A good shorthand is:
+
+- Threads are one way to get concurrency.
+
+- Async is another way to structure concurrency, usually with many tasks
+ multiplexed onto a small number of threads.
+
+So the precise answer is: async programming does not mean “no threads at all”;
+it usually means “not one thread per unit of work.”
diff --git a/async_programming/queuing.md b/async_programming/queuing.md
@@ -0,0 +1,17 @@
+# Queuing
+
+Concurrency and queuing with tokio in rust must be explicitly introduced. Ways
+to do this include:
+
+- tokio::spawn
+- select!
+- join!
+- mpsc::channel
+
+When doing so, take care to ensure the total amount of concurrency is bounded.
+For example, when writing a TCP accept loop, ensure that the total number of
+open sockets is bounded. When using mpsc::channel, pick a manageable channel
+capacity. Specific bound values will be application specific.
+
+Taking care and picking good bounds is a big part of writing reliable Tokio
+applications.
diff --git a/async_programming/send_and_sync.md b/async_programming/send_and_sync.md
@@ -0,0 +1,67 @@
+# Send and Sync
+
+`Send` and `Sync` are both **marker traits** in Rust that govern thread safety,
+but they address different aspects of how data crosses thread boundaries.
+
+## `Send`: Ownership Transfer
+
+A type is `Send` if it is safe to **transfer ownership** of a value to another
+thread. In other words, you can _move_ a `T` into a new thread. Most primitive
+types (`i32`, `bool`), `String`, `Vec<T>`, and `Arc<T>` are `Send`. Types that
+are **not** `Send` include:[^1][^6]
+
+- `Rc<T>` — its reference count is non-atomic, so moving it to another thread
+ risks a data race on the count[^3]
+- Raw pointers (`*const T`, `*mut T`) — no synchronization guarantees when
+ dereferenced across threads[^6]
+
+## `Sync`: Shared Reference Access
+
+A type is `Sync` if it is safe for **multiple threads to hold shared
+references** (`&T`) to it simultaneously. Formally, `T` is `Sync` if and only if
+`&T` is `Send`. Types that are **not** `Sync` include:[^5][^1]
+
+- `RefCell<T>` — its borrow-checking is not thread-safe (non-atomic interior
+ mutability)[^6]
+- `Cell<T>` — same reason; mutable access is not synchronized[^8]
+- Raw pointers[^6]
+
+## How They Relate
+
+The two traits are closely intertwined — `Sync` is almost _defined in terms of_
+`Send`. A useful mental model:[^8]
+
+| Trait | Asks… | Example: YES | Example: NO |
+| :----- | :----------------------------------------------- | :---------------- | :---------------------- |
+| `Send` | Can ownership move to another thread? | `Arc<Mutex<T>>` | `Rc<T>` |
+| `Sync` | Can a shared `&T` be used from multiple threads? | `Mutex<T>`, `i32` | `RefCell<T>`, `Cell<T>` |
+
+A type can be `Send` but not `Sync` (e.g., `Mutex<T>` where ownership transfers
+are fine but concurrent `&T` access needs the lock), or `Sync` but not `Send`
+(e.g., a `MutexGuard`, which can be shared but must be released on the
+originating thread). Both traits are **automatically derived** by the compiler —
+you only need to intervene when working with `unsafe` code or types that should
+explicitly opt out.[^3][^8]
+<span style="display:none">[^10][^2][^4][^7][^9]</span>
+
+<div align="center">⁂</div>
+
+[^1]: https://doc.rust-lang.org/nomicon/send-and-sync.html
+
+[^2]: https://www.reddit.com/r/rust/comments/ctdkyr/understanding_sendsync/
+
+[^3]: https://blog.masteringbackend.com/rust-send-and-sync-in-simple-terms
+
+[^4]: https://stackoverflow.com/questions/59428096/understanding-the-send-trait
+
+[^5]: https://www.reddit.com/r/rust/comments/1csrbhf/understanding_send_trait_in_rust/
+
+[^6]: https://leapcell.io/blog/understanding-send-and-sync-in-rust-async-handlers
+
+[^7]: https://masteringbackend.com/posts/rust-send-and-sync-in-simple-terms
+
+[^8]: https://www.youtube.com/watch?v=yOezcP-XaIw
+
+[^9]: https://web.mit.edu/rust-lang_v1.25/arch/amd64_ubuntu1404/share/doc/rust/html/nomicon/send-and-sync.html
+
+[^10]: https://doc.rust-lang.org/book/ch16-04-extensible-concurrency-sync-and-send.html
diff --git a/async_programming/synchronization_primitives.md b/async_programming/synchronization_primitives.md
@@ -0,0 +1,10 @@
+# Synchronization Primitives
+
+[“Concurrency primitives”](/async_programming/concurrency_primitives.md) is the
+broader term: it means basic building blocks for concurrent programming,
+including things like threads/tasks, queues, futures, and also synchronization
+mechanisms.
+
+“Synchronization primitives” is a subset that specifically deals with ordering
+and coordinated access to shared state (e.g. mutexes, rwlocks, semaphores,
+barriers, condition variables, atomics.
diff --git a/commands/docker.md b/commands/docker.md
@@ -0,0 +1,4 @@
+# Docker
+
+Pretty display docker ps:
+`docker ps --format "table {{.Names}}\t{{.Status}}\t{{.Ports}}\t{{.Networks}}"`
diff --git a/commands/index.md b/commands/index.md
@@ -0,0 +1,2 @@
+# Commands
+
diff --git a/content/commands/psql.md b/commands/psql.md
diff --git a/commands/rust.md b/commands/rust.md
@@ -0,0 +1,5 @@
+# Rust
+
+error: `error[E0658]: use of unstable library feature <library-name>`
+
+command: `rustup override set nightly`
diff --git a/commands/yay.md b/commands/yay.md
@@ -0,0 +1,46 @@
+# yay
+
+# `yay -Syu` nats-server update issue
+
+When run into permission denied issue updating `nats-server`, its due to how
+`go` works:
+
+### Quick Solution
+
+The fastest fix is to manually remove the problematic cache directory with
+elevated permissions:
+
+```bash
+sudo rm -rf ~/.cache/yay/nats-server
+```
+
+Then retry building the package:
+
+```bash
+yay -S nats-server
+```
+
+### Permanent Solution
+
+To prevent this from happening in future builds, modify how the package builds
+Go modules. If you maintain or can edit the `PKGBUILD`, add the `-modcacherw`
+flag:
+
+```bash
+export GOFLAGS="-buildmode=pie -trimpath -mod=readonly -modcacherw"
+```
+
+This flag ensures Go module cache files are created with read-write permissions,
+allowing yay to clean them up properly after builds.
+
+
+
+### Alternative Approach
+
+If you don't need to preserve the cache, you can clean all yay cache:
+
+```bash
+yay -Sc --aur
+```
+
+This removes all AUR build directories and untracked files.
diff --git a/compiler/compiler.md b/compiler/compiler.md
@@ -0,0 +1,33 @@
+# Compiler
+
+Programs are files that contain a bunch of text written by you, the programmer.
+That text is parsed by a special program called a compiler, which transforms it
+into a `abstract syntax tree` (`AST`), a data structure that ignores things like
+whitespaces, comments, and where you stand on the tabs versus spaces debate.
+
+The compiler then converts that AST to a lower-level representation called
+`bytecode`. You can feed that bytecode into another program called a `runtime`
+to evaluate it and get a result. So when you run a program, what you're really
+doing is telling the runtime to evaluate the bytecode generated by the compiler
+from the AST parsed from your source code. The details vary, but for most
+languages this is an accurate high-level view.
+
+Once again, the steps are:
+
+1. Program is parsed into an AST
+2. AST is compiled to bytecode
+3. Bytecode is evaluated by the runtime
+
+---
+
+As for TypeScript...
+
+Where TypeScript is special is that instead of compiling straight to bytecode,
+TypeScript compiles to... JavaScript Code! You then run that JavaScript code
+like you normally would - in your browser, or with NodeJS, or by hand with a
+paper and pen (for anyone reading this after the machine uprising has begun).
+
+After TypeScript Compiler generates an AST for your program - but before it
+emits code - it `typechecks` your code.
+
+- from Programming TypeScript by Boris Cherny
diff --git a/compiler/compiler_tokens.md b/compiler/compiler_tokens.md
@@ -0,0 +1,119 @@
+# Compiler Tokens
+
+1. Lexxing
+
+A “lex token” is one classified unit of source code that the lexer has
+recognized and tagged with a type.
+
+“lex” refers to lexical analysis (the first stage of a compiler/interpreter that
+breaks text into tokens).
+
+A “lex token” is simply one of those enum variants produced by that stage,
+representing a smallest unit of meaning such as a keyword, identifier, operator,
+or literal.
+
+```text
+use foo as bar
+```
+
+```rust
+[
+ LexTok::Use,
+ LexTok::Identifier("foo".into()),
+ LexTok::Alias,
+ LexTok::Identifier("bar".into()),
+]
+```
+
+2. Parsing
+
+Parsing is the step where a program takes a stream of tokens (like Use, Fn,
+identifiers, (, ), etc.) and checks whether they form a valid structure
+according to the language’s grammar.
+
+After lexing has turned raw text into tokens, parsing:
+
+- Reads the tokens in order and matches them against grammar rules (like “a
+ function definition is fn + name + params + body”).
+
+- Builds a tree structure (often called a parse tree or syntax tree) that
+ represents how the program is organized: expressions, statements, blocks,
+ functions, etc.
+
+- Reports syntax errors if the token sequence doesn’t fit the grammar (missing
+ ), extra ;, wrong keyword order, and so on).
+
+In the pipeline you’re looking at:
+
+- Lexing: characters → LexTok sequence
+
+- Parsing: LexTok sequence → syntax tree (AST) that later stages (like type
+ checking or code generation) will use
+
+3. Evaluating
+
+“evaluating” is the step where you actually run or compute the meaning of an
+expression or program.
+
+- After lexing: you have tokens.
+
+- After parsing: you have a syntax tree (often an AST).
+
+- Evaluating that AST means:
+
+ - Walking the tree,
+
+ - Applying operators (+, *, ==, etc.),
+
+ - Looking up variable values, calling functions, handling control flow (if,
+ while, etc.),
+
+ - And producing a result (like 42, "hello", or some side effect like printing
+ or updating state).
+
+```
++------------------+
+| Source code |
+| (text, .ling) |
++------------------+
+ |
+ | 1. Lexing (tokenization)
+ v
++------------------+
+| Tokens stream |
+| [LexTok::Use, |
+| LexTok::Fn, |
+| Identifier, ...]|
++------------------+
+ |
+ | 2. Parsing (syntax analysis)
+ v
++---------------------------+
+| AST (syntax tree) |
+| e.g. FunctionDef( |
+| name, params, body ) |
++---------------------------+
+ |
+ | 3. Evaluating / Executing
+ v
++---------------------------+
+| Program behavior |
+| - results / return vals |
+| - printed output |
+| - changed state, etc. |
++---------------------------+
+```
+
+### Question: What is the difference between an interpreter and a compiler?
+
+- Compiler: Translates the whole program into machine code (or bytecode) before
+ you run it, producing a separate executable or binary file.
+
+- Interpreter: Reads your source code and executes it directly, usually line by
+ line or statement by statement, without producing a standalone executable.
+
+- Compiled programs usually run faster because all translation work is done
+ ahead of time and the result is optimized machine code.
+
+- Interpreted programs usually run slower because translation happens as you
+ execute the code.
diff --git a/content/compiler/define_dal_transactions_macro.md b/compiler/define_dal_transactions_macro.md
diff --git a/compiler/dynamic_dispatch.md b/compiler/dynamic_dispatch.md
@@ -0,0 +1,116 @@
+# Dynamic Dispatch
+
+**Dynamic dispatch** is when you have multiple types that something might be,
+but you don't know which one until runtime. So you **dynamically** figure out
+which one of the methods to call on the type. Usually doing this requires the
+use of **dyn** trait.
+
+Dynamic dispatch is when the concrete type implementing a trait is resolved at
+**runtime** rather than compile time. In Rust, this is done via trait objects
+(`dyn Trait`), and the compiler implements it using a
+[**vtable**](/compiler/vtable.md) — a table of function pointers generated for
+each concrete type.
+
+## [Actors](/async_programming/actors.md) are dynamic dispatch
+
+When you use actors, you don't need dynamic dispatch, because actors provide
+dynamic dispatch on theri own.
+
+Storing something in an actor is an alternative to `Box<dyn Trait>`.
+
+```rust
+struct MyActor<T: AsyncRead + AsyncWrite> {
+ receiver: mpsc::Receiver<ActorMessage>,
+ connection: T,
+}
+```
+
+Here, the generic T doesn't leak to the sender.
+
+```rust
+#[derive(Clone)]
+pub struct MyActorHandle {
+ sender: mpsc::Sender<ActorMessage>,
+}
+```
+
+Here, T could be TcpStream or UnixStream depending on the connection. Remote
+connection uses TcpStream, whereas local connection uses UnixStream, for
+example.
+
+## Static vs Dynamic Dispatch
+
+In **static dispatch**, the compiler monomorphizes generic functions — it
+generates a separate copy of the function for each concrete type used. Calls are
+resolved at compile time and can be inlined.
+
+```rust
+fn area<T: Shape>(s: &T) -> f64 {
+ s.area() // resolved at compile time
+}
+```
+
+In **dynamic dispatch**, you use a trait object. The compiler doesn't know the
+concrete type at compile time, so it emits a vtable lookup at each call site.
+
+```rust
+fn area(s: &dyn Shape) -> f64 {
+ s.area() // resolved at runtime via vtable
+}
+```
+
+## How the Compiler Implements It
+
+A `dyn Trait` value is a **fat pointer** — two machine words:
+
+- A **data pointer** to the value itself
+- A **vtable pointer** to a static table of function pointers for the concrete
+ type
+
+The vtable is emitted by the compiler for each `(ConcreteType, Trait)` pair.
+When you call a method on a `dyn Trait`, Rust loads the function pointer from
+the vtable and calls it indirectly. This means no inlining and a small overhead,
+but it enables **heterogeneous collections** and **type erasure**.
+
+```
+Box<dyn Shape>
+├── data ptr ──► [ Circle { radius: 3.0 } ]
+└── vtable ptr ─► [ drop, size, align, area, ... ]
+```
+
+## Object Safety
+
+Not every trait can be used as `dyn Trait`. A trait must be **object-safe** for
+this. The key rules are:
+
+- Methods must not return `Self`
+- Methods must not have generic type parameters
+- The trait must not require `Sized`
+
+The compiler enforces this — if a trait is not object-safe, using `dyn Trait` is
+a compile error.
+
+## When to Use Dynamic Dispatch
+
+Prefer dynamic dispatch when:
+
+- You need a heterogeneous collection (e.g. `Vec<Box<dyn Plugin>>`)
+- You want to hide a concrete type behind an abstraction boundary (e.g. plugin
+ systems, renderers, handlers)
+- Binary size matters more than the last bit of performance (monomorphization
+ bloat is real)
+
+Prefer static dispatch when call-site performance and inlining are critical, or
+when the set of concrete types is small and known ahead of time.
+
+## Relationship to Ownership
+
+Dynamic dispatch is orthogonal to both concurrency and memory safety. You can
+wrap a trait object in any ownership primitive:
+
+- `Box<dyn Trait>` — heap-allocated, single owner
+- `Rc<dyn Trait>` — reference-counted, single-threaded
+- `Arc<dyn Trait + Send + Sync>` — reference-counted, multi-threaded
+
+The ownership wrapper controls lifetime and thread safety; the vtable only
+controls how methods are dispatched.
diff --git a/compiler/dynamic_libraries.md b/compiler/dynamic_libraries.md
@@ -0,0 +1,268 @@
+# Dynamic Libraries
+
+A **dynamic library** is a compiled binary that is loaded into a program's
+address space at either load time or runtime, rather than being copied into the
+final executable by the linker. On Linux these files typically have a `.so`
+extension (shared object), on Windows `.dll` (dynamic-link library), and on
+macOS `.dylib` (dynamic library).
+
+Dynamic libraries enable **code sharing** between multiple processes: the
+operating system maps a single physical copy of the library into the virtual
+address space of every process that needs it. This reduces total memory
+footprint and allows a library to be updated independently of the executables
+that depend on it.
+
+## Dynamic vs. Static Libraries
+
+| Feature | Static Library | Dynamic Library |
+| :------ | :------------- | :---------------- |
+| Linking phase | Compile / link time | Load time or runtime |
+| Binary size | Larger executable (library copied in) | Smaller executable (reference only) |
+| Memory sharing | Each process gets its own copy | OS shares one physical copy |
+| Updates | Recompile executable to update library | Replace library file, restart process |
+| Portability | Self-contained executable | Requires compatible library present at runtime |
+
+A static library (`.a` on Unix, `.lib` on Windows) is essentially an archive of
+object files. The linker extracts the needed object files and copies them into
+the final executable. Once linked, the static library is no longer needed to run
+the program.
+
+A dynamic library remains a separate file on disk. The executable contains a
+**reference** to the library — typically a recorded name and a symbol table of
+needed functions — and the operating system's dynamic loader resolves that
+reference when the process starts (load-time linking) or when the program
+explicitly requests it (runtime linking).
+
+## How the OS Loads Dynamic Libraries
+
+When a program starts, the operating system's dynamic loader (e.g., `ld.so` on
+Linux, `dyld` on macOS, or the Windows loader) inspects the executable's
+**interpreter** and **dynamic section** to determine which shared libraries are
+required. It performs the following steps:
+
+1. **Dependency resolution** — read the list of needed libraries from the
+ executable headers.
+2. **Library search** — locate each library on the search path (`LD_LIBRARY_PATH`,
+ system cache `/etc/ld.so.cache`, `rpath`, `runpath`, or default system
+ directories).
+3. **Loading and mapping** — `mmap` the library into the process's address space.
+4. **Symbol resolution** — walk the relocation tables and patch addresses so
+ that function calls in the executable point to the correct offsets in the
+ loaded library.
+5. **Initialization** — run constructor functions (e.g., `__attribute__((constructor))`
+ in C) registered in the library.
+
+This process is known as **dynamic linking**. If a required library cannot be
+found, the loader aborts and the program fails to start.
+
+## Runtime Loading with `dlopen` and Friends
+
+Programs can also load libraries explicitly after they have already started.
+This is **runtime dynamic linking** and is the mechanism behind plugin systems,
+extensible applications, and language interpreters that load native extensions.
+
+On POSIX systems the C standard library provides four key functions:
+
+- `dlopen(path, flags)` — load a shared object into the current address space
+- `dlsym(handle, symbol)` — retrieve the address of a named symbol (function or
+ variable)
+- `dlclose(handle)` — decrement the reference count and possibly unload the library
+- `dlerror()` — return a human-readable string describing the last error
+
+On Windows the analogous APIs are `LoadLibraryA`, `GetProcAddress`, and
+`FreeLibrary`.
+
+### Example: A Minimal Dynamic C Library
+
+Imagine a small C library that computes a checksum. Save this as `checksum.c`:
+
+```c
+#include <stdint.h>
+
+uint32_t checksum(const uint8_t *data, size_t len) {
+ uint32_t sum = 0;
+ for (size_t i = 0; i < len; i++) {
+ sum = (sum << 1) | (sum >> 31); // rotate left
+ sum += data[i];
+ }
+ return sum;
+}
+```
+
+Compile it into a shared object on Linux:
+
+```bash
+gcc -shared -fPIC -o libchecksum.so checksum.c
+```
+
+The `-fPIC` flag tells the compiler to emit **position-independent code** —
+machine code that can execute correctly regardless of where in memory it is
+mapped. This is mandatory for shared libraries because the OS may load them at
+different base addresses in different processes (or in the same process across
+restarts) for security reasons such as ASLR (Address Space Layout Randomization).
+
+## Loading a Dynamic C Library from Rust
+
+Rust can interact with dynamic C libraries through two mechanisms:
+
+1. **Compile-time dynamic linking** — declare `#[link(name = "checksum")]` and
+ let the Rust linker record a dependency on `libchecksum.so`. The OS loader
+ resolves it automatically when the program starts.
+2. **Runtime dynamic loading** — use a crate such as `libloading` to `dlopen`
+ the library manually and look up symbols on demand.
+
+Runtime loading is more flexible because the program can decide at execution time
+whether to load a library, handle failures gracefully, and even swap
+implementations without restarting.
+
+### Using `libloading`
+
+Add `libloading` to your `Cargo.toml`:
+
+```toml
+[dependencies]
+libloading = "0.8"
+```
+
+Then load the library and call its function:
+
+```rust
+use libloading::{Library, Symbol};
+use std::ffi::c_void;
+
+type ChecksumFn = unsafe extern "C" fn(*const u8, usize) -> u32;
+
+fn main() -> Result<(), Box<dyn std::error::Error>> {
+ unsafe {
+ let lib = Library::new("./libchecksum.so")?;
+
+ let checksum: Symbol<ChecksumFn> = lib.get(b"checksum\0")?;
+
+ let data = b"hello dynamic world";
+ let result = checksum(data.as_ptr(), data.len());
+
+ println!("Checksum: {}", result);
+ }
+
+ Ok(())
+}
+```
+
+Key observations about this code:
+
+- The `unsafe` block is required because the compiler cannot verify the
+ correctness of a C library's ABI, pointer usage, or thread safety.
+- The symbol name `b"checksum\0"` is a null-terminated byte string, matching the
+ C ABI expectation.
+- `Library::new` calls `dlopen` (or `LoadLibrary` on Windows) under the hood.
+- `Symbol` is essentially a smart pointer to a function loaded from a
+ [**vtable**](/compiler/vtable.md)-like structure inside the dynamic loader.
+
+### Safety and ABI Mismatches
+
+When calling into a dynamic C library, Rust's usual memory-safety guarantees do
+not apply across the FFI boundary. The C library operates on raw pointers and
+expects a specific **application binary interface (ABI)**. If the Rust side
+misdeclares a function signature — for example, using `u64` instead of `usize`,
+or omitting `extern "C"` — the behavior is undefined and may result in a
+[segfault](/memory_safety/segfault.md) or silent data corruption.
+
+Best practices for dynamic C library interop:
+
+- Define a thin, audited FFI module that mirrors the C headers exactly.
+- Use `std::os::raw` or the `libc` crate for C types (`c_int`, `c_char`, etc.).
+- Keep `unsafe` blocks as small as possible; validate inputs before crossing the
+ boundary.
+- Never pass Rust references (`&T`) directly to C code expecting mutable access
+ unless you have proven alias safety manually.
+
+## Relocation and the Global Offset Table
+
+Because a shared library can be loaded at any address, it cannot contain
+absolute addresses for its own functions or global data. Instead, the compiler
+emits **position-independent code** that refers to a **Global Offset Table
+(GOT)** and a **Procedure Linkage Table (PLT)**.
+
+- **GOT** — an array of pointers to global data. The code reads data indirectly
+ through the GOT so that only the table entries need to be patched at load time,
+ not every instruction that references the data.
+- **PLT** — a trampoline for external function calls. The first time a function
+ is called through the PLT, the dynamic linker resolves the real address and
+ patches the GOT entry; subsequent calls jump directly to the resolved target.
+
+This lazy resolution (often called **lazy binding**) improves startup time for
+large programs with many shared libraries because not every symbol needs to be
+resolved immediately.
+
+## Rust as a Dynamic Library Producer
+
+Rust can also produce dynamic libraries for consumption by other languages. Two
+crate types are relevant:
+
+- **`cdylib`** — produces a C-compatible dynamic library (`.so`, `.dll`, `.dylib`).
+ Use this when a C program (or Python via ctypes, or another language) needs to
+ load your Rust code dynamically. The Rust compiler strips Rust-specific metadata
+ and exports only functions marked `#[no_mangle]` and `pub extern "C"`.
+- **`dylib`** — produces a Rust-native dynamic library. This is primarily used
+ for rustc plugins and is rarely appropriate for general FFI.
+
+Example `Cargo.toml` for a C-callable Rust library:
+
+```toml
+[package]
+name = "rust_checksum"
+version = "0.1.0"
+edition = "2021"
+
+[lib]
+crate-type = ["cdylib"]
+```
+
+And the corresponding Rust source:
+
+```rust
+#[no_mangle]
+pub extern "C" fn rust_checksum(data: *const u8, len: usize) -> u32 {
+ if data.is_null() {
+ return 0;
+ }
+ let slice = unsafe { std::slice::from_raw_parts(data, len) };
+ slice.iter().fold(0u32, |acc, &b| acc.wrapping_mul(31).wrapping_add(b as u32))
+}
+```
+
+Other languages can now `dlopen` the resulting `.so` and call `rust_checksum`
+just as if it were a C function.
+
+## When to Use Dynamic Libraries
+
+Prefer dynamic libraries when:
+
+- Multiple executables share the same code and you want to reduce disk and
+ memory usage.
+- You need a plugin architecture where third-party code is loaded at runtime.
+- You want to update a library (e.g., a security patch) without rebuilding every
+ dependent executable.
+- You are shipping a large framework and want consumers to link against a stable
+ ABI without caring about your internal implementation details.
+
+Prefer static linking when:
+
+- You need a self-contained binary that runs without external dependencies
+ (e.g., containers, embedded systems, or CLI tools distributed to unknown
+ environments).
+- Maximum startup performance is critical and you want to avoid symbol
+ resolution overhead.
+- You want whole-program optimization (LTO) to inline across library boundaries.
+
+## Summary
+
+Dynamic libraries are separate binaries loaded by the OS loader either at
+process startup or on demand via `dlopen` / `LoadLibrary`. They reduce memory
+usage through sharing and enable runtime extensibility, but they introduce
+complexities around symbol resolution, ABI stability, and distribution.
+
+When loading a dynamic C library from Rust, you cross an **FFI boundary** where
+the compiler's usual safety checks no longer apply. Tools like `libloading`
+make runtime loading ergonomic, but correctness depends on matching the C ABI
+exactly and carefully auditing every `unsafe` call site.
diff --git a/compiler/index.md b/compiler/index.md
@@ -0,0 +1,36 @@
+# Compiler
+
+# Compiler Internals
+
+This folder contains documentation about Rust compiler internals and low-level
+programming concepts.
+
+## Topics
+
+### The Compiler
+
+- [The Compiler](compiler.md) - Overview of how a compiler works
+
+### Token Processing
+
+- [Compiler Tokens](compiler_tokens.md) - Understanding lexxing and parsing
+ stages
+
+### Low-Level Mechanics
+
+- [Lifetime.md](lifetime.md) - Variable lifetimes and borrowing rules
+- [Non-Lexical Lifetimes](non_lexical_lifetimes.md) - NLL and borrow checker
+ improvements
+- [VTable.md](vtable.md) - Virtual function tables and dynamic dispatch
+- [Dynamic Libraries](dynamic_libraries.md) - Shared objects, runtime loading,
+ and FFI with C libraries
+
+### Macros & Procedural Macros
+
+- [define_dal_transactions! Macro](define_dal_transactions_macro.md) - Trait
+ generation for database access layer operations
+
+### Advanced Topics
+
+- [Dynamic Dispatch](dynamic_dispatch.md) - Runtime polymorphism and trait
+ objects
diff --git a/compiler/lifetime.md b/compiler/lifetime.md
@@ -0,0 +1,62 @@
+# Lifetime
+
+# static
+
+### Question: Whats the difference between 'static reference and 'static trait bound?
+
+The two uses of 'static in Rust look similar but mean very different things:
+
+- ``&'static T` — A Static Reference:
+
+A &'static T is a reference that lives for the entire duration of the program.
+The data it points to must be stored somewhere that never gets freed — typically
+in the binary itself (e.g., string literals) or in a static variable.
+[rust-lang](https://internals.rust-lang.org/t/idea-aliasing-the-static-lifetime-for-lifetime-parameters-trait-bounds-e-g-auto/19117)
+
+```rust
+let s: &'static str = "hello world"; // baked into the binary
+```
+
+The key constraint here is on the reference itself: the pointed-to data must
+outlive everything.
+[rust-lang](https://doc.rust-lang.org/rust-by-example/scope/lifetime/static_lifetime.html)
+
+- `T: 'static` — A Static Trait Bound:
+
+A T: 'static bound does not mean T is a reference at all. It means the type T
+contains no non-static references — i.e., T holds no borrowed data that could
+expire.
+
+```rust
+fn generic<T: 'static>(x: T) { ... }
+```
+
+Crucially, any fully owned type (like String, Vec<i32>, u32) automatically
+satisfies T: 'static, because it has no internal borrowed references that could
+go stale. A &'static str also satisfies it, but a &'a str (with a non-static
+lifetime) does not.
+[rust-lang](https://doc.rust-lang.org/rust-by-example/scope/lifetime/static_lifetime.html)
+
+> It is a common misconception that 'static always means "lives forever", but
+> this is not the case. Just because a value is 'static does not mean that you
+> have a memory leak. You can read more in
+> [Common Rust Lifetime Misconceptions](https://github.com/pretzelhammer/rust-blog/blob/master/posts/common-rust-lifetime-misconceptions.md#2-if-t-static-then-t-must-be-valid-for-the-entire-program).
+
+### Question: When `T: 'static` Is Needed?
+
+The T: 'static bound is commonly required when values must escape their creation
+scope, such as when spawning threads with thread::spawn. Since a spawned thread
+can outlive the caller, Rust requires that anything sent into it contains no
+short-lived references — which is exactly what T: 'static guarantees.
+[Rust Traits are not interfaces](https://www.jamessturtevant.com/posts/rust-traits-are-not-interfaces-and-a-little-on-lifetimes/),
+[Learning Rust: static trait bounds](https://codeandbitters.com/static-trait-bound/)
+
+```rust
+// thread::spawn requires T: Send + 'static
+fn run_in_background<T: Send + 'static>(val: T) {
+ std::thread::spawn(move || { /* use val */ });
+}
+```
+
+A useful mental shortcut: &'static T is about where data lives; T: 'static is
+about whether a type is safe to keep indefinitely.
diff --git a/compiler/non_lexical_lifetimes.md b/compiler/non_lexical_lifetimes.md
@@ -0,0 +1,90 @@
+# Non-lexical lifetimes (NLL)
+
+Non-Lexical Lifetimes (NLL) is a significant improvement to Rust's borrow
+checker, introduced in Rust 2018 and fully stabilised by default in Rust 1.63.
+It changed how the compiler reasons about _how long a reference lives_.[^8]
+
+## The Problem: Lexical Lifetimes (Before NLL)
+
+Before NLL, a reference's lifetime lasted until the **end of its enclosing
+scope** (the closing `}`), regardless of whether it was actually used after a
+certain point. This caused the borrow checker to reject perfectly safe code:[^1]
+
+```rust
+let mut data = vec![1, 2, 3];
+let first = &data[^0]; // immutable borrow starts here
+println!("{}", first); // last actual use of `first`
+
+// OLD borrow checker: `first` still "alive" until `}`, so this fails ❌
+// Even though `first` is never used again!
+data.push(4);
+```
+
+## The Fix: NLL
+
+With NLL, the compiler tracks the **last point where a reference is actually
+used**, and ends its lifetime there — not at the closing brace. This is based on
+analysing the **control-flow graph** of your code rather than just the syntactic
+block structure.[^6][^1]
+
+```rust
+let mut data = vec![1, 2, 3];
+let first = &data[^0]; // immutable borrow starts
+println!("{}", first); // ✅ last use — borrow ENDS HERE (NLL)
+
+data.push(4); // ✅ now safe to mutate
+```
+
+## Lexical vs. Non-Lexical Lifetimes
+
+| | Lexical Lifetimes (Old) | Non-Lexical Lifetimes (NLL) |
+| :----------------- | :------------------------- | :------------------------------- |
+| Lifetime ends at | Closing `}` of scope | Last actual use of the reference |
+| Based on | Syntax tree (scope blocks) | Control-flow graph |
+| Rejects safe code? | Yes, in many common cases | Much less often |
+| Introduced | Original Rust | Rust 2018, stable in Rust 1.63 |
+
+## NLL and Control Flow
+
+NLL is smart enough to handle branching logic too. A borrow is only considered
+"live" at a point if its value **could be used in the future** from that point.
+This means in conditional branches where a reference is only used in one path,
+it won't block mutations on the other path.[^7]
+
+```rust
+let mut s = String::from("hello");
+
+let r = &s;
+if some_condition {
+ println!("{r}"); // r used here in one branch
+}
+// NLL knows r MAY still be live here, so mutation below could still fail
+// depending on the control flow — the compiler reasons through each path
+```
+
+In short, NLL makes Rust's borrow checker significantly more ergonomic without
+compromising any of its safety guarantees — it simply became smarter about
+_when_ a borrow truly ends.[^1]
+<span style="display:none">[^10][^2][^3][^4][^5][^9]</span>
+
+<div align="center">⁂</div>
+
+[^1]: https://oneuptime.com/blog/post/2026-01-25-non-lexical-lifetimes-rust/view
+
+[^2]: https://users.rust-lang.org/t/about-non-lexical-lifetimes/111614
+
+[^3]: https://www.reddit.com/r/rust/comments/wgxr9q/nonlexical_lifetimes_nll_fully_stable_rust_blog/
+
+[^4]: https://stackoverflow.com/questions/50251487/what-are-non-lexical-lifetimes
+
+[^5]: https://www.youtube.com/watch?v=XD-nc28-8Fw
+
+[^6]: https://rust-lang.github.io/rfcs/2094-nll.html
+
+[^7]: https://www.reddit.com/r/rust/comments/6brtsu/eli5_nonlexical_lifetimes/
+
+[^8]: https://blog.rust-lang.org/2022/08/05/nll-by-default.html
+
+[^9]: https://smallcultfollowing.com/babysteps/blog/2017/02/21/non-lexical-lifetimes-using-liveness-and-location/
+
+[^10]: https://smallcultfollowing.com/babysteps/blog/2016/04/27/non-lexical-lifetimes-introduction/
diff --git a/compiler/vtable.md b/compiler/vtable.md
@@ -0,0 +1,6 @@
+# Vtable
+
+A **vtable** (virtual function table) in Rust is a data structure used to enable
+dynamic dispatch with trait objects (`dyn Trait`). It is essentially a lookup
+table of function pointers that allows the correct method implementation to be
+called at runtime, even when the concrete type isn't known at compile time.
diff --git a/content/async_programming/actors.md b/content/async_programming/actors.md
@@ -1,12 +0,0 @@
----
-title: Actors
----
-
-# What are actors?
-
-Basic definition in the context of tokio in rust:
-
-An actor is **a background task** that uses **message passing channels** to
-communicate with the rest of the program.
-
-[Actors with Tokio - a lesson in ownership - Alice Ryhl](https://www.youtube.com/watch?v=fTXuGRP1ee4)
diff --git a/content/async_programming/async_futures.md b/content/async_programming/async_futures.md
@@ -1,63 +0,0 @@
----
-title: Async Futures
----
-
-These two signatures are functionally equivalent — `async fn` is just syntactic
-sugar that the compiler desugars into the explicit `impl Future` form.
-
-```rust
-trait T {
- async fn f(&self) -> u32;
-}
-```
-
-```rust
-trait T {
- fn f(&self) -> impl Future<Output = u32>;
-}
-```
-
-### Question: whats the different between:
-
-```rust
-fn create_user(user: NewUser, pool: &Pool<Postgres>) -> impl std::future::Future<Output = sqlx::Result<User>> + Send;
-```
-
-and
-
-```rust
-async fn create_user(user: NewUser, pool: &Pool<Postgres>) -> sqlx::Result<User>;
-```
-
-## What the compiler does
-
-When you write:
-
-```rust
-async fn create_user(user: NewUser, pool: &Pool<Postgres>) -> sqlx::Result<User>
-```
-
-The compiler automatically rewrites it to something like:
-(rust-lang)[https://blog.rust-lang.org/inside-rust/2022/11/17/async-fn-in-trait-nightly.html]
-
-```rust
-fn create_user(user: NewUser, pool: &Pool<Postgres>) -> impl Future<Output = sqlx::Result<User>> + '_
-```
-
-So the first signature in your example is just the explicit version of what
-async fn does implicitly.
-
-The + Send detail The most important practical difference in your specific
-example is the explicit `+ Send` bound on the first signature. This guarantees
-the returned future is safe to send across threads, which is required when
-spawning tasks with `tokio::spawn`. With `async fn`, whether the future is
-`Send` is inferred from the function body — if any non-`Send` type is held
-across an .await` point, the compiler will reject it without giving you an
-explicit contract.
-[rust-lang](https://rust-lang.github.io/async-book/part-guide/more-async-await.html)
-
-
-So the explicit form is useful when you're writing trait objects, function
-pointers, or want to enforce `Send` as part of a public API contract, while
-`async
-fn` is preferred for everyday use.
diff --git a/content/async_programming/busy_loop.md b/content/async_programming/busy_loop.md
@@ -1,38 +0,0 @@
----
-title: Busy Loop
----
-
-### Question: 什么是 Busy Loop（忙等待循环）？
-
-Busy loop（忙循环），也叫 busy waiting（忙等待） 或
-spinning（自旋），指的是程序在等待某个条件满足时，不断地反复检查这个条件，而不是暂停让出
-CPU 资源 。
-
-#### 通俗理解
-
-想象你在等外卖，有两种等法：
-
-- Busy loop（忙等）：你每隔一秒就跑去门口看一眼，"到了没？到了没？到了没？" —
- 你一直在消耗精力，什么别的事也做不了。
-
-- 正常等待（阻塞/回调）：你去做别的事，门铃响了再去开门 — 不浪费任何精力。
-
-#### 你代码里的问题
-
-帖子中的代码就是典型的 busy loop ：
-
-```rust
-let my_val = loop {
- if let Some(val) = shared.try_lock() {
- break val;
- }
- // 没拿到锁？继续循环，反复尝试...
-};
-```
-
-这段代码在拿不到锁时会不停地循环尝试，持续占用 CPU，毫无效率
-。[Busy Waiting](https://www.geeksforgeeks.org/operating-systems/busy-waiting-in-os/)
-
-关键原则：在 Rust 异步程序中，只要你不在持有锁的情况下调用 .await，用
-std::sync::Mutex 是完全安全且高效的，因为 Rust 的异步运行时只会在 .await
-点才切换任务 。
diff --git a/content/async_programming/channels.md b/content/async_programming/channels.md
@@ -1,21 +0,0 @@
----
-title: Channels
----
-
-Channels: A [communication primitive](/async_programming/concurrency_primitives.md)
-used in many languages (Rust, Go, .NET, etc.) to pass messages between
-concurrent pieces of code (goroutines, tasks, threads).
-
-# Tokio's channel primitives
-
-- mpsc: multi-producer, single-consumer channel. Many values can be sent.
-
-- oneshot: single-producer, single consumer channel. A single value can be sent.
-
-- broadcast: multi-producer, multi-consumer. Many values can be sent. Each
- receiver sees every value.
-
-- watch: multi-producer, multi-consumer. Many values can be sent, but no history
- is kept. Receivers only see the most recent value.
-
- [tokio-tutorial](/exercises/tokio-tutorial/index.md)
diff --git a/content/async_programming/concurrency_primitives.md b/content/async_programming/concurrency_primitives.md
@@ -1,60 +0,0 @@
----
-title: Concurrency Primitives
----
-
-**Concurrency primitives** in Rust are the fundamental building blocks that
-allow multiple parts of a program to run simultaneously — safely and without
-data races. They are the low-level tools you use to coordinate concurrent tasks,
-share data between threads, and synchronize execution.
-
-## Why Rust Is Special Here
-
-Rust's ownership and type system enforce concurrency safety at **compile time**,
-not at runtime. Many bugs that would silently corrupt data in other languages
-become compile errors in Rust, which is why Rust calls its approach "fearless
-concurrency".
-[doc.rust-lang](https://doc.rust-lang.org/book/ch16-00-concurrency.html)
-
-## The Core Primitives
-
-- [**Threads**](/operating_systems/thread.md) — the most basic primitive;
- independent paths of execution that run concurrently, letting you exploit
- multi-core processors
- [earthly](https://earthly.dev/blog/rust-concurrency-patterns-parallel-programming/)
-- **Channels (`mpsc`)** — typed message-passing pipes with a sender and receiver
- handle; one thread sends data, another receives it, avoiding shared memory
- entirely [news.ycombinator](https://news.ycombinator.com/item?id=7851274)
-- [**Mutex (`Mutex<T>`)**](/memory_safety/mutex.md) — short for _mutual
- exclusion_; only one thread can access the protected data at a time,
- preventing data races on shared state
- [earthly](https://earthly.dev/blog/rust-concurrency-patterns-parallel-programming/)
-- [**Arc (`Arc<T>`)**](/memory_safety/arc.md) — _Atomic Reference Counting_;
- lets multiple threads share ownership of a value safely
- [doc.rust-lang](https://doc.rust-lang.org/book/ch16-03-shared-state.html)
-- [**`RwLock<T>`**](/memory_safety/rwlock_pattern.md) — like a Mutex, but allows
- many simultaneous readers or one exclusive writer
-- [**Atomic types**](/memory_safety/atomic.md) — low-level primitives (e.g.,
- `AtomicUsize`) for lock-free, thread-safe operations on simple values
- [web.mit](https://web.mit.edu/rust-lang_v1.25/arch/amd64_ubuntu1404/share/doc/rust/html/book/first-edition/concurrency.html)
-
-## Key Traits: `Send` and `Sync`
-
-Rust enforces concurrency rules through two marker traits:
-[web.mit](https://web.mit.edu/rust-lang_v1.25/arch/amd64_ubuntu1404/share/doc/rust/html/book/first-edition/concurrency.html)
-
-- **`Send`** — a type can be transferred (moved) to another thread
-- **`Sync`** — a type can be safely _referenced_ from multiple threads
- simultaneously
-
-These traits are automatically implemented by the compiler where safe, and
-withheld where they aren't — so you can't accidentally send a non-thread-safe
-type across a thread boundary.
-[google.github](https://google.github.io/comprehensive-rust/concurrency/welcome.html)
-
-## A Simple Mental Model
-
-Think of concurrency primitives as traffic rules for threads. Channels say
-_"pass the data by handing it off"_, while `Mutex`/`Arc` say _"share the data,
-but take turns"_. Rust's compiler acts as the traffic enforcer, rejecting unsafe
-patterns before your code ever runs.
-[dzone](https://dzone.com/articles/concurrency-in-rust-safe-and-efficient-code)
diff --git a/content/async_programming/cpu.md b/content/async_programming/cpu.md
@@ -1,7 +0,0 @@
----
-title: CPU
----
-
-<img src="/whiteboards/cpu.png" alt="CPU" width="100%">
-
-[What are semiconductors?](https://newsroom.intel.com/de/tech101/what-are-semiconductors)
diff --git a/content/async_programming/git_actors_case_study.md b/content/async_programming/git_actors_case_study.md
@@ -1,642 +0,0 @@
----
-title: Git Actors Case Study
----
-
-# Why Custom Git Management Uses an Actor System Instead of HTTP Requests
-
-## Executive Summary
-
-A real world project uses an actor-based architecture for git management, not
-primarily because of WebSocket integration, but because git operations are
-inherently stateful, long-running, and require shared mutable state that HTTP's
-stateless request-response model cannot efficiently handle.
-
-See also:
-[DAL Architecture Overview](/system_design/dal_architecture_overview.md)
-
----
-
-## 1. The Core Problem: Stateful Git Operations
-
-### What HTTP Gives You
-
-```
-Client → HTTP Request → Server → HTTP Response → Client
-```
-
-- Stateless: Each request is independent
-- Fire-and-forget: No persistent connection
-- No shared context between requests
-
-### What Git Actually Requires
-
-```
-Client → Load Repo → Edit Files → Save → Commit → Switch Branch → ...
- ↓ ↓ ↓ ↓
- [Actor maintains working directory state]
-```
-
-Git operations are stateful:
-
-1. You load a repository once
-2. Make multiple edits over time
-3. The edits persist in a working directory
-4. You compile, stage, commit incrementally
-5. The state persists until explicitly saved or the session ends
-
----
-
-## 2. Architectural Analysis of the Codebase
-
-### The Three-Layer Architecture
-
-```
-┌──────────────────────────────────────────────────────────────────────────┐
-│ FRONTEND (Svelte) │
-│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
-│ │ FileTree │ │ CodeEditor │ │ GitEditor │ │ Canvas │ │
-│ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ │
-│ │ │ │ │ │
-│ └────────────────┼────────────────┼────────────────┘ │
-│ │ │
-│ ┌───────────▼───────────────┐ │
-│ │ WebSocket Connection │◄──── Persistent Connection │
-│ │ (TypeScript Client) │ │
-│ └─────────────┬─────────────┘ │
-└────────────────────────────┼─────────────────────────────────────────────┘
- │ Binary Protocol (MessagePack)
- ▼
-┌─────────────────────────────────────────────────────────────────────────┐
-│ BACKEND (Rust) │
-│ ┌─────────────────────────────────────────────────────────────────┐ │
-│ │ WebSocket Actor (per-connection) │ │
-│ │ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │ │
-│ │ │ StaticSession │ │ DynamicSession │ │ Ingress Router │ │ │
-│ │ │ (auth, sender) │ │ (actor sender) │ │ │ │ │
-│ │ └────────┬────────┘ └────────┬────────┘ └─────────────────┘ │ │
-│ └───────────┼────────────────────┼────────────────────────────────┘ │
-│ │ │ │
-│ └──────────┬─────────┘ │
-│ │ │
-│ ┌──────────────────────▼──────────────────────────────────────────┐ │
-│ │ ALLOCATOR ACTOR (Singleton per Server) │ │
-│ │ │ │
-│ │ HashMap<(project_id, branch) → (GitActorSender, JoinHandle)> │ │
-│ │ │ │
-│ │ Messages: Register | DeRegister | Kill | GetSender | GC │ │
-│ └─────────────────────────┬───────────────────────────────────────┘ │
-│ │ │
-│ ┌──────────────────┼──────────────────┐ │
-│ ▼ ▼ ▼ │
-│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
-│ │ Git Files │ │ Git Files │ │ Git Files │ ... │
-│ │ Actor 1 │ │ Actor 2 │ │ Actor 3 │ │
-│ │ (proj:1, │ │ (proj:2, │ │ (proj:1, │ │
-│ │ branch:a) │ │ branch:x) │ │ branch:b) │ │
-│ └──────┬──────┘ └──────┬──────┘ └──────┬──────┘ │
-│ │ │ │ │
-│ ▼ ▼ ▼ │
-│ ┌─────────────┐ ┌─────────────┐ ┌─────────────┐ │
-│ │ Working Dir │ │ Working Dir │ │ Working Dir │ │
-│ │ (TempDir) │ │ (TempDir) │ │ (TempDir) │ │
-│ │ + Compiler │ │ + Compiler │ │ + Compiler │ │
-│ │ State │ │ State │ │ State │ │
-│ └─────────────┘ └─────────────┘ └─────────────┘ │
-│ │
-│ ┌─────────────────────────────────────────────────────────────┐ │
-│ │ DATABASE (PostgreSQL) │ │
-│ │ ┌────────────┐ ┌────────────┐ ┌────────────────────────┐ │ │
-│ │ │ Git Blobs │ │ Git File │ │ Auth Sessions │ │ │
-│ │ │ (tarballs) │ │ Actor │ │ WebSocket Sessions │ │ │
-│ │ │ │ │ Sessions │ │ │ │ │
-│ │ └────────────┘ └────────────┘ └────────────────────────┘ │ │
-│ └─────────────────────────────────────────────────────────────┘ │
-└─────────────────────────────────────────────────────────────────────────┘
-```
-
----
-
-## 3. Why Actors, Not HTTP?
-
-### 3.1 Stateful Working Directory
-
-```rust
- // From git_files/actor.rs
- pub async fn git_files_actor<X: ...>(
- mut rx: mpsc::Receiver<IncomingFileActorMessage>,
- project_id: i32,
- branch: String,
- Storage handle: Arc<dyn GitDataTransfer + 'static>,
- ) -> Result<(), NanoServiceError> {
-
- // 1. Load git data from storage ONCE
- let git_data = storage_handle.load_git_data(project_id, branch.clone()).await?;
-
- // 2. Unpack to temp directory ONCE
- unpack_tar_gz(git_data, source_dir.clone())?;
-
- // 3. Cache compiler state in memory
- let mut compiler_state = EntryPointStates::new();
- let dep_graph = load_from_disk(&source_dir).unwrap_or(DependencyGraph::new());
-
- // 4. Listen for operations on this specific working directory
- while let Some(message) = rx.recv().await {
- match message {
- IncomingFileActorMessage::ReadFile(tx, path) => { ... }
- IncomingFileActorMessage::WriteFile(tx, path, data) => { ... }
- IncomingFileActorMessage::Compile(tx, path) => { ... }
- // ...
- }
- }
- }
-```
-
-If this were HTTP:
-
-```
-HTTP POST /files/read → Must reload repo, unpack tarball, return file
-HTTP POST /files/write → Must reload repo, unpack tarball, write, repack, save
-HTTP POST /compile → Must reload repo, unpack tarball, load graph, compile
-```
-
-Each request would:
-
-- Download the entire repo from DB (expensive)
-- Unpack the tarball (slow)
-- Perform tiny operation
-- Save back to DB
-- No caching of compilation state
-
-### 3.2 Compiler State Persistence
-
-```rust
- // The actor maintains compilation state across requests
- let mut compiler_state = EntryPointStates::new();
-
- // First compile: builds dependency graph from scratch
- let outcome = compile_entry_point(source_dir.as_path(), file_path, &mut compiler_state).await;
-
- // Second compile: reuses cached graph, only recompiles changed nodes
- let outcome = compile_entry_point(source_dir.as_path(), file_path, &mut compiler_state).await;
-```
-
-Why this matters:
-
-- Dependency graphs can be megabytes for complex CAD projects
-- Incremental compilation: Change one file → only recompile affected nodes
-- HTTP cannot do this: No shared state between requests
-
-### 3.3 Reference Counting & Session Management
-
-```rust
- // From allocator/actor.rs
- pub async fn websocket_allocator_actor(...) {
- let mut allocator = AllocatorMap::new(); // In-memory state
-
- while let Some(message) = rx.recv().await {
- match message {
- IncomingAllocatorMessage::Register(tx, project_id, branch) => {
- // Check if actor already exists
- // If yes: increment ref_count, return existing sender
- // If no: spawn new actor, return new sender
- }
- IncomingAllocatorMessage::DeRegister(tx, project_id, branch) => {
- // Decrement ref_count
- // If ref_count == 0: set time_zeroed for garbage collection
- }
- }
- }
- }
-```
-
-```rust
- // From state.rs
- pub type AllocatorMap = HashMap<AllocatorKey, (GitActorSender, ActorJoinHandle)>;
-
- #[derive(Debug, PartialEq, Hash, Eq)]
- pub struct AllocatorKey {
- pub project_id: i32,
- pub branch: String
- }
-```
-
-The Session Model:
-
-```
-User A opens project 1, branch "main" → ref_count = 1
-User B opens project 1, branch "main" → ref_count = 2, SAME actor
-User A closes → ref_count = 1, actor stays alive
-User B closes → ref_count = 0, actor marked for GC
-```
-
-HTTP Alternative Problems:
-
-```
-HTTP: No persistent state. Each request is independent.
-- User A opens project: start session
-- User A makes 100 edits: 100 independent requests
-- User B opens same project: start ANOTHER session
-- Database: Two separate copies of the repo loaded
-- Memory: Double memory usage
-- Coherence: Two separate working directories, no shared state
-```
-
-### 3.4 Locking & Consistency
-
-```rust
- // From git_files/actor.rs - AcquireLock message
- IncomingFileActorMessage::AcquireLock(sender, rx) => {
- let _ = sender.send(OutgoingFileActorMessage::LockAcquired);
-
- // Mutex-like behavior: hold lock until ReleaseLock
- let outcome = match rx.await {
- Ok(message) => message,
- Err(_) => continue,
- };
- match outcome {
- IncomingFileActorMessage::ReleaseLock(tx) => {
- let _: Result<(), OutgoingFileActorMessage> =
- tx.send(OutgoingFileActorMessage::LockReleased);
- },
- _ => continue,
- }
- }
-```
-
-Why locks matter:
-
-- Two users editing the same file simultaneously
-- User A's write must complete before User B's write
-- Actor ensures sequential processing of messages
-- No race conditions, no lost updates
-
-### 3.5 Garbage Collection & Cleanup
-
-```rust
-// From garbage_collector.rs
-pub async fn garbage_collector<X>(alloc_sender: AllocatorMessageSender) {
- loop {
- sleep(Duration::from_secs(20)).await;
- let _ = send_gc_request(&alloc_sender).await;
- }
-}
-```
-
-The lifecycle:
-
-1. User opens project → Actor spawned, ref_count = 1
-2. Multiple users open → ref_count incremented
-3. User closes → ref_count decremented
-4. Last user closes → ref_count = 0, time_zeroed set
-5. GC runs every 20 seconds → deletes actors with time_zeroed > 120 seconds ago
-6. Grace period: If user reopens within 2 minutes, actor still exists
-
-HTTP can't do this: No state to clean up, but also no caching benefits.
-
----
-
-## 4. The WebSocket Integration (Secondary Benefit)
-
-You asked: "Is it because of its integration with WebSocket frontend?"
-
-Partially, but it's not the primary reason. Here's the relationship:
-
-### WebSocket Benefits (Secondary)
-
-```
-┌───────────────────────────────────────────────────────────────┐
-│ HTTP vs WebSocket │
-├───────────────────────────────────────────────────────────────┤
-│ HTTP: │
-│ - Open connection, send request, get response, close │
-│ - Good for: auth, project CRUD, one-off operations │
-│ │
-│ WebSocket: │
-│ - Persistent connection, bidirectional messaging │
-│ - Good for: real-time file editing, compilation feedback, │
-│ multiplayer sync, server-initiated notifications │
-└───────────────────────────────────────────────────────────────┘
-```
-
-### But HTTP Could Also Work with Actors!
-
-```rust
- // Hypothetical HTTP approach with actors (NOT how it's done here)
- // HTTP endpoints would still talk to actors internally:
-
- async fn read_file_handler(
- Query((project_id, branch)): Query<(i32, String)>,
- ) -> impl IntoResponse {
- // 1. Get actor sender from allocator
- let sender = get_actor_sender(project_id, &branch).await?;
-
- // 2. Send message to actor
- let path = "file.txt";
- send_read_file_request(path, &sender).await
- }
-```
-
-So why WebSocket?
-
-| Requirement | HTTP | WebSocket | Actor System |
-| ------------------------ | ---- | --------- | ------------ |
-| Stateful file operations | ❌ | ⚠️ | ✅ |
-| Compiler state caching | ❌ | ❌ | ✅ |
-| Session ref counting | ❌ | ⚠️ | ✅ |
-| Real-time updates | ❌ | ✅ | ✅ |
-| Locking/concurrency | ❌ | ⚠️ | ✅ |
-| Server→client push | ❌ | ✅ | ✅ |
-
-## 5. System Design Patterns Used
-
-### 5.1 Actor Pattern (Erlang-style)
-
-```rust
-// Single-threaded message processing per actor
-while let Some(message) = rx.recv().await {
- // Process ONE message at a time
- // No locks needed within the actor
- // Actor owns all its state
-}
-```
-
-### 5.2 Resource Pool Pattern (Allocator)
-
-```rust
-// One actor per (project, branch) tuple
-// Reused across multiple websocket sessions
-// Ref counting prevents premature cleanup
-```
-
-### 5.3 Supervisor Pattern (implied)
-
-```rust
-// If actor panics, it's isolated
-// WebSocket handler aborts ping actor
-// Cleanup still runs on session end
-```
-
-### 5.4 Message Passing Concurrency
-
-```rust
-// No shared mutable state
-// Communication via channels only
-// Type-safe message protocols
-```
-
-### 5.5 Event Sourcing (implied)
-
-```rust
-// Changes don't modify stored data immediately
-// SaveSnapshot packages entire working dir
-// Stored as tarball in DB (immutable blob)
-```
-
----
-
-## 6. Comparison: Actor vs HTTP Architectures
-
-### Actor-Based (Current)
-
-```
-┌────────────────────────────────────────────────────────────────────┐
-│ ACTOR SYSTEM │
-├───────┬───┬──────────────┬───────────────┬───────────────┬─────────┤
-│ │ │
-│ Session 1 ─┬─► DynamicSession ─┬─► Allocator ──┬─► GitActor ──────┼──► TempDir
-│ │ │ │ │ + State
-│ Session 2 ─┤ │ │ │
-│ │ │ └─► GitActor ──────┼──► TempDir
-│ Session 3 ─┘ │ │ + State
-│ │ │
-│ Global GC ────────────────────────────────────────────────────────┼──► Cleanup
-│ │
-│ ✓ Single copy of repo in memory per (project, branch) │
-│ ✓ Compiler state persists across requests │
-│ ✓ Atomic operations with locks │
-│ ✓ Graceful cleanup via ref counting │
-│ ✓ WebSocket naturally maps to actor sessions │
-└────────────────────────────────────────────────────────────────────┘
-```
-
-### HTTP-Based Alternative (Theoretical)
-
-```
-┌─────────────────────────────────────────────────────────────────┐
-│ HTTP STATELESS SYSTEM │
-├───────┬───┬──────────────┬───────────────┬───────────────┬──────┤
-│ │ │
-│ HTTP/1 ──► Load Project ──► Edit ──► Save ──► Compile ──► ... │
-│ │ │ │ │ │
-│ ▼ ▼ ▼ ▼ │
-│ Download Upload Upload Upload │
-│ tarball tarball tarball tarball │
-│ │ │ │ │ │
-│ └───────────┴────────┴─┬─────┘ │
-│ DB (every operation) │
-│ │
-│ ✗ Download entire repo on every request │
-│ ✗ Unpack/repack tarball on every operation │
-│ ✗ No compiler state caching │
-│ ✗ No incremental compilation │
-│ ✗ Multiple users = multiple copies of same repo │
-│ ✗ Slow response times │
-│ ✗ High database load │
-└─────────────────────────────────────────────────────────────────┘
-```
-
----
-
-## 7. Summary: Why Actor System Wins
-
-| Factor | HTTP | Actor System |
-| ------------------------ | ------------------------------ | ------------------------------- |
-| Stateful file operations | ❌ Would need external cache | ✅ Actors own working directory |
-| Compiler state | ❌ Must reload on each compile | ✅ Cached in memory |
-| Incremental compilation | ❌ Full rebuild every time | ✅ Only changed nodes |
-| Session management | ⚠️ External session store | ✅ Ref counting built-in |
-| Locking | ⚠️ Database locks | ✅ Message queue serializes |
-| Memory efficiency | ❌ N copies for N users | ✅ 1 copy shared via sender |
-| Real-time updates | ❌ Long-polling/comet | ✅ WebSocket native |
-| Graceful cleanup | ⚠️ TTL-based | ✅ GC with grace period |
-| WebSocket integration | ⚠️ Request-response over WS | ✅ Native message passing |
-| Multiplayer sync | ❌ Complex broadcast logic | ✅ Actors as session boundaries |
-
-The actor system is used because git operations are fundamentally stateful, and
-the actor model provides:
-
-1. Stateful working directories — Load once, edit many times
-2. Compiler state caching — Incremental compilation via dependency graphs
-3. Session multiplexing — One actor, many websocket connections
-4. Reference counting — Memory-efficient session management
-5. Locking — Sequential
-
----
-
-## 9. Reference: HashMap Allocator Details
-
-### What the HashMap Allocator Does for the Git Management Actor
-
-The AllocatorMap (`HashMap<AllocatorKey, (GitActorSender, ActorJoinHandle)>`)
-acts as the in-memory registry for all running git file actors on this server.
-Here's what it does:
-
-#### Core Responsibilities
-
-1. **Tracks Live Actors**: Every time a new git file actor is spawned (on first
- Register), its (sender, join_handle) tuple gets inserted into the HashMap
- keyed by (project_id, branch).
-2. **Enables Actor Reuse**: When a subsequent Register comes in for the same
- project/branch, instead of spawning a new actor, the allocator:
- - Increments the ref_count in the DB (for tracking how many clients are using
- it)
- - Looks up the existing sender in the HashMap and returns it (no new actor
- spawned)
- - This is why both senders in your multi-client tests work — they point to
- the same actor.
-3. **Provides Fast O(1) Sender Lookup**: The `get_sender` process does a HashMap
- lookup to retrieve a cloned sender. This is a synchronous, non-DB operation —
- critical for low-latency websocket routing.
-4. **Cleans Up on Server Restart**: The actor starts by wiping all sessions in
- the DB for this server tag, ensuring stale state from a previous crashed
- instance is gone.
-5. **Enables Targeted Kill**: The kill process removes the entry from the
- HashMap, then waits on the join handle to confirm the actor stopped.
-
-#### Relationship Between Websocket Actor Allocator and Git File Actor Allocator
-
-They are the same allocator — **there is only one actor managing everything**.
-Here's how they relate:
-
-```
-┌────────────────────────────────────────────────────────────────────┐
-│ WEBSOCKET CONNECTION #1 │
-│ (one per connected client browser) │
-│ │
-│ - Owns its own ping actor (health monitoring) │
-│ - Owns DynamicSession + StaticSession state │
-│ - Communicates with the allocator via mpsc channel │
-└────────────────────────────────────────────────────────────────────┘
- │
- │ send_register_request()
- ▼
-┌────────────────────────────────────────────────────────────────────┐
-│ ALLOCATOR ACTOR (Single Global Actor) │
-│ ┌───────────────────────────────────────────────────────────────┐ │
-│ │ AllocatorMap HashMap (in-memory state) │ │
-│ │ Key: AllocatorKey { project_id, branch } │ │
-│ │ Value: (GitActorSender, JoinHandle) │ │
-│ │ │ │
-│ │ Example entries: │ │
-│ │ (project:42, "main") -> (sender_A, handle_1) │ │
-│ │ (project:42, "dev") -> (sender_B, handle_2) │ │
-│ │ (project:99, "main") -> (sender_C, handle_3) │ │
-│ └───────────────────────────────────────────────────────────────┘ │
-│ │
-│ Receives messages: Register, DeRegister, Kill, GetSender, GC │
-└────────────────────────────────────────────────────────────────────┘
- │
- ┌─────────────┴────────────┐
- │ │
- ▼ ▼
-┌──────────────────────────────────┐ ┌──────────────────────────────────┐
-│ GIT FILE ACTOR │ │ GIT FILE ACTOR │
-│ (project:42, branch:"main") │ │ (project:42, branch:"dev") │
-│ │ │ │
-│ - Owns temp dir on disk │ │ - Owns temp dir on disk │
-│ - Handles file operations │ │ - Handles file operations │
-│ - Has compiler state │ │ - Has compiler state │
-│ - Persists to DB on save │ │ - Persists to DB on save │
-└──────────────────────────────────┘ └──────────────────────────────────┘
-```
-
-#### The Message Flow
-
-```
-Client Browser Websocket Actor
- │ │
- │──── websocket connect ─────────────────>│
- │ │
- │ │ 1. auth check
- │ │ 2. calls allocator_actor_constructor()
- │ │ (static singleton, runs once per server)
- │ │
- │ │ 3. send_register_request(project_id, "main")
- │ │ to allocator via mpsc::Sender
- │ │
- │ ▼
- │ ┌───────────────────┐
- │ │ ALLOCATOR ACTOR │
- │ │ │
- │ │ checks DB → no existing session
- │ │ spawns git_files_actor_constructor()
- │ │ inserts into HashMap
- │ │ creates DB session (ref_count=1)
- │ └───────────────────┘
- │ │
- │ │ returns GitActorSender
- │ │
- │ ▼
- │ ┌───────────────────┐
- │ │ GIT FILE ACTOR │
- │ │ (project:42, main)│
- │ │ - loads tarball │
- │ │ - extracts files │
- │ │ - in-memory state │
- │ └───────────────────┘
- │ │
- │<────── websocket messages ──────────────┤
- │ (routed to git file actor) │
- │ │
- │ ─────── disconnect ────────────────────>│
- │ │ cleanup() → send_deregister_request()
- │ │ ref_count decremented in DB
- │ │ HashMap entry NOT removed (actor stays alive)
- │ │
- │ [if ref_count == 0, background GC sends Kill
- │ → removes from HashMap, deletes DB session]
-```
-
-#### Key Distinction: Two Different Session Types
-
-| Session Type | Storage | Purpose | Managed By |
-| ---------------------- | ---------------------------------- | -------------------------------------------------------------------- | ------------------------------------ |
-| Websocket Session | DB only (websocket_sessions table) | Track which users are connected, server tag for cleanup | Websocket actor's cleanup() function |
-| Git File Actor Session | DB + HashMap | Track actor lifecycle, ref_count for sharing, server tag for cleanup | Allocator actor |
-
-The websocket session exists purely in the DB to survive server restarts (so you
-know a user was connected before the crash). The git file actor session lives in
-both DB and HashMap — the DB for persistence across restarts, the HashMap for
-fast in-memory access.
-
-#### Why One Allocator Handles Both
-
-The name "websocket allocator" in some comments is a bit misleading — it
-**doesn't allocate websocket connections**. It allocates git file actors on
-behalf of websocket connections. All websocket connections on the server share
-the same allocator actor because:
-
-1. **Actor Model**: The allocator is a single tokio task with an mpsc channel.
- All websocket actors send messages to it.
-2. **Shared State**: The HashMap needs to be shared across all websocket
- connections so they can find/reuse the same git file actor for a given
- project/branch.
-3. **Efficiency**: One allocator means one place to manage actor lifecycles,
- reference counting, and cleanup — avoiding distributed state synchronization
- issues.
-
-#### The Reference Counting Mechanism
-
-```
-Register → ref_count++
-DeRegister → ref_count-- (DB sets time_zeroed when it hits 0)
- ↓
- Background GC polls DB every 30s
- ↓
- Finds sessions where time_zeroed IS NOT NULL
- ↓
- Sends Kill to allocator → removes from HashMap, deletes DB row
-```
-
-This is how the system handles clients disconnecting at different times — the
-actor only dies when the last client deregisters.
diff --git a/content/async_programming/index.md b/content/async_programming/index.md
@@ -1,94 +0,0 @@
----
-title: Async Programming
----
-
-Concurrent programming is the practice of performing multiple tasks during
-overlapping periods of time. It’s not the same as parallelism; in concurrency,
-different sequences of operations progress independently, though not necessarily
-simultaneously. This concept is quite advanced in programming, as it often
-involves managing [threads](/content/operating_systems/thread.md), locks, and
-avoiding pitfalls like race conditions and
-[deadlocks](https://tokio.rs/tokio/tutorial/shared-state#holding-a-mutexguard-across-an-await)—making
-concurrent code challenging to implement correctly. By structuring programs as
-independent processes that cooperate in a defined manner, developers can achieve
-more flexible and efficient designs. While these designs aren’t always parallel,
-the concurrency approach provides significant benefits in scalability and
-responsiveness.
-
-### Question: Whats the difference between concurrency and parallelism?
-
-First, start to understand why we want to make a distinction between parallel
-and concurrent in the first place!
-
-> The why has everything to do with resource utilization and efficiency.
-
-> Efficiency is the (often measurable) ability to avoid wasting materials,
-> energy, effort, money, and time in doing something or in producing a desired
-> result.
-
-> Parallelism is increasing the resources we use to solve a task. It has nothing
-> to do with efficiency.
-
-> Concurrency has everything to do with efficiency and resource utilization.
-> Concurrency can never make one single task go faster. It can only help us
-> utilize our resources better and thereby finish a set of tasks faster.
-
-(Asynchronous Programming in Rust)
-
-### Question: Whats the difference between async programming and concurrent programming?
-
-[Async programming](/async_programming/async_futures.md) is a way to
-implement concurrency without relying on OS threads, while concurrent
-programming is the broader idea of making a program do multiple things at once
-or appear to do so. In the Rust async book, concurrent programming includes
-thread-based models, but async programming keeps concurrency inside the program
-and uses an async runtime plus explicit yielding with await instead of
-OS-managed thread scheduling.
-
-#### Core difference:
-
-The Rust async book says concurrent programming means a program does multiple
-things at the same time, or at least appears to. It also notes that threads are
-one form of concurrent programming, where each thread is written sequentially
-and the operating system runs those threads concurrently.
-
-By contrast, async programming moves that coordination into your program rather
-than the operating system. In Rust, an async runtime manages tasks, and tasks
-give up control explicitly when they hit .await, which lets other tasks make
-progress.
-
-#### A simple way to think about it is this:
-
-Concurrent programming asks, “How can multiple things make progress?”
-
-Async programming answers, “We’ll do that inside one runtime, using lightweight
-tasks and await.”
-
-### Question: So does async programming avoid using threads?
-
-Not exactly. Async programming does not require one OS thread per task, but it
-can still use threads underneath. In Rust, the async model lets many lightweight
-tasks run without creating extra threads for each task, while the runtime may
-still use a small number of threads behind the scenes to drive those tasks.
-[rust-book](https://rust-book.cs.brown.edu/ch17-02-concurrency-with-async.html),
-[rust-lang](https://rust-lang.github.io/async-book/01_getting_started/02_why_async.html)
-
-What async avoids What async mainly avoids is the “one task = one thread” model.
-The Rust async material explains that you can run concurrent async work without
-spawning extra operating-system threads for each small job, which is why async
-can handle far more tasks with lower CPU and memory overhead.
-
-What may still use threads An async runtime often does use threads internally.
-The Rust book notes that, in practice, an async runtime might rely on
-operating-system threads under the hood even though your async code is written
-in terms of tasks and await, not manually managed threads.
-
-#### Simple way to think about it A good shorthand is:
-
-- Threads are one way to get concurrency.
-
-- Async is another way to structure concurrency, usually with many tasks
- multiplexed onto a small number of threads.
-
-So the precise answer is: async programming does not mean “no threads at all”;
-it usually means “not one thread per unit of work.”
diff --git a/content/async_programming/queuing.md b/content/async_programming/queuing.md
@@ -1,19 +0,0 @@
----
-title: Queuing
----
-
-Concurrency and queuing with tokio in rust must be explicitly introduced. Ways
-to do this include:
-
-- tokio::spawn
-- select!
-- join!
-- mpsc::channel
-
-When doing so, take care to ensure the total amount of concurrency is bounded.
-For example, when writing a TCP accept loop, ensure that the total number of
-open sockets is bounded. When using mpsc::channel, pick a manageable channel
-capacity. Specific bound values will be application specific.
-
-Taking care and picking good bounds is a big part of writing reliable Tokio
-applications.
diff --git a/content/async_programming/send_and_sync.md b/content/async_programming/send_and_sync.md
@@ -1,69 +0,0 @@
----
-title: Send and Sync
----
-
-`Send` and `Sync` are both **marker traits** in Rust that govern thread safety,
-but they address different aspects of how data crosses thread boundaries.
-
-## `Send`: Ownership Transfer
-
-A type is `Send` if it is safe to **transfer ownership** of a value to another
-thread. In other words, you can _move_ a `T` into a new thread. Most primitive
-types (`i32`, `bool`), `String`, `Vec<T>`, and `Arc<T>` are `Send`. Types that
-are **not** `Send` include:[^1][^6]
-
-- `Rc<T>` — its reference count is non-atomic, so moving it to another thread
- risks a data race on the count[^3]
-- Raw pointers (`*const T`, `*mut T`) — no synchronization guarantees when
- dereferenced across threads[^6]
-
-## `Sync`: Shared Reference Access
-
-A type is `Sync` if it is safe for **multiple threads to hold shared
-references** (`&T`) to it simultaneously. Formally, `T` is `Sync` if and only if
-`&T` is `Send`. Types that are **not** `Sync` include:[^5][^1]
-
-- `RefCell<T>` — its borrow-checking is not thread-safe (non-atomic interior
- mutability)[^6]
-- `Cell<T>` — same reason; mutable access is not synchronized[^8]
-- Raw pointers[^6]
-
-## How They Relate
-
-The two traits are closely intertwined — `Sync` is almost _defined in terms of_
-`Send`. A useful mental model:[^8]
-
-| Trait | Asks… | Example: YES | Example: NO |
-| :----- | :----------------------------------------------- | :---------------- | :---------------------- |
-| `Send` | Can ownership move to another thread? | `Arc<Mutex<T>>` | `Rc<T>` |
-| `Sync` | Can a shared `&T` be used from multiple threads? | `Mutex<T>`, `i32` | `RefCell<T>`, `Cell<T>` |
-
-A type can be `Send` but not `Sync` (e.g., `Mutex<T>` where ownership transfers
-are fine but concurrent `&T` access needs the lock), or `Sync` but not `Send`
-(e.g., a `MutexGuard`, which can be shared but must be released on the
-originating thread). Both traits are **automatically derived** by the compiler —
-you only need to intervene when working with `unsafe` code or types that should
-explicitly opt out.[^3][^8]
-<span style="display:none">[^10][^2][^4][^7][^9]</span>
-
-<div align="center">⁂</div>
-
-[^1]: https://doc.rust-lang.org/nomicon/send-and-sync.html
-
-[^2]: https://www.reddit.com/r/rust/comments/ctdkyr/understanding_sendsync/
-
-[^3]: https://blog.masteringbackend.com/rust-send-and-sync-in-simple-terms
-
-[^4]: https://stackoverflow.com/questions/59428096/understanding-the-send-trait
-
-[^5]: https://www.reddit.com/r/rust/comments/1csrbhf/understanding_send_trait_in_rust/
-
-[^6]: https://leapcell.io/blog/understanding-send-and-sync-in-rust-async-handlers
-
-[^7]: https://masteringbackend.com/posts/rust-send-and-sync-in-simple-terms
-
-[^8]: https://www.youtube.com/watch?v=yOezcP-XaIw
-
-[^9]: https://web.mit.edu/rust-lang_v1.25/arch/amd64_ubuntu1404/share/doc/rust/html/nomicon/send-and-sync.html
-
-[^10]: https://doc.rust-lang.org/book/ch16-04-extensible-concurrency-sync-and-send.html
diff --git a/content/async_programming/synchronization_primitives.md b/content/async_programming/synchronization_primitives.md
@@ -1,12 +0,0 @@
----
-title: Synchronization Primitives
----
-
-[“Concurrency primitives”](/async_programming/concurrency_primitives.md) is the
-broader term: it means basic building blocks for concurrent programming,
-including things like threads/tasks, queues, futures, and also synchronization
-mechanisms.
-
-“Synchronization primitives” is a subset that specifically deals with ordering
-and coordinated access to shared state (e.g. mutexes, rwlocks, semaphores,
-barriers, condition variables, atomics.
diff --git a/content/commands/docker.md b/content/commands/docker.md
@@ -1,6 +0,0 @@
----
-title: Docker
----
-
-Pretty display docker ps:
-`docker ps --format "table {{.Names}}\t{{.Status}}\t{{.Ports}}\t{{.Networks}}"`
diff --git a/content/commands/index.md b/content/commands/index.md
@@ -1,3 +0,0 @@
----
-title: Commands
----
diff --git a/content/commands/rust.md b/content/commands/rust.md
@@ -1,7 +0,0 @@
----
-title: Rust
----
-
-error: `error[E0658]: use of unstable library feature <library-name>`
-
-command: `rustup override set nightly`
diff --git a/content/commands/yay.md b/content/commands/yay.md
@@ -1,48 +0,0 @@
----
-title: yay
----
-
-# `yay -Syu` nats-server update issue
-
-When run into permission denied issue updating `nats-server`, its due to how
-`go` works:
-
-### Quick Solution
-
-The fastest fix is to manually remove the problematic cache directory with
-elevated permissions:
-
-```bash
-sudo rm -rf ~/.cache/yay/nats-server
-```
-
-Then retry building the package:
-
-```bash
-yay -S nats-server
-```
-
-### Permanent Solution
-
-To prevent this from happening in future builds, modify how the package builds
-Go modules. If you maintain or can edit the `PKGBUILD`, add the `-modcacherw`
-flag:
-
-```bash
-export GOFLAGS="-buildmode=pie -trimpath -mod=readonly -modcacherw"
-```
-
-This flag ensures Go module cache files are created with read-write permissions,
-allowing yay to clean them up properly after builds.
-
-
-
-### Alternative Approach
-
-If you don't need to preserve the cache, you can clean all yay cache:
-
-```bash
-yay -Sc --aur
-```
-
-This removes all AUR build directories and untracked files.
diff --git a/content/compiler/compiler.md b/content/compiler/compiler.md
@@ -1,35 +0,0 @@
----
-title: Compiler
----
-
-Programs are files that contain a bunch of text written by you, the programmer.
-That text is parsed by a special program called a compiler, which transforms it
-into a `abstract syntax tree` (`AST`), a data structure that ignores things like
-whitespaces, comments, and where you stand on the tabs versus spaces debate.
-
-The compiler then converts that AST to a lower-level representation called
-`bytecode`. You can feed that bytecode into another program called a `runtime`
-to evaluate it and get a result. So when you run a program, what you're really
-doing is telling the runtime to evaluate the bytecode generated by the compiler
-from the AST parsed from your source code. The details vary, but for most
-languages this is an accurate high-level view.
-
-Once again, the steps are:
-
-1. Program is parsed into an AST
-2. AST is compiled to bytecode
-3. Bytecode is evaluated by the runtime
-
----
-
-As for TypeScript...
-
-Where TypeScript is special is that instead of compiling straight to bytecode,
-TypeScript compiles to... JavaScript Code! You then run that JavaScript code
-like you normally would - in your browser, or with NodeJS, or by hand with a
-paper and pen (for anyone reading this after the machine uprising has begun).
-
-After TypeScript Compiler generates an AST for your program - but before it
-emits code - it `typechecks` your code.
-
-- from Programming TypeScript by Boris Cherny
diff --git a/content/compiler/compiler_tokens.md b/content/compiler/compiler_tokens.md
@@ -1,121 +0,0 @@
----
-title: Compiler Tokens
----
-
-1. Lexxing
-
-A “lex token” is one classified unit of source code that the lexer has
-recognized and tagged with a type.
-
-“lex” refers to lexical analysis (the first stage of a compiler/interpreter that
-breaks text into tokens).
-
-A “lex token” is simply one of those enum variants produced by that stage,
-representing a smallest unit of meaning such as a keyword, identifier, operator,
-or literal.
-
-```text
-use foo as bar
-```
-
-```rust
-[
- LexTok::Use,
- LexTok::Identifier("foo".into()),
- LexTok::Alias,
- LexTok::Identifier("bar".into()),
-]
-```
-
-2. Parsing
-
-Parsing is the step where a program takes a stream of tokens (like Use, Fn,
-identifiers, (, ), etc.) and checks whether they form a valid structure
-according to the language’s grammar.
-
-After lexing has turned raw text into tokens, parsing:
-
-- Reads the tokens in order and matches them against grammar rules (like “a
- function definition is fn + name + params + body”).
-
-- Builds a tree structure (often called a parse tree or syntax tree) that
- represents how the program is organized: expressions, statements, blocks,
- functions, etc.
-
-- Reports syntax errors if the token sequence doesn’t fit the grammar (missing
- ), extra ;, wrong keyword order, and so on).
-
-In the pipeline you’re looking at:
-
-- Lexing: characters → LexTok sequence
-
-- Parsing: LexTok sequence → syntax tree (AST) that later stages (like type
- checking or code generation) will use
-
-3. Evaluating
-
-“evaluating” is the step where you actually run or compute the meaning of an
-expression or program.
-
-- After lexing: you have tokens.
-
-- After parsing: you have a syntax tree (often an AST).
-
-- Evaluating that AST means:
-
- - Walking the tree,
-
- - Applying operators (+, *, ==, etc.),
-
- - Looking up variable values, calling functions, handling control flow (if,
- while, etc.),
-
- - And producing a result (like 42, "hello", or some side effect like printing
- or updating state).
-
-```
-+------------------+
-| Source code |
-| (text, .ling) |
-+------------------+
- |
- | 1. Lexing (tokenization)
- v
-+------------------+
-| Tokens stream |
-| [LexTok::Use, |
-| LexTok::Fn, |
-| Identifier, ...]|
-+------------------+
- |
- | 2. Parsing (syntax analysis)
- v
-+---------------------------+
-| AST (syntax tree) |
-| e.g. FunctionDef( |
-| name, params, body ) |
-+---------------------------+
- |
- | 3. Evaluating / Executing
- v
-+---------------------------+
-| Program behavior |
-| - results / return vals |
-| - printed output |
-| - changed state, etc. |
-+---------------------------+
-```
-
-### Question: What is the difference between an interpreter and a compiler?
-
-- Compiler: Translates the whole program into machine code (or bytecode) before
- you run it, producing a separate executable or binary file.
-
-- Interpreter: Reads your source code and executes it directly, usually line by
- line or statement by statement, without producing a standalone executable.
-
-- Compiled programs usually run faster because all translation work is done
- ahead of time and the result is optimized machine code.
-
-- Interpreted programs usually run slower because translation happens as you
- execute the code.
diff --git a/content/compiler/dynamic_dispatch.md b/content/compiler/dynamic_dispatch.md
@@ -1,118 +0,0 @@
----
-title: Dynamic Dispatch
----
-
-**Dynamic dispatch** is when you have multiple types that something might be,
-but you don't know which one until runtime. So you **dynamically** figure out
-which one of the methods to call on the type. Usually doing this requires the
-use of **dyn** trait.
-
-Dynamic dispatch is when the concrete type implementing a trait is resolved at
-**runtime** rather than compile time. In Rust, this is done via trait objects
-(`dyn Trait`), and the compiler implements it using a
-[**vtable**](/compiler/vtable.md) — a table of function pointers generated for
-each concrete type.
-
-## [Actors](/async_programming/actors.md) are dynamic dispatch
-
-When you use actors, you don't need dynamic dispatch, because actors provide
-dynamic dispatch on theri own.
-
-Storing something in an actor is an alternative to `Box<dyn Trait>`.
-
-```rust
-struct MyActor<T: AsyncRead + AsyncWrite> {
- receiver: mpsc::Receiver<ActorMessage>,
- connection: T,
-}
-```
-
-Here, the generic T doesn't leak to the sender.
-
-```rust
-#[derive(Clone)]
-pub struct MyActorHandle {
- sender: mpsc::Sender<ActorMessage>,
-}
-```
-
-Here, T could be TcpStream or UnixStream depending on the connection. Remote
-connection uses TcpStream, whereas local connection uses UnixStream, for
-example.
-
-## Static vs Dynamic Dispatch
-
-In **static dispatch**, the compiler monomorphizes generic functions — it
-generates a separate copy of the function for each concrete type used. Calls are
-resolved at compile time and can be inlined.
-
-```rust
-fn area<T: Shape>(s: &T) -> f64 {
- s.area() // resolved at compile time
-}
-```
-
-In **dynamic dispatch**, you use a trait object. The compiler doesn't know the
-concrete type at compile time, so it emits a vtable lookup at each call site.
-
-```rust
-fn area(s: &dyn Shape) -> f64 {
- s.area() // resolved at runtime via vtable
-}
-```
-
-## How the Compiler Implements It
-
-A `dyn Trait` value is a **fat pointer** — two machine words:
-
-- A **data pointer** to the value itself
-- A **vtable pointer** to a static table of function pointers for the concrete
- type
-
-The vtable is emitted by the compiler for each `(ConcreteType, Trait)` pair.
-When you call a method on a `dyn Trait`, Rust loads the function pointer from
-the vtable and calls it indirectly. This means no inlining and a small overhead,
-but it enables **heterogeneous collections** and **type erasure**.
-
-```
-Box<dyn Shape>
-├── data ptr ──► [ Circle { radius: 3.0 } ]
-└── vtable ptr ─► [ drop, size, align, area, ... ]
-```
-
-## Object Safety
-
-Not every trait can be used as `dyn Trait`. A trait must be **object-safe** for
-this. The key rules are:
-
-- Methods must not return `Self`
-- Methods must not have generic type parameters
-- The trait must not require `Sized`
-
-The compiler enforces this — if a trait is not object-safe, using `dyn Trait` is
-a compile error.
-
-## When to Use Dynamic Dispatch
-
-Prefer dynamic dispatch when:
-
-- You need a heterogeneous collection (e.g. `Vec<Box<dyn Plugin>>`)
-- You want to hide a concrete type behind an abstraction boundary (e.g. plugin
- systems, renderers, handlers)
-- Binary size matters more than the last bit of performance (monomorphization
- bloat is real)
-
-Prefer static dispatch when call-site performance and inlining are critical, or
-when the set of concrete types is small and known ahead of time.
-
-## Relationship to Ownership
-
-Dynamic dispatch is orthogonal to both concurrency and memory safety. You can
-wrap a trait object in any ownership primitive:
-
-- `Box<dyn Trait>` — heap-allocated, single owner
-- `Rc<dyn Trait>` — reference-counted, single-threaded
-- `Arc<dyn Trait + Send + Sync>` — reference-counted, multi-threaded
-
-The ownership wrapper controls lifetime and thread safety; the vtable only
-controls how methods are dispatched.
diff --git a/content/compiler/dynamic_libraries.md b/content/compiler/dynamic_libraries.md
@@ -1,270 +0,0 @@
----
-title: Dynamic Libraries
----
-
-A **dynamic library** is a compiled binary that is loaded into a program's
-address space at either load time or runtime, rather than being copied into the
-final executable by the linker. On Linux these files typically have a `.so`
-extension (shared object), on Windows `.dll` (dynamic-link library), and on
-macOS `.dylib` (dynamic library).
-
-Dynamic libraries enable **code sharing** between multiple processes: the
-operating system maps a single physical copy of the library into the virtual
-address space of every process that needs it. This reduces total memory
-footprint and allows a library to be updated independently of the executables
-that depend on it.
-
-## Dynamic vs. Static Libraries
-
-| Feature | Static Library | Dynamic Library |
-| :------ | :------------- | :---------------- |
-| Linking phase | Compile / link time | Load time or runtime |
-| Binary size | Larger executable (library copied in) | Smaller executable (reference only) |
-| Memory sharing | Each process gets its own copy | OS shares one physical copy |
-| Updates | Recompile executable to update library | Replace library file, restart process |
-| Portability | Self-contained executable | Requires compatible library present at runtime |
-
-A static library (`.a` on Unix, `.lib` on Windows) is essentially an archive of
-object files. The linker extracts the needed object files and copies them into
-the final executable. Once linked, the static library is no longer needed to run
-the program.
-
-A dynamic library remains a separate file on disk. The executable contains a
-**reference** to the library — typically a recorded name and a symbol table of
-needed functions — and the operating system's dynamic loader resolves that
-reference when the process starts (load-time linking) or when the program
-explicitly requests it (runtime linking).
-
-## How the OS Loads Dynamic Libraries
-
-When a program starts, the operating system's dynamic loader (e.g., `ld.so` on
-Linux, `dyld` on macOS, or the Windows loader) inspects the executable's
-**interpreter** and **dynamic section** to determine which shared libraries are
-required. It performs the following steps:
-
-1. **Dependency resolution** — read the list of needed libraries from the
- executable headers.
-2. **Library search** — locate each library on the search path (`LD_LIBRARY_PATH`,
- system cache `/etc/ld.so.cache`, `rpath`, `runpath`, or default system
- directories).
-3. **Loading and mapping** — `mmap` the library into the process's address space.
-4. **Symbol resolution** — walk the relocation tables and patch addresses so
- that function calls in the executable point to the correct offsets in the
- loaded library.
-5. **Initialization** — run constructor functions (e.g., `__attribute__((constructor))`
- in C) registered in the library.
-
-This process is known as **dynamic linking**. If a required library cannot be
-found, the loader aborts and the program fails to start.
-
-## Runtime Loading with `dlopen` and Friends
-
-Programs can also load libraries explicitly after they have already started.
-This is **runtime dynamic linking** and is the mechanism behind plugin systems,
-extensible applications, and language interpreters that load native extensions.
-
-On POSIX systems the C standard library provides four key functions:
-
-- `dlopen(path, flags)` — load a shared object into the current address space
-- `dlsym(handle, symbol)` — retrieve the address of a named symbol (function or
- variable)
-- `dlclose(handle)` — decrement the reference count and possibly unload the library
-- `dlerror()` — return a human-readable string describing the last error
-
-On Windows the analogous APIs are `LoadLibraryA`, `GetProcAddress`, and
-`FreeLibrary`.
-
-### Example: A Minimal Dynamic C Library
-
-Imagine a small C library that computes a checksum. Save this as `checksum.c`:
-
-```c
-#include <stdint.h>
-
-uint32_t checksum(const uint8_t *data, size_t len) {
- uint32_t sum = 0;
- for (size_t i = 0; i < len; i++) {
- sum = (sum << 1) | (sum >> 31); // rotate left
- sum += data[i];
- }
- return sum;
-}
-```
-
-Compile it into a shared object on Linux:
-
-```bash
-gcc -shared -fPIC -o libchecksum.so checksum.c
-```
-
-The `-fPIC` flag tells the compiler to emit **position-independent code** —
-machine code that can execute correctly regardless of where in memory it is
-mapped. This is mandatory for shared libraries because the OS may load them at
-different base addresses in different processes (or in the same process across
-restarts) for security reasons such as ASLR (Address Space Layout Randomization).
-
-## Loading a Dynamic C Library from Rust
-
-Rust can interact with dynamic C libraries through two mechanisms:
-
-1. **Compile-time dynamic linking** — declare `#[link(name = "checksum")]` and
- let the Rust linker record a dependency on `libchecksum.so`. The OS loader
- resolves it automatically when the program starts.
-2. **Runtime dynamic loading** — use a crate such as `libloading` to `dlopen`
- the library manually and look up symbols on demand.
-
-Runtime loading is more flexible because the program can decide at execution time
-whether to load a library, handle failures gracefully, and even swap
-implementations without restarting.
-
-### Using `libloading`
-
-Add `libloading` to your `Cargo.toml`:
-
-```toml
-[dependencies]
-libloading = "0.8"
-```
-
-Then load the library and call its function:
-
-```rust
-use libloading::{Library, Symbol};
-use std::ffi::c_void;
-
-type ChecksumFn = unsafe extern "C" fn(*const u8, usize) -> u32;
-
-fn main() -> Result<(), Box<dyn std::error::Error>> {
- unsafe {
- let lib = Library::new("./libchecksum.so")?;
-
- let checksum: Symbol<ChecksumFn> = lib.get(b"checksum\0")?;
-
- let data = b"hello dynamic world";
- let result = checksum(data.as_ptr(), data.len());
-
- println!("Checksum: {}", result);
- }
-
- Ok(())
-}
-```
-
-Key observations about this code:
-
-- The `unsafe` block is required because the compiler cannot verify the
- correctness of a C library's ABI, pointer usage, or thread safety.
-- The symbol name `b"checksum\0"` is a null-terminated byte string, matching the
- C ABI expectation.
-- `Library::new` calls `dlopen` (or `LoadLibrary` on Windows) under the hood.
-- `Symbol` is essentially a smart pointer to a function loaded from a
- [**vtable**](/compiler/vtable.md)-like structure inside the dynamic loader.
-
-### Safety and ABI Mismatches
-
-When calling into a dynamic C library, Rust's usual memory-safety guarantees do
-not apply across the FFI boundary. The C library operates on raw pointers and
-expects a specific **application binary interface (ABI)**. If the Rust side
-misdeclares a function signature — for example, using `u64` instead of `usize`,
-or omitting `extern "C"` — the behavior is undefined and may result in a
-[segfault](/memory_safety/segfault.md) or silent data corruption.
-
-Best practices for dynamic C library interop:
-
-- Define a thin, audited FFI module that mirrors the C headers exactly.
-- Use `std::os::raw` or the `libc` crate for C types (`c_int`, `c_char`, etc.).
-- Keep `unsafe` blocks as small as possible; validate inputs before crossing the
- boundary.
-- Never pass Rust references (`&T`) directly to C code expecting mutable access
- unless you have proven alias safety manually.
-
-## Relocation and the Global Offset Table
-
-Because a shared library can be loaded at any address, it cannot contain
-absolute addresses for its own functions or global data. Instead, the compiler
-emits **position-independent code** that refers to a **Global Offset Table
-(GOT)** and a **Procedure Linkage Table (PLT)**.
-
-- **GOT** — an array of pointers to global data. The code reads data indirectly
- through the GOT so that only the table entries need to be patched at load time,
- not every instruction that references the data.
-- **PLT** — a trampoline for external function calls. The first time a function
- is called through the PLT, the dynamic linker resolves the real address and
- patches the GOT entry; subsequent calls jump directly to the resolved target.
-
-This lazy resolution (often called **lazy binding**) improves startup time for
-large programs with many shared libraries because not every symbol needs to be
-resolved immediately.
-
-## Rust as a Dynamic Library Producer
-
-Rust can also produce dynamic libraries for consumption by other languages. Two
-crate types are relevant:
-
-- **`cdylib`** — produces a C-compatible dynamic library (`.so`, `.dll`, `.dylib`).
- Use this when a C program (or Python via ctypes, or another language) needs to
- load your Rust code dynamically. The Rust compiler strips Rust-specific metadata
- and exports only functions marked `#[no_mangle]` and `pub extern "C"`.
-- **`dylib`** — produces a Rust-native dynamic library. This is primarily used
- for rustc plugins and is rarely appropriate for general FFI.
-
-Example `Cargo.toml` for a C-callable Rust library:
-
-```toml
-[package]
-name = "rust_checksum"
-version = "0.1.0"
-edition = "2021"
-
-[lib]
-crate-type = ["cdylib"]
-```
-
-And the corresponding Rust source:
-
-```rust
-#[no_mangle]
-pub extern "C" fn rust_checksum(data: *const u8, len: usize) -> u32 {
- if data.is_null() {
- return 0;
- }
- let slice = unsafe { std::slice::from_raw_parts(data, len) };
- slice.iter().fold(0u32, |acc, &b| acc.wrapping_mul(31).wrapping_add(b as u32))
-}
-```
-
-Other languages can now `dlopen` the resulting `.so` and call `rust_checksum`
-just as if it were a C function.
-
-## When to Use Dynamic Libraries
-
-Prefer dynamic libraries when:
-
-- Multiple executables share the same code and you want to reduce disk and
- memory usage.
-- You need a plugin architecture where third-party code is loaded at runtime.
-- You want to update a library (e.g., a security patch) without rebuilding every
- dependent executable.
-- You are shipping a large framework and want consumers to link against a stable
- ABI without caring about your internal implementation details.
-
-Prefer static linking when:
-
-- You need a self-contained binary that runs without external dependencies
- (e.g., containers, embedded systems, or CLI tools distributed to unknown
- environments).
-- Maximum startup performance is critical and you want to avoid symbol
- resolution overhead.
-- You want whole-program optimization (LTO) to inline across library boundaries.
-
-## Summary
-
-Dynamic libraries are separate binaries loaded by the OS loader either at
-process startup or on demand via `dlopen` / `LoadLibrary`. They reduce memory
-usage through sharing and enable runtime extensibility, but they introduce
-complexities around symbol resolution, ABI stability, and distribution.
-
-When loading a dynamic C library from Rust, you cross an **FFI boundary** where
-the compiler's usual safety checks no longer apply. Tools like `libloading`
-make runtime loading ergonomic, but correctness depends on matching the C ABI
-exactly and carefully auditing every `unsafe` call site.
diff --git a/content/compiler/index.md b/content/compiler/index.md
@@ -1,38 +0,0 @@
----
-title: Compiler
----
-
-# Compiler Internals
-
-This folder contains documentation about Rust compiler internals and low-level
-programming concepts.
-
-## Topics
-
-### The Compiler
-
-- [The Compiler](compiler.md) - Overview of how a compiler works
-
-### Token Processing
-
-- [Compiler Tokens](compiler_tokens.md) - Understanding lexxing and parsing
- stages
-
-### Low-Level Mechanics
-
-- [Lifetime.md](lifetime.md) - Variable lifetimes and borrowing rules
-- [Non-Lexical Lifetimes](non_lexical_lifetimes.md) - NLL and borrow checker
- improvements
-- [VTable.md](vtable.md) - Virtual function tables and dynamic dispatch
-- [Dynamic Libraries](dynamic_libraries.md) - Shared objects, runtime loading,
- and FFI with C libraries
-
-### Macros & Procedural Macros
-
-- [define_dal_transactions! Macro](define_dal_transactions_macro.md) - Trait
- generation for database access layer operations
-
-### Advanced Topics
-
-- [Dynamic Dispatch](dynamic_dispatch.md) - Runtime polymorphism and trait
- objects
diff --git a/content/compiler/lifetime.md b/content/compiler/lifetime.md
@@ -1,64 +0,0 @@
----
-title: Lifetime
----
-
-# static
-
-### Question: Whats the difference between 'static reference and 'static trait bound?
-
-The two uses of 'static in Rust look similar but mean very different things:
-
-- ``&'static T` — A Static Reference:
-
-A &'static T is a reference that lives for the entire duration of the program.
-The data it points to must be stored somewhere that never gets freed — typically
-in the binary itself (e.g., string literals) or in a static variable.
-[rust-lang](https://internals.rust-lang.org/t/idea-aliasing-the-static-lifetime-for-lifetime-parameters-trait-bounds-e-g-auto/19117)
-
-```rust
-let s: &'static str = "hello world"; // baked into the binary
-```
-
-The key constraint here is on the reference itself: the pointed-to data must
-outlive everything.
-[rust-lang](https://doc.rust-lang.org/rust-by-example/scope/lifetime/static_lifetime.html)
-
-- `T: 'static` — A Static Trait Bound:
-
-A T: 'static bound does not mean T is a reference at all. It means the type T
-contains no non-static references — i.e., T holds no borrowed data that could
-expire.
-
-```rust
-fn generic<T: 'static>(x: T) { ... }
-```
-
-Crucially, any fully owned type (like String, Vec<i32>, u32) automatically
-satisfies T: 'static, because it has no internal borrowed references that could
-go stale. A &'static str also satisfies it, but a &'a str (with a non-static
-lifetime) does not.
-[rust-lang](https://doc.rust-lang.org/rust-by-example/scope/lifetime/static_lifetime.html)
-
-> It is a common misconception that 'static always means "lives forever", but
-> this is not the case. Just because a value is 'static does not mean that you
-> have a memory leak. You can read more in
-> [Common Rust Lifetime Misconceptions](https://github.com/pretzelhammer/rust-blog/blob/master/posts/common-rust-lifetime-misconceptions.md#2-if-t-static-then-t-must-be-valid-for-the-entire-program).
-
-### Question: When `T: 'static` Is Needed?
-
-The T: 'static bound is commonly required when values must escape their creation
-scope, such as when spawning threads with thread::spawn. Since a spawned thread
-can outlive the caller, Rust requires that anything sent into it contains no
-short-lived references — which is exactly what T: 'static guarantees.
-[Rust Traits are not interfaces](https://www.jamessturtevant.com/posts/rust-traits-are-not-interfaces-and-a-little-on-lifetimes/),
-[Learning Rust: static trait bounds](https://codeandbitters.com/static-trait-bound/)
-
-```rust
-// thread::spawn requires T: Send + 'static
-fn run_in_background<T: Send + 'static>(val: T) {
- std::thread::spawn(move || { /* use val */ });
-}
-```
-
-A useful mental shortcut: &'static T is about where data lives; T: 'static is
-about whether a type is safe to keep indefinitely.
diff --git a/content/compiler/non_lexical_lifetimes.md b/content/compiler/non_lexical_lifetimes.md
@@ -1,92 +0,0 @@
----
-title: Non-lexical lifetimes (NLL)
----
-
-Non-Lexical Lifetimes (NLL) is a significant improvement to Rust's borrow
-checker, introduced in Rust 2018 and fully stabilised by default in Rust 1.63.
-It changed how the compiler reasons about _how long a reference lives_.[^8]
-
-## The Problem: Lexical Lifetimes (Before NLL)
-
-Before NLL, a reference's lifetime lasted until the **end of its enclosing
-scope** (the closing `}`), regardless of whether it was actually used after a
-certain point. This caused the borrow checker to reject perfectly safe code:[^1]
-
-```rust
-let mut data = vec![1, 2, 3];
-let first = &data[^0]; // immutable borrow starts here
-println!("{}", first); // last actual use of `first`
-
-// OLD borrow checker: `first` still "alive" until `}`, so this fails ❌
-// Even though `first` is never used again!
-data.push(4);
-```
-
-## The Fix: NLL
-
-With NLL, the compiler tracks the **last point where a reference is actually
-used**, and ends its lifetime there — not at the closing brace. This is based on
-analysing the **control-flow graph** of your code rather than just the syntactic
-block structure.[^6][^1]
-
-```rust
-let mut data = vec![1, 2, 3];
-let first = &data[^0]; // immutable borrow starts
-println!("{}", first); // ✅ last use — borrow ENDS HERE (NLL)
-
-data.push(4); // ✅ now safe to mutate
-```
-
-## Lexical vs. Non-Lexical Lifetimes
-
-| | Lexical Lifetimes (Old) | Non-Lexical Lifetimes (NLL) |
-| :----------------- | :------------------------- | :------------------------------- |
-| Lifetime ends at | Closing `}` of scope | Last actual use of the reference |
-| Based on | Syntax tree (scope blocks) | Control-flow graph |
-| Rejects safe code? | Yes, in many common cases | Much less often |
-| Introduced | Original Rust | Rust 2018, stable in Rust 1.63 |
-
-## NLL and Control Flow
-
-NLL is smart enough to handle branching logic too. A borrow is only considered
-"live" at a point if its value **could be used in the future** from that point.
-This means in conditional branches where a reference is only used in one path,
-it won't block mutations on the other path.[^7]
-
-```rust
-let mut s = String::from("hello");
-
-let r = &s;
-if some_condition {
- println!("{r}"); // r used here in one branch
-}
-// NLL knows r MAY still be live here, so mutation below could still fail
-// depending on the control flow — the compiler reasons through each path
-```
-
-In short, NLL makes Rust's borrow checker significantly more ergonomic without
-compromising any of its safety guarantees — it simply became smarter about
-_when_ a borrow truly ends.[^1]
-<span style="display:none">[^10][^2][^3][^4][^5][^9]</span>
-
-<div align="center">⁂</div>
-
-[^1]: https://oneuptime.com/blog/post/2026-01-25-non-lexical-lifetimes-rust/view
-
-[^2]: https://users.rust-lang.org/t/about-non-lexical-lifetimes/111614
-
-[^3]: https://www.reddit.com/r/rust/comments/wgxr9q/nonlexical_lifetimes_nll_fully_stable_rust_blog/
-
-[^4]: https://stackoverflow.com/questions/50251487/what-are-non-lexical-lifetimes
-
-[^5]: https://www.youtube.com/watch?v=XD-nc28-8Fw
-
-[^6]: https://rust-lang.github.io/rfcs/2094-nll.html
-
-[^7]: https://www.reddit.com/r/rust/comments/6brtsu/eli5_nonlexical_lifetimes/
-
-[^8]: https://blog.rust-lang.org/2022/08/05/nll-by-default.html
-
-[^9]: https://smallcultfollowing.com/babysteps/blog/2017/02/21/non-lexical-lifetimes-using-liveness-and-location/
-
-[^10]: https://smallcultfollowing.com/babysteps/blog/2016/04/27/non-lexical-lifetimes-introduction/
diff --git a/content/compiler/vtable.md b/content/compiler/vtable.md
@@ -1,8 +0,0 @@
----
-title: Vtable
----
-
-A **vtable** (virtual function table) in Rust is a data structure used to enable
-dynamic dispatch with trait objects (`dyn Trait`). It is essentially a lookup
-table of function pointers that allows the correct method implementation to be
-called at runtime, even when the concrete type isn't known at compile time.
diff --git a/content/engineering_practice/index.md b/content/engineering_practice/index.md
@@ -1,3 +0,0 @@
----
-title: Engineering Practice
----
diff --git a/content/engineering_practice/software-entropy.md b/content/engineering_practice/software-entropy.md
@@ -1,219 +0,0 @@
----
-title: Software Entropy
----
-
-# Software Entropy — Why Codebases Rot and How Constant Tidying Keeps Them Alive
-
-There is a law that every engineer eventually learns, not from a textbook but
-from the gut-sinking moment they open a file they haven't touched in six months.
-The code is worse than they left it. Not because anyone vandalized it
-deliberately, but because software, left unattended, tends toward disorder. This
-is software entropy — and it is not a metaphor. It is the dominant force shaping
-the long-term economics of any engineering project.
-
-## What Is Software Entropy?
-
-Entropy, borrowed from thermodynamics, describes the natural tendency of closed
-systems toward disorder. In software, every new feature, every rushed hotfix,
-every "I'll clean this up later" decision adds a small quantum of disorder to
-the codebase. Individually, these moments are invisible. Collectively, they
-compound into a system that nobody fully understands anymore.
-
-Kent Beck, in _Tidy First?_, frames the economic reality with stark clarity:
-**the cost of software is approximately equal to the cost of changing it**. He
-calls this _Constantine's Equivalence_ — named after Larry Constantine, who,
-with Ed Yourdon, wrote _Structured Design_ (1975), the foundational text on
-coupling and cohesion. Beck's formulation is:
-
-> cost(software) ≈ cost(change) ≈ cost(big changes) ≈ coupling
-
-Coupling — the degree to which changing one element forces changes in others —
-is the engine of entropy. When code is tangled, a superficially simple change
-detonates a cascade: change this, and you have to change that, and that, and
-that. Over time the cost distribution follows a power law: the handful of most
-expensive changes, driven by accumulated coupling, dwarf everything else
-combined.
-
-## The Productivity Death Spiral
-
-Software entropy does not kill a project all at once. It kills it slowly, via
-cognitive creep.
-
-Early in a project, a single engineer can hold the entire system in their head.
-They move fast. Then the system grows. New abstractions are introduced
-inconsistently. Dead code accumulates. Nested conditionals deepen. Functions
-sprawl across hundreds of lines. Variables are declared far from where they are
-used. Names drift from their meaning.
-
-Each of these issues adds a small tax to every future change. Reading time
-increases. Onboarding new engineers takes longer. Pull requests grow larger
-because each change requires touching more files. Eventually, the team is
-spending more energy on archaeology — _what does this code even do?_ — than on
-delivery. Productivity trends toward zero. Not a sudden collapse, but a slow,
-grinding asymptote.
-
-This is the scenario Beck is writing against in _Tidy First?_: a codebase that
-has become "messy" in the technical sense — not aesthetically displeasing, but
-genuinely resistant to change.
-
-## Structure Versus Behavior: The Core Distinction
-
-One of the most important conceptual contributions in _Tidy First?_ is the sharp
-separation between **structure changes** and **behavior changes**.
-
-- A **behavior change** alters what the system does: a new feature, a bug fix, a
- performance optimization.
-- A **structure change** alters how the system is organized without changing
- what it computes.
-
-Mixing these two kinds of changes in a single commit is a primary driver of
-entropy. When a reviewer sees a pull request that simultaneously refactors a
-module _and_ adds a feature, they cannot easily reason about either change. The
-refactoring masks potential bugs in the feature; the feature masks the intent of
-the refactoring. Beck's prescription is explicit: **separate tidying commits
-from behavior change commits**. Keep them in their own pull requests, as small
-as possible.
-
-This separation also unlocks a key economic property: structure changes are
-almost always **reversible**. You extract a helper function and don't like it?
-Inline it back. It's as if it never existed. Behavior changes can be
-irreversible — you can't un-send 100,000 tax notices with the wrong figure on
-them. Treating these two categories of change with the same process and the same
-review overhead is, as Beck puts it, "a waste."
-
-## Tidyings: Janitorial Work as Engineering Practice
-
-Beck introduces the concept of a **tidying** — a "cute, fuzzy little refactoring
-that nobody could possibly hate on." Tidyings are small, safe, structure-only
-changes that reduce local disorder. They include:
-
-- **Guard Clauses** — Replacing deeply nested conditionals with early returns,
- making preconditions explicit and reducing the mental indentation a reader
- must maintain.
-- **Dead Code Removal** — Deleting code that is never executed. Every line of
- code is a line someone has to read. Dead code is pure cognitive tax with zero
- return.
-- **Explaining Variables and Constants** — Extracting a complex sub-expression
- into a named variable, or replacing a magic number with a symbolic constant.
- This puts hard-won understanding _back into the code_, so the next reader
- doesn't have to rediscover it.
-- **Normalize Symmetries** — When the same pattern is implemented multiple
- different ways across a codebase, pick one, and convert the others. Readers
- expect that difference means difference; incidental variation destroys that
- expectation.
-- **Cohesion Order** — Moving coupled elements next to each other. If you have
- to change three things every time you touch a feature, move those three things
- adjacent before you touch them.
-- **Extract Helper** — Pulling a block of code with a clear, limited purpose
- into a named routine. The name is the tidying; it replaces an implicit "what"
- with an explicit one.
-- **Chunk Statements** — Inserting a blank line between logically distinct
- blocks of code. This may be the simplest tidying in existence. It is also
- surprisingly powerful: it visually signals structure that was previously
- invisible.
-
-None of these are dramatic interventions. Each can be done in minutes. Each
-makes the next change slightly easier. And because software design enables more
-software design, these small improvements compound — what Beck calls the
-**avalanche effect**: "you tidy this bit, and that bit, and then the tidyings
-start to compound... suddenly, without you ever straining, a giant
-simplification becomes the matter of a stroke or two of your pen."
-
-## Cognitive Creep and the Cost of Coupling
-
-Coupling is the mechanism by which entropy becomes expensive. Ed Yourdon and
-Larry Constantine, studying programs in the 1970s, observed that expensive
-programs shared a common property: changing one element required changing
-others. Cheap programs required localized changes. This observation, formalized
-as **coupling**, remains the most predictive metric of software maintainability
-fifty years later.
-
-Beck extends this with _Constantine's Equivalence_: if the cost of software
-equals the cost of change, and the cost of change is dominated by the cost of
-big cascading changes, and cascading changes are caused by coupling, then:
-
-> **reducing coupling is the primary lever for reducing the long-term cost of a
-> software system**
-
-Coupling is also the mechanism of cognitive creep. To understand a piece of
-code, a reader must also understand everything it depends on. High coupling
-means a large, sprawling context that must be loaded into working memory before
-any change can be safely made. Low coupling means a small, bounded context — the
-cognitive load stays manageable.
-
-Cohesion is coupling's companion. A cohesive module contains elements that
-change together. Coupled elements that live far apart in the codebase (different
-files, different directories, different repositories) force engineers to scatter
-their attention. Moving coupled things together — increasing cohesion — is often
-sufficient to make a change tractable even before the coupling itself is
-resolved.
-
-## The Economics of Tidying: When to Tidy First
-
-Beck is not dogmatic. The title of the book ends in a question mark for a
-reason. Tidying is not always the right first move.
-
-The time value of money pushes toward tidying _after_: earn revenue from
-behavior changes sooner, spend money on structure later. Options theory pushes
-toward tidying _first_: a cleaner structure creates more optionality — a larger
-portfolio of behaviors that can be implemented next, each one cheaper because
-the structure supports it. The correct answer is contingent.
-
-The heuristic Beck offers is straightforward:
-
-> If
-> `cost(tidying) + cost(behavior change after tidying) < cost(behavior change without tidying)`,
-> tidy first. Always.
-
-At the scale of minutes to hours — the scale of individual tidyings — this
-calculation is rarely precise but always directionally useful. The deeper
-practice is developing the _taste_ to make the call quickly and correctly,
-preparing for the larger structural decisions that govern weeks and months of
-development.
-
-## The Rhythm of Housekeeping
-
-The failure mode Beck identifies in teams that have learned about tidying is the
-tidying binge: discovering that you can make your work better by cleaning, and
-then disappearing into a refactoring spiral that delays the features others are
-waiting for. "Coupling conducts one tidying to the next," he writes. "Tidyings
-are the Pringles of software design. When you're tidying first, resist the urge
-to eat the next one."
-
-The sustainable practice is **rhythm**: small, constant, incremental
-housekeeping, woven into daily development. Never a big deal. Never reported,
-tracked, planned, or scheduled as a separate initiative. Just the Scout Rule,
-applied continuously: leave the code slightly better than you found it.
-
-This rhythm is what prevents entropy from compounding. It does not eliminate
-disorder — disorder is the natural state — but it continuously reverses small
-accumulations before they become structural debt. A codebase maintained this way
-never becomes the kind of mess that requires a multi-quarter "refactoring
-initiative," the kind that is always de-prioritized in favor of features and
-therefore never happens.
-
-## The Human Dimension
-
-There is a deeper argument in _Tidy First?_ that goes beyond economics. Beck
-opens with the framing that **software design is an exercise in human
-relationships**: between the programmer and themselves, between teammates,
-between engineers and the business. The state of a codebase reflects and shapes
-those relationships.
-
-A messy codebase is demoralizing. It signals that nobody cares, which makes the
-next person care a little less, which makes it messier still. A codebase that is
-tidied continuously signals the opposite: that the people working here respect
-each other's time and attention. That respect compounds just as surely as the
-entropy does — but in the other direction.
-
-Tidying is, in this sense, professional self-care. "You can't be your best self
-if you're always rushing, if you're always changing code that's painful to
-change." The discipline of small, continuous housekeeping is not bureaucratic
-overhead. It is the engineering practice that keeps the codebase — and the team
-working in it — alive.
-
----
-
-_References: Kent Beck, Tidy First? A Personal Exercise in Empirical Software
-Design (O'Reilly, 2023). Ed Yourdon and Larry Constantine, Structured Design
-(Prentice Hall, 1979)._
diff --git a/content/engineering_practice/spotify_model.md b/content/engineering_practice/spotify_model.md
@@ -1,18 +0,0 @@
----
-title: The Spotify Model
----
-
-## Engineering Squads
-
-[Spotify Model](https://www.youtube.com/watch?v=Yvfz4HGtoPc)
-
-- A squad has autonomy of what to build
-- Squad mission
-- Product strategy
-- Short-term goals
-- Localize decision within the squad
-- Be aligned with other squads regarding overall mission
-
-Be autonomous but don't suboptimize!
-
-Loosely coupled, but tightly aligned squads.
diff --git a/content/food/chinese-tomato-chicken-soup.md b/content/food/chinese-tomato-chicken-soup.md
@@ -1,35 +0,0 @@
----
-title: Chinese Tomato Chicken Soup
----
-
-A classic, light and tangy Chinese tomato chicken soup ready in ~30 minutes.
-
-## Ingredients (serves 4)
-
-- 2 chicken breasts or thighs, thinly sliced
-- 3 medium ripe tomatoes, cut into wedges
-- 4 cups (1 litre) chicken broth
-- 2 cloves garlic, minced
-- 3 slices fresh ginger
-- 2 spring onions, cut into 5cm pieces
-- 1 tsp light soy sauce
-- ½ tsp sesame oil
-- Salt and white pepper to taste
-- 1 tbsp cornstarch + 2 tbsp cold water (optional, to thicken)
-- 2–3 eggs, beaten (optional, for egg ribbons)
-
-## Instructions
-
-1. Toss chicken with a pinch of salt and cornstarch. Marinate 10 mins.
-2. Bring broth to a boil with ginger and spring onions.
-3. Add tomatoes, simmer 10 mins until softened.
-4. Add chicken, cook 5–8 mins until done.
-5. Slowly drizzle in beaten eggs while stirring for silky egg ribbons.
-6. Season with soy sauce, sesame oil, white pepper, and salt. Serve with rice.
-
-## Tips
-
-- Use white pepper for authentic Chinese flavour.
-- Ripe tomatoes create the best natural sweet-sour broth.
-- Add a splash of rice vinegar to brighten if tomatoes aren't tart.
-- Add noodles or bok choy in the last 5 minutes for a heartier meal.
diff --git a/content/food/cinnamon-ginger-tea.md b/content/food/cinnamon-ginger-tea.md
@@ -1,87 +0,0 @@
----
-title: Cinnamon & Ginger Tea
----
-
-Cinnamon and ginger tea is one of those timeless home remedies that has been
-used across cultures for centuries. Beyond its warming, spiced flavour, both
-ingredients carry well-documented anti-inflammatory and antimicrobial properties
-that make this tea a reliable ally when you're fighting a cold.
-
----
-
-## Ingredients
-
-- 2 cups (500ml) water
-- 1 cinnamon stick (or ½ tsp ground cinnamon)
-- 1-inch (2–3cm) piece of fresh ginger, peeled and sliced (or ½ tsp ground
- ginger)
-- 1 tbsp honey _(optional, but recommended — adds antibacterial properties)_
-- A squeeze of fresh lemon _(optional)_
-
----
-
-## Instructions
-
-1. **Bring water to a boil** in a small saucepan.
-2. **Add the ginger slices and cinnamon stick** to the boiling water.
-3. **Reduce heat and simmer** for 10–15 minutes to let the flavours and
- compounds steep fully.
-4. **Strain** the tea into a mug.
-5. **Stir in honey and lemon** if using, once the tea has cooled slightly (very
- hot liquid can degrade honey's beneficial enzymes).
-6. **Drink warm** — ideally 2–3 times a day when you're feeling under the
- weather.
-
-> **Tip:** For a stronger brew, use fresh ginger rather than ground, and simmer
-> for the full 15 minutes.
-
----
-
-## How It Helps With a Cold
-
-### Ginger
-
-Ginger contains **gingerols** and **shogaols** — bioactive compounds with potent
-anti-inflammatory and antioxidant effects. These help:
-
-- Soothe a sore throat by reducing local inflammation
-- Act as a natural decongestant, loosening mucus and easing nasal congestion
-- Calm nausea, which can accompany some viral illnesses
-- Provide a mild antiviral effect, potentially inhibiting viral replication
-
-### Cinnamon
-
-Cinnamon is rich in **cinnamaldehyde**, the compound responsible for its
-distinctive smell and much of its medicinal value. It helps by:
-
-- Offering antibacterial and antifungal properties, supporting the immune system
-- Reducing inflammation in the throat and airways
-- Warming the body from within, which may help ease chills and improve
- circulation
-
-### Honey & Lemon (Bonus)
-
-- **Honey** is a well-known demulcent — it coats the throat, reduces coughing,
- and has antimicrobial properties. A 2021 systematic review found honey
- superior to usual care (including antihistamines and cough suppressants) for
- upper respiratory tract symptoms.
-- **Lemon** provides a dose of Vitamin C and acts as a mild astringent, helping
- to clear mucus.
-
----
-
-## When to Drink It
-
-| Timing | Benefit |
-| -------------------------- | -------------------------------------- |
-| First thing in the morning | Kickstarts immunity and warms the body |
-| Before bed | Soothes a sore throat overnight |
-| After meals | Aids digestion alongside cold symptoms |
-
----
-
-## A Note of Caution
-
-This tea is a **complementary remedy**, not a replacement for medical treatment.
-If symptoms are severe, persist beyond 10 days, or are accompanied by high fever
-or difficulty breathing, consult a healthcare professional.
diff --git a/content/food/coffee_and_cacao_comparison.md b/content/food/coffee_and_cacao_comparison.md
@@ -1,86 +0,0 @@
----
-title: Coffee, Matcha, and Cacao Comparison
----
-
-## Average caffeine content per typical serving
-
-| Beverage | Typical serving size | Caffeine content (≈) | Notes |
-| ------------------------------------ | ---------------------------------- | -------------------- | -------------------------------------------------------------- |
-| **Matcha** | 1 g powder (≈ 1 tsp) ≈ 1 oz brewed | **70 mg** | Uses the whole leaf, so higher than regular tea. |
-| **Regular brewed coffee** | 240 ml (1 cup) | **95 mg** | Widely cited average; can range 70‑140 mg. |
-| **Decaf coffee** | 240 ml | **3–5 mg** | Most decaf retains 2–5 % of the original caffeine. |
-| **Black tea** | 240 ml | **20 – 40 mg** | Depends on steep time and leaf strength. |
-| **Green tea** | 240 ml | **20 – 45 mg** | Similar to black tea but usually a touch lower. |
-| **Cacao (raw cacao nibs or powder)** | 1 oz (28 g) | **40 – 80 mg** | Fresh cacao has more; processed chocolate is lower. |
-| **Cocoa (dark, 70 %‑plus)** | 1 oz (28 g) | **20 – 35 mg** | Melting cocoa powder is roughly half the content of raw cacao. |
-
-### Quick comparison (per serving)
-
-- **Matcha**: ≈ 70 mg – higher than any tea, about the same as a standard cup of
- coffee.
-- **Regular coffee**: ≈ 95 mg – the highest among the listed drinks.
-- **Decaf coffee**: ≈ 3–5 mg – minimal caffeine.
-- **Black tea**: ≈ 20–40 mg – moderate.
-- **Green tea**: ≈ 20–45 mg – similar range, usually a bit lower than black tea.
-- **Cacao nibs**: ≈ 40–80 mg – comparable to matcha if a large portion is used.
-- **Cocoa powder**: ≈ 20–35 mg – less than cacao but still a small caffeine hit.
-
----
-
-## Cacao vs. Cocoa – What’s the difference?
-
-| Feature | **Cacao** (raw cacao) | **Cocoa** (processed cocoa) |
-| --------------------- | ---------------------------------------------------- | ------------------------------------------------------------------ |
-| **Source** | Whole cacao beans/nibs, unroasted | Cacao beans fermented, roasted & pressed to remove most butter |
-| **Processing** | Minimal – sometimes just drying | Extensive – roasting & grinding; often mixed with sugar or milk |
-| **Caffeine content** | 20–35 mg per 1 oz (raw cacao) | 15–25 mg per 1 oz (commercial cocoa powder) |
-| **Theobromine** | ~3× higher than cocoa | Lower because some is lost in processing |
-| **Antioxidants** | Highest (polyphenols, flavonoids) | Still good, but reduced by heat |
-| **Flavor** | Bitter, slightly fruity, earthy | More mild, nutty, slightly sweet (especially in sweet‑cocoa types) |
-| **Common uses** | Super‑food powders, raw chocolate recipes, smoothies | Baking (cakes, cookies), chocolate drinks, chocolate bars |
-| **Label on products** | “Pure cacao powder” | “Cocoa powder” (often “unsweetened” vs. “sweetened”) |
-
-### Bottom line
-
-- Cacao is the raw, nutrient‑dense product with more caffeine and theobromine.
-- Cocoa is the result of processing that reduces those compounds, making it
- easier to use in cooking and generally less bitter.
-
----
-
-## Caffeine, Cerebral Blood Flow, and Cognition
-
-Caffeine is a potent adenosine receptor antagonist that produces several
-neurovascular effects. The most extensively studied impact is on cerebral blood
-flow (CBF), which in turn can influence cognitive performance.
-
-| Effect | Typical caffeine dose | Evidence | Cognitive implication |
-| ------------------------------------------------------------------ | ---------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- |
-| **Acute CBF reduction** | 200–400 mg (≈ 2 cups coffee) | Grigor, R., et al. _Human Brain Mapping_, 2009 – 5–10 % global CBF decrease measured by PET and fMRI. | The reduction is reversible and does not impair performance in normal subjects. |
-| **Regional CBF changes** | Same dose | O’Connell, K.R.M., et al. _Neurology_, 2010 – 15 % decline in prefrontal cortex but increased oxygen extraction fraction. | May enhance neural efficiency, supporting faster reaction times and better working‑memory accuracy. |
-| **Chronic tolerance** | Habitual users (≥ 2 g/day) | Smith, D.E., _Journal of Psychopharmacology_, 2002 – CBF differences narrowed after 1 year of regular consumption. | Cognitive benefits persist even when vascular changes diminish. |
-| **Cognitive enhancement** | 100–300 mg | A meta‑analysis by Chang, Y., et al. _Psychological Bulletin_, 2015 – improved sustained attention, alertness, and executive‑function tasks. | Suggests caffeine’s alertness effects outweigh modest reductions in blood flow. |
-| **Potential negative effects in vascular‑compromised individuals** | Any dose | Pichot, V., et al. _Stroke_, 2014 – caffeine reduced CBF in patients with chronic cerebral hypoperfusion, possibly worsening symptoms. | Caution advised for stroke or Alzheimer's patients. |
-
-### How to interpret the data
-
-- **Short‑term**: A moderate dose (< 400 mg) causes a small drop in overall CBF
- but still keeps the brain well‑oxygenated; neurocognitive tests generally show
- _enhanced_ performance thanks to increased alertness and decreased adenosine
- tone.
-- **Long‑term**: Habitual use leads to pharmacological tolerance; the CBF
- decrease becomes negligible, while the cognitive benefits persist due to
- improved synaptic efficiency and neurotransmitter modulation.
-- **Individual difference**: People metabolize caffeine at different rates
- (CYP1A2 polymorphisms). Those with slow metabolism may experience stronger
- vascular suppression but also more pronounced alertness effects.
-
-### Practical take‑aways
-
-- For healthy adults, a daily dose of 200–400 mg is unlikely to adversely affect
- cognition, and may even help, especially during demanding mental work.
-- Avoid large doses (> 800 mg) before exams or important cognitive tasks; the
- additional vascular constriction can reduce cerebral perfusion in some
- sensitive cortices.
-- Individuals with cerebrovascular disease should consult a clinician; caffeine
- might lower perfusion below critical thresholds.
diff --git a/content/food/dragon-breath-chai.md b/content/food/dragon-breath-chai.md
@@ -1,65 +0,0 @@
----
-title: Dragon Breath Chai
----
-
-Dragon Breath Chai is a bold, fiery reimagining of the traditional Indian Masala Chai. While a standard chai is warming and comforting, the "Dragon Breath" variant is designed to ignite the senses and stimulate the metabolism, blending the aromatic depth of traditional spices with a sharp, spicy kick.
-
----
-
-## Ingredients (serves 2)
-
-- 2 cups (500ml) water
-- 2 tbsp loose leaf black tea (Assam or Ceylon preferred for a strong base)
-- 1 cinnamon stick
-- 4–5 green cardamom pods, lightly crushed
-- 3–4 whole cloves
-- 1-inch (2–3cm) piece of fresh ginger, peeled and sliced
-- 1 small dried bird's eye chili (whole) or ¼ tsp cayenne pepper
-- ½ tsp black pepper, freshly ground
-- 1 cup (250ml) whole milk or a creamy plant-based alternative (e.g., oat milk)
-- 1–2 tbsp honey, maple syrup, or jaggery (to balance the heat)
-
----
-
-## Instructions
-
-1. **Infuse the Aromatics**: In a small saucepan, combine the water, sliced ginger, cinnamon stick, crushed cardamom, cloves, and the dried chili (or cayenne).
-2. **Simmer**: Bring the mixture to a boil, then reduce the heat and simmer for about 10 minutes. This allows the "heat" components and woody spices to fully extract.
-3. **Steep the Tea**: Add the loose leaf black tea to the pot. Simmer for another 3–5 minutes, depending on your preferred strength.
-4. **Add the Creaminess**: Pour in the milk and stir in the freshly ground black pepper.
-5. **Gentle Heat**: Bring the mixture back up to a gentle simmer—do not let it reach a rolling boil once the milk is added to avoid scorching.
-6. **Strain and Serve**: Strain the chai through a fine-mesh sieve into mugs.
-7. **Sweeten**: Stir in your chosen sweetener while the tea is hot.
-
-> **Tip:** If the "breath" is too intense, increase the amount of milk or honey; the fats and sugars help neutralize the capsaicin.
-
----
-
-## The Science of the Heat
-
-The distinctive "fire" of Dragon Breath Chai comes from a synergy of three different types of pungent compounds:
-
-### Capsaicin (Chili)
-Found in the bird's eye chili, **capsaicin** binds to TRPV1 receptors in the mouth, creating a sensation of heat. This not only provides the "kick" but can also temporarily increase metabolic rate and improve blood flow.
-
-### Gingerols & Shogaols (Ginger)
-Fresh ginger provides a different kind of heat—a sharp, zesty warmth known as **gingerol**. When simmered, some of this is converted into **shogaol**, which is even more pungent and provides potent anti-inflammatory benefits.
-
-### Piperine (Black Pepper)
-The **piperine** in black pepper adds a subtle, earthy bite. More importantly, piperine is known to enhance the bioavailability of other compounds, potentially helping the body absorb the beneficial antioxidants from the tea and spices.
-
----
-
-## Recommended Pairings
-
-| Pairing | Why it Works |
-| :--- | :--- |
-| **Dark Chocolate** | The bitterness of the cacao complements the heat and sweetness. |
-| **Buttery Shortbread** | The fat in the butter coats the tongue, tempering the spicy "breath." |
-| **Savory Samosas** | The traditional pairing; the spices in the chai echo the flavors of the snack. |
-
----
-
-## A Note of Caution
-
-Due to the inclusion of chili and high concentrations of ginger, this brew is significantly more potent than standard tea. Those with sensitive stomachs or a low tolerance for spicy foods should start with a pinch of cayenne rather than a whole chili.
diff --git a/content/food/garlic-antibiotic-resistance-and-infections.md b/content/food/garlic-antibiotic-resistance-and-infections.md
@@ -1,152 +0,0 @@
----
-title: Garlic as Natural Antibiotic
----
-
-[See talk by Herbal Medicine Expert Simon Mills](https://www.youtube.com/watch?v=5Jk5XCLAr6w)
-
-[Amazing Spices, Herbs & Drinks That Repair The Body & Fight Disease | Simon Mills](https://www.youtube.com/watch?v=LcB5duDjMvA)
-
-This note describes how **raw, uncooked garlic** (_Allium sativum_) is
-understood within **herbal medicine**, **traditional healing systems**, and
-**holistic health** when people turn to food and plants for support during or
-after **bacterial infection**. It is for reflection on traditions and self-care
-philosophy—not a substitute for urgent or conventional medical care when
-infection is severe, spreading, or not improving.
-
-For millennia, well before the industrial manufacture of modern drugs, healing
-across cultures leaned heavily on **plants**—knowledge refined in kitchens,
-gardens, monastic scriptoria, and lineage holders rather than in pharmacies
-alone. That long continuity is difficult to explain if plants had offered
-**nothing** reliable: empires and villages alike depended on remedies that often
-**worked well enough** to be remembered, repeated, and written down. This is not
-an argument against **conventional medicine**, which has transformed trauma,
-infection in crisis, and many diagnoses once untreatable; it is simply to say
-that **herbal medicine** and holistic care belong to the same deep human effort
-to stay well. In holistic practice, one of the most remarked-on traits of herbs
-is how **fast** they can register in **health as it is actually lived**: warmth,
-circulation, openness of the breath, settling of digestion, mental brightness,
-or a lifted sense of **vitality and resistance** often show up within **minutes
-to hours**, not only after weeks of waiting. For many acute or everyday
-imbalances, whole-plant preparations are experienced as **pretty much
-immediate**—they engage physiology **directly**, without needing industrial
-chemistry to “switch the body on.” That speed is why kitchens and healers
-reached for plants first: the feedback loop is **right now** in nose, mouth,
-belly, and skin. The **herb–body conversation** starts **almost immediately**
-and, in skillful use, is **swift, palpable, and repeated** whenever the plant is
-brought back in—so immediacy is not a rare exception but part of what people
-mean when they say herbs **work**. **Raw garlic** is one of the plants herbal
-traditions still point to for **sharp, activating support** at the early edge of
-sickness, while **serious or worsening infection** still belongs with
-**conventional care**.
-
-## Why raw garlic is central in these traditions
-
-Holistic and herbal lineages usually treat garlic as a **potent, “hot” and
-dispersing** ally: something that **moves stagnation**, **opens circulation**,
-and helps the body **clear** what does not belong. In practical terms, those
-ideas meet the kitchen fact that garlic’s most famous sulfur chemistry appears
-when the clove is **cut or crushed while still living tissue**, so that
-**alliin** meets **alliinase** and **allicin** and related compounds form. Heat
-tends to **quiet or destroy** that enzyme-driven burst, which is why recipes
-framed as “medicinal” so often insist on **raw or minimally heated**
-preparation—not because cooked garlic has no merit as food, but because the
-**sharp, pungent, allicin-forward** profile is what herbal literature usually
-points to for the strongest **cleansing** or **microbe-aware** emphasis.
-
-From a holistic angle, **raw garlic** is therefore not simply a flavor choice;
-it is tied to **full expression of the plant’s character**—the same pungency
-that Western herbalists historically linked to **antiseptic** and **protective**
-qualities in topical and internal folk use.
-
-## How infection is seen through a holistic lens
-
-Herbal and holistic models rarely stop at naming a bacterium alone. They also
-ask about **terrain**: digestion, sleep, stress, elimination, nutrient density,
-and whether vitality feels **drained** or **stubbornly stuck**. Bacterial
-illness may be discussed in terms of **heat** or **toxic buildup**, **dampness**
-(in **Traditional Chinese Medicine** practice theory), or **aggravated fire**
-(in some **Ayurvedic** frameworks)—always as **patterns** for individualized
-work with a qualified practitioner, not labels you assign yourself in place of
-diagnosis.
-
-**Garlic** in those maps is often classed as **warming**, **penetrating**, and
-**downward- and outward-moving**: supporting the body’s urge to **sweat**,
-**expectorate**, or **eliminate**, and to **defend boundaries** when “invasion”
-is part of the verbal picture healers use. None of this replaces
-**microbiology** or **antibiotics** when they are clearly indicated; it situates
-the plant as one thread in a larger picture of **recovery**, **prevention**, and
-**daily rhythm**.
-
-## The “effect” attributed to raw garlic for bacterial challenges
-
-Across herbal and holistic writing (popular and professional), **raw crushed or
-chopped garlic** is commonly credited with helping the body:
-
-1. **Meet microbes with a strong, sulfur-rich signal** — Traditions emphasize
- pungency as **cleansing** and **guarding** membranes and tissues,
- metaphorically “burning off” what overwhelms normal balance.
-
-2. **Support immune vigilance** — Not as a drug dose, but as **tonic** or
- **acute** food medicine: small amounts taken with meals or in **oxymels**,
- **honeys**, or **fresh pestos** where raw garlic stays enzymatically active.
-
-3. **Aid drainage and clearance** — Especially where **congestion**, **sluggish
- digestion**, or **stagnant** states accompany illness; warmth and pungency
- are thought to **wake** sluggish processes.
-
-4. **Complement rest and warmth** — Broths, soups finished at the table with raw
- garlic stirred in, or traditional **fire cider**–style preparations keep the
- emphasis on **whole-food** support rather than isolated “extract”
- thinking—though extracts appear in commerce, the **uncooked clove** remains
- the archetype in folk herbalism.
-
-Professional herbalists vary widely in how literally they speak about “natural
-antibiotics.” A measured holistic stance: **garlic is a respected historical
-antimicrobial plant** with **laboratory interest** in its sulfur chemistry,
-while **human outcomes** for serious resistant infections are **not** something
-responsible holistic literature promises from **kitchen garlic alone**.
-
-## Working with raw garlic in everyday holistic practice
-
-Common patterns (always adapted to personal tolerance and professional
-guidance):
-
-- **Crush or finely chop** fresh cloves and let them **sit briefly** before
- eating, so enzyme-driven chemistry can develop, then add to food **after**
- cooking cools slightly, or eat with bread, salad, or dips to reduce stomach
- upset.
-
-- **Small, repeated amounts** rather than heroic single doses—holistic care
- usually favors **digestible integration** over shock loading unless a
- practitioner specifies otherwise.
-
-- **Pair with rest, fluids, and simple foods** so “supporting clearance” is not
- fighting a depleted system.
-
-- **Respect contraindications** discussed in herbals: **sensitive digestion**,
- **GERD**, some **bleeding-related** contexts or **surgery** planning, and
- **medication combinations**—holistic practitioners routinely screen these;
- self-experimenters should not ignore them.
-
-## Limits of the holistic frame (stated plainly)
-
-Fever that won’t break, spreading redness, breathing difficulty, confusion,
-severe pain, **urinary symptoms with fever or back pain**, pregnancy, infancy,
-or **immune compromise** warrant **prompt conventional evaluation**—herbal and
-holistic layers can still matter for convalescence, but they do not replace
-timely diagnosis and prescribed care when the picture is acute or high-risk.
-
-**Antibiotic resistance** is a public-health and clinical problem; holistic
-writers who mention it usually do so to **encourage completion of prescribed
-courses**, **prevention**, and **stewardship**, not to imply that **raw garlic**
-should replace **culture-directed therapy** when a physician has indicated
-antibiotics.
-
-## Closing orientation
-
-In herbal medicine and holistic health, **raw uncooked garlic** is cherished as
-a **living, pungent** expression of a plant long associated with **protection**,
-**cleansing**, and **vitality** in the face of bacterial illness. Its effect, in
-that worldview, is as much about **supporting the whole person’s capacity to
-recover** as about any single compound—while serious infection remains a setting
-where **merging** traditional wisdom with **modern care** is the wisest path.
diff --git a/content/food/index.md b/content/food/index.md
@@ -1,3 +0,0 @@
----
-title: Food & Nutrition
----
diff --git a/content/food/lemony-chicken-green-olives.md b/content/food/lemony-chicken-green-olives.md
@@ -1,34 +0,0 @@
----
-title: Lemony Chicken with Green Olives
----
-
-Braised boneless chicken thighs with lemon, olives, and herbs—a one-pot move from stovetop sear to gentle oven bake. Step 5 in the source refers to a mixed-leaf salad and dressing “below”; those details were not on the scanned page, so serve with any green salad and dressing you like.
-
-## Ingredients (serves 4)
-
-- 8 boneless chicken thighs, skin on (about 3 oz each), seasoned well with salt and pepper
-- 2 tbsp duck fat
-- 2 medium onions, chopped or sliced
-- 1 tsp salt
-- ¼ tsp ground black pepper
-- Pinch of saffron (optional)
-- ½ cup bone broth
-- ¾ cup pitted olives
-- 2 lemons, peeled and sliced
-- About 20 sprigs fresh parsley, chopped (or to taste)
-- About 20 sprigs fresh coriander (cilantro), chopped (or to taste)
-
-## Instructions
-
-1. Preheat the oven to 350°F (about 177°C).
-2. Use a heavy, lidded pot that moves safely from hob to oven. Brown the chicken skin-side down in duck fat over medium-high heat. Remove from the pan and set aside.
-3. Reduce heat to medium, add the onions, cover, and sweat for about 5 minutes.
-4. Add the salt, pepper, and saffron; stir well. Return the chicken to the pot. Add broth, olives, and lemons. Bring to a simmer, cover, and transfer to the oven for about 40 minutes.
-5. While the chicken finishes, prepare a simple side salad and dressing if you like (the printed recipe referenced a separate salad section not shown here).
-6. When the chicken is done, uncover and stir in parsley and coriander. Spoon the sauce over the chicken and serve.
-
-## Tips
-
-- A tight-fitting lid keeps the braising liquid from reducing too aggressively; uncover only for step 6 if you want the sauce slightly thicker—watch it so olives and lemon do not burn.
-- If thighs vary in size, check doneness earlier; meat should reach a safe internal temperature for poultry and feel firm but still moist in the centre.
-- Pitted olives make eating easier; choose a mild green olive or swap in partly for niçoise-style if you prefer less salt—taste before adding extra salt at the table.
diff --git a/content/food/liver-cakes.md b/content/food/liver-cakes.md
@@ -1,31 +0,0 @@
----
-title: Liver Cakes
----
-
-Savory baked liver “muffins” blended with apple, onion, and herbs—rich from duck fat and eggs, easy to portion. They go well with a bright salad (pomegranate, mint, and fennel is a natural match); add dressing if you have a recipe you like.
-
-## Ingredients (serves 4, about 12 liver muffins)
-
-- 15 oz chicken liver
-- 4 eggs
-- ½ cup duck fat (about 3 oz)
-- 4 tbsp olive oil
-- 1 medium apple (about 6 oz)
-- 2 small onions (about 4 oz)
-- 1 tsp salt
-- ¼ tsp ground black pepper
-- 1 tsp dried thyme
-
-## Instructions
-
-1. Preheat the oven to 325°F (about 163°C). Put all ingredients into a strong blender and pulse until you have a smooth, fairly liquid paste.
-2. Pour the mixture into a silicone 12-muffin pan and bake for 15–25 minutes, until set and cooked through. If you do not have a muffin pan, use a small baking dish lined with parchment; the layer will be thinner and baking time will be shorter—check early.
-3. Invert the liver cakes onto a plate and serve with pomegranate, mint, and fennel salad (and dressing, if you have one).
-
-## Tips
-
-- A high-power blender gives the smoothest texture; scrape down the sides and pulse until no large liver or apple pieces remain.
-- Silicone pans make unmolding easier; if using metal, grease well or use parchment liners.
-- Doneness: the centres should not look wet or pink; when unsure, an instant-read thermometer in the centre should reach a safe temperature for cooked liver mixtures (follow local food-safety guidance).
-- Duck fat adds depth; if you substitute another fat, expect a milder flavour.
-- For the salad pairing, a simple mix of shaved fennel, pomegranate seeds, fresh mint, lemon or vinegar, and olive oil complements the richness of the cakes.
diff --git a/content/food/nutrients.md b/content/food/nutrients.md
@@ -1,389 +0,0 @@
----
-title: Micronutrients
----
-
-As discussed in chapters 4 and 5 of the Book: Change Your Diet Change Your Mind
-by Georgia Ede , micronutrients are indispensable to the building and burning
-pathways that make up the miracle of brain metabolism. The goal of this resource
-is to bring this list of players to life by describing the fascinating tasks
-each nutrient performs, paying special attention to the contributions each one
-makes to maintaining mental health.
-
-**Try not to think of this as a list of supplements—these are essential
-ingredients we are supposed to be able to obtain from our diets.** We are best
-adapted to absorbing, utilizing, and achieving proper balance of these nutrients
-when we consume them in whole foods, not as isolated, concentrated extracts.
-
-## Vitamin A (Retinol and related compounds)
-
-True vitamin A compounds (not to be confused with beta-carotene and similar
-compounds found in carrots and certain other colorful plant foods) belong to a
-family of fat-soluble hormones best known for their role in eye health,
-specifically for making both night vision and color perception possible. Less
-commonly known is that vitamin A influences the genes involved in the growth and
-development of all cells, and therefore is crucial to the development and
-maintenance of the entire brain.
-
-Learning and memory also rely heavily on vitamin A to help connect neurons in
-new patterns to solidify knowledge. Vitamin A deficiency in early life increases
-the risk for autism, and problems with vitamin A signaling may play a role in
-the development of schizophrenia. Vitamin A signaling can decline with age,
-contributing to cognitive deficits later in life.
-
-> [!info] Source Code:
-> - **Preformed vitamin A (retinol):** liver (beef, lamb, poultry, cod); fish liver oils; egg yolks; vitamin A–containing dairy (whole milk, butter, cream, cheese as consumed in traditional diets); fatty fish (herring, mackerel, salmon, sardines).
-> - **Provitamin A carotenoids (β-carotene and related):** convert to vitamin A with variable efficiency—sweet potato; carrot; pumpkin; winter squash; dark leafy greens (kale, spinach, collards); red peppers; mango; apricot; cantaloupe (whole fruit, not juice-only patterns).
-
-## The Busy B's
-
-Even though they look nothing alike and have unique responsibilities, the B
-vitamins are often lumped together and referred to as "B complex," because they
-all serve as coenzymes (enzyme assistants) that help cells extract energy from
-food and assemble vital molecules, so they are indispensable to multiple burning
-and building pathways.
-
-We must consume foods rich in B vitamins regularly because most of them can't be
-stored in our tissues, with the notable exception of vitamin B12, which we store
-in the liver. Since the brain is a high-energy organ, even brief deficiency of a
-single member of the B vitamin family can slow the brain's machinery and lead to
-vague, nonspecific psychiatric symptoms, such as fatigue, apathy, or insomnia.
-
-### Vitamin B1 (Thiamine)
-
-Vitamin B1 helps make and break the bonds that hold molecules together. It plays
-essential roles in both glycolysis (Engine G) and the citric acid cycle (part of
-Engine M), and connects the two pathways by converting pyruvate to acetyl CoA.
-It also helps the pentose phosphate pathway to make DNA and RNA, and assists in
-the construction of the neurotransmitters acetylcholine, glutamate, and GABA.
-Since glucose processing requires thiamine, diets high in carbohydrate increase
-thiamine requirements. Deficiency remains widespread in parts of the world that
-rely heavily on starchy staples like rice and have poor access to thiamine-rich
-protein sources.
-
-> [!info] Source Code:
-> - Pork (loin, ham); beef and lamb muscle meat; poultry; fish (especially tuna, salmon); whole-grain and enriched cereals and breads; legumes (black beans, navy beans, lentils); nuts and seeds (sunflower seeds, macadamia); peas and soy foods.
-
-### Vitamin B2 (Riboflavin)
-
-Vitamin B2 is an integral part of FAD, which carries electrons to the electron
-transport chain. Riboflavin helps build antioxidants and synthesize vitamin B3,
-and is also required to activate vitamins B6 and B9.
-
-> [!info] Source Code:
-> - Organ meats (liver, kidney); dairy milk and yogurt; eggs; lean beef; mushrooms (portobello, white button); almonds; spinach; whole grains; poultry (dark meat somewhat richer than white).
-
-### Vitamin B3 (Niacin)
-
-Vitamin B3 is an integral part of NAD, which is used by hundreds of enzymes to
-help transfer electrons between molecules. Severe, advanced niacin deficiency
-causes pellagra, a disease which can lead to depression, psychosis, delirium,
-and dementia.
-
-> [!info] Source Code:
-> - Poultry (chicken, turkey); beef; pork; fish (tuna, salmon); peanuts and peanut butter; mushrooms (portobello, crimini); whole grains; legumes; potatoes; fortified/enriched grains where culturally used; seeds (sesame, sunflower).
-
-### Vitamin B5 (Pantothenic acid)
-
-Vitamin B5 is an integral part of CoA (coenzyme A), which changes the shape of
-large molecules to help them undergo chemical reactions more easily. Vitamin B5
-participates widely in metabolism but is particularly important to the assembly
-of components needed for growth and is required to activate folate (vitamin B9).
-
-> [!info] Source Code:
-> - Animal foods across the board (beef, poultry, pork, organ meats, eggs, fish); whole grains; shiitake and other mushrooms; avocado; sweet potato; broccoli; sunflower seeds; peanuts; dairy milk and yogurt.
-
-### Vitamin B6 (Pyridoxine)
-
-Vitamin B6 is used by dozens of enzymes to help transfer carbon building blocks
-between molecules. Vitamin B6 is required for gluconeogenesis, amino acid
-processing, and to make vitamin B3, DNA, RNA, serotonin, dopamine,
-norepinephrine, and GABA.
-
-> [!info] Source Code:
-> - Chickpeas and other legumes; fish (tuna, salmon); poultry; potatoes (starchy); bananas; organ meats; lean beef; fortified cereals where used; nuts (pistachio); whole grains; sunflower seeds; dark leafy greens.
-
-### Vitamin B7 (Biotin)
-
-Vitamin B7 helps add carbon building blocks to molecules, regulates gene
-activity, and supports gluconeogenesis, the production of blood glucose.
-
-> [!info] Source Code:
-> - Egg yolks; beef liver; salmon; pork; sunflower seeds; sweet potato; almonds; spinach and other leafy greens; whole grains; legumes; cauliflower; cheese (modest amounts vary by type).
-
-### Vitamin B9 (Folate)
-
-Vitamin B9 supplies the carbon building blocks needed to help vitamin B12 make
-DNA, myelin, and certain neurotransmitters including serotonin, dopamine, and
-norepinephrine. Cells can't multiply without DNA, so folate requirements are
-much higher in times of rapid growth and development (such as pregnancy), and in
-parts of the body that replenish cells frequently, such as the bone marrow where
-red blood cells are produced. This is why folate deficiency can cause anemia
-(low red blood cell count) and neural tube defects such as spina bifida.
-
-Flour and cereal fortification programs in many countries have helped prevent
-deficiencies, but these use synthetic folic acid rather than the natural folate
-found in foods. Best animal food source is liver; good plant sources include
-spinach, asparagus, and avocado.
-
-> [!info] Source Code:
-> - Liver (beef, lamb, poultry); edamame and mature soybeans; lentils, black beans, kidney beans, chickpeas; asparagus, Brussels sprouts, broccoli; spinach, romaine, mustard greens; avocado; citrus (orange), strawberries; beets; sunflower seeds; peanuts; wheat germ; quinoa and whole wheat when part of a mixed diet (concentrations vary with processing).
-
-### Vitamin B12 (Cobalamin)
-
-Whereas most B vitamins participate in dozens to hundreds of metabolic
-reactions, B12 serves only two enzymes, so we need very little B12 and it can
-take years to deplete our reserves. The first enzyme helps vitamin B9 (folate)
-move carbon building blocks between molecules to make neurotransmitters and DNA
-(which is why B12 deficiency, like folate deficiency, can cause anemia), and the
-other enzyme is used to build myelin.
-
-Vitamin B12 deficiency is not uncommon even in affluent countries, partly
-because so many medications and health conditions can interfere with B12
-absorption, and partly because diets low in animal foods are becoming more
-prevalent. B12 deficiency can lead to a variety of psychiatric symptoms,
-including depression, psychosis, memory problems, and personality changes.
-
-B12 is only found in animal foods; good sources include shellfish, meat, fish,
-poultry, eggs, and low-fat dairy products.
-
-> [!info] Source Code:
-> - **Very high (examples):** clams; mussels; oysters; liver (beef, lamb, veal); sardines and other small fatty fish; crab and other shellfish.
-> - **Commonly useful sources:** salmon, trout, tuna; beef and lamb; poultry; eggs (especially whole eggs); milk, yogurt, and cheese (content varies). **Note:** reliable plant-only B12 generally requires intentional microbial sources (e.g., certain fermented foods or fortified foods), not ordinary unwashed produce alone.
-
-## Vitamin C (L-ascorbic acid)
-
-Vitamin C (L-ascorbic acid) is a coenzyme required to build collagen (a
-component of the blood-brain barrier), and helps regulate the production of
-myelin, which insulates brain circuits.
-
-> [!info] Source Code:
-> - Citrus (oranges, grapefruit, lemon, lime); kiwifruit; strawberries, raspberries; papaya; guava; bell peppers (especially red); broccoli; Brussels sprouts; cauliflower; tomatoes; potatoes (white and sweet); cantaloupe; cabbage; kohlrabi; fresh herbs like parsley.
-
-## Vitamin D3 (cholecalciferol)
-
-Strictly speaking, vitamin D3 is not essential in the diet because your skin can
-produce it if exposed to enough sunlight. Vitamin D is a fat-soluble hormone
-that influences brain development, calcium balance, antioxidant defenses, and
-neuroplasticity—the creation of new neuron networks in response to new
-experiences, which is key to learning and memory. Vitamin D deficiency is very
-common, particularly in people with insulin resistance, and deficiency during
-pregnancy increases risk for developmental and neuropsychiatric complications in
-offspring.
-
-> [!info] Source Code:
-> - **D3 in whole foods (typically modest unless eaten often):** fatty fish and fish roe (salmon, mackerel, herring, sardines, tuna); beef liver; egg yolks; pork (small contributions).
-> - **UV-exposed mushrooms** can provide vitamin D2 (ergocalciferol), not D3, but still contribute to vitamin D status depending on exposure and species.
-> - **Primary non-food source for many people:** sensible sunlight exposure enabling cutaneous synthesis of D3.
-
-## Vitamin E (alpha-tocopherol)
-
-Vitamin E helps protect unsaturated fatty acids (MUFAs and PUFAs) from oxidative
-damage, helps maintain the shape of cell membranes, and regulates genes involved
-in protecting cell membranes.
-
-> [!info] Source Code:
-> - Wheat germ; sunflower seeds and almonds; hazelnuts; peanuts; avocado; spinach; Swiss chard; beet greens; broccoli; mango; kiwi; olives; whole grains (varies); vegetable oils are concentrated sources in modern diets but are not “whole foods” in the strict sense.
-
-## Vitamin K1
-
-Vitamin K1 helps add carbon groups to clotting proteins, allowing them to bind
-calcium and initiate the “coagulation cascade” to prevent uncontrolled bleeding.
-
-> [!info] Source Code:
-> - Dark leafy greens (kale, spinach, collards, mustard greens, turnip greens, Swiss chard); broccoli, Brussels sprouts, cabbage; parsley; asparagus; prunes; soybean and canola oils as used in food prep (concentrated).
-
-## Vitamin K2
-
-Vitamin K2 is a much-overlooked fat-soluble hormone that activates proteins
-involved in brain cell growth and survival and participates in the production of
-vital membrane fats (sphingolipids), including those needed to make myelin.
-Several different forms of K2 exist, but 98 percent of the K2 in the human brain
-exists in a form called MK-4. Best dietary sources are liver and egg yolks.
-
-> [!info] Source Code:
-> - **MK-4–rich patterns:** liver (chicken, goose, beef); egg yolks; butter and full-fat dairy from pasture-raised/forage-fed animals (variable with feeding).
-> - **Longer-chain menaquinones (e.g., MK-7):** natto (fermented soy); certain aged cheeses; some fermented dairy and animal products (microbial K2 varies).
-
-## Calcium
-
-Like a scout, calcium carries high-priority messages about energy demands,
-neurotransmitter supply, and cell health from the outer reaches of the neuron to
-deep inside the mitochondria and nucleus (the cell’s command center), so they
-can rapidly adapt to changing circumstances. Glutamate and GABA receptors use
-calcium, and calcium signaling is critical for learning and memory
-(neuroplasticity), neurotransmitter release, and even cell survival. If the cell
-is in dire straits due to viral infection, lack of oxygen, or other serious
-threats, large amounts of calcium rush in, initiating the cell’s suicide program
-(apoptosis).
-
-> [!info] Source Code:
-> - Dairy (milk, yogurt, kefir), cheese; canned sardines and salmon with bones; anchovies; tofu set with calcium salts; sesame seeds and tahini; almonds; white beans and other legumes; leafy greens (collard, kale, bok choy—bioavailability varies with oxalate); figs; blackstrap molasses (used sparingly); some mineral waters (label-dependent).
-
-## Chloride
-
-Chloride is the dominant negatively charged ion in the brain. It helps regulate
-fluid balance and cell volume and cooperates with sodium to maintain neurons’
-readiness to fire.
-
-> [!info] Source Code:
-> - Chloride is supplied mainly as **salt (sodium chloride)** in real-world diets; whole-food contributors include seaweed, celery, tomatoes, olives, beets, shellfish, rye, and many preserved or brined whole foods where salt is intrinsic to preparation.
-
-## Choline
-
-The vast majority of choline is used to make phosphatidylcholine, an essential
-component of cell membranes. Choline is also used to build myelin, DNA
-molecules, and the neurotransmitter acetylcholine. Choline was only recognized
-as an essential nutrient in 1998, so little is known about how deficiency
-affects mental health, but early studies suggest choline deficiency may affect
-attention and memory, perhaps because acetylcholine is so important to these
-brain functions. Studies find that most people in the United States do not
-consume adequate choline; best sources are red meat, liver, eggs, and fish roe.
-
-> [!info] Source Code:
-> - Egg yolks; beef, chicken, and cod liver; fish roe; muscle meats (beef, pork, chicken); wheat germ (for those who include grains); soybeans and roasted soy nuts; shiitake mushrooms; lima beans and other legumes; Brussels sprouts; long-simmered meat or poultry broth (connective tissue releases phospholipids).
-
-## Copper
-
-The electron transport chain relies on copper to pull electrons through one of
-its large enzyme complexes (cytochrome c oxidase) as it works to make ATP. The
-enzyme that transforms dopamine into norepinephrine depends on copper as well.
-
-> [!info] Source Code:
-> - Liver (beef, calf, lamb); oysters and other shellfish; crab and lobster; shiitake mushrooms; dark chocolate/cacao (whole bean preparations); cashews, almonds, sesame; sunflower seeds; chickpeas; potatoes (with skin); whole grains and wheat germ.
-
-## Iodine
-
-Iodine is an integral part of thyroid hormone, which is not only a major
-orchestrator of brain development in early life but also supports healthy brain
-metabolism throughout the lifespan. Iodine deficiency causes hypothyroidism (low
-thyroid hormone activity); when this occurs during pregnancy, it can lead to
-irreversible cognitive deficits in the developing baby. Iodine deficiency is
-widespread, affecting up to two billion people, including in the United States
-and Europe, and is a leading cause of preventable intellectual disabilities. In
-adults, hypothyroidism can cause symptoms of depression such as apathy and
-fatigue, and can even cause reversible dementia, likely due in part to sluggish
-brain glucose metabolism. Best food sources are fish, shrimp, seaweed, and
-iodized salt.
-
-> [!info] Source Code:
-> - **Rich marine patterns:** seaweed (kelp, nori, kombu—potency varies widely, excess possible); cod, haddock, pollock, tuna; shrimp, scallops; milk and yogurt (iodine varies with farm practices and season); eggs; **iodized salt** as a practical public-health vehicle where seafood intake is low; some coastal soils and produce (variable).
-
-## Iron
-
-When we think of iron, we think of blood, but this mineral’s responsibilities
-extend far beyond carrying oxygen to the brain in red blood cells. Iron is
-gifted with the ability to exist in two different charged states, so it can give
-and receive electrons easily. This special talent makes it indispensable to the
-electron transport chain and many other pathways, including those used to
-construct DNA, myelin, and the neurotransmitters serotonin, dopamine, and
-norepinephrine. Iron deficiency is the most common nutrient deficiency in the
-world, affecting more than 25 percent of the world’s population, most of whom
-are pregnant women and very young children. Since iron is needed to build DNA
-and myelin, iron deficiency during pregnancy can have irreversible effects on
-intelligence, memory, and attention, and can increase risk for autism. Best
-dietary sources are red meat, liver, mussels, and oysters.
-
-> [!info] Source Code:
-> - **Heme iron (generally well absorbed):** liver; beef and lamb; dark poultry meat; sardines; oysters, clams, mussels; venison and other game.
-> - **Non-heme iron (absorption improved with vitamin C and careful food pairing):** lentils, chickpeas, white beans; tofu and tempeh; spinach and Swiss chard; pumpkin seeds; quinoa; blackstrap molasses (sparing use).
-
-## Magnesium
-
-Magnesium’s compact size and strong positive charge give it magnetic properties
-useful in hundreds of chemical reactions, helping to generate energy, build
-proteins, and stabilize genes. Magnesium exists in balance with calcium and
-zinc, which keeps their destructive influences in check. One of magnesium’s most
-intriguing tasks is to sit stubbornly inside glutamate receptors (NMDA
-receptors, to be exact), plugging them up and preventing positive ions from
-entering the cell. Only when a strong electrical signal comes along will
-magnesium pop out like a champagne cork and allow those ions to pour in so the
-neuron can fire. NMDA receptors are particularly important for learning, memory,
-and healthy circadian rhythm (sleep-wake patterns).
-
-> [!info] Source Code:
-> - Pumpkin, chia, and flax seeds; almonds and cashews; peanuts; spinach and Swiss chard; black beans, edamame, lima beans; quinoa; brown rice and whole wheat; avocado; yogurt and milk; banana; dark chocolate (cacao solids); mackerel and halibut (examples of fish contributing).
-
-## Manganese
-
-The antioxidant enzyme superoxide dismutase, which shields mitochondria from
-free-radical damage, contains manganese. The multipurpose enzyme glutamine
-synthetase, which is used to manufacture glutamine, glutamate, and GABA, as well
-as to detoxify glutamate and ammonia in the brain, also requires manganese.
-
-> [!info] Source Code:
-> - Whole grains (oat, brown rice, bulgur); mussels and clams; hazelnuts, pecans; chickpeas and soy; spinach and beet greens; black tea (beverage from leaf); pineapple; sweet potato.
-
-## Molybdenum
-
-Only four enzymes in the body require molybdenum. These enzymes help prevent DNA
-mutations and support healthy uric acid levels (which protects the brain against
-oxidative stress).
-
-> [!info] Source Code:
-> - Legumes (lentils, black beans, split peas); whole grains (especially oats); peanuts and other nuts; leafy vegetables; potatoes (with skin); tomatoes; seeds (sunflower, sesame); liver (modest).
-
-## Phosphorus
-
-Phosphorus is an essential component of cell membranes, DNA and RNA, ATP
-molecules (the P stands for phosphate), and bone. It also participates in
-multiple chemical reactions—including electron transfer in the electron
-transport chain—and helps regulate the pH of the blood.
-
-> [!info] Source Code:
-> - Dairy (milk, cheese, yogurt); eggs; fish and shellfish; poultry; pork and beef muscle; lentils, chickpeas, soybeans; pumpkin seeds, sunflower seeds; quinoa; whole wheat, oatmeal; potatoes; tree nuts (almonds, cashews, walnuts—content varies).
-
-## Potassium
-
-Potassium is the dominant positively charged ion inside neurons, with
-concentrations maintained at roughly thirty times higher inside than outside to
-help maintain neurons’ readiness to fire. The enzyme that releases energy from
-ATP also requires potassium.
-
-> [!info] Source Code:
-> - White and sweet potatoes with skin; beet greens, spinach, Swiss chard; tomato products (paste, puree—whole-tomato based); oranges, orange juice (whole-fruit patterns preferred), bananas, cantaloupe, kiwi, apricots (dried unsweetened in moderation); avocado; white beans, lentils, soybeans; yogurt and milk; salmon, halibut, tuna; coconut water (natural).
-
-## Selenium
-
-Several antioxidant enzymes contain selenium, including glutathione peroxidase,
-which helps protect the brain against stress, including oxidative stress.
-
-> [!info] Source Code:
-> - **Brazil nuts (very high—portion control matters);** seafood (tuna, halibut, sardines, oysters); organ meats; pork, beef, turkey, chicken; sunflower seeds; shiitake and crimini mushrooms; brown rice and whole wheat; eggs; dairy milk and yogurt.
-
-## Sodium
-
-Sodium is the dominant positively charged ion outside neurons, with
-concentrations maintained at roughly ten times higher outside than inside to
-help maintain neurons’ readiness to fire.
-
-> [!info] Source Code:
-> - Sodium occurs widely as **sodium chloride** and natural sodium in whole foods: celery, beets, carrots; milk and yogurt; shellfish; anchovies; pickles and fermented vegetables (salt-mediated); tomato juice; many cheeses and cured whole-meat preparations (traditional preservation).
-
-## Sulfur
-
-Sulfur is an essential component of insulin and glutathione (one of the most
-important antioxidants in the brain). It is also required to build two amino
-acids (cysteine and methionine) and helps guide protein folding and numerous
-metabolic pathways.
-
-> [!info] Source Code:
-> - **Sulfur mainly arrives as protein-bound amino acids and organosulfur compounds:** meats, poultry, fish, eggs; milk, yogurt, cheese; legumes (especially mature soy); alliums (onion, garlic, leek, shallot); crucifers (broccoli, cauliflower, Brussels sprouts, cabbage, kale); mustard family plants.
-
-## Zinc
-
-Zinc allows certain proteins to fold into their correct shapes and assists
-certain enzymes in their catalytic duties. Zinc is required for healthy immune
-system function and neurotransmitter activity. One of zinc’s unique
-responsibilities is to burst out of tiny storage compartments into the synapse
-alongside glutamate (a stimulating neurotransmitter) to buffer its signal. Zinc
-also behaves as a natural dopamine reuptake inhibitor, prolonging dopamine
-signaling in the synapse. Zinc ripens young BDNF (brain-derived growth factor)
-molecules to maturity so they may fertilize developing neurons, supporting the
-process of neuroplasticity. Zinc is also central to the process of autophagy;
-when mitochondria or other critical cell components are damaged beyond repair
-and need to be destroyed, zinc helps calcium flip the kill switch, partly by
-intentionally generating oxygen free radicals to attack them from within and
-finish them off.
-
-> [!info] Source Code:
-> - Oysters (very high); crab, lobster; beef and lamb; pumpkin seeds, hemp seeds; cashews, almonds; chickpeas, lentils, black beans; yogurt, cheese; oats, quinoa; pork and dark poultry meat; eggs.
diff --git a/content/food/pan-fried-king-prawns.md b/content/food/pan-fried-king-prawns.md
@@ -1,33 +0,0 @@
----
-title: Pan-Fried Shell-On King Prawns
----
-
-A simple, flavourful pan-fried king prawn recipe ready in ~10 minutes.
-
-## Ingredients (serves 2)
-
-- 8–12 raw king prawns, shell on (heads on or off)
-- 1–2 tbsp olive oil or vegetable oil
-- 1–2 tbsp butter
-- 2–3 garlic cloves, finely chopped or crushed
-- Salt and black pepper to taste
-- Lemon wedges and chopped parsley or chives (optional)
-
-## Instructions
-
-1. Rinse the prawns under cold water, then pat completely dry with paper towel.
-2. Heat a large frying pan over medium-high heat and add the oil.
-3. Once hot, add the prawns in a single layer. Season with salt and pepper.
-4. Fry for about 3 minutes on the first side until the shells start to turn pink.
-5. Flip and cook for a further 2–3 minutes until the shells are fully pink and the flesh is opaque throughout.
-6. Reduce heat to medium. Push prawns to one side, add the butter and garlic to the empty space.
-7. Let the garlic sizzle for 30–60 seconds without browning, then toss the prawns in the garlic butter for about 1 minute.
-8. Finish with a squeeze of lemon and a sprinkle of parsley or chives. Serve immediately.
-
-## Tips
-
-- Dry the prawns well before cooking so they fry rather than steam.
-- Cook in one layer; overcrowding the pan will cause them to stew.
-- The prawns are done when fully pink with no grey or translucent parts — firm but still juicy.
-- Add a pinch of chili flakes with the garlic for a little heat.
-- Serve with crusty bread, rice, or just on their own and peel at the table.
diff --git a/content/food/salmon-muffins.md b/content/food/salmon-muffins.md
@@ -1,25 +0,0 @@
----
-title: Salmon Muffins
----
-
-Simple baked salmon “muffins” blended with duck fat and salt—similar in spirit to other fish or liver muffin recipes, but kept minimal for a carnivore-style plate. The printed card labels the dish as carnivore; scale up if you want more than one serving.
-
-## Ingredients (makes 1 serving)
-
-- 4 tbsp duck fat
-- 1 × 16 oz can wild salmon, in brine, drained
-- 1 tsp salt
-
-## Instructions
-
-1. Preheat the oven to 325°F (about 163°C).
-2. Mix all ingredients in a blender until smooth.
-3. Divide the mixture into a silicone muffin pan and bake for no longer than about 20 minutes to avoid drying out.
-4. Season with salt to taste and serve.
-
-## Tips
-
-- The source suggests a **maximum** bake time; start checking a few minutes earlier if your pan runs hot or the batter is spread thin.
-- Canned salmon packaged in brine is usually saltier than water-packed; taste after blending before adding the full teaspoon of salt if you are sensitive to sodium.
-- **Customize your protein (from the printed note):** about 1 oz canned wild salmon supplies roughly 6 g protein—useful if you swap can size or split the batch.
-- **Printed nutrition (one serving, from the source card):** about 63 g fat, 60 g protein, 0 g total carbohydrates, 0 g fiber, about 800 kcal; macronutrient split stated as about 71% fat, 29% protein, 0% carbohydrates. Treat these as approximate labels on a single recipe card, not a laboratory analysis.
diff --git a/content/food/steamed-white-clams.md b/content/food/steamed-white-clams.md
@@ -1,33 +0,0 @@
----
-title: Steamed White Clams
----
-
-A simple, flavourful steamed white clam recipe ready in ~30 minutes.
-
-## Ingredients (serves 2)
-
-- 1 kg fresh white clams, scrubbed and rinsed
-- 2–3 tbsp olive oil or 2 tbsp butter
-- 3–4 cloves garlic, finely chopped
-- Small pinch chili flakes (optional)
-- 120 ml dry white wine or water
-- Salt and black pepper to taste
-- Lemon wedges and chopped parsley (optional)
-
-## Instructions
-
-1. Soak clams in cold salted water for 20–30 minutes, then rinse well.
-2. Discard any clams with broken shells or that stay open when tapped.
-3. Heat oil or butter in a large pot or deep pan over medium heat.
-4. Add garlic (and chili if using), cook 1–2 minutes until fragrant but not browned.
-5. Pour in white wine or water, bring just to a boil.
-6. Add clams, stir once, cover with a lid, and cook 5–8 minutes until most clams open, shaking the pan occasionally.
-7. Discard any clams that did not open.
-8. Season with salt and pepper, squeeze lemon over, and garnish with parsley. Serve with crusty bread.
-
-## Tips
-
-- Use only water and lemon instead of wine for an alcohol-free version.
-- Add a chopped shallot or onion with the garlic for extra sweetness.
-- The broth is excellent for dipping bread — don't waste it.
-- Clams cook fast; don't overcook or they turn rubbery.
diff --git a/content/index.md b/content/index.md
@@ -1,65 +0,0 @@
----
-title: Ling's Notebook
----
-
-Personal study notes for software engineering topics with Rust and TypeScript.
-
-[](https://github.com/ling0x/notes/actions/workflows/deploy-pages.yaml)
-
-[Also backed up on codeberg](https://codeberg.org/ling0x/notes)
-
-## Software Engineering
-
-- [Compiler](compiler/)
-
-- [Async Programming](async_programming/)
-
-- [System Design](system_design/)
-
-- [Machine Learning](machine_learning/)
-
-- [Memory Safety](memory_safety/)
-
-- [Networking](networking/)
-
-- [Observability](observability/)
-
-- [Operating Systems](operating_systems/)
-
-- [Refactoring](refactoring/)
-
-- [System Administration](system_administration/)
-
-- [Test Driven Development](test_driven_development/)
-
-- [Web Development](web_development/)
-
-- [Engineering Practice](engineering_practice/)
-
-- [Useful Commands](commands/)
-
-## Broader Knowledge
-
-- [Linear Algebra](linear_algebra/)
-
-- [Quantum Mechanics](quantum_mechanics/)
-
-- [Personal Development](personal_development/)
-
-- [References](references/)
-
-- [Food Recipes](food/)
-
-## Coding Exercises
-
-- [Spinlock](exercises/spinlock/)
-
-- [Tokio Tutorial](exercises/tokio-tutorial/)
-
-- [Channels](exercises/channels/)
-
-- [My Vec](exercises/my_vec/)
-
-- [Actors](exercises/actors/)
-
-- [Design Patterns in Rust](exercises/design_patterns_in_rust/)
diff --git a/content/linear_algebra/index.md b/content/linear_algebra/index.md
@@ -1,7 +0,0 @@
----
-title: Linear Algebra
----
-
-1. Vectors
- - [Vectors](linear_algebra/vectors/vectors.md)
- - [Dot Product](linear_algebra/vectors/dot_product.md)
diff --git a/content/linear_algebra/vectors/cross_product.md b/content/linear_algebra/vectors/cross_product.md
@@ -1,307 +0,0 @@
----
-title: Cross Product
----
-
-The **cross product** is an operation that takes two vectors in
-$$ \mathbb{R}^3 $$
-
-and returns another vector in $$ \mathbb{R}^3 $$
-
-, written $\mathbf{a} \times \mathbf{b}$. Unlike the dot product, the result is
-a vector, not a scalar. The cross product is only defined in three dimensions
-(and in a generalized sense in seven dimensions; here we restrict to
-$\mathbb{R}^3$).
-
-## Geometric meaning
-
-- **Direction:** $\mathbf{a} \times \mathbf{b}$ is perpendicular to both
- $\mathbf{a}$ and $\mathbf{b}$, following the right-hand rule: if you point
- your fingers along $\mathbf{a}$ and curl them toward $\mathbf{b}$, your thumb
- points in the direction of $\mathbf{a} \times \mathbf{b}$.
-- **Magnitude:** $\|\mathbf{a} \times \mathbf{b}\| =
- \|\mathbf{a}\|\,\|\mathbf{b}\|\sin\theta$, where $\theta$ is the angle between
- $\mathbf{a}$ and $\mathbf{b}$. So the length equals the area of the
- parallelogram spanned by $\mathbf{a}$ and $\mathbf{b}$.
-
-## Algebraic definition
-
-For vectors
-
-$$
-\mathbf{a} =
-\begin{pmatrix}
-a_1\\
-a_2\\
-a_3
-\end{pmatrix},
-\quad
-\mathbf{b} =
-\begin{pmatrix}
-b_1\\
-b_2\\
-b_3
-\end{pmatrix},
-$$
-
-the cross product is
-
-$$
-\mathbf{a} \times \mathbf{b} =
-\begin{pmatrix}
-a_2 b_3 - a_3 b_2\\
-a_3 b_1 - a_1 b_3\\
-a_1 b_2 - a_2 b_1
-\end{pmatrix}.
-$$
-
-This can be remembered using the determinant of a formal $3\times 3$ matrix:
-
-$$
-
-\mathbf{a} \times \mathbf{b} = \begin{vmatrix} \mathbf{e}_1 & \mathbf{e}_2 &
-\mathbf{e}_3\\ a_1 & a_2 & a_3\\ b_1 & b_2 & b_3 \end{vmatrix}
-
-$$
-
-$$
-
-\mathbf{e}_1(a_2 b_3 - a_3 b_2)
-
-- \mathbf{e}_2(a_1 b_3 - a_3 b_1)
-
-* \mathbf{e}_3(a_1 b_2 - a_2 b_1),
-
-$$
-
-where $\mathbf{e}_1, \mathbf{e}_2, \mathbf{e}_3$ are the standard unit vectors
-in $\mathbb{R}^3$.
-
-### The "Cross-Out" Method (Fastest)
-
-The shorthand calculation for this is:
-
-1. Stack them: Write the components of the first vector over the second vector
- twice.
-2. Cross out the first and last columns.
-3. Multiply in an 'X' pattern (top-left bot-right minus top-right bot-left) for
- each remaining pair:
-
-<img src="/assets/cross_product_shorthand.png" alt="Cross Product Calculation" width="500">
-
-## Rules of calculation (with examples in LaTeX)
-
-Let $\mathbf{a}, \mathbf{b}, \mathbf{c} \in \mathbb{R}^3$ and $\lambda \in
-\mathbb{R}$.
-
----
-
-**1. Anticommutativity**
-
-Swapping the order flips the sign:
-
-$$
-\mathbf{a} \times \mathbf{b} = -\bigl(\mathbf{b} \times \mathbf{a}\bigr).
-$$
-
-Example:
-
-$$
-\begin{pmatrix} 1\\ 0\\ 0 \end{pmatrix} \times \begin{pmatrix} 0\\ 1\\ 0 \end{pmatrix}
-= \begin{pmatrix} 0\\ 0\\ 1 \end{pmatrix},
-\quad
-\begin{pmatrix} 0\\ 1\\ 0 \end{pmatrix} \times \begin{pmatrix} 1\\ 0\\ 0 \end{pmatrix}
-= \begin{pmatrix} 0\\ 0\\ -1 \end{pmatrix}.
-$$
-
----
-
-**2. Distributivity over addition**
-
-$$
-\mathbf{a} \times (\mathbf{b} + \mathbf{c})
-= \mathbf{a} \times \mathbf{b} + \mathbf{a} \times \mathbf{c},
-\qquad
-(\mathbf{a} + \mathbf{b}) \times \mathbf{c}
-= \mathbf{a} \times \mathbf{c} + \mathbf{b} \times \mathbf{c}.
-$$
-
-Example (second component of $\mathbf{a} \times (\mathbf{b}+\mathbf{c})$):
-
-$$
-\mathbf{a} = \begin{pmatrix} 1\\ 2\\ 0 \end{pmatrix},\;
-\mathbf{b} = \begin{pmatrix} 0\\ 1\\ 1 \end{pmatrix},\;
-\mathbf{c} = \begin{pmatrix} 1\\ 0\\ 1 \end{pmatrix}
-\;\Rightarrow\;
-\mathbf{b}+\mathbf{c} = \begin{pmatrix} 1\\ 1\\ 2 \end{pmatrix}.
-$$
-
-$$
-\mathbf{a} \times \mathbf{b} = \begin{pmatrix} 2\\ -1\\ 1 \end{pmatrix},\quad
-\mathbf{a} \times \mathbf{c} = \begin{pmatrix} 2\\ -1\\ -2 \end{pmatrix}
-\;\Rightarrow\;
-\mathbf{a} \times \mathbf{b} + \mathbf{a} \times \mathbf{c} = \begin{pmatrix} 4\\ -2\\ -1 \end{pmatrix}.
-$$
-
-$$
-\mathbf{a} \times (\mathbf{b}+\mathbf{c}) = \begin{pmatrix} 2\cdot 2 - 0\cdot 1\\ 0\cdot 1 - 1\cdot 2\\ 1\cdot 1 - 2\cdot 1 \end{pmatrix} = \begin{pmatrix} 4\\ -2\\ -1 \end{pmatrix}.
-$$
-
----
-
-**3. Scalar multiplication (homogeneity)**
-
-A scalar can be factored out of either slot:
-
-$$
-(\lambda \mathbf{a}) \times \mathbf{b}
-= \mathbf{a} \times (\lambda \mathbf{b})
-= \lambda (\mathbf{a} \times \mathbf{b}).
-$$
-
-Example: with $\mathbf{a} = \begin{pmatrix} 1\\ 0\\ 0 \end{pmatrix}$,
-$\mathbf{b} = \begin{pmatrix} 0\\ 1\\ 0 \end{pmatrix}$, $\lambda = 3$,
-
-$$
-(3\mathbf{a}) \times \mathbf{b}
-= \begin{pmatrix} 3\\ 0\\ 0 \end{pmatrix} \times \begin{pmatrix} 0\\ 1\\ 0 \end{pmatrix}
-= \begin{pmatrix} 0\\ 0\\ 3 \end{pmatrix}
-= 3 \begin{pmatrix} 0\\ 0\\ 1 \end{pmatrix}
-= 3(\mathbf{a} \times \mathbf{b}).
-$$
-
----
-
-**4. Cross product with the zero vector**
-
-$$
-\mathbf{a} \times \mathbf{0} = \mathbf{0} \times \mathbf{a} = \mathbf{0}.
-$$
-
----
-
-**5. Parallel vectors**
-
-$\mathbf{a}$ and $\mathbf{b}$ are parallel (or one is zero) if and only if
-
-$$
-\mathbf{a} \times \mathbf{b} = \mathbf{0}.
-$$
-
-Example: $\mathbf{a} = \begin{pmatrix} 2\\ 4\\ 6 \end{pmatrix}$, $\mathbf{b}
-= \begin{pmatrix} 1\\ 2\\ 3 \end{pmatrix} = \tfrac{1}{2}\mathbf{a}$, so
-
-$$
-\mathbf{a} \times \mathbf{b}
-= \begin{pmatrix} 4\cdot 3 - 6\cdot 2\\ 6\cdot 1 - 2\cdot 3\\ 2\cdot 2 - 4\cdot 1 \end{pmatrix}
-= \begin{pmatrix} 0\\ 0\\ 0 \end{pmatrix}.
-$$
-
----
-
-**6. Self-cross product**
-
-$$
-\mathbf{a} \times \mathbf{a} = \mathbf{0}.
-$$
-
-(Special case of the parallel-vectors rule.)
-
----
-
-**7. Jacobi identity**
-
-$$
-
-\mathbf{a} \times (\mathbf{b} \times \mathbf{c})
-
-- \mathbf{b} \times (\mathbf{c} \times \mathbf{a})
-- \mathbf{c} \times (\mathbf{a} \times \mathbf{b}) = \mathbf{0}.
-
-$$
-
----
-
-**8. Relation to dot product (vector triple product expansion)**
-
-$$
-\mathbf{a} \times (\mathbf{b} \times \mathbf{c})
-= (\mathbf{a} \cdot \mathbf{c})\mathbf{b} - (\mathbf{a} \cdot \mathbf{b})\mathbf{c}.
-$$
-
-Example: $\mathbf{a} = \mathbf{e}_1$, $\mathbf{b} = \mathbf{e}_2$,
-$\mathbf{c} = \mathbf{e}_3$:
-
-$$
-\mathbf{a} \cdot \mathbf{c} = 0,\quad \mathbf{a} \cdot \mathbf{b} = 0
-\;\Rightarrow\;
-\mathbf{a} \times (\mathbf{b} \times \mathbf{c}) = 0\cdot \mathbf{b} - 0\cdot \mathbf{c} = \mathbf{0}.
-$$
-
-$$
-\mathbf{b} \times \mathbf{c} = \mathbf{e}_1
-\;\Rightarrow\;
-\mathbf{e}_1 \times \mathbf{e}_1 = \mathbf{0}.
-$$
-
----
-
-**9. Magnitude and angle**
-
-$$
-\|\mathbf{a} \times \mathbf{b}\|^2
-= \|\mathbf{a}\|^2 \|\mathbf{b}\|^2 - (\mathbf{a} \cdot \mathbf{b})^2.
-$$
-
-Equivalently, $\|\mathbf{a} \times \mathbf{b}\| =
-\|\mathbf{a}\|\,\|\mathbf{b}\|\sin\theta$.
-
----
-
-**10. Relation to the dot product (scalar triple product)**
-
-$$
-\mathbf{a} \cdot (\mathbf{b} \times \mathbf{c})
-= \mathbf{b} \cdot (\mathbf{c} \times \mathbf{a})
-= \mathbf{c} \cdot (\mathbf{a} \times \mathbf{b}).
-$$
-
-This value is the (signed) volume of the parallelepiped spanned by $\mathbf{a},
-\mathbf{b}, \mathbf{c}$. Example:
-
-$$
-\mathbf{a} = \begin{pmatrix} 1\\ 0\\ 0 \end{pmatrix},\;
-\mathbf{b} = \begin{pmatrix} 0\\ 1\\ 0 \end{pmatrix},\;
-\mathbf{c} = \begin{pmatrix} 0\\ 0\\ 1 \end{pmatrix}
-\;\Rightarrow\;
-\mathbf{b} \times \mathbf{c} = \begin{pmatrix} 1\\ 0\\ 0 \end{pmatrix},\quad
-\mathbf{a} \cdot (\mathbf{b} \times \mathbf{c}) = 1.
-$$
-
-## Worked example
-
-Compute $\mathbf{u} \times \mathbf{v}$ for
-
-$$
-\mathbf{u} = \begin{pmatrix} 2\\ -1\\ 3 \end{pmatrix},\qquad
-\mathbf{v} = \begin{pmatrix} 1\\ 4\\ -2 \end{pmatrix}.
-$$
-
-$$
-\mathbf{u} \times \mathbf{v}
-= \begin{pmatrix}
-(-1)(-2) - (3)(4)\\
-(3)(1) - (2)(-2)\\
-(2)(4) - (-1)(1)
-\end{pmatrix}
-= \begin{pmatrix}
-2 - 12\\
-3 + 4\\
-8 + 1
-\end{pmatrix}
-= \begin{pmatrix} -10\\ 7\\ 9 \end{pmatrix}.
-$$
-
-Check: $\mathbf{u} \cdot (\mathbf{u} \times \mathbf{v}) = 2(-10) + (-1)(7) +
-3(9) = -20 - 7 + 27 = 0$, and $\mathbf{v} \cdot (\mathbf{u} \times \mathbf{v})
-= 1(-10) + 4(7) + (-2)(9) = -10 + 28 - 18 = 0$, so the result is perpendicular
-to both $\mathbf{u}$ and $\mathbf{v}$.
diff --git a/content/linear_algebra/vectors/dot_product.md b/content/linear_algebra/vectors/dot_product.md
@@ -1,233 +0,0 @@
----
-title: Dot Product
----
-
-The dot product is an operation that takes two vectors of the same dimension and
-returns a single real number (a scalar), often written with a centered dot like
-\(\mathbf{a} \cdot \mathbf{b}\).
-
-## Algebraic definition
-
-For vectors in $$ \mathbb{R}^n $$
-
-$$
-\mathbf{a} =
-\begin{pmatrix}
-a_1\\
-a_2\\
-\vdots\\
-a_n
-\end{pmatrix},
-\quad
-\mathbf{b} =
-\begin{pmatrix}
-b_1\\
-b_2\\
-\vdots\\
-b_n
-\end{pmatrix},
-$$
-
-their dot product is
-
-$$
-
-\mathbf{a} \cdot \mathbf{b} =
-
-a_1 b_1 + a_2 b_2 + \dots + a_n b_n.
-
-$$
-
-Example in $$ \mathbb{R}^3 $$
-
-$$
-
-\begin{pmatrix} 1\\ 3\\ -5 \end{pmatrix} \cdot \begin{pmatrix} 4\\ -2\\ -1
-\end{pmatrix} =
-
-1\cdot 4 + 3\cdot(-2) + (-5)\cdot(-1) =
-
-4 - 6 + 5 = 3.
-
-$$
-
-## Geometric definition
-
-If $$ \mathbf{a}, \mathbf{b} \in \mathbb{R}^n $$
-
-and $$ \theta $$
-
-is the angle between them, then
-
-$$ \mathbf{a} \cdot \mathbf{b} $$
-
-$$ \|\mathbf{a}\|\;\|\mathbf{b}\|\cos\theta, $$
-
-where $$ \|\mathbf{a}\| $$
-
-is the Euclidean length (norm) of $$ \mathbf{a} $$
-
-From this, you also get
-
-$$
-\mathbf{a} \cdot \mathbf{a} = \|\mathbf{a}\|^2,
-\quad
-\|\mathbf{a}\| = \sqrt{\mathbf{a} \cdot \mathbf{a}}.
-$$
-
-## Basic calculation rules
-
-Let $$ \mathbf{a}, \mathbf{b}, \mathbf{c} \in \mathbb{R}^n $$
-
-and $$ \lambda \in
-\mathbb{R} $$
-
-Then:
-
-- Commutativity:
-
- $$
- \mathbf{a} \cdot \mathbf{b} = \mathbf{b} \cdot \mathbf{a}.
- $$
-
-- Distributivity over addition:
-
- $$
- \mathbf{a} \cdot (\mathbf{b} + \mathbf{c}) =
- \mathbf{a} \cdot \mathbf{b} + \mathbf{a} \cdot \mathbf{c}.
- $$
-
-- Homogeneity (scalar multiplication in one slot):
-
- $$
- (\lambda \mathbf{a}) \cdot \mathbf{b} =
- \lambda (\mathbf{a} \cdot \mathbf{b}), \quad \mathbf{a} \cdot (\lambda \mathbf{b}) =
- \lambda (\mathbf{a} \cdot \mathbf{b}).
- $$
-
-- Positivity:
-
- $$
- \mathbf{a} \cdot \mathbf{a} \ge 0
- \quad \text{and} \quad
- \mathbf{a} \cdot \mathbf{a} = 0 \iff \mathbf{a} = \mathbf{0}.
- $$
-
-## Worked examples
-
-1. Simple 2D example
-
-Let
-
-$$
-\mathbf{u} =
-\begin{pmatrix}
-2\\
--1
-\end{pmatrix},
-\quad
-\mathbf{v} =
-\begin{pmatrix}
-3\\
-4
-\end{pmatrix}.
-$$
-
-Then
-
-$$ \mathbf{u} \cdot \mathbf{v}
-2\cdot 3 + (-1)\cdot 4 =
-6 - 4 =
-2.
-$$
-
-2. 4D example
-
-Let
-
-$$
-\mathbf{x} =
-\begin{pmatrix}
-2\\
-0\\
--3\\
-1
-\end{pmatrix},
-\quad
-\mathbf{y} =
-\begin{pmatrix}
--1\\
-3\\
-1\\
-2
-\end{pmatrix}.
-$$
-
-Then
-
-$$
-\mathbf{x} \cdot \mathbf{y} =
-2(-1) + 0(3) + (-3)(1) + 1(2) =
--2 + 0 - 3 + 2 =
--3.
-$$
-
-3. Using the geometric form to find an angle
-
-Let
-
-$$
-\mathbf{a} =
-\begin{pmatrix}
-1\\
-2
-\end{pmatrix},
-\quad
-\mathbf{b} =
-\begin{pmatrix}
-2\\
-1
-\end{pmatrix}.
-$$
-
-Compute
-
-$$
-
-\mathbf{a} \cdot \mathbf{b} = 1\cdot 2 + 2\cdot 1 4,
-
-$$
-
-$$
-
-\|\mathbf{a}\| =
-
-\sqrt{1^2 + 2^2} =
-
-\sqrt{5}, \quad \|\mathbf{b}\| =
-
-\sqrt{2^2 + 1^2} =
-
-\sqrt{5}.
-
-$$
-
-So
-
-$$
-
-\cos\theta =
-
-\frac{\mathbf{a} \cdot \mathbf{b}}{\|\mathbf{a}\|\,\|\mathbf{b}\|} =
-
-\frac{4}{\sqrt{5}\sqrt{5}} =
-
-\frac{4}{5},
-
-$$
-
-and hence
-
-$$
-\theta = \arccos\!\left(\frac{4}{5}\right).
-$$
diff --git a/content/linear_algebra/vectors/index.md b/content/linear_algebra/vectors/index.md
@@ -1,8 +0,0 @@
----
-title: Vectors
----
-
-- Vectors
-- Dot Product
-- Cross Product
-- Lines and Planes
diff --git a/content/linear_algebra/vectors/vectors.md b/content/linear_algebra/vectors/vectors.md
@@ -1,51 +0,0 @@
----
-title: Vectors
----
-
-A vector is an object that has both **magnitude** (length) and **direction** and
-is often represented as an ordered list of numbers, like components along
-coordinate axes. In $\mathbb{R}^n$, a vector is typically written as a column or
-row of $n$ real numbers and can be added to other vectors or scaled by real
-numbers.
-
-1. Basic vector notation (inline):
-
-A vector in 2D can be written as $\vec{v} = (v_1, v_2)$.
-
-2. Column vector (display):
-
-A column vector in 3D:
-
-$$
-\vec{v} = \begin{bmatrix} v_1 \\ v_2 \\ v_3 \end{bmatrix}
-$$
-
-3. Vector in $\mathbb{R}^n$:
-
-In general, a vector in $\mathbb{R}^n$ is
-
-$$
-\vec{v} = \begin{bmatrix} v_1 \\ v_2 \\ \dots \\ v_n \end{bmatrix}.
-$$
-
-4. Vector addition and scalar multiplication:
-
-If $\vec{u} = (u_1, u_2)$ and $\vec{v} = (v_1, v_2)$, then
-
-$$
-\vec{u} + \vec{v} = (u_1 + v_1,\; u_2 + v_2)
-$$
-
-and for a scalar $a$,
-
-$$
-a\vec{v} = (av_1,\; av_2).
-$$
-
-5. Magnitude (length) of a vector:
-
-The length of $\vec{v} = (v_1, v_2, v_3)$ is
-
-$$
-\|\vec{v}\| = \sqrt{v_1^2 + v_2^2 + v_3^2}.
-$$
diff --git a/content/machine_learning/concepts.md b/content/machine_learning/concepts.md
@@ -1,17 +0,0 @@
----
-title: Concepts
----
-
-## Concepts
-
-### ONNX
-
-Machine learning data are often stored in `.onnx` format
-
-[ONNX Documentation](https://onnx.ai/onnx/index.html)
-
-### Image processing kernal
-
-They are basically matrices
-
-[Image processing kernel](https://en.wikipedia.org/wiki/Kernel_(image_processing))
diff --git a/content/machine_learning/index.md b/content/machine_learning/index.md
@@ -1,3 +0,0 @@
----
-title: Machine Learning
----
diff --git a/content/memory_safety/Static Functions with Traits and Generics vs. Arc and Instance Methods in Rust.md b/content/memory_safety/Static Functions with Traits and Generics vs. Arc and Instance Methods in Rust.md
@@ -1,129 +0,0 @@
----
-title: Static Dispatch vs Dynamic Dispatch
----
-
-# Static Functions with Traits and Generics vs. Arc and Instance Methods in Rust
-
-Rust offers powerful mechanisms for achieving polymorphism and code reuse, primarily through **traits** and **generics**. When designing systems, developers often face a choice between using **static dispatch** (achieved with generics and trait bounds) and **dynamic dispatch** (achieved with trait objects, often combined with smart pointers like `Arc`). This document will compare these two approaches, highlighting their characteristics, trade-offs, and suitable use cases.
-## 1. Static Functions with Traits and Generics (Static Dispatch)
-
-This approach leverages Rust's generics and trait bounds to achieve polymorphism at compile time. The compiler generates specialized versions of functions or methods for each concrete type that satisfies the trait bounds. This process is known as **monomorphization**.
-
-### Characteristics:
-
-* **Compile-time polymorphism**: The specific method implementation is determined at compile time.
-* **Zero-cost abstractions**: There is no runtime overhead for dispatching methods, as the calls are direct.
-* **Performance**: Generally results in faster code due to direct function calls and opportunities for compiler optimizations (e.g., inlining).
-* **Type safety**: All type checks are performed at compile time.
-* **Larger binary size**: Monomorphization can lead to code bloat if a generic function is used with many different types, as a separate version of the code is generated for each type.
-* **No runtime type flexibility**: The types must be known at compile time. You cannot store a collection of different types that implement the same trait without using trait objects.
-
-### Example:
-
-```rust
-trait Printable {
- fn print(&self);
-}
-
-struct Dog { name: String }
-impl Printable for Dog {
- fn print(&self) {
- println!("Dog: {}", self.name);
- }
-}
-
-struct Cat { lives: u8 }
-impl Printable for Cat {
- fn print(&self) {
- println!("Cat with {} lives", self.lives);
- }
-}
-
-// Generic function using static dispatch
-fn print_animal<T: Printable>(animal: &T) {
- animal.print();
-}
-
-fn main() {
- let dog = Dog { name: "Buddy".to_string() };
- let cat = Cat { lives: 9 };
-
- print_animal(&dog);
- print_animal(&cat);
-}
-```
-
-## 2. Arc and Instance Methods (Dynamic Dispatch with Trait Objects)
-
-This approach uses **trait objects** (e.g., `dyn Trait`) combined with smart pointers like `Arc` (Atomically Reference Counted) to achieve polymorphism at runtime. `Arc` is used for shared ownership across multiple threads. When you have an `Arc<dyn Trait>`, you are essentially holding a pointer to a type that implements `Trait`, but the exact concrete type is not known at compile time. The method calls are resolved via a **vtable** (virtual table) at runtime.
-
-### Characteristics:
-
-* **Runtime polymorphism**: The specific method implementation is determined at runtime.
-* **Runtime overhead**: There is a small overhead for method dispatch due to vtable lookups.
-* **Flexibility**: Allows storing collections of different types that implement the same trait, and working with types whose concrete identity is not known until runtime.
-* **Smaller binary size**: Avoids code bloat as only one version of the code is generated for the trait object.
-* **Type safety**: Type checks are still performed, but the specific concrete type is not known until runtime.
-* **Sized constraint**: Trait objects must be
-sized, meaning they must have a known size at compile time. Since `dyn Trait` itself is unsized, it must always be behind a pointer (like `&`, `Box`, `Rc`, or `Arc`).
-* **Shared ownership and thread safety**: `Arc` provides shared, thread-safe ownership, making it suitable for scenarios where multiple parts of your program (potentially across threads) need to own and interact with the same trait object.
-
-### Example:
-
-```rust
-use std::sync::Arc;
-
-trait Printable {
- fn print(&self);
-}
-
-struct Dog { name: String }
-impl Printable for Dog {
- fn print(&self) {
- println!("Dog: {}", self.name);
- }
-}
-
-struct Cat { lives: u8 }
-impl Printable for Cat {
- fn print(&self) {
- println!("Cat with {} lives", self.lives);
- }
-}
-
-fn main() {
- let dog = Arc::new(Dog { name: "Buddy".to_string() });
- let cat = Arc::new(Cat { lives: 9 });
-
- // We can put different types implementing Printable into a collection of Arc<dyn Printable>
- let animals: Vec<Arc<dyn Printable>> = vec![dog, cat];
-
- for animal in animals {
- animal.print(); // Dynamic dispatch occurs here
- }
-}
-```
-
-## 3. Trade-offs and When to Use Which
-
-| Feature | Static Functions with Traits and Generics (Static Dispatch) | Arc and Instance Methods (Dynamic Dispatch) |
-| :------------------ | :---------------------------------------------------------- | :---------------------------------------------------------- |
-| **Polymorphism** | Compile-time | Runtime |
-| **Performance** | Higher (zero-cost abstraction, direct calls) | Lower (vtable lookup overhead) |
-| **Binary Size** | Potentially larger (monomorphization/code bloat) | Potentially smaller (single code version for trait object) |
-| **Flexibility** | Less (types must be known at compile time) | More (can work with unknown types at runtime, heterogeneous collections) |
-| **Type Safety** | Compile-time enforced | Compile-time enforced (but concrete type unknown at runtime) |
-| **Ownership** | Borrowed or owned values | Shared, thread-safe ownership with `Arc` |
-| **Use Cases** | Performance-critical code, when types are known at compile time, libraries where API stability is less critical, or when you need to avoid runtime overhead. |
-| | Building heterogeneous collections, when working with external data or configurations where types are determined at runtime, plugin architectures, or when shared, thread-safe ownership is required. |
-
-## Conclusion
-
-The choice between static functions with traits/generics and `Arc` with instance methods (trait objects) in Rust depends heavily on the specific requirements of your project. Static dispatch offers superior performance and compile-time guarantees but sacrifices runtime flexibility and can lead to larger binaries. Dynamic dispatch, on the other hand, provides immense flexibility and smaller binary sizes at the cost of a minor runtime performance overhead. Understanding these trade-offs is crucial for writing efficient, maintainable, and idiomatic Rust code.
-
-## References
-
-[1] The Rust Programming Language - Generic Types, Traits, and Lifetimes: [https://doc.rust-lang.org/book/ch10-00-generics.html](https://doc.rust-lang.org/book/ch10-00-generics.html)
-[2] Trait Objects and Dynamic Dispatch - The Rust How-to Book: [https://john-cd.com/rust_howto/language/trait_objects_and_dynamic_dispatch.html](https://john-cd.com/rust_howto/language/trait_objects_and_dynamic_dispatch.html)
-[3] Understanding Box<dyn Trait> in Rust: Dynamic Dispatch Done Right: [https://medium.com/@adamszpilewicz/understanding-box-dyn-trait-in-rust-dynamic-dispatch-done-right-4ebc185d4b40](https://medium.com/@adamszpilewicz/understanding-box-dyn-trait-in-rust-dynamic-dispatch-done-right-4ebc185d4b40)
-[4] Rust Static vs. Dynamic Dispatch: [https://softwaremill.com/rust-static-vs-dynamic-dispatch/](https://softwaremill.com/rust-static-vs-dynamic-dispatch/)
diff --git a/content/memory_safety/arc.md b/content/memory_safety/arc.md
@@ -1,50 +0,0 @@
----
-title: Arc
----
-
-In Rust, Arc is a smart pointer type that provides thread-safe, shared ownership
-of data on the heap. It’s used when multiple threads need to own the same value
-at the same time.
-
-## Core idea
-
-- Arc stands for Atomically Reference Counted.
-
-- Type: std::sync::Arc<T>.
-
-- Cloning an Arc<T> creates another handle to the same data and increments an
- internal atomic reference count.
-
-- When the last Arc<T> for a value is dropped, the data is deallocated.
-
-### Question: What "Atomically" actually mean in Arc?
-
-The "atomically" refers specifically to how the reference count itself is
-incremented and decremented, not to blocking other threads from reading.
-
-An atomic operation is a CPU-level instruction that completes entirely as a
-single, indivisible step, with no possibility of another thread observing it in
-a half-finished state. So when two threads clone or drop an Arc simultaneously,
-the reference count updates cannot interleave or corrupt each other.
-
-### Question: Does it mean that "another thread cannot read [the reference count] while one thread is reading it."?
-
-No, this is the description of a mutex/lock, not an atomic operation.
-
-The key distinction between Arc and Mutex/Lock:
-
-| Mechanism | How it works | Blocking? |
-| :------------------- | :----------------------------------------------------------------- | :---------------------------------- |
-| **Mutex** | Only one thread accesses data at a time; others wait | Yes — threads are blocked |
-| **Atomic operation** | The CPU guarantees the operation is indivisible; no waiting needed | No — threads don't block each other |
-
-Atomic operations use special CPU instructions (like `fetch_add` and
-`fetch_sub`) that make the increment/decrement happen in one uninterruptible
-step. Multiple threads can operate concurrently — they just can't _partially_
-observe each other's changes.[^3]
-
-### Question: What Arc does NOT guarantee?
-
-Arc only makes the **reference count** thread-safe, it does not make the
-underlying datat T thread-safe. That's why is you need multiple thread to mutate
-the shared data, you still need `Arc<Mutex<T>>` or `Arc<RwLock<T>>`
diff --git a/content/memory_safety/atomic.md b/content/memory_safety/atomic.md
@@ -1,84 +0,0 @@
----
-title: Atomic
----
-
-In software engineering, **atomic** comes from the Greek word _atomos_, meaning
-**indivisible** — and that's exactly what it means in code. An atomic operation
-is one that either fully completes or hasn't happened at all; it can never be
-observed in a half-finished state by another thread.[^3][^5]
-
-## Why It Matters in Concurrency
-
-Without atomicity, even a simple operation like `x += 1` compiles down to
-multiple CPU instructions (load, add, store), which can be interrupted mid-way
-by another thread. This creates **race conditions** where two threads read the
-same value, both increment it, and one update is silently lost. Atomic
-operations solve this by guaranteeing the entire read-modify-write cycle happens
-as one uninterruptible unit.[^5][^6]
-
-On a multi-core CPU, when a core begins an atomic operation, it pauses memory
-access from other cores for that location until the operation completes.[^4]
-
-## Atomics in Rust
-
-Rust exposes atomic operations through the `std::sync::atomic` module. The types
-all start with `Atomic`, for example:[^15]
-
-- `AtomicBool` — atomic boolean
-- `AtomicI32`, `AtomicUsize` — atomic integers
-- `AtomicPtr` — atomic pointer
-
-Common operations on these types include:[^3]
-
-- **`load`** — reads the value atomically
-- **`store`** — writes a value atomically
-- **`fetch_add`** — atomically increments and returns the old value
-- **`compare_exchange`** — atomically checks if a value matches an expectation,
- and only then replaces it
-
-## Memory Ordering
-
-Every atomic operation in Rust requires an `Ordering` argument (e.g., `Relaxed`,
-`Acquire`, `Release`, `SeqCst`). This controls how the CPU and compiler may
-reorder instructions around the atomic operation relative to other memory
-accesses — a subtle but critical detail for correctness in concurrent code. Rust
-inherits this memory model from C++20.[^9][^10]
-
-## Role in the Ecosystem
-
-Atomic types are the **lowest-level building block** for concurrency in Rust.
-Higher-level primitives like `Mutex` and `RwLock` are themselves implemented
-using atomic operations under the hood.[^3]
-<span style="display:none">[^1][^11][^12][^13][^14][^2][^7][^8]</span>
-
-<div align="center">⁂</div>
-
-[^1]: https://doc.rust-lang.org/std/sync/atomic/
-
-[^2]: https://doc.rust-lang.org/nomicon/atomics.html
-
-[^3]: https://marabos.nl/atomics/atomics.html
-
-[^4]: https://leapcell.io/blog/rust-atomics-explained
-
-[^5]: https://whenderson.dev/blog/implementing-atomics-in-rust/
-
-[^6]: https://stackoverflow.com/questions/53587866/what-is-the-difference-between-this-atomic-rust-code-and-its-non-atomic-coun
-
-[^7]: https://www.reddit.com/r/rust/comments/hskm11/having_a_hard_time_understanding_atomic/
-
-[^8]: http://blog.rustbr.org/entendendo-atomicos/
-
-[^9]: https://cfsamsonbooks.gitbook.io/explaining-atomics-in-rust
-
-[^10]: https://www.youtube.com/watch?v=rMGWeSjctlY
-
-[^11]: https://blog.rustbr.org/en/understanding-atomics/
-
-[^12]: https://www.reddit.com/r/rust/comments/1ksqo9i/mastering_rust_atomic_types_a_guide_to_safe/
-
-[^13]: https://dev.to/leapcell/rust-concurrency-atomic-explained-58cl
-
-[^14]: https://rust-lang.guide/guide/learn-async-rust/rust-atomics-and-locs.html
-
-[^15]: https://web.mit.edu/rust-lang_v1.25/arch/amd64_ubuntu1404/share/doc/rust/html/std/sync/atomic/index.html
diff --git a/content/memory_safety/borrowing_rules.md b/content/memory_safety/borrowing_rules.md
@@ -1,74 +0,0 @@
----
-title: Borrowing Rules
----
-
-Rust enforces two hard rules at compile time:
-
-1. You can have any number of immutable references (&T) at the same time
-
-2. You can have only one mutable reference (&mut T) at a time — and when a
- mutable reference is active, no immutable references may exist either
-
-### String example:
-
-#### Question: Why does rust only allow one muatble reference at a time?
-
-A String in Rust is heap-allocated and stores a pointer, a length, and a
-capacity internally. If you hold an immutable reference r1 to a String and then
-mutate it via a second mutable reference r2 — say by pushing characters — the
-String may reallocate its internal buffer to a new heap address. At that point,
-r1 would be pointing to freed memory, which is a dangling pointer and causes
-[undefined behavior](/memory_safety/segfault.md). Rust prevents this entirely at
-compile time.
-
-```rust
-let mut s = String::from("hello");
-
-let r1 = &s; // ✅ immutable borrow
-let r2 = &s; // ✅ another immutable borrow — fine!
-let r3 = &mut s; // ❌ compile error: can't borrow mutably while r1/r2 exist
-```
-
-### Furthermore:
-
-The single mutable reference rule actually solves several classes of bugs
-simultaneously:
-
-- Dangling pointers — mutation causes reallocation, invalidating old references
- (your example)
-
-- Data races — two threads mutating the same memory simultaneously leads to
- unpredictable results
-
-- [Iterator invalidation](iterator_invalidation.md) — modifying a collection
- while iterating over it (a common bug in C++ and Java)
-
-- Compiler optimisation safety — the compiler can safely optimise and even
- vectorise (SIMD) code because it knows no two mutable aliases can overlap
-
-## Mutable Reference [Lifetime](/memory_safety/lifetime.md) Is Scoped
-
-The borrow checker is smart enough to track when a reference's last use is, not
-just its scope. This means a mutable reference can be created once the immutable
-ones are no longer actively used:
-
-```rust
-let mut s = String::from("hello");
-
-let r1 = &s;
-let r2 = &s;
-println!("{r1}, {r2}"); // r1 and r2 last used here — they're effectively dropped
-
-let r3 = &mut s; // ✅ now safe, no active immutable refs
-r3.push_str(", world");
-```
-
-This feature is called
-[Non-Lexical Lifetimes (NLL)](/memory_safety/non_lexical_lifetimes.md) and was
-introduced to make Rust's borrow checker less restrictive while keeping it safe.
-
-## References:
-
-[Why Does Rust Enforce the “One Mutable or Many Immutable References” Rule in Single-Threaded Programs?](https://users.rust-lang.org/t/why-does-rust-enforce-the-one-mutable-or-many-immutable-references-rule-in-single-threaded-programs/121017/2)
-
-[The Problem With Single-threaded Shared Mutability](https://manishearth.github.io/blog/2015/05/17/the-problem-with-shared-mutability/)
diff --git a/content/memory_safety/box.md b/content/memory_safety/box.md
@@ -1,40 +0,0 @@
----
-title: Box
----
-
-All values in Rust are stack allocated by default. Box is basically a pointer to
-a heap allocated value. When Box<T> goes out of scope, its destructor is called,
-freeing up the memory on the heap.
-
-### Question: when / why to do box in rust?
-
-We use Box whenever we need to put something on the heap and have a pointer to
-it. Because by default Rust values live on the stack, and their size must be
-known at compile time.
-
-## Use Box<T> when:
-
-- Recursive types - without Box the type woul dhave inifnite size:
-
-```rust
-enum List { Cons(i32, Box<List>), Nil }
-```
-
-- Types whose size can’t be known at compile time but you need a sized handle:
-
-Trait objects like `Box<dyn Error>` or `Box<dyn MyTrait>` put the unknown‑sized
-value on the heap and give you a pointer of known size so functions can return
-or store them easily.
-
-- Moving large data efficiently:
-
-If you have a big struct and want to move ownership around without copying all
-its bytes on the stack, you can store it in a Box and move just the pointer
-(cheap copy).
-
-## Don't need Box
-
-For example, Vec<T>, String, HashMap<K, V>, etc., these already have a pointer,
-so dont need Box for them.
-
-[Reference](https://rustwiki.org/en/rust-by-example/std/box.html)
diff --git a/content/memory_safety/drop.md b/content/memory_safety/drop.md
@@ -1,106 +0,0 @@
----
-title: Drop
----
-
-Rust’s Drop trait lets a type run custom cleanup code when a value is about to
-go out of scope.
-
-The drop method is called automatically when an object goes out of scope. It is
-part of the std library in Rust. Box, Vec, String, File, and Process are some
-examples of types that implement the Drop trait to free resources.
-
-## Examples
-
-There are three way to manage drop:
-
-1. **Inner block (idiomatic, no extra types)**
-
-Put everything that holds the DB in a nested scope. When the block ends, service
-and db are dropped automatically, then you delete the file.
-
-```rust
-let path = create_test_db_path();
-{
- let db = Arc::new(open_test_db(&path)); let
- service = setup_test_service(db.clone()).await.unwrap();
- // … test body …
-}
-cleanup_test_db(path);
-```
-
-No manual drop—the closing } defines when the handles go away.
-
-2. **RAII “temp file” guard**
-
-Use a small type that implements Drop and calls remove_file in drop, and declare
-it before the `Arc<DB>` so the DB is dropped first (last declared → dropped
-first):
-
-```rust
-let path = create_test_db_path();
-let _tmp = TempRedb::new(path.clone()); // deletes path on Drop
-let db = Arc::new(open_test_db(&path));
-// … when function/block ends: db drops, then _tmp deletes the file
-```
-
-Same idea as tempfile::TempPath / NamedTempFile in the tempfile crate: the guard
-runs cleanup when it goes out of scope, after earlier locals are dropped.
-
-3. **Manual cleanup before the end of scope**
-
-```rust
-use std::fs;
-use std::path::Path;
-
-struct TestDb {
- path: String,
-}
-
-impl TestDb {
- fn new(path: &str) -> Self {
- println!("Opening DB at {}", path);
- Self { path: path.to_string() }
- }
-}
-
-impl Drop for TestDb {
- fn drop(&mut self) {
- println!("Dropping TestDb for {}", self.path);
- // In real code: close DB connection, etc.
- }
-}
-
-fn cleanup_db(path: &str) {
- if Path::new(path).exists() {
- fs::remove_file(path).unwrap();
- println!("Cleaned up DB file: {}", path);
- }
-}
-
-fn main() {
- let path = "test.db";
-
- let db = TestDb::new(path); // declared first → dropped last
- let _guard = std::fs::File::open(path).unwrap_or_else(|_| {
- // create dummy file for demo
- fs::write(path, "").unwrap();
- fs::File::open(path).unwrap()
- });
-
- // ... test body ...
-
- // Explicitly drop `db` *before* `_guard`, even though `db` was declared first
- drop(db); // ← explicit drop call
-
- // Now safe to clean up file (since db is gone)
- cleanup_db(path);
-
- // `_guard` is dropped here automatically at end of scope
-}
-```
-
-────────────────────────────────────────
-
-Summary: Prefer a block for tests; use a guard (or tempfile) if you want cleanup
-bundled in one place. You only need explicit drop when you want a non-default
-order without restructuring scopes.
diff --git a/content/memory_safety/index.md b/content/memory_safety/index.md
@@ -1,16 +0,0 @@
----
-title: Memory Safety
----
-
-Notes on Rust's memory safety model, covering ownership, borrowing, and
-concurrency primitives.
-
-## Topics
-
-- **Ownership & Borrowing** — borrowing rules, non-lexical lifetimes, shared and
- exclusive references, interior mutability
-- **Smart Pointers** — `Box`, `Arc`, and the `Drop` trait
-- **Concurrency** — `Mutex`, `RwLock`, atomics, spinlocks, `Send`/`Sync` traits,
- and synchronization/concurrency primitives
-- **Memory Model** — stack vs. heap, segfaults, iterator invalidation
-- **Practical Patterns** — connection pooling, PostgreSQL connections
diff --git a/content/memory_safety/interior_mutability.md b/content/memory_safety/interior_mutability.md
@@ -1,52 +0,0 @@
----
-title: Interior Mutability
----
-
-Interior mutability in Rust is a pattern that lets you change the internal state
-of a value through a shared (immutable) reference, something that normal
-borrowing rules do not allow.
-
-## Core idea
-
-- Normally, &T means “shared, read-only” and &mut T means “unique, writable”.
-
-- A type has interior mutability if you can call methods that mutate its
- internal data even when you only have &self.
-
-- This is implemented by special wrapper types that enforce safety at runtime
- instead of purely at compile time (often via unsafe inside but a safe public
- API).
-
-## Common types used
-
-- Rust’s standard library provides several types that rely on interior
- mutability:
-
-- Cell<T> and RefCell<T> in std::cell for single-threaded code.
-
-- Mutex<T> and RwLock<T> in std::sync for synchronized mutation across threads.
-
-- Atomic types like AtomicUsize for lock-free concurrent mutation.
-
-## Example
-
-A simple example is a struct that keeps a usage counter even when you only pass
-around
-
-```rust
-use std::cell::Cell;
-
-struct Counter {
- value: Cell<u32>,
-}
-
-impl Counter {
- fn inc(&self) {
- self.value.set(self.value.get() + 1);
- }
-}
-```
-
-Here, inc takes &self but still mutates the internal value.
-
-[Reference](https://mara.nl/atomics/basics.html#interior-mutability)
diff --git a/content/memory_safety/iterator_invalidation.md b/content/memory_safety/iterator_invalidation.md
@@ -1,56 +0,0 @@
----
-title: Iterator Invalidation
----
-
-Iterator invalidation is a bug where an iterator becomes invalid because the
-underlying collection it references is modified during iteration. Rust prevents
-this at compile time through its borrow checker, unlike languages like C++ where
-it causes undefined behaviour or crashes at runtime.
-
-When you iterate over a collection (e.g., a Vec), the iterator holds a reference
-to that collection's memory. If the collection is modified — say, by pushing a
-new element — the Vec may need to reallocate its memory to grow, freeing the old
-memory and leaving the iterator pointing to a dangling address. At best this
-causes a crash (segfault); at worst it silently corrupts memory.
-
-Rust's borrow checker enforces that you cannot hold an immutable borrow and a
-mutable borrow at the same time. When you call v.iter(), Rust creates an
-immutable borrow on the entire collection. Any attempt to call v.push() or
-v.remove() during the loop requires a mutable borrow — which the compiler
-outright refuses to compile:
-
-```rust
-fn main() {
- let mut vec = vec![1, 2, 3, 4, 5];
- for elem in &vec {
- vec.push(elem * 2); // COMPILE ERROR: cannot borrow `vec` as mutable
- // because it is also borrowed as immutable
- }
-}
-```
-
-The error is caught at compile time, not at runtime.
-
-## What You _Can_ Do Safely
-
-| Pattern | Allowed? | Why |
-| :--------------------------------------- | :------- | :----------------------------------------- |
-| `v.iter()` + read elements | ✅ | Multiple immutable borrows are fine |
-| `v.iter_mut()` + modify each element | ✅ | One mutable borrow, no structural changes |
-| `v.iter()` + `v.push()` in the same loop | ❌ | Immutable + mutable borrow conflict |
-| Collect indices, then modify after loop | ✅ | Iterator is dropped before mutation begins |
-
-A common safe workaround is to use `retain` or collect the changes you need into
-a separate `Vec`, then apply them after the loop ends — by which point the
-iterator has been dropped and the borrow is released.
-
-## It's effectively threaded
-
-> Aliasing with mutability in a sufficiently complex, single-threaded program is
-> effectively the same thing as accessing data shared across multiple threads
-> without a lock
-
-> My intuition is that code far away from my code might as well be in another
-> thread, for all I can reason about what it will do to shared mutable state.
-
-[Reference](http://manishearth.github.io/blog/2015/05/17/the-problem-with-shared-mutability/)
diff --git a/content/memory_safety/mutex.md b/content/memory_safety/mutex.md
@@ -1,209 +0,0 @@
----
-title: Mutex
----
-
-Mutex is the most commonly used tool for sharing (mutable) data between threads.
-Mutex is short for "Mutural Exclusion". It uses UnsafeCell under the hood, and
-provides a safe interface MutexGuard for threads to access the data with
-"Interior Mutability". It is a thread-safe wrapper around some data T that
-ensures only one thread can access that data at a time
-
-```
- +-----------------------------+
- | Arc<Mutex<T>> |
- | (shared across threads) |
- | |
- | +---------------------+ |
- | | Mutex<T> | |
- | | (lock + the data) | |
- | | | |
- | | +-------------+ | |
- | | | value T | | |
- | | +-------------+ | |
- | +----------^----------+ |
- +--------------|--------------+
- |
- clones of Arc |
- (Arc::clone(&mutex)) |
- sent to threads |
- |
- +------------------+------------------+
- | | |
- v v v
-+----------------+ +----------------+ +----------------+
-| thread 1 | | thread 2 | | thread 3 |
-| | | | | |
-| lock() | | lock() | | lock() |
-| | | | | | | | |
-| v | | v | | v |
-| MutexGuard<T> | | MutexGuard<T> | | MutexGuard<T> |
-| (exclusive | | (exclusive | | (exclusive |
-| access) | | access) | | access) |
-+----------------+ +----------------+ +----------------+
-
-Only ONE MutexGuard can exist at a time:
-- When a thread calls lock(), it blocks until it gets a MutexGuard.
-- While a thread holds the guard, others wait.
-- When the guard is dropped (goes out of scope), the lock is released.
-```
-
-## Basic Example:
-
-```rust
-use std::sync::Mutex;
-
-fn main() {
- let my_mutex = Mutex::new(5);
-
- {
- let mut guard = my_mutex.lock().unwrap();
- *guard = 6; // modify protected data
- } // guard dropped here, mutex unlocked
-
- println!("{:?}", my_mutex); // prints: Mutex { data: 6 }
-}
-```
-
-## Poisoning
-
-- If a thread panics while holding the lock, the mutex becomes poisoned to
- signal that the data may be in an inconsistent state.
-
-- After that, `.lock()` returns an `Err(PoisonError)` instead of
- `Ok(MutexGuard)`; you often handle this with `unwrap()` (propagating the
- panic) or `unwrap_or_else` to recover.
-
-### Question: whats the difference between Arc and Mutex?
-
-Arc and Mutex solve different problems: Arc gives shared ownership of data
-across threads, while Mutex controls exclusive access to data (usually for
-mutation) so only one thread touches it at a time.
-
-## How Arc and Mutex are used together
-
-A very common pattern in Rust is Arc<Mutex<T>>, where:
-
-- Arc lets many threads share ownership of the same Mutex<T>.
-
-- Mutex ensures that each access to T is exclusive, so concurrent mutation is
- safe.
-
-## Quick example intuition
-
-If you want several threads to read the same configuration that never changes,
-you typically use Arc<Config>.
-
-If you want several threads to increment a shared counter, you typically use
-Arc<Mutex<u32>> so they can all own the counter and take turns mutating it
-safely.
-
-## On using `std::sync::Mutex` and `tokio::sync::Mutex`
-
-> Note that std::sync::Mutex and not tokio::sync::Mutex is used to guard the
-> HashMap. A common error is to unconditionally use tokio::sync::Mutex from
-> within async code. An async mutex is a mutex that is locked across calls to
-> .await.
-
-> A synchronous mutex will block the current thread when waiting to acquire the
-> lock. This, in turn, will block other tasks from processing. Switching to
-> tokio::sync::Mutex will cause the task to yield control back to the executor,
-> but this will usually not help with performance as the asynchronous mutex uses
-> a synchronous mutex internally.
-
-> As a rule of thumb, using a synchronous mutex from within asynchronous code is
-> fine as long as contention remains low and the lock is not held across calls
-> to .await.
-
-### Question: What does “Locked across calls to .await” mean?
-
-It means: you acquired the mutex lock before an `.await`, and you still hold
-that lock while the future is suspended at that `.await` and possibly resumed
-later on another poll.
-
-In async Rust, every .await is a point where your function can be suspended and
-other tasks can run on the same thread.
-
-Code that hold lock across the await:
-
-```rust
-async fn f(m: &std::sync::Mutex<Data>) {
- let mut guard = m.lock().unwrap(); // lock acquired here
-
- do_something_async().await; // <-- await while still holding `guard`
-
- guard.value += 1; // lock released only when `guard` is dropped
-}
-```
-
-the Mutex remains locked for the entire time between taking guard and dropping
-it, including the whole period when the function is paused at
-do_something_async().await. The lock is “held across the await”.
-[How to Use async Rust Without Blocking the RUntime](https://oneuptime.com/blog/post/2026-01-07-rust-async-without-blocking/view)
-
-By contrast, code that does not hold the lock across an .await acquires and
-releases it entirely before the first .await:
-
-```rust
-async fn g(m: &std::sync::Mutex<Data>) {
- {
- let mut guard = m.lock().unwrap();
- guard.counter += 1;
- // lock is dropped at the end of this block
- } // <--- lock released here
-
- do_something_async().await; // no lock held during await
-}
-```
-
-Here the critical section is synchronous and short, so using std::sync::Mutex is
-fine because the lock is never held while the task is suspended at .await.
-[Sync mutex in async program](https://users.rust-lang.org/t/sync-mutex-in-async-program/66118)
-
-A quick way to think about it:
-
-- “Held/locked across `.await`” = the lifetime of your `MutexGuard` spans over
- an `.await` expression.
-
-- “Not held across `.await`” = you drop the guard (end its scope) before you hit
- any `.await`.
-
-However, the `tokio::sync::Mutex` is designed to be held across `await` points:
-
-```rust
-async fn g(m: &tokio::sync::Mutex<Data>) {
- let mut guard = m.lock().await; // yields if needed, no thread block
- do_something_async().await; // still holding the lock
- guard.counter += 1;
-} // lock released when guard is dropped
-```
-
-is legal in terms of the runtime: you’re not blocking a worker thread while
-waiting for I/O, other tasks can still run.
-
-So “locked across calls to .await” means:
-
-- With std::sync::Mutex: strongly discouraged; can stall threads and deadlock.
-
-- With tokio::sync::Mutex: allowed and supported; the runtime knows how to park
- and wake tasks that are waiting on the async mutex.
-
-#### When to choose which
-
-- Use std::sync::Mutex in async code when:
-
- - You only lock for very short, synchronous work.
-
- - You always drop the guard before any .await.
-
-- Use tokio::sync::Mutex when:
-
- - You really need to keep the lock across one or more .awaits, or
-
- - The critical section involves async operations that can’t be easily
- refactored to be purely synchronous.
-
-```
-## Reference
-
-[Spinlock Considered Harmful](https://matklad.github.io/2020/01/02/spinlocks-considered-harmful.html)
-```
diff --git a/content/memory_safety/rwlock_pattern.md b/content/memory_safety/rwlock_pattern.md
@@ -1,84 +0,0 @@
----
-title: RWLock Pattern
----
-
-The RWLock (Readers-Writer Lock) pattern is a concurrency synchronization
-primitive that allows multiple threads to read shared data simultaneously, but
-requires exclusive access for any write operation. It's ideal for scenarios
-where reads are frequent and writes are rare, as it avoids the unnecessary
-serialization that a plain Mutex would impose.
-
-## Core Rules
-
-- Multiple readers can hold the lock at the same time — reads are non-exclusive
-
-- Only one writer can hold the lock at a time — writes are exclusive
-
-- A write lock blocks all readers and other writers until it is released
-
-- Once a writer is active, new readers must wait
-
-## Rust RwLock Example
-
-Rust provides std::sync::RwLock<T> in its standard library. Here's a practical
-example with multiple reader threads and one writer thread:
-
-```rust
-use std::sync::{Arc, RwLock};
-use std::thread;
-
-fn main() {
- // Wrap data in Arc<RwLock<T>> for shared ownership across threads
- let data = Arc::new(RwLock::new(0u32));
-
- // --- Writer thread: exclusive access ---
- let data_writer = Arc::clone(&data);
- let writer = thread::spawn(move || {
- let mut w = data_writer.write().unwrap(); // blocks until no readers/writers
- *w += 42;
- println!("Writer set value to: {}", *w);
- }); // write lock dropped here
-
- writer.join().unwrap();
-
- // --- Multiple reader threads: concurrent access ---
- let reader_handles: Vec<_> = (0..4).map(|i| {
- let data_reader = Arc::clone(&data);
- thread::spawn(move || {
- let r = data_reader.read().unwrap(); // multiple readers can hold this simultaneously
- println!("Reader {} sees value: {}", i, *r);
- })
- }).collect();
-
- for handle in reader_handles {
- handle.join().unwrap();
- }
-}
-```
-
-### Output (reader order may vary):
-
-```text
-Writer set value to: 42
-Reader 0 sees value: 42
-Reader 2 sees value: 42
-Reader 1 sees value: 42
-Reader 3 sees value: 42
-```
-
-## When to Use vs. Mutex
-
-RwLock is only better than Mutex when you have many concurrent readers competing
-at the same time. For a single reader or low concurrency, an uncontended RwLock
-is no faster than a Mutex because readers still acquire the lock and update
-internal state. A good rule of thumb: if your workload is read-heavy (e.g., 90%+
-reads) with rare writes, RwLock gives a meaningful throughput boost.
-
-## Watch Out: Lock Poisoning
-
-In Rust, if a thread panics while holding a write lock, the RwLock becomes
-poisoned. Subsequent calls to .read() or .write() will return an Err, which you
-handle via .unwrap() or .into_inner(). This is a safety feature to prevent
-access to potentially corrupt data.
-
-[RWLock](https://dev-doc.rust-lang.org/beta/std/sync/struct.RwLock.html)
diff --git a/content/memory_safety/segfault.md b/content/memory_safety/segfault.md
@@ -1,90 +0,0 @@
----
-title: Segfault
----
-
-A segmentation fault (segfault) is a runtime error that occurs when a program
-tries to access a memory location it is not permitted to access — such as
-reading or writing outside its allocated memory bounds. The operating system
-catches this illegal access and terminates the offending process, sending a
-SIGSEGV signal on Unix-like systems or raising a STATUS_ACCESS_VIOLATION
-exception on Windows.
-
-## How It Happens
-
-Program memory is divided into segments — text (instructions), data (global
-variables), stack (local variables), and heap (dynamically allocated memory). A
-segfault occurs when a reference falls outside the segment where a variable
-resides, or when a write is attempted to a read-only segment. The most common
-causes are:
-
-- Null pointer dereference — accessing memory at address 0x0
-
-- Buffer overflow — reading/writing past the end of an array
-
-- Dangling pointer — using a pointer to memory that has already been freed
-
-- Stack overflow — infinite recursion exhausting the stack
-
-## Rust and Segfaults
-
-Rust's ownership and borrow checker system is specifically designed to eliminate
-segfaults in safe code at compile time. According to the Rust team, a Rust
-program can only segfault in two scenarios: you used unsafe code that violates
-memory safety guarantees, or the Rust compiler itself has a bug.
-
-### Triggering a segfault with unsafe Rust
-
-The most direct way is dereferencing a raw null or invalid pointer inside an
-unsafe block:
-
-```rust
-fn main() {
- // Dereference an invalid memory address — instant segfault
- unsafe { *(0x1 as *mut i32) = 1 };
-}
-```
-
-This writes to memory address 0x1, which is not mapped, causing the OS to send
-SIGSEGV.
-
-### Stack overflow via infinite recursion
-
-```rust
-fn recurse() {
- recurse(); // infinite recursion → stack overflow → crash
-}
-
-fn main() {
- recurse();
-}
-```
-
-Rust's runtime catches this and typically raises SIGABRT with a "thread has
-overflowed its stack" message rather than a raw SIGSEGV, but it is the same
-underlying mechanism.
-
-### Writing to read-only memory via unsafe
-
-```rust
-fn main() {
- let x: &str = "hello"; // stored in read-only memory
- let ptr = x.as_ptr() as *mut u8;
- unsafe {
- *ptr = b'H'; // writing to read-only segment → segfault
- }
-}
-```
-
-## Safe Rust vs. Unsafe Rust
-
-| Scenario | Safe Rust | `unsafe` Rust |
-| :----------------------- | :------------------------------------ | :----------------------------------- |
-| Null pointer dereference | Impossible — `Option<T>` used instead | Possible with raw pointers |
-| Buffer overflow | Panics with bounds check | Possible with raw pointer arithmetic |
-| Dangling pointer | Prevented by borrow checker | Possible |
-| Stack overflow | Handled gracefully (abort) | Same behaviour |
-
-The key takeaway is that safe Rust **prevents segfaults by design** — the borrow
-checker enforces memory safety rules at compile time that languages like C/C++
-leave to the programmer. When you do need low-level control, `unsafe` blocks opt
-out of these guarantees and reintroduce the risk.
diff --git a/content/memory_safety/shared_and_exclusive_reference.md b/content/memory_safety/shared_and_exclusive_reference.md
@@ -1,13 +0,0 @@
----
-title: Shared and Exclusive Reference
----
-
-# Shared vs. Exclusive References in Rust
-
-In Rust, there are two kinds of references:
-
-- `&T` — a shared reference: multiple parties can hold one at the same time, but
- mutation is normally forbidden
-
-- `&mut T` — an exclusive reference: only one party can hold it, and mutation is
- allowed
diff --git a/content/memory_safety/spinlock.md b/content/memory_safety/spinlock.md
@@ -1,116 +0,0 @@
----
-title: Spinlock Mechanism
----
-
-A spinlock works by repeatedly "spinning" — checking an atomic `locked` flag in
-a tight loop until it successfully flips it from `false` to `true`. The key
-operations are:[^1]
-
-- **`lock()`** — atomically swaps `locked` from `false` to `true`. If `locked`
- is _already_ `true`, it keeps retrying (the "spin")
-- **`unlock()`** — stores `false` back into `locked`, releasing it for another
- thread
-
-```
-THREAD 1 THREAD 2
-────────────────────────────────────────────────────────────
-
- ┌──────────────────────────┐
- │ swap locked │
- │ false → true (acquire) │
- └────────────┬─────────────┘
-lock() │
- ▼
- ┌──────────────────────────┐ ┌──────────────────────────┐
- │ access protected data │ │ swap locked │
- └────────────┬─────────────┘ │ true → true (acquire) │
- │ └────────────┬─────────────┘
-unlock() │ │ (spinning...)
- ▼ ┌────────────▼─────────────┐
- ┌──────────────────────────┐ │ swap locked │
- │ store false in locked │ │ true → true (acquire) │
- │ (release) │ └────────────┬─────────────┘
- └────────────┬─────────────┘ │ (spinning...)
- │ │
- │ happens-before │
- └───────────────────────────────────▼
- ┌──────────────────────────┐
- │ swap locked │
- │ false → true (acquire) │ ← lock() succeeds
- └────────────┬─────────────┘
- lock() │
- ▼
- ┌──────────────────────────┐
- │ access protected data │
- └────────────┬─────────────┘
- unlock() │
- ▼
- ┌──────────────────────────┐
- │ store false in locked │
- │ (release) │
- └──────────────────────────┘
-```
-
-- Thread 1 (left column) acquires the lock immediately (swaps false → true),
- accesses the protected data, then releases by storing false.
-
-- Thread 2 (right column) spins repeatedly — each swap returns true → true,
- meaning the lock is still held — until Thread 1's unlock() stores false.
-
-- The happens-before arrow (the pink curved arrow in the original) is shown as
- the └──────────────────────────────────────────────────────▶ line connecting
- Thread 1's release to Thread 2's successful acquire. This is the critical
- guarantee that prevents concurrent data access.
-
-## Why the Second Thread is Blocked
-
-Looking at the diagram, Thread 2 (right column) attempts its own `lock()` call
-but keeps getting `true → true` swaps, meaning it sees the lock is still held.
-It can only proceed when Thread 1's `unlock()` stores `false` — at that point,
-Thread 2's next swap finally succeeds (`false → true`), and _only then_ does it
-access the protected data.[^2]
-
-This is the **happens-before relationship** mentioned in the text: Thread 1's
-`unlock()` (release) _happens before_ Thread 2's successful `lock()` (acquire),
-which guarantees that any memory writes Thread 1 made to the shared data are
-visible to Thread 2.[^3]
-
-## The Concrete Guarantee
-
-| Moment | Thread 1 | Thread 2 |
-| :----- | :----------------------------- | :--------------------------- |
-| T1 | Acquires lock (`false→true`) | — |
-| T2 | Accessing protected data | Spinning (sees `true→true`) |
-| T3 | Releases lock (stores `false`) | Still spinning |
-| T4 | Done | Acquires lock (`false→true`) |
-| T5 | — | Accessing protected data |
-
-The data access blocks (pink/maroon boxes in the diagram) are never active at
-the same time — Thread 2 is stuck spinning until Thread 1 is completely done and
-has unlocked. That's what "can't access the data concurrently" means: the
-spinlock enforces **serial, not parallel** access to that critical section.[^1]
-<span style="display:none">[^10][^11][^4][^5][^6][^7][^8][^9]</span>
-
-<div align="center">⁂</div>
-
-[^1]: https://www.shadecoder.com/topics/a-spinlock-a-comprehensive-guide-for-2025
-
-[^2]: https://wiki.osdev.org/Spinlock
-
-[^3]: https://en.wikipedia.org/wiki/Lock_(computer_science)
-
-[^4]: image.jpg
-
-[^5]: https://www.productteacher.com/quick-product-tips/understanding-mutual-exclusion-mutex
-
-[^6]: https://stackoverflow.com/questions/53919851/how-spinlock-prevents-the-process-to-be-interrupted
-
-[^7]: https://forums.swift.org/t/pitch-synchronous-mutual-exclusion-lock/69889
-
-[^8]: https://student.cs.uwaterloo.ca/~cs350/F19/notes/synchronization-2up.pdf
-
-[^9]: https://student.cs.uwaterloo.ca/~cs350/F20/notes/synchronization-2up.pdf
-
-[^10]: https://de.wikipedia.org/wiki/Spinlock
-
-[^11]: https://spcl.inf.ethz.ch/Teaching/2020-pp/lectures/PP-l17-BeyondLocks.pdf
diff --git a/content/memory_safety/stack-and-heap.md b/content/memory_safety/stack-and-heap.md
@@ -1,37 +0,0 @@
----
-title: Stack and Heap
----
-
-# Stack
-
-## Call Stack
-
-A region of memory the CPU uses automatically to track function calls, memory
-pointers, local variables, and return addresses
-
-## Rust-specific rule
-
-[Borrowing Rules](../memory_safety/borrowing_rules.md)
-
-# Heap
-
-The heap is a region of a computer's memory used for dynamic memory allocation —
-memory that is reserved and released at runtime rather than at compile time. It
-is one of the two main memory areas programs use (alongside the stack), and it
-gives programmers flexible control over how much memory to use and for how long.
-
-## Heap vs. Stack
-
-| Feature | Heap | Stack |
-| :---------------- | :--------------------------- | :-------------------- |
-| Allocation timing | Runtime (dynamic) | Compile time (static) |
-| Size | Large, flexible | Small, fixed |
-| Management | Manual or GC | Automatic (LIFO) |
-| Access scope | Global (anywhere in program) | Local to function |
-| Speed | Slightly slower | Faster |
-| Risk | Memory leaks, fragmentation | Stack overflow |
-
-The stack follows a strict last-in, first-out (LIFO) structure, making it fast
-but limited. The heap is more flexible but requires careful management — failure
-to deallocate memory causes **memory leaks**, where memory becomes permanently
-unavailable.
diff --git a/content/memory_safety/string_vs_str.md b/content/memory_safety/string_vs_str.md
@@ -1,90 +0,0 @@
----
-title: String vs Str
----
-
-Here is how `String` source code look like:
-
-```rust
-#[derive(PartialEq, PartialOrd, Eq, Ord)]
-#[stable(feature = "rust1", since = "1.0.0")]
-#[lang = "String"]
-pub struct String {
- vec: Vec<u8>,
-}
-```
-
-In Rust, String is implemented as a wrapper around a `Vec<u8>` because it is
-designed to be a dynamically growable, heap-allocated buffer of UTF-8 encoded
-bytes. The standard library leverages the existing `Vec` type to manage the
-string's underlying memory while enforcing specific character encoding rules.
-
-## The Stack Layout
-
-When you create a String, Rust places a small, fixed-size struct on the stack.
-On a 64-bit system, this struct is exactly 24 bytes and consists of three
-machine-word fields:
-
-- **Pointer**: A reference pointing to the memory address on the heap where the
- actual UTF-8 text data is stored.
-
-- **Length**: The amount of memory (in bytes) currently being used by the
- string's text.
-
-- **Capacity**: The total amount of heap memory currently allocated for the
- string.
-
-Because this control block is small, moving a String from one variable to
-another is incredibly fast. Rust only copies these 24 bytes of metadata on the
-stack rather than copying the actual text data on the heap.
-
-## String memory allocation
-
-- **p** is the address of the String object itself, so text keeps the same p the
- whole time because it is the same variable.
-
-- **ptr** is the address of the actual character buffer.
-
-- **len** is the current string size.
-
-- **capacity** is how much buffer space is available before another growth is
- needed.
-
-```rust
-fn main() {
- let mut text = String::new();
- println!("p: {:p} ptr: {:<15?} len: {:2}, capacity: {:2} '{}'", &text, text.as_ptr(), text.len(), text.capacity(), text);
-
- text.push_str("Hello foo!");
- println!("p: {:p} ptr: {:<15?} len: {:2}, capacity: {:2} '{}'", &text, text.as_ptr(), text.len(), text.capacity(), text);
-
- text.replace_range(6..9, "bar");
- println!("p: {:p} ptr: {:<15?} len: {:2}, capacity: {:2} '{}'", &text, text.as_ptr(), text.len(), text.capacity(), text);
-
- let temp = String::from("The black cat");
- println!("{temp}");
- println!("p: {:p} ptr: {:<15?} len: {:2}, capacity: {:2} '{}'", &temp, temp.as_ptr(), temp.len(), temp.capacity(), temp);
-
- text.replace_range(6..9, "qqrq");
- println!("p: {:p} ptr: {:<15?} len: {:2}, capacity: {:2} '{}'", &text, text.as_ptr(), text.len(), text.capacity(), text);
-
- text.replace_range(6..9, "123456");
- println!("p: {:p} ptr: {:<15?} len: {:2}, capacity: {:2} '{}'", &text, text.as_ptr(), text.len(), text.capacity(), text);
-
- text.replace_range(6..9, "12345678901234567890");
- println!("p: {:p} ptr: {:<15?} len: {:2}, capacity: {:2} '{}'", &text, text.as_ptr(), text.len(), text.capacity(), text);
-
-}
-```
-
-```rust
-p: 0x7ffdcc572770 ptr: 0x1 len: 0, capacity: 0 ''
-p: 0x7ffdcc572770 ptr: 0x5de087754ba0 len: 10, capacity: 10 'Hello foo!'
-p: 0x7ffdcc572770 ptr: 0x5de087754ba0 len: 10, capacity: 10 'Hello bar!'
-The black cat
-p: 0x7ffdcc573260 ptr: 0x5de087754bc0 len: 13, capacity: 13 'The black cat'
-p: 0x7ffdcc572770 ptr: 0x5de087754ba0 len: 11, capacity: 20 'Hello qqrq!'
-p: 0x7ffdcc572770 ptr: 0x5de087754ba0 len: 14, capacity: 20 'Hello 123456q!'
-p: 0x7ffdcc572770 ptr: 0x5de087754be0 len: 28, capacity: 40 'Hello 12345678901234567890q!'
-```
-
-[String and memory allocation](https://rust.code-maven.com/strings-and-memory-allocation)
diff --git a/content/memory_safety/vec.md b/content/memory_safety/vec.md
@@ -1,85 +0,0 @@
----
-title: Vec
----
-
-# `Vec<u8>`
-
-Getting the size of a Vec<u8> in Rust is straightforward — since each element is
-exactly 1 byte, .len() gives you the byte count directly.
-
-## `len()` vs `capacity()`
-
-A `Vec<u8>` internally holds three things: a pointer to heap memory, a length,
-and a capacity.
-
-- `len()` — the number of actual bytes of data present. This is the file size.
-
-- `capacity()` — the total memory reserved on the heap, which may be larger than
- len() to avoid frequent reallocations as the vector grows. This is an internal
- memory management detail.
-
- For example, a `Vec<u8>` with 500 bytes of real data might have a capacity of
- 512 — those extra 12 bytes are pre-allocated empty slots Rust reserved
- speculatively. They contain no real data.
- [rust-lang](https://doc.rust-lang.org/std/vec/struct.Vec.html)
-
-```
- Stack Heap
-───────────────── ──────────────────────────────────────────────────
-┌───────────────┐ ┌─────────────────────────────┬──────────────────┐
-│ ptr │──────────► │ REAL DATA (500 bytes) │ EMPTY (12 bytes)│
-├───────────────┤ │ │ │
-│ len = 500 │ │ [0x01][0xFF][0x3A]... │ [ ][ ][ ]... │
-├───────────────┤ │ │ │
-│ cap = 512 │ │ ◄────── len = 500 ────────►│◄─── cap-len ────►│
-└───────────────┘ └─────────────────────────────┴──────────────────┘
- ◄──────────────── cap = 512 ──────────────────────►
-```
-
-- `ptr` points to the start of the heap block
-
-- `len = 500` marks how far into the block contains real, valid data — this is
- your file size
-
-- `cap = 512` is the total reserved memory; the trailing 12 bytes are allocated
- but uninitialised and ignored
- [stackoverflow](https://stackoverflow.com/questions/54889521/whats-the-difference-between-len-and-capacity)
-
-
-- When you call `.len()`, you get `500` — exactly the bytes fetched from the
- database, nothing more
-
-### Question: Why `512`?
-
-512 was just an illustrative, round-number example, not a special Rust rule.
-
-In reality:
-
-- Rust does not guarantee “500 bytes → capacity 512”. The allocator chooses how
- much memory to give you, and `Vec` typically grows by some factor (often ~2×),
- but the exact value is an implementation detail.
-
-- When a `Vec` needs more space (because `len == capacity` and you push again),
- it reallocates to a larger capacity to reduce how often it has to reallocate
- in the future. That larger capacity might be 512, 640, 1000, etc., depending
- on the previous capacity and the growth strategy.
-
-- So “500 data, 512 capacity” was just to show the idea: **capacity ≥ len**, and
- the extra part is reserved space for future pushes, not real data.
-
-### Question: Whats the difference between "padding" and "pre-allocated empty slot"?
-
-“Padding” is a term we use for extra bytes inserted inside or between fields of
-a struct to satisfy alignment requirements of the CPU (e.g., to align a u64 on
-an 8‑byte boundary). However, in a `Vec<T>`:
-
-- `len` is how many initialized elements you have.
-
-- `capacity` is how many elements the heap allocation can hold.
-
-- The bytes between `le`n and `capacity` are logically **uninitialized storage
- for future elements**, not alignment padding.
-
- So for a `Vec<u8>` with `len = 500` and `capacity = 512`, those 12 bytes are
- just unused, reserved space that Rust can fill later if you push more bytes;
- they’re not considered padding in the usual memory-layout sense.
diff --git a/content/memory_safety/vec_vs_hashset.md b/content/memory_safety/vec_vs_hashset.md
@@ -1,79 +0,0 @@
----
-title: Vec vs HashSet
----
-
-[HashSet - Rust By Example](https://doc.rust-lang.org/rust-by-example/std/hash/hashset.html)
-
-Sets have 4 primary operations (all of the following calls return an iterator):
-
-- union: get all the unique elements in both sets.
-
-- difference: get all the elements that are in the first set but not the second.
-
-- intersection: get all the elements that are only in both sets.
-
-- symmetric_difference: get all the elements that are in one set or the other,
- but not both.
-
-### Question: Is HashSet O(1) and Vec O(n) in rust?
-
-In Rust, the O(1) versus O(n) comparison between a `HashSet` and a `Vec`
-primarily refers to lookup operations, such as checking if a collection contains
-a specific element. While a `HashSet` provides O(1) average time complexity for
-lookups, a `Vec` requires O(n) time because it must perform a linear search.
-
-| Operation | `Vec<T>` | `HashSet<T>` |
-| :---------------------- | :----------------------------------------------- | :--------------------------- |
-| **Lookup (`contains`)** | O(n) [^7] | O(1) average [^7][^9] |
-| **Add Element** | O(1) amortized (`push`) [^8] | O(1) average (`insert`) [^9] |
-| **Remove Element** | O(n) (`remove`), O(1) (`pop`/`swap_remove`) [^8] | O(1) average (`remove`) |
-
-## Lookup Performance
-
-Checking if an element exists using `contains()` is where the O(n) vs O(1)
-difference is most prominent. A `HashSet` computes a hash to instantly find the
-element's bucket, offering O(1) average lookup time regardless of the
-collection's size. In contrast, a `Vec` must iterate through its elements one by
-one until a match is found, resulting in O(n) time complexity where the search
-time grows linearly with the number of elements.[^3][^7][^10]
-
-## Insertion and Removal
-
-Appending an element to a `Vec` using `push()` operates in amortized O(1) time,
-though occasional capacity reallocations can cause brief performance spikes.
-`HashSet` insertions are also O(1) on average, but the operation carries a
-higher "constant factor" because computing the hash and handling potential
-bucket collisions requires more CPU work than simply placing an item at the end
-of a vector. For removals, `HashSet` operates in O(1) average time, whereas
-removing an element from a specific index in a `Vec` using `remove()` is O(n)
-because it requires shifting all subsequent elements to fill the gap.[^8][^10][^3]
-
-[^1]: https://users.rust-lang.org/t/hashset-foo-vs-vec-foo-as-a-key/103281/2
-
-[^2]: https://users.rust-lang.org/t/iterating-through-hashmap-approx-twice-the-time-of-vec/75964/4
-
-[^3]: https://stackoverflow.com/questions/72877598/hashset-is-slower-than-bruteforce-in-rust
-
-[^4]: https://stackoverflow.com/questions/3185226/huge-performance-difference-between-vector-and-hashset
-
-[^5]: https://wkaisertexas.github.io/blog/hashmaps-versus-vectors/
-
-[^6]: https://stackoverflow.com/questions/39803237/build-hashset-from-a-vector-in-rust
-
-[^7]: https://zenn.dev/iriko/articles/d06fdc5a3de863
-
-[^8]: https://oneuptime.com/blog/post/2026-02-01-rust-collections/view
-
-[^9]: https://dev.to/alexmercedcoder/working-with-collections-in-rust-a-comprehensive-guide-3c9f
-
-[^10]: https://nindalf.com/posts/optimising-rust/
-
-[^11]: https://users.rust-lang.org/t/vec-vs-hashmap-vs-hashset-for-unique-named-items/106615
-
-[^12]: https://users.rust-lang.org/t/question-about-hashset/7779/2
-
-[^13]: https://www.reddit.com/r/rust/comments/1c6h18a/hashset_method_slower_than_naive_method_when/
-
-[^14]: https://doc.rust-lang.org/rust-by-example/std/hash/hashset.html
-
-[^15]: https://users.rust-lang.org/t/idiomatic-way-to-get-difference-between-two-vecs/48396
diff --git a/content/networking/cache.md b/content/networking/cache.md
@@ -1,7 +0,0 @@
----
-title: Cache
----
-
-## Three types of web cache
-
-<img src="/assets/cache.png" alt="Server Setup" width="100%">
diff --git a/content/networking/ccna.md b/content/networking/ccna.md
@@ -1,193 +0,0 @@
----
-title: CCNA Exam Topics
----
-
-1.0 Networking Fundamentals
-
-```
-1.1 Explain the role and function of network components
-1.1.a Routers
-1.1.b Layer 2 and Layer 3 switches
-1.1.c Next-generation firewalls and IPS
-1.1.d Access points
-1.1.e Controllers (Cisco DNA Center and WLC)
-1.1.f Endpoints
-1.1.g Servers
-1.1.h PoE
-
-1.2 Describe characteristics of network topology architectures
-1.2.a Two-tier
-1.2.b Three-tier
-1.2.c Spine-leaf
-1.2.d WAN
-1.2.e Small office/home office (SOHO)
-1.2.f On-premise and cloud
-
-1.3 Compare physical interface and cabling types
-1.3.a Single-mode fiber, multimode fiber, copper
-1.3.b Connections (Ethernet shared media and point-to-point)
-
-1.4 Identify interface and cable issues (collisions, errors, mismatch duplex, and/or speed)
-
-1.5 Compare TCP to UDP
-
-1.6 Configure and verify IPv4 addressing and subnetting
-
-1.7 Describe private IPv4 addressing
-
-1.8 Configure and verify IPv6 addressing and prefix
-
-1.9 Describe IPv6 address types
-1.9.a Unicast (global, unique local, and link local)
-1.9.b Anycast
-1.9.c Multicast
-1.9.d Modified EUI 64
-
-1.10 Verify IP parameters for Client OS (Windows, Mac OS, Linux)
-
-1.11 Describe wireless principles
-1.11.a Nonoverlapping Wi-Fi channels
-1.11.b SSID
-1.11.c RF
-1.11.d Encryption
-
-1.12 Explain virtualization fundamentals (server virtualization, containers, and VRFs)
-
-1.13 Describe switching concepts
-1.13.a MAC learning and aging
-1.13.b Frame switching
-1.13.c Frame flooding
-1.13.d MAC address table
-```
-
-2.0 Network Access
-
-```
-2.1 Configure and verify VLANs (normal range) spanning multiple switches
-2.1.a Access ports (data and voice)
-2.1.b Default VLAN
-2.1.c InterVLAN connectivity
-
-2.2 Configure and verify interswitch connectivity
-2.2.a Trunk ports
-2.2.b 802.1Q
-2.2.c Native VLAN
-
-2.3 Configure and verify Layer 2 discovery protocols (Cisco Discovery Protocol and LLDP)
-
-2.4 Configure and verify (Layer 2/Layer 3) EtherChannel (LACP)
-
-2.5 Interpret basic operations of Rapid PVST+ Spanning Tree Protocol
-2.5.a Root port, root bridge (primary/secondary), and other port names
-2.5.b Port states (forwarding/blocking)
-2.5.c PortFast
-2.5.d Root guard, loop guard, BPDU filter, and BPDU guard
-
-2.6 Describe Cisco Wireless Architectures and AP modes
-
-2.7 Describe physical infrastructure connections of WLAN components (AP, WLC, access/trunk ports, and LAG)
-
-2.8 Describe network device management access (Telnet, SSH, HTTP, HTTPS, console, TACACS+/RADIUS, and cloud managed)
-
-2.9 Interpret the wireless LAN GUI configuration for client connectivity, such as WLAN creation, security settings, QoS profiles, and advanced settings
-```
-
-3.0 IP Connectivity
-
-```
-3.1 Interpret the components of routing table
-3.1.a Routing protocol code
-3.1.b Prefix
-3.1.c Network mask
-3.1.d Next hop
-3.1.e Administrative distance
-3.1.f Metric
-3.1.g Gateway of last resort
-
-3.2 Determine how a router makes a forwarding decision by default
-3.2.a Longest prefix match
-3.2.b Administrative distance
-3.2.c Routing protocol metric
-
-3.3 Configure and verify IPv4 and IPv6 static routing
-3.3.a Default route
-3.3.b Network route
-3.3.c Host route
-3.3.d Floating static
-
-3.4 Configure and verify single area OSPFv2
-3.4.a Neighbor adjacencies
-3.4.b Point-to-point
-3.4.c Broadcast (DR/BDR selection)
-3.4.d Router ID
-
-3.5 Describe the purpose, functions, and concepts of first hop redundancy protocols
-```
-
-4.0 IP Services
-
-```
-4.1 Configure and verify inside source NAT using static and pools
-
-4.2 Configure and verify NTP operating in a client and server mode
-
-4.3 Explain the role of DHCP and DNS within the network
-
-4.4 Explain the function of SNMP in network operations
-
-4.5 Describe the use of syslog features including facilities and levels
-
-4.6 Configure and verify DHCP client and relay
-
-4.7 Explain the forwarding per-hop behavior (PHB) for QoS, such as classification, marking, queuing, congestion, policing, and shaping
-
-4.8 Configure network devices for remote access using SSH
-
-4.9 Describe the capabilities and functions of TFTP/FTP in the network
-```
-
-5.0 Security Fundamentals
-
-```
-5.1 Define key security concepts (threats, vulnerabilities, exploits, and mitigation techniques)
-
-5.2 Describe security program elements (user awareness, training, and physical access control)
-
-5.3 Configure and verify device access control using local passwords
-
-5.4 Describe security password policies elements, such as management, complexity, and password alternatives (multifactor authentication, certificates, and biometrics)
-
-5.5. Describe IPsec remote access and site-to-site VPNs
-
-5.6 Configure and verify access control lists
-
-5.7 Configure and verify Layer 2 security features (DHCP snooping, dynamic ARP inspection, and port security)
-
-5.8 Compare authentication, authorization, and accounting concepts
-
-5.9 Describe wireless security protocols (WPA, WPA2, and WPA3)
-
-5.10 Configure and verify WLAN within the GUI using WPA2 PSK
-```
-
-6.0 Automation and Programmability
-
-```
-6.1 Explain how automation impacts network management
-
-6.2 Compare traditional networks with controller-based networking
-
-6.3 Describe controller-based, software defined architecture (overlay, underlay, and fabric)
-6.3.a Separation of control plane and data plane
-6.3.b Northbound and Southbound APIs
-
-6.4 Explain AI (generative and predictive) and machine learning in network operations
-
-6.5 Describe characteristics of REST-based APIs (authentication types, CRUD, HTTP verbs, and data encoding)
-
-6.6 Recognize the capabilities of configuration management mechanisms, such as Ansible and Terraform
-
-6.7 Recognize components of JSON-encoded data
-```
-
-[CCNA](https://learningnetwork.cisco.com/s/ccna-exam-topics)
diff --git a/content/networking/edge_infra.md b/content/networking/edge_infra.md
@@ -1,168 +0,0 @@
----
-title: Edge Infra
----
-
-Building a globally distributed edge infrastructure for software package
-delivery requires a synergy of multi-tiered caching, intelligent edge
-computation, and robust networking. By combining these technologies, we can
-ensure that developers worldwide can pull packages, binaries, and containers
-with minimal latency and high availability.[^1][^2]
-
-Here is the design and implementation strategy for a global edge network
-tailored for package delivery.
-
-### Edge Architecture Overview
-
-Our system utilizes a tiered architecture to move data as close to the end user
-as possible while protecting the origin from traffic spikes.[^3][^1]
-
-```text
- +-------------------+
- | Developer |
- | (npm/docker pull) |
- +---------+---------+
- |
- v
- +-------------------+
- | Anycast / Geo-DNS |
- +---------+---------+
- | (Routes to nearest PoP)
- v
-+-----------------------------------------------------------+
-| EDGE PoP |
-| |
-| +-------------+ +---------------+ +--------------+ |
-| | Edge Compute|--->| L1 Cache | | WAF / DDoS | |
-| | (Auth/Route)| | (Memory/NVMe) | | Protection | |
-| +-------------+ +-------+-------+ +--------------+ |
-+-----------------------------|-----------------------------+
- | (Cache Miss)
- v
-+-----------------------------------------------------------+
-| REGIONAL SHIELD CACHE |
-| +-----------------------------------------------------+ |
-| | L2 Cache (High Capacity SSD, Request Collapsing) | |
-| +--------------------------+--------------------------+ |
-+-----------------------------|-----------------------------+
- | (Cache Miss)
- v
-+-----------------------------------------------------------+
-| ORIGIN INFRASTRUCTURE |
-| +--------------------+ +------------------------+ |
-| | Blob Storage | | Global Metadata DB | |
-| | (S3 / GCS) | | (Spanner / DynamoDB) | |
-| +--------------------+ +------------------------+ |
-+-----------------------------------------------------------+
-```
-
-### The Full Stack
-
-To achieve this scale, the technology stack must be highly concurrent and
-lightweight:
-
-- **Edge Routing \& Proxy:** NGINX, Envoy, or Rust-based proxies to handle
- millions of concurrent TCP connections and perform TLS termination.[^4]
-- **Edge Compute:** WebAssembly (Wasm) or V8 Isolates running directly on the
- CDN edge to execute custom logic like authentication, A/B testing, and request
- filtering without routing back to the origin.[^3][^1]
-- **Caching Layer:** Varnish or custom memory-mapped file systems for L1 edge
- caching, backed by high-capacity NVMe drives for L2 regional shields.
-- **Data \& Origin:** Geographically replicated object storage (like AWS S3) for
- immutable package blobs, and a globally distributed database (like Google
- Cloud Spanner) for mutable package metadata and user entitlements.
-
-### Edge Caching and CDN Optimizations
-
-Software packages often experience "thundering herd" traffic patterns, such as
-when a popular CI/CD pipeline kicks off thousands of identical container pulls
-simultaneously.
-
-- **Tiered Cache Hierarchy:** Implementing an L1 edge cache and an L2 regional
- shield reduces origin calls and optimizes global latency.[^3]
-- **Request Collapsing:** If 10,000 clients request the same uncached package
- simultaneously, the edge node collapses these into a single origin request,
- preventing origin overload.
-- **Predictive Caching:** By analyzing package dependency trees (e.g.,
- `package.json`), edge servers can pre-cache required dependencies before the
- client explicitly requests them.[^1]
-- **Cache Invalidation:** Using stale-while-revalidate headers allows the CDN to
- serve a slightly outdated metadata file (like a Docker `latest` tag) while
- asynchronously fetching the updated version in the background.
-
-### Distributed Networking Solutions
-
-Routing users efficiently is critical for minimizing latency and ensuring high
-availability during regional outages.
-
-- **Anycast IP Routing:** Advertising the same IP address from multiple global
- locations allows the Border Gateway Protocol (BGP) to naturally route the
- user's TCP connection to the topologically closest datacenter.
-- **Dynamic Server Selection:** The system actively monitors Round Trip Time
- (RTT) and CPU usage to dynamically route traffic away from congested or
- degraded Points of Presence (PoPs).[^5][^1]
-- **Protocol Optimizations:** Utilizing TCP BBR congestion control and HTTP/3
- (QUIC) reduces connection setup time and mitigates the impact of packet loss
- on unstable mobile networks.
-
-### System Data Flows
-
-When a user pulls a package, the request follows a strict path to ensure
-authorization and speed:
-
-1. **Resolution:** The client's DNS query hits a Geo-DNS provider, returning the
- Anycast IP of the nearest Edge PoP.
-2. **Edge Auth:** The request reaches the Edge Proxy. An Edge Function executes
- immediately, verifying the user's API token against a highly cached subset of
- the metadata database.[^1]
-3. **Cache Lookup:** The proxy checks the L1 Cache. If the package is found, it
- is returned instantly.
-4. **Shield Fallback:** On an L1 miss, the request goes to the Regional Shield.
- If the package is present in the L2 cache, it is returned and populated in
- L1.
-5. **Origin Fetch:** On an L2 miss, the shield fetches the blob from Origin
- Storage, caches it, and streams it back down the chain to the client.
-
-### Performance Impact Chart
-
-This tiered networking approach dramatically reduces latency across the
-distribution lifecycle.
-
-```text
-Average Response Latency (ms) by Retrieval Tier
-------------------------------------------------------------
-Origin Fetch |################################ (250ms)
-Regional Shield L2 |########### (85ms)
-Edge PoP L1 |### (20ms)
-Predictive Cache |# (5ms)
-------------------------------------------------------------
-```
-
-[^1]: https://notionhive.com/blog/edge-computing-cdn-strategies
-
-[^2]: https://talents.studysmarter.co.uk/companies/cloudsmith-ltd/belfast/senior-software-engineer-edge-29145650/
-
-[^3]: https://www.daydreamsoft.com/blog/edge-caching-and-cdn-optimization-delivering-lightning-fast-web-experiences
-
-[^4]: https://builtin.com/job/senior-software-engineer-tech-platform/6578449
-
-[^5]: https://arxiv.org/html/2412.09474v1
-
-[^6]: https://www.geeksforgeeks.org/system-design/edge-caching-system-design/
-
-[^7]: https://ijrai.org/index.php/ijrai/article/view/180
-
-[^8]: https://networks.imdea.org/trade-offs-in-optimizing-the-cache-deployments-of-cdns/
-
-[^9]: https://careers.deliveroo.co.uk/role/senior-platform-engineer-edge-15f9904608d9/
-
-[^10]: https://www.sciencedirect.com/science/article/abs/pii/S0140366404002889
-
-[^11]: https://www.meegle.com/en_us/topics/content-delivery-network/cdn-caching-mechanisms
-
-[^12]: https://builtin.com/job/senior-software-engineer-edge-infrastructure/7113935
-
-[^13]: https://www.youtube.com/watch?v=zLblLu3rUC4
-
-[^14]: https://www.builtinla.com/job/sr-staff-software-engineer-edge-cdn-platform/7455324
-
-[^15]: https://www.dynadot.com/blog/global-cdn-strategies
diff --git a/content/networking/index.md b/content/networking/index.md
@@ -1,3 +0,0 @@
----
-title: Networking
----
diff --git a/content/networking/ipc.md b/content/networking/ipc.md
@@ -1,19 +0,0 @@
----
-title: IPC
----
-
-## IPC (Inter-Process Communication)
-
-IPC is the broad concept — it refers to any mechanism that allows separate
-processes to exchange data. It's not a specific technology but rather an
-umbrella term covering many approaches: shared memory, message queues, pipes,
-signals, and sockets. Both Unix sockets, [TCP](/networking/tcp.md) and
-[UDP](/networking/udp.md) can be used as IPC mechanisms.
-
-## IPC (narrow/common usage)
-
-Local-only mechanisms (Unix sockets, pipes, shared memory)
-
-## TCP & UDP
-
-Network protocols that can do IPC, but are designed for cross-machine use
diff --git a/content/networking/loopback_address.md b/content/networking/loopback_address.md
@@ -1,119 +0,0 @@
----
-title: Loopback Address
----
-
-In Linux system administration and networking fundamentals, a loopback address
-refers to a special, virtual network interface that allows a computer to
-communicate with itself [5]. This interface is typically named `lo` or `lo0` on
-Unix-like systems
-
-## IP Addressing and Standards
-
-The most common IPv4 address used for this purpose is 127.0.0.1, which is the
-standard address for IPv4 loopback traffic. This address is part of a reserved
-block of more than 16 million addresses ranging from 127.0.0.0 to
-127.255.255.255 (127.0.0.0/8) that are designated for loopback functionality.
-For IPv6, the address ::1 is also reserved for this purpose. Internet
-Engineering Task Force (IETF) standards reserve the IPv4 address block
-127.0.0.0/8 and the IPv6 address ::1/128 for this specific use.
-
-## Functionality and Behavior
-
-The primary purpose of the loopback interface is to return the packets sent to
-it; whatever is sent to it is looped back internally. Traffic destined for a
-loopback address allows a host to communicate with itself without sending
-traffic onto any physical network. When an application sends data to 127.0.0.1,
-the IP stack recognizes this as a loopback address and immediately redirects the
-traffic internally through the loopback interface. Because of this design, a
-system that is not connected to any network will still have this loopback device
-and hence a 127.0.0.0 address.
-
-## System Administration and Configuration
-
-In Linux, the loopback interface is managed by tools like NetworkManager, which
-assigns the IP addresses 127.0.0.1 and ::1 that are persistent across reboots.
-Administrators cannot override 127.0.0.1 and ::1, although they can assign
-additional IP addresses to the interface. The hostname localhost is simply a
-name that resolves to this IP address and is configured in /etc/hosts.
-
-## Usage and Availability
-
-Common uses for the loopback device include diagnostics, local service testing,
-and inter-process communication. System administrators and engineers rely on
-loopback addresses when setting up servers to ensure configurations are correct.
-While the majority of systems have loopback addresses by default, they typically
-have zero impact on whether the system is part of a distributed system or not.
-
-## Compare between Unix Socket and Loopback Address
-
-While both methods allow processes on the same machine to communicate, Unix
-domain sockets are technically superior for performance and resource management
-due to the lack of TCP/IP overhead and port limitations. However, loopback
-addresses provide a robust, network-stack-independent method that is often
-sufficient for testing and applications already designed around TCP/IP
-protocols.
-
-### Resource Usage
-
-Port Numbers: Using loopback addresses requires binding to local port numbers,
-which are a limited resource.
-
-File System: Unix sockets utilize the file system for their endpoints, avoiding
-the consumption of network ports.
-
-### Performance and Overhead
-
-Latency: Unix domain sockets generally offer lower latency than TCP loopback
-connections. Benchmarks on modern Linux systems show Unix domain sockets
-delivering approximately 2-3 microseconds of latency, compared to 3.6
-microseconds for TCP loopback, representing a 36% improvement.
-
-Overhead: Unix domain sockets are faster because they avoid the overhead of the
-TCP/IP stack. Local interprocess communication via Unix domain sockets is faster
-than loopback localhost connections because there is less TCP processing
-involved. The loopback device is considered more complicated than Unix sockets,
-resulting in higher relative overhead for loopback traffic.
-
-### Underlying Mechanism
-
-Unix Sockets: These are inter-process communication (IPC) mechanisms that use
-local files to send and receive data, rather than network interfaces and IP
-packets. They function as communication endpoints for data exchange between
-processes running on the same Unix or Unix-like operating system.
-
-Loopback Address: This relies on the TCP/IP stack using the loopback interface
-(typically lo with IP 127.0.0.1). While TCP/IP sockets using the loopback
-address are a natural way to pass messages between processes on the same host,
-they still traverse the network stack.
-
-### Reliability and Robustness
-
-NIC Dependency: Communication via loopback addresses is highly robust because
-the loopback interface (127.0.0.1) does not depend on the physical network
-interface card (NIC). It remains active regardless of whether the NIC is up,
-down, or reconfigured.
-
-Failure Scenarios: If services are configured to communicate via a physical NIC
-IP address, communication can fail if the NIC goes down or the IP changes. In
-contrast, using the loopback address ensures communication remains functional
-even if network hardware fails.
-
-### Typical Use Cases
-
-Loopback: The loopback interface is primarily used for diagnostics and testing
-local server applications. It is also commonly used for IPC when applications
-are designed to use TCP/IP sockets (e.g., Java applications).
-
-Unix Sockets: These are preferred for high-performance local IPC, such as in
-database configurations (e.g., PostgreSQL performance advice suggests Unix
-sockets over localhost for local connections).
-
-### References
-
-[UNIX sockets vs. localhost: PostgreSQL performance advice](https://www.cybertec-postgresql.com/en/postgresql-performance-advice-unix-sockets-vs-localhost/)
-
-## Why does Docker use 172.x.x.x?
-
-Docker uses IP ranges starting with 172.x.x.x (specifically 172.17.0.0/16 by
-default) because that is part of the private IP address space defined in RFC
-1918, which Docker reserves for internal container networking.
diff --git a/content/networking/tcp.md b/content/networking/tcp.md
@@ -1,46 +0,0 @@
----
-title: TCP
----
-
-## TCP (Transmission Control Protocol)
-
-A network protocol ensuring reliable, ordered data delivery between any two
-machines across a network.
-
-## TCP vs TCP/IP
-
-TCP (Transmission Control Protocol) is a single protocol that operates at the
-transport layer — it's responsible for breaking data into packets, ensuring they
-arrive in order, and retransmitting lost ones.
-
-TCP/IP is not a single protocol but a whole suite (stack) of protocols that work
-together to enable internet communication. It's named after its two most
-important protocols — TCP and IP — but it also includes UDP, HTTP, DNS, FTP, and
-many others.
-
-### The Role of Each Layer
-
-TCP/IP is organized into four layers, each with a distinct job:
-
-- Application layer — user-facing protocols like HTTP, FTP, SMTP, DNS
-
-- Transport layer — TCP or UDP (handles delivery, ordering, error checking)
-
-- Internet layer — IP (handles addressing and routing packets to the right
- machine)
-
-- Network Access layer — physical transmission over Ethernet, Wi-Fi, etc.
-
-### How TCP and IP Work Together
-
-IP and TCP are complementary: IP figures out where to send data (routing between
-machines using IP addresses), while TCP figures out how to send it reliably
-(ordering, retransmission, flow control). TCP doesn't send data directly to the
-recipient — it hands packets down to IP, which routes them across the network.
-
-A simple analogy: IP is like the postal system that routes packages to the right
-address, and TCP is the careful courier who numbers each box, confirms delivery,
-and resends anything that got lost.
-
-So in short: TCP is one piece inside TCP/IP. When people say "TCP/IP," they mean
-the entire networking framework that powers the internet.
diff --git a/content/networking/udp.md b/content/networking/udp.md
@@ -1,6 +0,0 @@
----
-title: UDP
----
-
-UDP (User Datagram Protocol) — a fast, connectionless network protocol that
-sends data without delivery guarantees between any machines across a network.
diff --git a/content/networking/unix_socket.md b/content/networking/unix_socket.md
@@ -1,186 +0,0 @@
----
-title: Unix Socket
----
-
-A Unix domain socket (UDS) is a special file on your filesystem (e.g.
-`/tmp/app.sock`) that acts as a communication endpoint between processes on the
-same machine. Unlike TCP sockets, data never leaves the OS kernel — it's copied
-directly between process buffers — making it significantly faster for local IPC.
-The server binds to the socket path, and one or more clients connect to it.
-
-Key properties:
-
-- Lives at a filesystem path, controlled by file permissions
-
-- Supports stream mode (like TCP — ordered, reliable) and datagram mode (like
- UDP — message-based)
-
-- Commonly used between e.g. Nginx ↔ PHP-FPM, apps ↔ local databases, or
- microservices on the same host
-
-## Rust Examples: Multiple Producers → Single Receiver
-
-The pattern here is: **multiple client threads each connect to the same Unix
-socket**, and a single server receives all their messages. We'll use tokio for
-async I/O.
-[Unix Listener](https://docs.rs/tokio/latest/tokio/net/struct.UnixListener.html),
-[Unix Stream](https://docs.rs/tokio/latest/tokio/net/struct.UnixStream.html)
-
-```toml
-[dependencies]
-tokio = { version = "1", features = ["full"] }
-```
-
-### Example 1 — Basic Server (Single Receiver)
-
-The server binds to a socket path and handles each incoming connection, printing
-whatever data it receives.
-[docs](https://docs.rs/tokio/latest/tokio/net/struct.UnixListener.html)
-
-```rust
-// server.rs
-use tokio::net::UnixListener;
-use tokio::io::AsyncReadExt;
-
-#[tokio::main]
-async fn main() {
- let socket_path = "/tmp/mpsr.sock";
- let _ = std::fs::remove_file(socket_path); // clean up stale socket
-
- let listener = UnixListener::bind(socket_path).unwrap();
- println!("Server listening on {}", socket_path);
-
- loop {
- match listener.accept().await {
- Ok((mut stream, _addr)) => {
- // Spawn a task per connection — all funnel into this single server
- tokio::spawn(async move {
- let mut buf = vec![0u8; 1024];
- match stream.read(&mut buf).await {
- Ok(n) if n > 0 => {
- let msg = String::from_utf8_lossy(&buf[..n]);
- println!("[Server] Received: {}", msg);
- }
- _ => eprintln!("[Server] Connection closed or error"),
- }
- });
- }
- Err(e) => eprintln!("Accept error: {}", e),
- }
- }
-}
-```
-
-### Example 2 — Multiple Producers (Clients)
-
-Each producer runs in its own thread, connects to the socket, and sends a
-message.
-
-```rust
-// client.rs
-use tokio::net::UnixStream;
-use tokio::io::AsyncWriteExt;
-
-#[tokio::main]
-async fn main() {
- let socket_path = "/tmp/mpsr.sock";
- let num_producers = 5;
-
- let mut handles = vec![];
-
- for i in 0..num_producers {
- let path = socket_path.to_string();
- let handle = tokio::spawn(async move {
- match UnixStream::connect(&path).await {
- Ok(mut stream) => {
- let msg = format!("Hello from producer {}", i);
- stream.write_all(msg.as_bytes()).await.unwrap();
- println!("[Producer {}] Sent: {}", i, msg);
- }
- Err(e) => eprintln!("[Producer {}] Failed to connect: {}", i, e),
- }
- });
- handles.push(handle);
- }
-
- for h in handles {
- h.await.unwrap();
- }
-}
-```
-
-### Example 3 — Combined: Server + Producers in One Binary
-
-A self-contained example where the server is spawned as a background task, and 5
-producers send concurrently.
-
-```rust
-use tokio::net::{UnixListener, UnixStream};
-use tokio::io::{AsyncReadExt, AsyncWriteExt};
-use tokio::time::{sleep, Duration};
-
-#[tokio::main]
-async fn main() {
- let socket_path = "/tmp/mpsr_combined.sock";
- let _ = std::fs::remove_file(socket_path);
-
- // --- Spawn the single receiver (server) ---
- let server_path = socket_path.to_string();
- tokio::spawn(async move {
- let listener = UnixListener::bind(&server_path).unwrap();
- println!("[Server] Listening...");
- loop {
- if let Ok((mut stream, _)) = listener.accept().await {
- tokio::spawn(async move {
- let mut buf = vec![0u8; 256];
- if let Ok(n) = stream.read(&mut buf).await {
- println!("[Server] Got: {}", String::from_utf8_lossy(&buf[..n]));
- }
- });
- }
- }
- });
-
- // Give the server a moment to bind
- sleep(Duration::from_millis(50)).await;
-
- // --- Spawn multiple producers ---
- let mut handles = vec![];
- for i in 0..5 {
- let path = socket_path.to_string();
- handles.push(tokio::spawn(async move {
- let mut stream = UnixStream::connect(&path).await.unwrap();
- let msg = format!("Message from producer {}", i);
- stream.write_all(msg.as_bytes()).await.unwrap();
- println!("[Producer {}] Sent.", i);
- }));
- }
-
- for h in handles {
- h.await.unwrap();
- }
-
- sleep(Duration::from_millis(100)).await; // let server print all messages
-}
-```
-
-Expected output (order may vary since tasks run concurrently):
-
-```text
-[Server] Listening...
-[Producer 0] Sent.
-[Producer 2] Sent.
-[Server] Got: Message from producer 0
-[Server] Got: Message from producer 2
-...
-```
-
-### How the MPSR Pattern Works Here
-
-Each producer independently connects and sends — the Unix socket listener
-naturally queues incoming connections. The tokio::spawn per connection is what
-makes this a multiple-producer, single-receiver funnel: one server loop accepts
-all connections, but each connection is handled concurrently. If you also want
-to aggregate messages into a single channel, you can pair this with
-std::sync::mpsc or tokio::sync::mpsc inside the server's accept loop.
-[mpsc](https://blog.softwaremill.com/multithreading-in-rust-with-mpsc-multi-producer-single-consumer-channels-db0fc91ae3fa)
diff --git a/content/observability/index.md b/content/observability/index.md
@@ -1,3 +0,0 @@
----
-title: Observability
----
diff --git a/content/observability/span.md b/content/observability/span.md
@@ -1,145 +0,0 @@
----
-title: Span
----
-
-## Question: What is a span in relation to traces, logs and metrics?
-
-A **span** is the foundational unit of **distributed tracing**, representing a
-single logical operation (e.g., an HTTP request, a database call, or a function
-invocation) with timing and contextual metadata [7]. To understand spans
-deeply—and how they relate to **traces**, **logs**, and **metrics**—we must
-examine their structural, semantic, and operational relationships.
-
----
-
-### **1. Span as the Atomic Unit of a Trace**
-
-A **trace** is a directed acyclic graph (DAG) of spans that captures the
-end-to-end journey of a request across services [1]. Each span:
-
-- Has a **start/end timestamp** and **duration**,
-- Contains **attributes** (key-value metadata, e.g., HTTP status, user ID),
-- May include **events** (timestamped annotations like “query started”),
-- Has a **parent-child relationship** with other spans (e.g., a gateway span may
- have child spans for auth and DB calls) [[3], [8]].
-
-Example trace structure[8]:
-
-```
-Trace
-├── Span (API Gateway)
-│ ├── Span (Auth Service)
-│ └── Span (User Service)
-│ └── Span (Database Query)
-└── Span (Response Formatting)
-```
-
----
-
-### **2. Relationship to Logs**
-
-- **Logs** are discrete, timestamped records of events (e.g., “error: connection
- timeout”), often unstructured or semi-structured.
-- **Spans can embed logs**: When instrumentation libraries (e.g., OpenTelemetry)
- integrate with logging frameworks, log statements can be attached to spans as
- **structured events** or **log records**, enriching them with trace context
- (trace ID, span ID) [10].
-- This enables **correlation**: You can view logs _within the context_ of a
- specific span—e.g., see all logs from a database query span during a failed
- request [10].
-
-> “When adding OpenTelemetry instrumentation on top of your existing log
-> libraries, the log becomes a dot on a trace span” [10].
-
----
-
-### **3. Relationship to Metrics**
-
-- **Metrics** are aggregated numerical measurements over time (e.g., request
- rate, latency percentiles, error counts).
-- **Spans feed into metrics indirectly**:
- - Span durations can be used to compute **latency histograms** (e.g.,
- `http.server.request.duration`).
- - Span attributes (e.g., `http.status_code`) can be aggregated into
- **counters** (e.g., `http_requests_total{status="500"}`).
-- While spans are _individual_, _context-rich_ units, metrics are _aggregated_,
- _summarized_—but both are essential for the **RED method** (Rate, Errors,
- Duration) [7].
-
-> “Developers can acquire a comprehensive perspective of their software
-> environment by combining distributed traces, metrics, events, and logs” [7].
-
----
-
-### **4. Relationship to Traces (Recap & Nuance)**
-
-- A **trace** is a _collection of spans_ that together represent a single
- request’s path through a distributed system [3].
-- Spans in a trace are linked via:
- - **Trace ID** (identifies the full trace),
- - **Span ID** (identifies the span),
- - **Parent Span ID** (enables tree-like nesting) [[1], [8]].
-- Spans may also have **links** to spans in _other traces_ (e.g., for batch
- processing or async workflows) [1].
-
----
-
-### **5. Practical Implications**
-
-- **Troubleshooting**: A trace gives you a _map_; logs give you _narrative
- detail_; metrics give you _signal-level trends_. For example:
- - A metric alert (e.g., high error rate) → drill into traces to find failing
- spans → inspect embedded logs for root cause [14].
-- **Context propagation**: Spans carry trace context (trace ID, span ID,
- sampling flags) across service boundaries, enabling distributed correlation
- [9].
-
----
-
-### **Summary**
-
-| Concept | Role | Relationship to Span |
-| ----------- | ------------------------------------------ | ------------------------------------------------------------------- |
-| **Span** | Smallest unit of work in a trace | — |
-| **Trace** | Collection of spans forming a request path | Spans are its building blocks [3] |
-| **Logs** | Event records with timestamps | Logs can be attached to spans as events or structured metadata [10] |
-| **Metrics** | Aggregated numerical signals | Span data (duration, status) is used to derive metrics [7] |
-
-In essence, **spans unify the three pillars of observability**—they are the
-_contextual glue_ that lets you correlate logs (what happened), metrics (how
-often/long), and traces (how it flows) into actionable insights [[4], [14]].
-
-## References
-
-1. [Traces | OpenTelemetry](https://opentelemetry.io/docs/concepts/signals/traces/)
- _(brave)_
-2. [OpenTelemetry - Understanding Traces vs. Spans | SigNoz](https://signoz.io/comparisons/opentelemetry-trace-vs-span/)
- _(brave)_
-3. [Logs vs Metrics vs Traces - Engineering Fundamentals Playbook](https://microsoft.github.io/code-with-engineering-playbook/observability/log-vs-metric-vs-trace/)
- _(google)_
-4. [Observability primer | OpenTelemetry](https://opentelemetry.io/docs/concepts/observability-primer/)
- _(brave)_
-5. [Unpacking Observability: Understanding Logs, Events, Spans, and Traces | Dzero Labs](https://medium.com/dzerolabs/observability-journey-understanding-logs-events-traces-and-spans-836524d63172)
- _(google)_
-6. [OpenTelemetry demystified: a deep dive into distributed tracing | CNCF](https://www.cncf.io/blog/2023/05/03/opentelemetry-demystified-a-deep-dive-into-distributed-tracing/)
- _(google)_
-7. [What Are Spans in Distributed Tracing? - LogicMonitor](https://www.logicmonitor.com/blog/what-are-spans-in-distributed-tracing)
- _(startpage)_
-8. [Traces & Spans: Observability Basics You Should Know - Last9](https://last9.io/blog/traces-spans-observability-basics/)
- _(startpage)_
-9. [software-skills/skills/system-design/references/key-concepts ...](https://github.com/itzcull/software-skills/blob/master/skills/system-design/references/key-concepts/distributed-tracing.md)
- _(aol)_
-10. [Tracing the Line: Understanding Logs vs. Traces - Honeycomb](https://www.honeycomb.io/blog/understanding-logs-vs-traces)
- _(google)_
-11. [A Deep Dive into OpenTelemetry. Part 1 - AWS in Plain English](https://aws.plainenglish.io/opentelemetry-deep-dive-part-1-6ebbd2362bd3)
- _(google)_
-12. [Deep Dive into OpenTelemetry in Saleor](https://saleor.io/blog/otel-deep-dive)
- _(google)_
-13. [Logging Observability - OpenClaw AI Agent Skill | LLMBase](https://llmbase.ai/openclaw/logging-observability/)
- _(aol)_
-14. [Learning Observability from Scratch: Logs, Metrics, and Traces | by Milind Nair | Mar, 2026 | Medium](https://medium.com/@nairmilind3/learning-observability-from-scratch-c36d9771003b)
- _(brave)_
-15. [A Deep Dive Into OpenTelemetry Metrics | Tiger Data](https://www.tigerdata.com/blog/a-deep-dive-into-open-telemetry-metrics)
- _(aol)_
-16. [GitHub - tokio-rs/tracing: Application level tracing for Rust.](https://github.com/tokio-rs/tracing)
- _(aol)_
diff --git a/content/observability/tracing-logs-metrics.md b/content/observability/tracing-logs-metrics.md
@@ -1,234 +0,0 @@
----
-title: Tracing, Logs, & Metrics
----
-
-## Question: Whats the difference between tracing, logs, and metrics?
-
----
-
-**Tracing, Logs, and Metrics** are three core components of **observability**,
-each serving distinct but complementary roles in understanding system behavior.
-
----
-
-### 🔹 **Logs**
-
-- **What they are**: Chronological records of discrete events that occur during
- program execution (e.g., “User logged in”, “Database query failed”) [[7]].
-- **Purpose**: Primarily used for error tracking, debugging, and auditing. Logs
- help answer _“What happened?”_ and _“When did it happen?”_ [[9]].
-- **Format**: Usually unstructured or semi-structured text, often timestamped
- and tagged with severity levels (e.g., INFO, WARN, ERROR) [[7]].
-
----
-
-### 🔹 **Tracing**
-
-- **What it is**: A technique to track the flow of a single request as it
- traverses multiple services or components (e.g., from API gateway → auth
- service → database) [[13]].
-- **Purpose**: Provides end-to-end visibility into request paths, latency, and
- failure points—especially critical in distributed systems [[1], [11]].
-- **Structure**: Composed of _spans_ (units of work) nested in _traces_;
- visualized as flame graphs or Gantt charts [[11], [13]].
-- **Use case**: Diagnosing slow responses or failures across service boundaries
- [[1], [5]].
-
----
-
-### 🔹 **Metrics**
-
-- **What they are**: Quantitative, time-series measurements (e.g., request rate,
- CPU usage, error counts) [[6]], [[9]].
-- **Purpose**: Ideal for trend analysis, alerting, and capacity planning.
- Metrics answer _“How is the system performing?”_ [[6]].
-- **Format**: Structured and aggregatable (e.g., counters, gauges, histograms)
- [[6]].
-
----
-
-### 🔸 **When to Use Metrics, Logs, or Traces**
-
-Metrics, logs, and traces are complementary observability signals—not
-replacements for each other. Each answers different questions and excels in
-specific scenarios.
-
-| Consider Metrics When... | Consider Logs When... | Consider Traces When... |
-| ----------------------------------------------------- | -------------------------------------------------------- | ------------------------------------------------------- |
-| You need to monitor system health over time | You need to investigate specific errors or events | You need to track a request across service boundaries |
-| You want to set up alerting (e.g., "error rate > 1%") | You need to debug a specific request using IDs | You need to find the exact span where latency occurs |
-| You care about trends, aggregations, and patterns | You need raw context, stack traces, or user actions | You're diagnosing slow responses in distributed systems |
-| You need low storage/processing overhead | You're doing post-mortem analysis or compliance auditing | You need to understand request flow and dependencies |
-| You're capacity planning or auto-scaling | You need to match errors across different services | You want to visualize dependencies and request paths |
-
----
-
-### 🎯 **Why Metrics Instead of Just Logs?**
-
-Logs capture every event but are **too noisy and expensive** for real-time
-monitoring:
-
-1. **Aggregation Efficiency**: Metrics pre-compute values (sums, averages,
- percentiles), while querying logs for aggregated insights requires scanning
- massive datasets. For example, calculating error rates from logs across
- millions of events per second is computationally expensive; metrics do this
- inline.
-
-2. **Real-time Alerting**: Metrics are time-series data designed for rapid
- aggregation. You can quickly detect anomalies using threshold-based alerts.
- With logs, you'd need to constantly parse and aggregate in
- real-time—introducing latency and cost.
-
-3. **Storage Cost**: Storing every log line is expensive (GBs/TBs/day). Metrics
- are orders of magnitude smaller because they store only numerical summaries.
-
-4. **Dashboard Visualization**: Trending data over time (e.g., "requests/second
- over 24 hours") works naturally with metrics. Extracting this from logs
- requires complex aggregation queries.
-
-**Example**: Monitoring a web service. Metrics tell you the error rate spiked to
-5% across all endpoints. Logs alone wouldn't reveal this pattern—you'd need to
-manually scan countless entries to notice the trend.
-
- _Prometheus time-series database
-showing system metrics over time_
-
----
-
-### 🎯 **Why Traces Instead of Just Logs?**
-
-While logs record individual events, traces provide **context and causality**
-across distributed systems:
-
-1. **End-to-End Request Flow**: A single user request may traverse 10+ services.
- Logs from each service are disconnected unless correlated by trace IDs.
- Traces natively capture this relationship with parent-child span
- relationships.
-
-2. **Latency Breakdown**: Logs can tell you each service took 100ms, but traces
- show exactly where time was spent (e.g., 50ms in auth, 80ms in database).
- This is critical for identifying bottlenecks.
-
-3. **Failure Context**: If a request fails, traces show the entire path—what
- succeeded, what failed, and where. Logs from individual services lack this
- cross-service context.
-
-4. **Dependency Visualization**: Traces reveal service boundaries and call
- sequences, helping you understand system architecture and potential single
- points of failure.
-
-**Example**: A user reports a slow page load. Logs from the API gateway show a
-2-second response. Logs from the auth service show 200ms. Logs from the database
-show 100ms. Without traces, you can't correlate these or see that the auth
-service was actually waiting on the database (1.5s of wait time hidden in
-individual logs).
-
- _Jaeger trace visualization showing a
-distributed request across multiple services with latency breakdown_
-
- _Flame graph view in Jaeger
-highlighting which spans consume the most time_
-
----
-
-### 🎯 **Why Logs Instead of Just Metrics or Traces?**
-
-Logs provide **context and detail** that metrics and traces cannot:
-
-1. **Rich Context**: Metrics are numbers; traces show flow. Logs contain the
- actual messages: error stack traces, variable values, user IDs, request
- payloads. This is essential for debugging.
-
-2. **Unpredictable Events**: You can't pre-aggregate unknown error conditions in
- metrics. Logs capture everything—including rare, unexpected errors that
- metrics might miss in the noise.
-
-3. **Debugging Speed**: When tracing identifies a problematic span, logs provide
- the exact log lines from that span (with context like user ID, request
- parameters, error messages) to pinpoint the cause.
-
-4. **Compliance & Audit**: Logs provide an immutable record of what happened,
- often required for regulatory compliance. Metrics summarize but don't replay
- events.
-
-**Example**: A trace shows a 500 error in the payment service. The metric shows
-0.1% error rate (acceptable). But only logs contain the actual error: "Payment
-declined: insufficient funds" with the user's transaction ID—needed to
-investigate and communicate with the user.
-
- _Grafana Loki aggregated logs with
-label-based filtering and search_
-
----
-
-### 🔗 **The Synergistic Workflow**
-
-The three signals work best together:
-
-1. **Metrics detect**: Threshold-based alerts tell you something is wrong
-2. **Traces localize**: Identify which request/path/service is affected
-3. **Logs debug**: Provide context to understand and fix the root cause
-
-**Example workflow**:
-
-- ✅ Metrics show error rate increased from 0.1% to 5% → **Something is wrong**
-- ✅ Traces reveal the spike is from `/api/checkout` requests in the payment
- service → **Where is it wrong?**
-- ✅ Logs with that trace ID show:
- `PaymentProviderError: Connection timeout to Stripe` → **Why is it wrong?**
-
----
-
-### 🌐 **Observability Concepts**
-
-Observability is the _property_ of a system that allows engineers to understand
-its internal state _from its external outputs_ [[4], [10], [12]]. It goes beyond
-monitoring by enabling _root-cause analysis_ of unexpected behavior, not just
-detection [[16]].
-
-The **three pillars of observability** are:
-
-1. **Metrics** – for high-level system health and trends
-2. **Logs** – for detailed event-level debugging
-3. **Traces** – for request-level flow and latency analysis
-
-Together, they provide a comprehensive view:
-
-- Metrics tell you _something is wrong_
-- Traces tell you _which request/path is affected_
-- Logs tell you _what happened and why_
-
-This layered approach prevents log overload while maintaining debugging
-capability.
-
----
-
-### ⚖️ Key Differences Summary
-
-| Aspect | Logs | Traces | Metrics |
-| ----------------- | ---------------------------------------- | ------------------------------------------- | ------------------------------------------ |
-| **Granularity** | Event-level (e.g., function call, error) | Request-level (end-to-end path) | Aggregated (e.g., avg latency, error rate) |
-| **Structure** | Semi-structured text | Hierarchical spans (parent/child) | Numerical, time-series |
-| **Primary Use** | Debugging, auditing | Performance analysis, distributed debugging | Alerting, trend analysis |
-| **Cost/Overhead** | Moderate | High (esp. in distributed systems) [[5]] | Low (aggregated) |
-
----
-
-## Sources
-
-1. [java - What is the difference between Tracing and Logging? - Stack Overflow](https://stackoverflow.com/questions/27244807/what-is-the-difference-between-tracing-and-logging)
-2. [What is observability? Not just logs, metrics, and traces - Dynatrace](https://www.dynatrace.com/news/blog/what-is-observability-2/)
-3. [Tracing Vs. Logging – Key Differences + Examples - Edge Delta](https://edgedelta.com/company/blog/tracing-vs-logging-differences-with-examples)
-4. [What Is Observability? | IBM](https://www.ibm.com/think/topics/observability)
-5. [Logging vs Tracing in real projects — how deep do you actually go?](https://www.reddit.com/r/Backend/comments/1r45scd/logging_vs_tracing_in_real_projects_how_deep_do/)
-6. [Observability in 2025: How It Works, Challenges and Best Practices](https://lumigo.io/what-is-observability-concepts-use-cases-and-technologies/)
-7. [Tracing vs Logging vs Monitoring: What's the Difference? – BMC Helix](https://blogs.helixops.ai/monitoring-logging-tracing/)
-8. [What Is Observability & How Does it Work?](https://www.datadoghq.com/knowledge-center/observability/)
-9. [Logging vs Metrics vs Tracing: What's the Difference? | Better Stack Community](https://betterstack.com/community/guides/observability/logging-metrics-tracing/)
-10. [What Is Observability? Concepts, Use Cases & Technologies - Tigera.io](https://www.tigera.io/learn/guides/observability/)
-11. [Monitoring explained: What is the difference between Logging, Tracing & Profiling? - greeeg.com](https://greeeg.com/en/issues/differences-between-logging-tracing-profiling)
-12. [Observability That Works: Understand System Failures and Drive Better Business Outcomes | Splunk](https://www.splunk.com/en_us/blog/learn/observability.html)
-13. [Traces - OpenTelemetry](https://opentelemetry.io/docs/concepts/signals/traces/)
-14. [What Is Observability? Key Components and Best Practices](https://www.honeycomb.io/blog/what-is-observability-key-components-best-practices)
-15. [Screening and Assessment of Contaminated Sediment](https://extapps.dec.ny.gov/docs/fish_marine_pdf/screenasssedfin.pdf)
-16. [What is Observability? A Guide to Success - Cribl](https://cribl.io/blog/what-is-observability/)
diff --git a/content/operating_systems/data_streams.md b/content/operating_systems/data_streams.md
@@ -1,217 +0,0 @@
----
-title: Data Streams
----
-
-## Data Streams and Pipe
-
-<img src="/whiteboards/data_stream.png" alt="Data Stream" width="100%">
-
-## Passing Data Streams between rust binaries
-
-We can pass data directly between binaries, they don't have to go through a
-networking layers http server with ports, which adds unecessary overhead when it
-is not needed.
-
-<img src="/whiteboards/pipe_between_binaries.png" alt="Data Stream" width="100%">
-
-## Making your own “filter” binary
-
-If you want your Rust binary to be the thing that “processes stdout from another
-binary”, just read from stdin and write to stdout:
-
-```rust
-use std::io::{self, BufRead, Write};
-
-fn main() -> io::Result<()> {
- let stdin = io::stdin();
- let stdout = io::stdout();
- let mut out = stdout.lock();
-
- for line in stdin.lock().lines() {
- let line = line?;
- // transform line
- let processed = line.replace("foo", "bar");
- writeln!(out, "{processed}")?;
- }
-
- Ok(())
-}
-```
-
-Then in a shell you can do `producer_cmd | your_rust_filter | consumer_cmd`.
-
-## Use processed data as stdin to another binary
-
-To “chain” binaries but keep Rust in the middle, you can run a second command
-and write your processed data into its stdin.
-
-```rust
-use std::io::{self, BufRead, BufReader, Write};
-use std::process::{Command, Stdio};
-
-fn main() -> io::Result<()> {
- // First binary: produces data
- let mut producer = Command::new("producer_cmd")
- .arg("some_arg")
- .stdout(Stdio::piped())
- .spawn()?;
-
- let producer_stdout = producer
- .stdout
- .take()
- .expect("failed to capture producer stdout");
- let reader = BufReader::new(producer_stdout);
-
- // Second binary: consumes processed data
- let mut consumer = Command::new("consumer_cmd")
- .arg("another_arg")
- .stdin(Stdio::piped())
- .spawn()?;
-
- let consumer_stdin = consumer
- .stdin
- .take()
- .expect("failed to open consumer stdin");
-
- // Process loop: read from producer, transform, send to consumer
- let mut writer = consumer_stdin;
- for line in reader.lines() {
- let line = line?;
- let processed = format!("prefix: {line}\n"); // any transformation
- writer.write_all(processed.as_bytes())?;
- }
- // Close consumer stdin so it can finish
- drop(writer);
-
- // Wait for both processes to exit
- let prod_status = producer.wait()?;
- let cons_status = consumer.wait()?;
-
- eprintln!("producer: {prod_status}, consumer: {cons_status}");
-
- Ok(())
-}
-```
-
-Conceptually this is equivalent to
-`producer_cmd | my_rust_filter | consumer_cmd`, but Rust is the middle filter,
-so you never rely on shell piping.
-
-## Async Pipes with Tokio
-
-Use Tokio's async I/O to pipe binary data. `tokio::process::Command` mirrors the
-sync API but uses `AsyncRead/AsyncWrite` traits, letting you `.await` on reads
-and writes without blocking threads.
-
-```rust
-use tokio::process::Command;
-use tokio::io::{AsyncReadExt, AsyncWriteExt};
-use tokio::process::Stdio;
-
-#[tokio::main]
-async fn main() {
- let mut child = Command::new("xxd")
- .stdin(Stdio::piped())
- .stdout(Stdio::piped())
- .spawn()
- .expect("Failed to spawn");
-
- let binary_data: Vec<u8> = (0u8..=255).collect(); // 256 bytes of raw binary
-
- // Write asynchronously
- if let Some(mut stdin) = child.stdin.take() {
- stdin.write_all(&binary_data).await.expect("Write failed");
- // Drop stdin to close pipe (send EOF)
- }
-
- // Read asynchronously
- let mut output = Vec::new();
- if let Some(mut stdout) = child.stdout.take() {
- stdout.read_to_end(&mut output).await.expect("Read failed");
- }
-
- child.wait().await.expect("Child failed");
- println!("Received {} bytes of output", output.len());
-}
-```
-
-For streaming, long-running binaries, use tokio_process_tools which wraps
-Tokio's process API with real-time line/byte inspection via channels.
-
-## Fan-In: Many Binaries → One Receiver
-
-This is a classic multi-producer, single-consumer (MPSC) pattern. You don't need
-special OS-level listeners — Tokio's mpsc channel is purpose-built for this.
-
-The architecture:
-
-- Each producer binary is spawned as a child process with stdout(Stdio::piped())
-
-- Each gets a clone of the mpsc::Sender
-
-- A single async task owns the Receiver and processes all incoming data
-
-```rust
-use tokio::process::Command;
-use tokio::io::{AsyncReadExt, BufReader};
-use tokio::sync::mpsc;
-use std::process::Stdio;
-
-#[tokio::main]
-async fn main() {
- let (tx, mut rx) = mpsc::channel::<(usize, Vec<u8>)>(32);
-
- let producers = vec!["producer_a", "producer_b", "producer_c"];
-
- for (id, binary) in producers.iter().enumerate() {
- let tx = tx.clone(); // clone sender for each producer
-
- let mut child = Command::new(binary)
- .stdout(Stdio::piped())
- .spawn()
- .expect("Failed to spawn producer");
-
- tokio::spawn(async move {
- let mut stdout = BufReader::new(child.stdout.take().unwrap());
- let mut buf = Vec::new();
-
- loop {
- let mut chunk = vec![0u8; 4096];
- match stdout.read(&mut chunk).await {
- Ok(0) => break, // EOF
- Ok(n) => {
- chunk.truncate(n);
- tx.send((id, chunk)).await.unwrap();
- }
- Err(e) => { eprintln!("Error from producer {}: {}", id, e); break; }
- }
- }
- child.wait().await.unwrap();
- });
- }
-
- drop(tx); // drop original sender so rx knows when all producers are done
-
- // Single receiver handles all producers
- while let Some((producer_id, data)) = rx.recv().await {
- println!("Producer {}: {} bytes", producer_id, data.len());
- // process binary data here...
- }
-}
-```
-
-Do You Need an Event Listener? Not in the traditional sense. Tokio's scheduler
-is itself an event-driven async runtime — when no data is available, tasks are
-suspended and CPU is yielded, not wasted in a spin loop.
-
-# Delimiter
-
-A delimiter is a character or group of characters that marks where one piece of
-data ends and the next one begins.
-
-In computing and data formats, common delimiters include commas, tabs, spaces,
-semicolons, and pipes, such as in CSV files where commas separate values. In
-general English dictionaries, it is defined as a character that marks the
-beginning or end of a unit of data. In maths and programming, symbols like
-parentheses (), quotes " " and braces {} also act as delimiters because they
-enclose or bound expressions or code blocks.
diff --git a/content/operating_systems/index.md b/content/operating_systems/index.md
@@ -1,22 +0,0 @@
----
-title: Operating Systems
----
-
-## OS-level view
-
-A [mutex (mutual exclusion lock)](/memory_safety/mutex.md) is a synchronization
-primitive that ensures only one thread at a time enters a critical section that
-uses some shared data.
-
-If a thread tries to lock a mutex that is already locked, the OS (or runtime)
-either makes it spin briefly or puts it to sleep and queues it until the mutex
-is unlocked.
-
-The OS uses low-level tools (like Linux futex on an integer in memory) plus
-atomics and memory fences to implement this reliably across cores.
-
-- [System Call(syscall)](/operating_systems/system_call.md)
-- [User Space and Kernel Space](/operating_systems/user_space_and_kernel_space.md)
-- [Thread](/operating_systems/thread.md)
-- [PostgreSQL Connection](/operating_systems/postgres/postgresql_connection.md)
-- [Connection Pool](/operating_systems/postgres/connection_pool.md)
diff --git a/content/operating_systems/postgres/connection_pool.md b/content/operating_systems/postgres/connection_pool.md
@@ -1,46 +0,0 @@
----
-title: Connection Pool
----
-
-A connection pool is a component (in your app or as a proxy like
-PgBouncer/Pgpool-II) that keeps a limited number of database connections open
-and reuses them across many client requests.
-
-## Key ideas:
-
-- Opening a [PostgreSQL connection](/operating_systems/postgres/postgresql_connection.md) is
- relatively expensive and each connection consumes memory; creating hundreds or
- thousands on demand is wasteful.
-
-- The pool maintains, say, 20–50 actual DB connections and lets many more
- logical clients "borrow" them for short periods.
-
-- When a request finishes, the connection is returned to the pool instead of
- being closed, so the next request can reuse it without the startup cost.
-
-- The pool enforces an upper bound on concurrent PostgreSQL connections, which
- indirectly bounds per-connection memory usage.
-
-### Question: what if the pool is full?
-
-#### General behavior when pool is full:
-
-- The pool has a max number of physical PostgreSQL connections it will open.
-
-- When all of them are checked out and a new request comes in, most pool
- implementations put the request in a queue and wait for a connection to be
- returned. If no connection becomes free before a timeout, they raise an
- error/exception (often something like "pool exhausted" or "timeout waiting for
- connection").
-
-- This is separate from PostgreSQL's own max_connections; the pool will usually
- hit its own limit first and throttle clients, which is the goal.
-
-### Question: What if the DB itself is at max connections?
-
-- PostgreSQL has a server-side max_connections limit; when that is reached, any
- new physical connection attempt fails with "too many connections".
-
-- If your pool tries to open more physical connections beyond what the server
- allows (e.g. under load or after restart), those attempts will fail and bubble
- up as errors to the application (connection failure rather than "queued").
diff --git a/content/operating_systems/postgres/index.md b/content/operating_systems/postgres/index.md
@@ -1,7 +0,0 @@
----
-title: Postgres
----
-
-- [PostgreSQL Connection](/operating_systems/postgres/postgresql_connection.md)
-- [Connection Pool](/operating_systems/postgres/connection_pool.md)
-- [Postgres Binaries](/operating_systems/postgres/postgres_binaries.md)
diff --git a/content/operating_systems/postgres/postgres_binaries.md b/content/operating_systems/postgres/postgres_binaries.md
@@ -1,5 +0,0 @@
----
-title: Postgres Binaries
----
-
-<img src="/whiteboards/postgres.png" alt="Postgres Structure" width="100%">
diff --git a/content/operating_systems/postgres/postgresql_connection.md b/content/operating_systems/postgres/postgresql_connection.md
@@ -1,29 +0,0 @@
----
-title: PostgreSQL Connection
----
-
-A PostgreSQL connection is the link between a client (like an app, script, or
-psql) and a PostgreSQL database server that lets them send queries and receive
-results.
-
-## Memory and Processes
-
-A PostgreSQL connection uses memory because each client gets its own backend
-process with its own state (caches, buffers, variables, temp data, etc.), and
-that state must live somewhere in RAM. This is also why too many connections can
-exhaust memory and hurt performance.
-
-PostgreSQL is a process‑per‑connection model (on typical
-[Unix-like systems](/operating_systems/README.md) a new OS process is forked for
-each client connection).
-
-Each backend process has its own private address space for session-local state,
-which avoids accidental shared mutable state between connections.
-
-Backends share only explicit shared-memory regions and files (e.g., shared
-buffer cache, WAL, locks), with well-defined synchronization, so there's no
-"mysterious" shared state like in a multi-threaded process with global
-variables.
-
-So, you do get isolation of most state per connection, with only controlled,
-intentional shared memory for common data structures.
diff --git a/content/operating_systems/system_call.md b/content/operating_systems/system_call.md
@@ -1,9 +0,0 @@
----
-title: System Call
----
-
-A system call is how a normal program asks the operating system kernel to do
-something privileged (like read a file, send data on a socket, allocate certain
-memory).
-
-“System call” is a concept from operating‑system interfaces and kernel design.
diff --git a/content/operating_systems/thread.md b/content/operating_systems/thread.md
@@ -1,147 +0,0 @@
----
-title: Thread
----
-
-### Question: What exactly is a thread?
-
-A thread is the smallest unit of execution that a CPU can run — essentially, an
-independent sequence of instructions within a program. The CPU distinguishes
-threads by their saved state and ID, not by “feeling” them electrically 😜.
-
-#### Every thread has a small bundle of CPU-related data called its context:
-
-- Program counter (which instruction to run next)
-
-- CPU registers (temporary values)
-
-- Stack pointer (where its stack lives)
-
-- A thread ID the OS uses to refer to it
-
-This context is stored in memory in a per-thread data structure managed by the
-OS (often called a TCB, “thread control block”).
-
-#### When the OS scheduler decides to run a different thread:
-
-- It takes the context of that thread (from memory).
-
-- It loads that context into the CPU’s registers and program counter.
-
-- From the CPU’s perspective, it is now executing that thread.
-
-So “which thread am I running?” = “which context did the OS most recently load
-into my registers and program counter?”
-
-### Question: Why This Matters for Arc?
-
-This is exactly why Arc exists in Rust — when multiple threads share the same
-data, they all access the same memory. Without atomic operations on the
-reference count, two threads incrementing it simultaneously could corrupt it,
-leading to use-after-free bugs or memory leaks. Arc's atomic reference counting
-prevents this without needing a lock.
-
-### Question: What is the relationship between a thread and a process and a program?
-
-```
- ,----------------, ,---------,
- ,-----------------------, ," ,"|
- ," ,"| ," ," |
- +-----------------------+ | ," ," |
- | .-----------------. | | +---------+ |
- | | | | | | -==----'| |
- | | I LOVE DOS! | | | | | |
- | | Bad command or | | |/----|`---= | |
- | | C:\>_ | | | ,/|==== ooo | ;
- | | | | | // |(((( [33]| ,"
- | `-----------------' |," .;'| |(((( | ,"
- +-----------------------+ ;; | | |,"
- /_)______________(_/ //' | +---------+
- ___________________________/___ `,
- / oooooooooooooooo .o. oooo /, \,"-----------
- / ==ooooooooooooooo==.o. ooo= // ,`\--{)B ,"
-/_==__==========__==_ooo__ooo=_/' /___________,"
-`-----------------------------'
- v
-+--------------------------------------------------------------+
-| PROGRAM |
-| (code on disk, not running) |
-+--------------------------------------------------------------+
- | |
- | OS loads program into memory, twice |
- v v
-+----------------------+ +----------------------+
-| PROCESS 1 | | PROCESS 2 |
-| (instance of prog) | | (instance of prog) |
-| Own address space | | Own address space |
-+----------------------+ +----------------------+
- | | | | | |
- | | +---------------+ | | +---------------+
- | +-------+ | | +-------+ |
- v v v v v v
-+--------+ +--------+ +--------+ +--------+ +--------+ +--------+
-|THREAD 1| |THREAD 2| |THREAD 3| |THREAD A| |TRHEAD B| |THREAD C|
-+--------+ +--------+ +--------+ +--------+ +--------+ +--------+
-
-- PROGRAM: one set of instructions on disk
-- PROCESS 1 and PROCESS 2: two running instances of the same program
-- THREADs: multiple execution flows inside PROCESS 1 (PROCESS 2 could also have threads)
-```
-
-The three concepts form a clear hierarchy: a program becomes a process when run,
-and a process contains one or more threads.
-
-Program → Process → Thread
-
-Think of it like this:
-
-- A program is a static set of instructions stored on disk — it's just a file,
- doing nothing
-
-- A process is what a program becomes when the OS loads it into memory and
- starts executing it — it's a living, running instance of the program
-
-- A thread is the actual unit of execution inside a process — the sequence of
- instructions the CPU is actively running
-
-### Question: Are threads allocated automatically to a process? How do I know how many threads a process runs? And how do I know how many process a program runs?
-
-Every process automatically gets exactly one thread when it starts — the main
-thread. This is the thread that begins executing at main(). Any additional
-threads beyond that must be explicitly created by the program itself — the OS
-does not add more threads on its own. So the number of threads a process has is
-entirely determined by what the programmer coded.
-
-There's no fixed number as to how many threads can a process run — it depends on
-system resources. On Linux for example, the maximum is calculated as
-
-```text
-max threads = virtual memory size ÷ (stack size × 1024 × 1024)
-```
-
-In practice, a Linux system can support tens of thousands of threads (e.g.
-~63,704 on a typical kernel). On Windows, the limit is also very high and
-practically constrained by available memory rather than a hard cap.
-
-A program typically runs as one process by default. It can spawn additional
-processes explicitly in code (e.g. using fork() in Unix or spawning subprocesses
-in Rust/Python) — but this is always a deliberate programmer choice, not
-automatic.
-
-## How to Check in Practice
-
-On Linux/macOS (terminal):
-
-- See threads for a specific process:
-
-```bash
-ps -o nlwp <pid> # shows number of threads
-cat /proc/<pid>/status | grep Threads
-```
-
-- See all processes from a program:
-
-```bash
-ps aux | grep <program_name>
-```
-
-- Live view with threads: run top, then press H to toggle thread view
diff --git a/content/operating_systems/user_space_and_kernel_space.md b/content/operating_systems/user_space_and_kernel_space.md
@@ -1,13 +0,0 @@
----
-title: User Space and Kernel Space
----
-
-User space is where your application code runs with limited privileges; kernel
-space is where the OS kernel runs with full access to hardware and all memory.
-
-“User space” and “kernel space” are concepts from OS memory protection,
-privilege levels, and CPU modes.
-
-In a curriculum, you’d see them in an Operating Systems course or a Systems
-Programming / OS Internals course, often alongside processes, threads,
-scheduling, virtual memory, and interrupts.
diff --git a/content/personal_development/4_disciplines_of_execution.md b/content/personal_development/4_disciplines_of_execution.md
@@ -1,41 +0,0 @@
----
-title: 4 Disciplines of Execution
----
-
-The 4 Disciplines of Execution (4DX) is a framework designed to help individuals
-and organizations achieve critical goals despite daily distractions. By
-emphasizing actionable metrics over past results, this methodology bridges the
-gap between high-level strategy and practical execution.
-
-## 1. Focus on the Wildly Important
-
-Selecting just one or two Wildly Important Goals (WIGs) ensures that your most
-critical objectives receive focused attention. The more priorities you attempt
-to juggle simultaneously, the less you will actually accomplish across the
-board. Narrowing your focus guarantees that these vital goals are not
-overshadowed by the distracting demands of routine work.
-
-## 2. Act on Lead Measures
-
-Effective execution requires distinguishing between the two types of tracking
-metrics: lag measures and lead measures. Lag measures, such as total revenue or
-weight lost, track final results that can no longer be changed once they are
-recorded. Conversely, lead measures are predictive, actionable daily
-activities—like making sales calls or exercising—that you can control to
-actively drive those end results.
-
-## 3. Keep a Compelling Scoreboard
-
-Maintaining a simple, highly visible scoreboard allows you to instantly track
-progress toward your primary objectives. People naturally perform better and
-stay more engaged when they are actively keeping score. Visualizing this
-progress provides the motivation necessary to maintain momentum and understand
-whether or not you are succeeding.
-
-## 4. Cadence of Accountability
-
-Consistent accountability is the driving force that ensures your strategic goals
-are actually executed. Establishing a regular rhythm, such as a brief weekly
-review, allows you to evaluate past commitments, update your scoreboard, and
-plan your next steps. This continuous reporting cycle prevents your primary
-goals from being swallowed up by the whirlwind of daily activities.
diff --git a/content/personal_development/attention_restoration_theory.md b/content/personal_development/attention_restoration_theory.md
@@ -1,260 +0,0 @@
----
-title: Attention Restoration Theory
----
-
-# Why a walk in the park makes your head quieter
-
-There is a particular kind of tiredness that has nothing to do with the body.
-You have been staring at a screen for hours; the work is not physically
-demanding; you have eaten, you have slept. Yet something in the machinery of
-focus has worn thin. You cannot hold a thought. You reread the same paragraph
-three times. You snap at a colleague over nothing. And then—sometimes—you
-step outside, walk among trees for twenty minutes, and return able to think
-again. Attention Restoration Theory (ART) is an attempt to explain, in
-cognitive terms, why that happens.
-
-## Two kinds of attention
-
-ART begins not with nature but with a distinction William James drew in 1890
-between two modes of attention: **voluntary** and **involuntary**.[^1][^2]
-Voluntary attention is the deliberate, effortful kind—what you use to read a
-dense paper, debug code, or hold a boring conversation. It is focused,
-goal-directed, and shielded by active inhibition of whatever else is competing
-for notice. Involuntary attention is the automatic kind, pulled by stimuli
-that are interesting in themselves: a bird flying past the window, a change in
-the light, the sound of water.[^2][^3]
-
-Stephen and Rachel Kaplan, working at the University of Michigan through the
-1970s and 1980s, proposed that the first kind is a finite resource and the
-second kind is not.[^1][^4] Voluntary attention—which they renamed **directed
-attention**—depends on mechanisms of cognitive inhibition that fatigue with
-sustained use. When directed attention is depleted, the symptoms are
-recognisable: errors, impulsivity, irritability, difficulty planning,
-difficulty holding information in working memory, and a subjective sense of
-mental clutter.[^4][^5] The Kaplans called this state **directed attention
-fatigue**.[^1]
-
-Modern urban and screen-mediated life, they argued, is unusually demanding of
-directed attention. Traffic, email, open-plan offices, and most of the
-ordinary stimuli of a city require constant filtering—each competing signal
-must be actively suppressed so the task at hand can proceed.[^1][^4] The
-inhibitory system wears out. The mind begins to leak.
-
-## The restoration claim
-
-ART's central proposal is that directed attention can recover if involuntary
-attention is allowed to take over for a while.[^1][^4] This is not the same as
-sleep, which rests the whole system, nor the same as doing nothing, which may
-leave directed attention engaged in rumination. What restores is engagement
-with an environment that captures involuntary attention **gently**—enough to
-occupy the mind, not so much that it demands effortful focus or provokes
-anxious vigilance.[^4][^6]
-
-Natural environments, the Kaplans argued, are unusually well suited to this.
-A forest, a river, a meadow—these are full of stimuli that are interesting
-without being urgent: moving leaves, shifting light, birdsong, the slow
-reconfiguration of clouds. None of them require you to do anything. None of
-them are trying to sell you anything or demanding a response within
-twenty-four hours. And yet they are not empty; the mind has somewhere to
-rest.[^1][^4]
-
-## Soft fascination
-
-The critical concept here is **fascination**—the quality of a stimulus that
-holds attention effortlessly. ART distinguishes two varieties.[^4][^6]
-
-**Hard fascination** is the total capture of attention by something intense:
-a sports match, an action film, a video game, a car crash. The stimulus is
-compelling enough that directed attention is not needed, but it leaves no
-cognitive room for anything else. Hard fascination can be restorative in a
-limited sense, but it does not support the kind of quiet reflection that the
-Kaplans considered part of full restoration.[^4]
-
-**Soft fascination** is gentler. The stimulus—a sunset, a fountain, wind in
-grass—engages involuntary attention, but leaves enough mental bandwidth for
-background thought to continue. The mind wanders, drifts, considers
-unresolved questions. Directed attention is not being used, and so it
-recovers; and at the same time, something else has space to happen—what the
-Kaplans called **reflection**.[^4][^6] This is why people often report that
-their best ideas come on walks, and why problems that seemed intractable at a
-desk sometimes resolve themselves on the way home.
-
-## The four properties of a restorative environment
-
-ART specifies four components that together make an environment
-restorative.[^4][^6][^7] All four need to be present, to some degree, for the
-full effect.
-
-1. **Being away.** The environment provides psychological distance from
- habitual demands. This can be literal (a different place) or conceptual
- (a different mode of engagement). The test is whether the mental content
- the place evokes is different from the content of your ordinary
- concerns.[^6][^7]
-2. **Extent.** The setting has enough scope and coherence to occupy the mind
- for a while. It feels like a "whole other world"—rich enough to explore,
- structured enough to make sense.[^6][^7] A well-designed Japanese garden
- can possess extent in a small space; a single houseplant usually cannot.
-3. **Fascination.** The environment engages involuntary attention through
- soft, aesthetically pleasing stimuli. This is the mechanism by which
- directed attention is spared.[^4][^6]
-4. **Compatibility.** The environment fits what you actually want to be
- doing. A forest is not restorative for someone who finds forests
- threatening; a library is not restorative for someone who wanted a walk.
- Restoration requires that the setting align with the person's
- inclinations and purposes.[^6][^7]
-
-Nature is not the only environment that can meet these criteria—places of
-worship, museums, long drives, certain cafés, and even well-designed indoor
-spaces can qualify—but the Kaplans argued that natural settings have an
-"aesthetic advantage" in supplying all four, particularly soft fascination
-and extent.[^1][^4]
-
-## The evidence
-
-The most cited experimental evidence for ART comes from a 2008 paper by Marc
-Berman, John Jonides, and Stephen Kaplan, published in _Psychological
-Science_.[^8][^9] Participants first completed a demanding backwards
-digit-span task, then took a fifty-minute walk either through the
-tree-lined Ann Arbor Arboretum or along a busy urban street, then repeated
-the task. Those who walked in nature improved their performance by roughly
-twenty percent; those who walked in the city showed no reliable
-improvement.[^8][^9] A second experiment reproduced the effect using only
-photographs of nature versus photographs of urban scenes, suggesting that
-some of the benefit survives even without physical exposure.[^8]
-
-A striking detail: the effect held in winter, when Chicago-area participants
-rated the nature walk as actively unpleasant due to cold. Whether the walk
-was enjoyable appeared not to matter for the cognitive benefit—the restoration
-was not mediated by mood.[^9][^10]
-
-Related findings have accumulated. A 2012 study by Berman and colleagues
-found that brief nature walks produced larger working-memory improvements in
-people with major depressive disorder than in healthy controls.[^9] Work on
-children aged four to eight found that a twenty-minute walk in a natural
-setting produced faster reaction times on attention tasks than an equivalent
-urban walk.[^10] Studies of window views have reported that office workers
-with views of trees and sky perform better on some cognitive and creative
-tasks than those without, and report better affect.[^11][^12]
-
-## Replication, caveats, and the messy middle
-
-ART is widely cited, but the empirical picture is less tidy than the popular
-summaries suggest. A 2016 systematic review and meta-analysis by Ohly and
-colleagues, synthesising thirty-one studies, found significant positive
-effects of nature exposure on three attention measures—Digit Span Forward,
-Digit Span Backward, and Trail Making Test B—but ten other meta-analyses
-within the same review showed no marked benefit.[^13][^14] A 2021 conceptual
-replication and meta-analysis by Stevenson and colleagues, focused on the
-Attention Network Test, concluded that simulated nature does not reliably
-restore executive attention.[^15]
-
-Several features of the literature complicate the picture. Sample sizes are
-often small. "Nature" and "urban" are defined inconsistently across
-studies—some compare forests with city centres, others compare potted plants
-with bare walls. The effect can be confounded by exercise, social
-interaction, air quality, and sunlight. The file-drawer problem—a bias
-toward publishing significant results—likely inflates the apparent
-consistency of the effect.[^1][^13][^15]
-
-The best current summary is probably this: exposure to natural environments
-does seem to benefit some aspects of cognition and mood, but the effect is
-smaller and more variable than the earliest framings suggested, and the
-exact mechanisms remain uncertain.[^13][^15] ART is a useful descriptive
-framework with real empirical support, not a precise quantitative law.
-
-## The default mode network
-
-A mechanistic bridge between ART and neuroscience has emerged through work
-on the **default mode network (DMN)**—a set of brain regions, including
-medial prefrontal cortex and posterior cingulate, that activate when
-attention is not focused on external tasks.[^1] The DMN is associated with
-mind-wandering, autobiographical memory, and creative thought. It is
-suppressed during demanding, externally focused work and rebounds during
-rest.[^1]
-
-Soft fascination, on this account, is the condition in which directed
-attention relaxes its grip enough for the DMN to come online, while
-involuntary attention remains lightly engaged with the environment. The mind
-is not idling in a closed loop—it is resting in a way that is externally
-anchored and internally active.[^1] This is why a walk in the park feels
-qualitatively different from lying in bed staring at a ceiling: both rest
-directed attention, but only the walk provides the gentle external
-scaffolding that keeps reflection productive rather than ruminative.
-
-## ART and screens
-
-One of the more practical questions raised by ART is what screen-mediated
-life does to the attention system. Most digital environments are rich in
-hard fascination rather than soft: they are engineered to capture attention
-fully, via novelty, social feedback, and intermittent reinforcement.[^1]
-They do not rest directed attention so much as bypass it—but they also do
-not leave room for reflection, and many involve precisely the kinds of
-filtering and inhibition (notifications, pop-ups, tab-switching) that
-deplete it.[^4]
-
-The testable prediction is straightforward: time spent on most social media
-and entertainment platforms should feel like rest but fail to restore
-directed attention, while time spent with soft-fascination media—slow
-nature documentaries, ambient walking videos, photographs of natural
-scenes—should produce measurable restoration. The evidence here is still
-limited, but the 2008 Berman result with photographs of nature, and later
-work on virtual nature exposure, offers preliminary support.[^8][^11]
-
-## Practical implications
-
-- **Take real breaks, not fake ones.** Scrolling a feed is not a break from
- directed attention—it is another demand on it. A short walk outside,
- even a few minutes at a window with a natural view, is more
- restorative.[^8][^9]
-- **Build soft fascination into your environment.** Plants, natural light,
- views of trees or water, nature sounds, and uncluttered visual fields all
- contribute. The effect is modest but cumulative.[^11][^12]
-- **Distinguish rest from stimulation.** Hard-fascination activities
- (intense films, fast games) feel refreshing but tend not to leave room
- for reflection. Soft-fascination activities (walking, gardening,
- unhurried conversation in a pleasant setting) do.[^4][^6]
-- **Match the environment to compatibility.** A restorative place is one
- that fits what you want to be doing; forcing yourself into "nature"
- against inclination may not help.[^6][^7]
-- **Use nature prophylactically before demanding work, not just after.**
- Restoration improves subsequent performance; starting from a replenished
- baseline is cheaper than recovering from a depleted one.[^8][^9]
-
-The underlying frame is worth holding onto even where the specific
-numbers are contested. Directed attention is a finite resource. Most of
-modern life is designed to consume it. Restoration is not optional, and
-most things that feel like restoration are not. Finding the environments
-that genuinely refill the tank—and spending time in them deliberately,
-not apologetically—is a small but serious discipline.
-
-<div align="center">⁂</div>
-
-[^1]: https://en.wikipedia.org/wiki/Attention_restoration_theory
-
-[^2]: https://hikingresearch.wordpress.com/2011/03/05/additional-benefits-of-attention-restoration/
-
-[^3]: https://kappanonline.org/merritt-going-outdoors-natural-antidote-attention-fatigue/
-
-[^4]: https://www.tandfonline.com/doi/full/10.1080/10937404.2016.1196155
-
-[^5]: https://www.sciencedirect.com/science/article/abs/pii/S0272494402001135
-
-[^6]: https://www.ecehh.org/research/attention-restoration-theory-a-systematic-review/
-
-[^7]: https://www.thrive.org.uk/how-we-help/social-therapeutic-horticulture-resource-centre/therapeutic-opportunities-in-sth/attention-restoration-theory
-
-[^8]: https://journals.sagepub.com/doi/abs/10.1111/j.1467-9280.2008.02225.x
-
-[^9]: https://psychology.uchicago.edu/news/marc-berman-thinks-you-should-take-walk-park
-
-[^10]: https://www.drjonslaughter.com/post/nature-s-power-to-restore-your-mind-the-science-behind-attention-restoration-theory
-
-[^11]: https://www.sciencedirect.com/science/article/abs/pii/S0272494422001682
-
-[^12]: https://pure.bond.edu.au/ws/portalfiles/portal/201246366/AM_Design_by_Nature.pdf
-
-[^13]: https://pubmed.ncbi.nlm.nih.gov/27668460/
-
-[^14]: https://www.sciencedirect.com/science/article/abs/pii/0272494495900012
-
-[^15]: https://www.sciencedirect.com/science/article/abs/pii/S0272494421001626
diff --git a/content/personal_development/do_less_to_do_more.md b/content/personal_development/do_less_to_do_more.md
@@ -1,131 +0,0 @@
----
-title: Do Less to Do More
----
-
-# You can only do more by doing less
-
-The counterintuitive truth at the heart of high performance is this: sustainable
-output is not a function of how much you attempt, but of how clearly you
-prioritise.[^1][^2] Doing less — deliberately, strategically — is what makes
-doing more possible.
-
-## The productivity trap
-
-Most people conflate busyness with progress. A packed calendar feels productive.
-A long to-do list feels responsible. But volume of activity is not a proxy for
-value created.[^3][^4]
-
-- Cognitive load is finite. Every item you hold in active attention draws from
- the same limited reservoir of focus.[^5][^6]
-- Context-switching compounds the cost: research consistently shows that moving
- between tasks doesn't just pause work, it degrades the quality of all tasks
- involved.[^7][^5]
-- Most output follows a Pareto-like distribution — roughly 20% of efforts
- produce 80% of meaningful results.[^8][^3]
-
-The trap is that the _feeling_ of doing more is itself a reward. Crossing items
-off a list, filling your hours, maintaining a constant state of motion — these
-are all emotionally satisfying in ways that bear little relation to actual
-progress.[^3][^4]
-
-## Why less creates more
-
-Doing less is not laziness. It is the act of concentrating energy rather than
-dispersing it.[^2][^9]
-
-Consider a lens focusing sunlight. A beam scattered across a surface warms it
-gently. The same energy, narrowed to a point, can start a fire. Subtraction is
-how you build that point.
-
-- **Depth over breadth**: dedicating uninterrupted time to a single task allows
- you to reach states of flow — where output quality and speed both increase
- dramatically.[^10][^9]
-- **Rest as a multiplier**: recovery is not the absence of work; it is the
- biological mechanism through which learning consolidates, creativity
- regenerates, and performance sustains.[^11][^12]
-- **Decision bandwidth**: every additional commitment consumes decision-making
- capacity. Reducing commitments preserves the clarity needed to act well on the
- ones that remain.[^6][^5]
-
-## The principle in practice
-
-Applying this requires active subtraction — deciding what _not_ to do is harder
-than adding more.[^2][^13]
-
-1. **Identify your high-leverage work**
- - Ask: "What is the one thing, that if done well, makes everything else
- easier or unnecessary?"[^14]
- - Protect that work with time blocks that are defended like appointments.[^9]
-2. **Prune ruthlessly**
- - Audit your commitments regularly. For each one ask: "Would I agree to this
- today if it were not already on my list?"[^13][^2]
- - If the answer is no, it is a candidate for elimination or delegation.
-3. **Embrace strategic incompleteness**
- - Not every email needs a reply. Not every meeting needs your presence. Not
- every idea needs to be pursued.[^4][^3]
- - Choosing to leave some things undone is a productive act, not a failure.
-4. **Work in cycles, not sprints**
- - Pair focused effort with genuine recovery. The ultradian rhythm — roughly
- 90 minutes of focused work followed by rest — aligns with natural peaks of
- cognitive performance.[^11][^12]
- - Pushing through fatigue does not extend your output; it degrades it.
-5. **Measure outcomes, not hours**
- - Shift your self-assessment from "how long did I work?" to "what did I
- actually move forward?"[^1][^8]
- - This reframes rest and leisure as inputs, not indulgences.
-
-## Where resistance comes from
-
-The practical obstacles to doing less are rarely logistical — they are
-psychological.[^3][^13]
-
-- **Identity**: many people have fused their sense of worth with their
- busyness. Slowing down can feel like a moral failure.[^4][^3]
-- **Social signalling**: being busy is culturally legible as being important.
- Choosing fewer things can feel socially costly.[^3]
-- **Fear of missing out**: every commitment declined feels like an opportunity
- lost, even when the real opportunity cost is the focus you could have applied
- elsewhere.[^2][^13]
-
-Recognising these forces allows you to choose differently — not because it is
-easy, but because you understand what the alternative actually costs.
-
-## A useful reframe
-
-Think of your attention as a currency. You can spend it in a hundred small
-places, or invest it in a few that compound. The goal is not to do as little as
-possible, but to do only what matters — and to do that well.[^1][^9][^14]
-
-The people who produce the most meaningful work over a lifetime are rarely those
-who attempted the most. They are those who maintained the discipline to say no
-clearly, and yes deeply.[^2][^9]
-
-<div align="center">⁂</div>
-
-[^1]: https://jamesclear.com/productivity
-
-[^2]: https://gregmckeown.com/books/essentialism/
-
-[^3]: https://www.apa.org/topics/mental-health/busyness-burnout
-
-[^4]: https://hbr.org/2019/03/to-be-more-productive-do-less
-
-[^5]: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7075496/
-
-[^6]: https://www.apa.org/research/action/multitask
-
-[^7]: https://www.ics.uci.edu/~gmark/CHI2005.pdf
-
-[^8]: https://tim.blog/2007/07/11/the-4-hour-workweek-first-2-chapters/
-
-[^9]: https://calnewport.com/books/deep-work/
-
-[^10]: https://www.researchgate.net/publication/224927532_Flow_The_Psychology_of_Optimal_Experience
-
-[^11]: https://www.sleepfoundation.org/sleep-science/ultradian-rhythm
-
-[^12]: https://hbr.org/2010/10/the-productivity-paradox-how-s
-
-[^13]: https://sive.rs/hellyeah
-
-[^14]: https://www.the1thing.com/
diff --git a/content/personal_development/egoless_programming.md b/content/personal_development/egoless_programming.md
@@ -1,95 +0,0 @@
----
-title: Egoless Programming
----
-
-1. **Understand and accept that you will make mistakes.**
-
- No one writes perfect code. Bugs, oversights, and misjudgments happen to
- everyone, including experts. The key is to treat mistakes as opportunities to
- learn rather than sources of embarrassment. An egoless programmer fixes
- issues calmly, analyzes why they happened, and improves their process instead
- of defending errors.
-
- Example: When a bug is found during testing, thank the person who caught it
- instead of feeling attacked.
-
-2. **You are not your code.**
-
- Code reviews exist to improve the product, not to judge the programmer. The
- goal is to find bugs, catch design flaws, and ensure consistency — not to
- criticize someone’s abilities. When your code is reviewed, remember it’s a
- normal, expected part of building reliable software. Feedback is about making
- the code better together, not about you being wrong.
-
- Example: Think, “We’re finding issues in the code so it’s stronger,” not
- “They’re finding faults in me.”
-
-3. **No matter how much “karate” you know, someone else will always know more.**
-
- There’s always a more experienced or specialized developer out there.
- Recognizing this keeps you humble and open to learning. Instead of competing
- for superiority, treat discussions as chances to improve together. Curiosity
- will make you a better programmer than pride ever could.
-
-4. **Don’t rewrite code without consultation.**
-
- Even if existing code looks inefficient or messy, consult the team before
- reworking it. There’s a fine line between fixing code and rewriting it — know
- the difference. Pursue changes as part of a code review, not as a lone
- enforcer. Unilateral rewrites can waste time, break dependencies, or erase
- important context.
-
- Example: Before refactoring a module, talk to the original author to
- understand the decisions and constraints behind it.
-
-5. **Treat people who know less than you with respect, deference, and
- patience.**
-
- Mentorship builds stronger teams. Talking down to less experienced developers
- discourages them and weakens collaboration. Share knowledge with patience and
- humility — remember, you were once a beginner too. Nontechnical colleagues
- often see developers as prima donnas or overly sensitive; don’t reinforce
- that stereotype with anger or impatience. The way you treat newcomers and
- teammates sets the tone for your team’s culture and reputation.
-
-6. **The only constant in the world is change.**
-
- Technology evolves quickly—languages, frameworks, and best practices
- constantly shift. Adaptability is more important than mastery of a single
- tool. Be ready to learn new approaches and let go of old habits.
-
- Example: Moving from monoliths to microservices, or from manual deployments
- to CI/CD pipelines, requires flexibility.
-
-7. **The only true authority stems from knowledge, not from position.**
-
- Titles and hierarchy shouldn’t override logic or facts. In a healthy
- engineering culture, ideas win on merit, not on who said them. Leaders should
- earn respect through understanding and clear reasoning, not through formal
- rank.
-
-8. **Fight for what you believe, but gracefully accept defeat.**
-
- If you think something is important, advocate for it with evidence and
- conviction. But once the team decides, support the outcome even if it wasn’t
- your proposal. This balance between assertiveness and teamwork builds trust
- and keeps projects moving forward.
-
-9. **Don’t be “the guy in the room.”**
-
- This means don’t act like you’re smarter than everyone else. The “guy in the
- room” (or “hero programmer”) dominates conversations, dismisses others, and
- resists compromise. Avoid that behavior; it alienates teammates and stifles
- creativity. Good programmers make the entire team better, not just
- themselves.
-
-10. **Critique code instead of people – be kind to the coder, not to the code.**
-
- When reviewing code, focus on improving it, not judging the person behind
- it. Tone matters: say “This function could be clearer if…” rather than “You
- didn’t write this well.” Constructive feedback builds confidence and shared
- understanding.
-
-([The Psychology of Computer Programming](https://archive.org/details/psychologyofcomp00unse)
-by Gerald M. Weinberg, describes principles for healthy, collaborative software
-development that values humility, learning, and teamwork over ego or rivalry.)
diff --git a/content/personal_development/engineering_competency.md b/content/personal_development/engineering_competency.md
@@ -1,31 +0,0 @@
----
-title: Engineering Competency
----
-
-## Levels of Competence
-
-- Level 1 - Unconscious Incompetence
-- Level 2 - Conscious Incompetence
-- Level 3 - Conscious Competence
-- Level 4 - Unconscious Competence
-
-## Zone of Proximal Development
-
-- Be able to identify gaps of knowledge and expanding gradually through Zone of
- Proximal Development.
-
-- So no jumping ahead, and no deliberately trying to be fast.
-
-- Need to work on accuracy first and then consistency so fluency / speed will
- follow naturally.
-
-- Also that trying to learn something too far away from current comfort zone is
- unachievable - need to expand gradually through both vertical and lateral
- learning challenges.
-
-- There is no such thing as unconscious learning, learning has to be conscious.
- Conscious Learning.
-
-## Engineering Competency Matrix
-
-[Circle CI Engineering Competency Matrix](/assets/circle_ci_engineering_competency_matrix.xlsx)
diff --git a/content/personal_development/first_principles.md b/content/personal_development/first_principles.md
@@ -1,166 +0,0 @@
----
-title: First Principles
----
-
-# Learning from first principle
-
-Learning from first principles means stripping away “this is how it’s done” and
-rebuilding your understanding from basic truths you can justify yourself.[^1][^2]
-
-## What “first principles” actually means
-
-- It is a way of thinking where you break a problem or topic into its
- **fundamental** components, then reason upward from those.[^2][^3]
-- Instead of copying existing methods (analogies, best practices, standard
- curricula), you ask what is absolutely true, what is just habit, and what
- follows logically from the basics.[^4][^1]
-- Elon Musk popularised this in business and engineering by breaking complex
- things (like rockets, batteries, car costs) into raw materials, physics, and
- constraints, then designing from there.[^5][^3][^2]
-
-A simple illustration: rather than asking “How do I build a better horse
-carriage?”, you ask “What is transportation fundamentally? Move people from A to
-B, with safety, speed, cost, comfort constraints” and you may end up with a car
-instead of a better carriage.[^6][^7]
-
-## The core steps (general recipe)
-
-Most guides converge on a similar practical loop you can apply to anything.[^3][^7][^1][^4]
-
-1. Clarify the problem or question
- - State what you’re trying to understand or improve in one clear sentence.[^8][^3]
- - Example: “I want to understand how learning actually works so I can study
- more effectively.”
-2. List and question your assumptions
- - Write down what you “think you know” or what everyone seems to believe
- about this topic.[^1][^4][^3]
- - For each item, ask:
- - “How do I know this is true?”
- - “What if the opposite were true?”[^4][^3][^1]
-3. Deconstruct to fundamental truths
- - Break the topic into the simplest elements that are as close as possible to
- facts of reality: physics, biology, logic, basic data, clear cause–effect.[^7][^2][^3]
- - Fundamental truths are things you could defend with evidence or clear
- reasoning, not just “everyone says so.”[^3][^1][^4]
-4. Rebuild from the ground up
- - Starting only from your fundamentals, reconstruct an explanation, method,
- or plan.[^2][^7][^3]
- - Ask: “Given only these basics, what follows? What’s the simplest solution
- that fits them?”[^2][^3]
-5. Test, get feedback, refine
- - Try your new understanding or method in reality, observe results, then
- adjust your fundamentals or reasoning if needed.[^5][^3]
-
-## How to use it to _learn_ any subject
-
-Here’s how to apply this when learning something new (math, physics,
-programming, finance, etc.).[^9][^7][^3]
-
-1. Start with ultra-basic questions
- - “What is this thing, in the simplest possible terms?”
- - “What problem is this concept solving?”[^7]
- - Example (calculus): “A derivative is fundamentally a way to describe how
- fast something changes.”
-2. Build a minimal foundation
- - Identify the prerequisite ideas you genuinely need, and learn those clearly
- (even if it means going “backwards” to simpler topics).[^9][^3][^7]
- - Use the Feynman approach: try explaining the concept as if to a child; gaps
- reveal what is not fundamental or not understood yet.[^7]
-3. Constantly interrogate explanations
- - When you read or watch something, pause and ask:
- - “Which parts are definitions or conventions, and which parts are
- unavoidable truths?”
- - “Could this be different in another system or context?”[^1][^4][^3]
-4. Connect to reality and intuition
- - Tie concepts to real or visual examples until they “click”.[^9][^7]
- - Example: for probability, think in terms of games, lotteries, or real-life
- risks; for physics, think in terms of everyday forces and motions.[^9]
-5. Practice “why” chains
- - Use “Five Whys” or similar: keep asking “why?” until you hit something you
- can’t reasonably question further (a first principle or definition).[^3]
- - Example:
- - “Why do spaced repetitions help memory?” → Because repeated recall
- strengthens neural connections.
- - “Why does recall matter?” → Because memory is about reactivating patterns
- of activity in the brain, and practice makes them easier to trigger.
-
-## Concrete daily exercises to build the habit
-
-To _learn how to learn_ from first principles, turning it into a daily habit is
-crucial.[^3]
-
-Try these simple exercises:
-
-- One-problem deconstruction (5–10 minutes)
- - Pick any small problem (how you organise your day, why you procrastinate,
- how you cook something).
- - Write: problem → assumptions → fundamentals → one improved solution.[^8][^3]
-- Explain-it-like-I’m-10 journal
- - After studying something, write a one-paragraph explanation as if to a
- child.
- - Highlight any sentence where you hid behind jargon; revisit that part of the
- material.[^7]
-- “Opposite day”
- - Take a common belief in your field (“you must take notes by hand”, “you need
- 8 hours to be productive”, etc.).
- - Argue the opposite for a few minutes; look for data or cases that support
- it.[^4][^1][^3]
-- Weekly “rebuild”
- - Once a week, choose one skill or topic you use often (how you study, how you
- work out, how you manage tasks).
- - Strip it to fundamentals and redesign your method from scratch.[^2][^3]
-
-## Example: learning more effectively (from first principles)
-
-Let’s walk through a quick example applying this to “how to study”.
-
-1. Clarify
- - “I want to learn faster and remember more from what I read.”
-2. Question assumptions
- - Assumptions might include: “Re-reading is good studying”, “long sessions
- are better than short ones”, “highlighting equals learning.”[^1][^4]
-3. Deconstruct learning to fundamentals
- - Basic truths from cognitive science:
- - Memory improves with active recall, not just exposure.
- - Spaced repetition beats cramming.
- - Understanding (connecting ideas) leads to longer retention than rote
- memorisation.[^3][^9][^7]
-4. Rebuild a study method
- - From those fundamentals, a study system might be:
- - Short sessions with frequent recall (close the book and explain from
- memory).
- - Use flashcards spaced over days instead of single long cram sessions.
- - After reading, make a simple concept map linking ideas instead of just
- highlighting.[^9][^7][^3]
-5. Test and refine
- - Try this for two weeks, track how much you remember on quick self-tests,
- then adjust which tactics actually move the needle.
-
-If you repeat this pattern on different parts of your life, you train yourself
-to default to first principles thinking rather than habits or expert “scripts”.[^7][^3]
-
-What area do you most want to apply first-principles learning to right now
-(e.g., math, career decisions, productivity, something else)?
-<span style="display:none">[^10]</span>
-
-<div align="center">⁂</div>
-
-[^1]: https://fs.blog/first-principles/
-
-[^2]: https://blog.btrax.com/first-principles-thinking/
-
-[^3]: https://www.maray.ai/posts/first-principles-thinking
-
-[^4]: https://www.readynorth.com/blog/what-is-first-principles-thinking
-
-[^5]: https://theinvisiblementor.com/using-first-principles-to-approach-difficult-problems-like-elon-musk/
-
-[^6]: https://www.youtube.com/watch?v=NV3sBlRgzTI
-
-[^7]: https://modelthinkers.com/mental-model/first-principle-thinking
-
-[^8]: https://www.rhysthedavies.com/first-principle-thinking/
-
-[^9]: https://www.reddit.com/r/Physics/comments/11nrl20/how_do_i_embrace_the_first_principles_reasoning/
-
-[^10]: https://www.youtube.com/watch?v=ooTnMMnrOTo
diff --git a/content/personal_development/index.md b/content/personal_development/index.md
@@ -1,5 +0,0 @@
----
-title: Personal Development
----
-
-Knowledge about Personal Development
diff --git a/content/personal_development/learn_how_to_learn.md b/content/personal_development/learn_how_to_learn.md
@@ -1,31 +0,0 @@
----
-title: Learn How To Learn
----
-
-> If you really want to learn, and you want to learn more quickly and more
-> deeply, pay attention to how you pay attention. Think about how you think.
-> Learn how you learn.
-
-> Most of us did not take courses on metacognition or learning theory when we
-> were growing up. We were expected to learn, but rarely taught to learn.
-
-(Head First Design Patterns)
-
-1. Slow down. The more you understand, the less you have to memorize. - Don’t
- just read. Stop and think.
-
-2. Do the exercises. Write your own notes. - Use a pencil. There’s plenty of
- evidence that physical activity while learning can increase the learning.
-
-3. There are No Dumb Questions
-
-4. Talk about it. - Speaking activates a different part of the brain.
-
-5. Listen to your brain. - Pay attention to whether your brain is getting
- overloaded.
-
-6. Feel something! Get involved with the stories. Make up your own captions or
- images about the topics.
-
-7. Design something! Apply the knowledge to something new you’re designing, or
- refactor an older project.
diff --git a/content/personal_development/rule_of_the_vital_few.md b/content/personal_development/rule_of_the_vital_few.md
@@ -1,67 +0,0 @@
----
-title: The Rule of the Vital Few
----
-
-# The Rule of the Vital Few: Leveraging the Pareto Principle for Deep Work
-
-The "Rule of the Vital Few" is a conceptual framework derived from the Pareto Principle, which observes that in many systems, roughly 80% of the effects come from 20% of the causes.[^1][^2] While often simplified as the "80/20 Rule," the term "Vital Few" was specifically coined by Joseph Juran to distinguish between the critical factors that drive a system and the "trivial many" that create noise without significant value.[^3]
-
-When applied to personal productivity and cognitive performance, this rule suggests that a small minority of your activities produce the vast majority of your meaningful progress.
-
-## The Dichotomy: Vital Few vs. Trivial Many
-
-To achieve peak performance, one must first categorize their workload into two distinct groups:
-
-- **The Vital Few**: High-leverage activities (HLAs). These are tasks that require intense concentration, complex problem-solving, and creative synthesis. They are the activities that move the needle on your long-term goals (e.g., writing a complex architecture doc, learning a new language, deep strategic planning).[^4]
-- **The Trivial Many**: Shallow work. These are logistical or administrative tasks that can be performed while distracted. They are necessary for maintenance but do not create significant value on their own (e.g., clearing an inbox, attending status meetings, routine data entry).[^4]
-
-The danger in most professional environments is the **"Efficiency Trap"**: the tendency to spend 80% of one's energy optimizing the trivial many, leaving only 20% for the vital few.
-
-## The Synergy with Deep Work
-
-Deep Work—the ability to focus without distraction on a cognitively demanding task—is the only mechanism capable of executing the "Vital Few" effectively.[^4]
-
-Because the vital few tasks are complex, they require a state of flow and the absence of "attention residue." Attention residue occurs when you switch from a task (even a trivial one) to another; a part of your attention remains stuck on the previous task, reducing your cognitive capacity for the new one.[^5]
-
-If you intersperse the vital few with the trivial many (e.g., checking email every 15 minutes while coding a complex algorithm), you never reach the depth required to produce high-quality output. You are essentially attempting to perform "Vital" work with "Trivial" cognitive resources.
-
-## Applying the Rule to Minimize Distraction
-
-To minimize distraction, you must shift your perspective: distraction is not just a lack of willpower, but a failure to protect the vital few from the trivial many.
-
-### 1. The Pareto Audit
-List every activity you perform in a typical week. Assign a "Value Score" (1-10) based on how much it contributes to your primary long-term objective.
-- **Vital Few**: Tasks scoring 8–10.
-- **Trivial Many**: Tasks scoring 1–5.
-
-### 2. Aggressive Pruning
-Once identified, apply the following strategies to the trivial many:
-- **Eliminate**: Does this task actually need to happen?
-- **Automate**: Can a script or tool handle this?
-- **Delegate**: Can this be handled by someone else?
-- **Batch**: Group all trivial tasks into a single "shallow work" block (e.g., 4 PM to 5 PM).
-
-### 3. Guarding the Deep Work Block
-Schedule your "Vital Few" tasks during your peak biological energy windows (typically the morning). During these blocks:
-- **Zero-Interrupt Policy**: Turn off all notifications.
-- **Physical Separation**: If possible, change your environment to signify the shift from "trivial" to "vital" mode.
-- **Single-Tasking**: Commit to one vital task. Multitasking is the fastest way to turn a vital activity into a trivial one.
-
-## The Practical Framework for Implementation
-
-| Step | Action | Objective |
-| :--- | :--- | :--- |
-| **Audit** | Log time for 5 days $\rightarrow$ Value Score | Identify the 20% (Vital Few) |
-| **Filter** | Prune $\rightarrow$ Batch $\rightarrow$ Automate | Shrink the 80% (Trivial Many) |
-| **Schedule** | Time-block morning for HLAs | Allocate peak energy to high value |
-| **Protect** | Digital detox $\rightarrow$ Do Not Disturb | Eliminate attention residue |
-
-By consciously ignoring the trivial many during your deep work windows, you are not being unproductive—you are optimizing your cognitive throughput for the activities that actually matter.
-
-<div align="center">⁂</div>
-
-[^1]: https://fs.blog/pareto-principle/
-[^2]: https://en.wikipedia.org/wiki/Pareto_principle
-[^3]: https://jerimaerowley.com/blog-topics/business/80-20-rule-of-the-vital-few
-[^4]: https://www.calnewport.com/books/deep-work/
-[^5]: https://hbr.org/2010/12/the-cost-of-fast-switching
diff --git a/content/personal_development/social_media_detox.md b/content/personal_development/social_media_detox.md
@@ -1,189 +0,0 @@
----
-title: Social Media Detox
----
-
-# Social Media Detox — Distancing Yourself to Reclaim Deep Focus
-
-> _"The ability to perform deep work is becoming increasingly rare at exactly
-> the same time it is becoming increasingly valuable in our economy."_ — Cal
-> Newport, _Deep Work_
-
----
-
-## The Fragmentation Problem
-
-Every time you open Twitter, scroll through Instagram, or check a notification,
-you are not just losing a few seconds — you are fragmenting the architecture of
-your attention. Cal Newport calls this the shift from **deep work** to **shallow
-work**: the replacement of cognitively demanding, value-producing concentration
-with noncognitively demanding, easily replicated tasks performed while
-distracted.
-
-The average knowledge worker, according to a 2012 McKinsey study Newport cites,
-spends more than 60 percent of their workweek engaged in electronic
-communication and internet searching. This is not productivity — it is the
-_simulacrum_ of productivity. Busyness masquerading as output.
-
-Social media amplifies this fragmentation to an extreme. Platforms are not
-neutral tools. They are engineered to colonize attention: personalized feeds,
-intermittent variable rewards, notification pulses, and infinite scroll are
-deliberate design choices made to maximize time-on-platform. Newport is blunt
-about this: these services are "products, developed by private companies, funded
-lavishly, marketed carefully, and designed ultimately to capture then sell your
-personal information and attention to advertisers."
-
----
-
-## The Deep Work Lens
-
-Newport defines **deep work** as: _"Professional activities performed in a state
-of distraction-free concentration that push your cognitive capabilities to their
-limit. These efforts create new value, improve your skill, and are hard to
-replicate."_
-
-Its opposite — **shallow work** — is what social media produces and demands.
-Liking posts, replying to threads, and composing 280-character opinions are
-shallow by definition. They require no cognitive depth, they produce no lasting
-value, and crucially, they _train_ your brain for distraction.
-
-This is not a metaphor. Newport references Nicholas Carr's argument that
-spending prolonged time in a state of frenetic shallowness permanently reduces
-your capacity to concentrate. Every hour you spend in distracted scrolling is an
-hour spent _practicing_ an inability to focus. The neurological muscle
-responsible for sustained concentration atrophies — not through disuse alone,
-but through active misuse.
-
----
-
-## The 30-Day Experiment — Newport's Prescription
-
-Newport proposes a direct intervention: a **30-day cold-turkey abstention** from
-all social media services. Not a deactivation announcement, not a dramatic
-goodbye post — just stop, quietly, without ceremony. He gives two specific
-reasons for the silence:
-
-1. Announcing it feeds the same validation-seeking behavior that social media
- exploits
-2. It forces an honest reckoning: will anyone notice? Will anything suffer?
-
-After 30 days, ask yourself two questions about each platform:
-
-- Would the last 30 days have been _notably_ better if I had used this service?
-- Did people _genuinely care_ that I was absent?
-
-For most people, the answer to both is no. This is not pessimism — it is a
-reality check on the inflated sense of importance that social media manufacture.
-Newport notes that the average Twitter user had around 208 followers at the time
-of writing. When you know 200 people opted in to receive your updates, it
-becomes easy to believe your activity matters. The 30-day gap dissolves that
-illusion.
-
----
-
-## Distancing from the Social Stream
-
-The 30-day experiment is a diagnostic. The deeper practice is structural
-distancing — rebuilding your life so that social media occupies the periphery
-rather than the center.
-
-This involves several shifts:
-
-**Treat tools instrumentally, not habitually.** Newport advocates the
-**craftsman approach** to tool selection: only adopt a network tool if its
-positive impacts on what you value substantially outweigh the negatives. This is
-a higher bar than "it might be occasionally useful." Apply it ruthlessly. Most
-social platforms fail it.
-
-**Protect your attention like a resource.** Newport observes that J.K. Rowling
-avoided social media throughout the writing of the Harry Potter series. Woody
-Allen never owned a computer. Neal Stephenson makes himself near-impossible to
-reach electronically. These are not eccentricities — they are strategic acts of
-cognitive self-preservation.
-
-**Replace the vacuum.** Newport draws on Arnold Bennett's insight from a century
-ago: if you leave leisure time unstructured, the lowest-friction option fills
-it. Social media thrives in vacuums. The solution is not willpower but
-pre-commitment — decide what your evenings and free hours are _for_ before they
-begin. Read a deliberately chosen book. Exercise. Pursue a structured hobby.
-Give your brain something real to do.
-
----
-
-## A Detox from Society's Noise
-
-There is a broader dimension beyond productivity. The case for distancing from
-social media is also a case for distancing from the **ambient noise of
-collective opinion** — the reflexive takes, the outrage cycles, the performative
-discourse that constitutes most of what flows through social feeds.
-
-Carl Jung did not retreat to his stone tower in Bollingen merely to write
-faster. He retreated because deep intellectual work requires a kind of
-**internal silence** that constant social input dissolves. Newport quotes Jung
-directly: _"In my retiring room I am by myself. I keep the key with me all the
-time — no one else is allowed in there except with my permission."_
-
-That room is not a geographic location — it is a cognitive state. Social media
-is the lock pick that opens it uninvited, dozens of times a day. Quitting, or
-severely limiting, these platforms is one way of taking the key back.
-
-The modern version of this retreat does not require a lakeside tower. It
-requires:
-
-- Deleting apps from your phone (friction is a surprisingly effective deterrent)
-- Establishing phone-free periods — mornings, meals, the first hour after waking
-- Using a dedicated browser profile with social media blocked during work hours
-- Treating your attention as something given deliberately, not taken
- automatically
-
----
-
-## What You Gain
-
-Newport's own life is the clearest data point. Without social media — no
-Twitter, no Facebook, no Instagram — he published four books in ten years,
-earned a PhD, wrote peer-reviewed academic papers at a high rate, and was hired
-as a tenure-track professor at Georgetown, all while rarely working past six
-p.m. His evenings were present, his reading was genuine, and — his phrase is
-worth keeping — "the lack of distraction tones down that background hum of
-nervous mental energy that seems to increasingly pervade people's daily lives."
-
-That background hum is familiar. It is the low-grade anxiety of an always-open
-loop: what did someone post, what did someone reply, what is happening right now
-that you are not watching. Closing that loop — not temporarily, but structurally
-— is a form of mental hygiene.
-
-Depth is not nostalgia. In an economy that increasingly rewards the ability to
-learn difficult things quickly and produce at an elite level, distraction is not
-a neutral lifestyle choice. It is a competitive disadvantage that accumulates
-quietly over years.
-
----
-
-## Where to Begin
-
-If a permanent quit feels too drastic, start with the structure Newport
-outlines:
-
-1. **Run the 30-day experiment** — cold turkey, no announcements, honest audit
- afterward
-2. **Delete the highest-pull apps** from your phone first — Instagram and TikTok
- in particular are designed for mobile, idle-moment consumption
-3. **Establish a daily deep work block** — even 90 minutes of phone-off,
- notification-free work will begin rebuilding the attention muscle
-4. **Fill leisure time intentionally** — books, exercise, craft, in-person
- conversation; anything that requires and rewards sustained engagement
-5. **Evaluate each platform by the craftsman standard** — does this tool serve
- my top-tier goals substantially, or just marginally? If marginally, remove it
-
-The goal is not asceticism. It is reclaiming the cognitive architecture that
-makes meaningful work — and meaningful rest — possible.
-
----
-
-## References
-
-- Newport, Cal. _Deep Work: Rules for Focused Success in a Distracted World_.
- Grand Central Publishing, 2016.
-- Carr, Nicholas. _The Shallows: What the Internet Is Doing to Our Brains_. W.W.
- Norton, 2010.
-- Bennett, Arnold. _How to Live on 24 Hours a Day_. 1908.
diff --git a/content/personal_development/theory_of_knowledge.md b/content/personal_development/theory_of_knowledge.md
@@ -1,56 +0,0 @@
----
-title: Theory of Knowledge
----
-
-# Philoshophy
-
-## I think therefore I am.
-
-René Descartes discovered certainty by doubting everything until he found
-something indubitable: the act of thinking itself. Because he was thinking (and
-doubting), he knew he must exist, famously concluding "Cogito, ergo sum" ("I
-think, therefore I am"). He rejected sensory evidence, using radical doubt to
-establish that only the existence of his own mind is certain.
-
-### Question: How do we know anything?
-
-How to Know (Descartes’ Method):
-
-- **Method of Radical Doubt:** Descartes decided to reject any idea that could
- be doubted, including the existence of his body, the physical world, and
- sensory input, which he noted could be deceptive.
-
-- **The Cogito ("I think, therefore I am"):** Even if a "demon" is deceiving
- him, or he is dreaming, he must exist to be deceived or to have those
- thoughts. The very act of doubting one's existence confirms it.
-
-- **Rationalism:** Knowledge is not gained through senses but by clear and
- distinct perceptions of the mind (reason).
-
-- **God as Guarantee:** Descartes argued that because he has an idea of a
- perfect God, that perfect being must exist and would not allow him to be
- fundamentally deceived about reality.
-
-## Propositions
-
-```
- ____________ ___________
- / \ / \
- / X \
- / / \ \
- / / \ \
- / / \ \
-/ / KNOW- \ \
-\ TRUTHS \ LEDGE / BELIEFS /
- \ \ / /
- \ \ / /
- \ \ / /
- \ X /
- \____________/ \____________/
-```
-
-- Left circle: Truths (all true propositions).
-
-- Right circle: Beliefs (all propositions someone believes).
-
-- Overlap: Knowledge (propositions that are both true and believed).
diff --git a/content/personal_development/unconscious_thought_theory.md b/content/personal_development/unconscious_thought_theory.md
@@ -1,214 +0,0 @@
----
-title: Unconscious Thought Theory
----
-
-# The case for doing less thinking on purpose
-
-There is a counterintuitive finding in decision science: for complex problems
-with many variables, people who are _distracted_ after absorbing the relevant
-information often make better choices than people who sit down and deliberately
-think it through.[^1][^2] This is the core claim of Unconscious Thought Theory
-(UTT), proposed by Ap Dijksterhuis and Loran Nordgren in 2006. The broader
-implication—that idleness, mind-wandering, and strategic laziness are not flaws
-but cognitive tools—connects to a much older philosophical thread about the
-value of doing nothing.
-
-## Unconscious Thought Theory
-
-UTT distinguishes between two modes of thought: conscious and unconscious.
-Conscious thought is deliberate, sequential, and constrained by working memory
-(you can hold roughly seven items at once). Unconscious thought runs in the
-background with no such bottleneck—it processes in parallel, weighs attributes
-more naturally, and operates without the distortions introduced by
-overthinking.[^1][^3]
-
-The key experimental paradigm works like this: participants are given complex
-information (e.g., twelve attributes about each of four apartments), then split
-into three groups. One group decides immediately. One deliberates consciously
-for several minutes. One is distracted by a puzzle, then asked to decide.
-Consistently, the distracted group—the one that could not consciously
-deliberate—chose the best option more often than either of the other
-groups.[^1][^2]
-
-Dijksterhuis and Nordgren called this the **deliberation-without-attention
-effect**. The principle: for simple decisions (few variables), conscious thought
-is fine and often better. For complex decisions (many variables, competing
-trade-offs), unconscious thought tends to produce superior outcomes because it
-can integrate more information without the serial bottleneck of conscious
-attention.[^2][^4]
-
-### The six principles of UTT
-
-UTT rests on six distinguishing principles between conscious and unconscious
-thought.[^1][^3]
-
-1. **Unconscious Thought Principle.** Conscious thought is thought _with_
- attention; unconscious thought is thought _without_ attention directed at the
- problem.
-2. **Capacity Principle.** Conscious thought has low capacity (serial
- processing); unconscious thought has high capacity (parallel processing).
-3. **Bottom-Up vs. Top-Down Principle.** Unconscious thought works bottom-up,
- integrating information as it comes. Conscious thought works top-down, guided
- by schemas and expectations—which can introduce bias.
-4. **Weighting Principle.** Unconscious thought naturally weights the relative
- importance of different attributes. Conscious thought tends to distort
- weightings, overemphasising whatever is most salient or recently attended to.
-5. **Rule Principle.** Conscious thought follows strict rules and is precise.
- Unconscious thought gives rough estimates. This makes conscious thought
- better for simple, rule-based problems.
-6. **Convergence vs. Divergence Principle.** Conscious thought converges on a
- solution; unconscious thought diverges and explores. This is why unconscious
- processing is linked to creativity.
-
-### Criticisms and nuance
-
-UTT is not without controversy. Several replication attempts have failed to
-reproduce the deliberation-without-attention effect, particularly when the
-experimental design includes a clearly dominant alternative.[^5] Critics argue
-that what appears to be "unconscious thought" may simply be the absence of
-conscious overthinking—that distraction prevents people from _worsening_ an
-already-formed first impression, rather than enabling a deeper unconscious
-process.[^6] The effect also disappears when participants are primed to feel
-powerful, suggesting the mechanism may be more about restraining conscious
-interference than empowering unconscious processing.[^5]
-
-Regardless of the exact mechanism, the practical takeaway is robust: for
-complex, multi-variable decisions, sleeping on it or stepping away tends to
-produce better outcomes than grinding through the options consciously.
-
-## Being lazy ambitious
-
-There is a concept—sometimes called "lazy ambitious"—that describes people who
-hold strong long-term vision but resist the compulsion to fill every hour with
-visible productivity.[^7] The phrase sounds like a contradiction, and that is
-the point. Productivity culture insists that ambition must be expressed through
-relentless action: early mornings, optimised routines, quantified output. The
-lazy ambitious person rejects this, not out of apathy, but from an intuition
-that the mind's best work happens in the gaps between effort.
-
-Bill Gates captured a related idea: "I choose a lazy person to do a hard job.
-Because a lazy person will find an easy way to do it."[^8] The "laziness" here
-is actually efficiency—the refusal to accept unnecessary complexity, the drive
-to find a more elegant path. It is not about doing nothing; it is about doing
-less of the wrong thing.
-
-What looks like laziness from the outside is often unconscious processing on
-the inside. Carl Jung called it "active imagination"—the conscious mind steps
-back, and the unconscious delivers insights that no amount of forced effort
-could produce.[^7] The lazy ambitious person's downtime is not a bug. It is the
-incubation period where unconscious connections form.
-
-## The neuroscience of idleness
-
-When the brain is not focused on an external task, it does not go quiet. It
-activates the **default mode network (DMN)**—a set of interconnected regions
-associated with daydreaming, self-reflection, memory consolidation, and
-creative thinking.[^9][^10]
-
-Research has shown that the DMN is causally linked to creative fluency. Direct
-cortical stimulation of DMN nodes disrupts the ability to generate original
-ideas, confirming the network's role is not incidental but functional.[^10]
-High-creative individuals show greater resting-state functional connectivity
-between the inferior frontal gyrus (associated with cognitive control) and the
-entire default mode network, suggesting that creativity depends on cooperation
-between controlled and spontaneous processes.[^11]
-
-This is the mechanism behind the shower thought, the walk-and-eureka moment,
-the answer that arrives the morning after you gave up. The incubation effect—
-stepping away from a problem to let a solution surface—relies on the DMN
-quietly sifting through memories, associations, and patterns while the
-conscious mind is occupied elsewhere.[^9]
-
-## In Praise of Idleness
-
-Bertrand Russell argued in his 1935 essay _In Praise of Idleness_ that "a
-great deal of harm is being done in the modern world by belief in the
-virtuousness of work."[^12] His case was both economic and philosophical:
-modern technology had made it possible to produce enough for everyone with far
-less labour, yet society insisted on overwork as a moral duty. Russell proposed
-a four-hour workday, with the remaining time devoted to whatever the individual
-found meaningful.[^12][^13]
-
-Russell's sharpest observation was about what overwork does to the mind:
-
-> The modern man thinks that everything ought to be done for the sake of
-> something else, and never for its own sake.[^13]
-
-This is the cult of productivity taken to its endpoint—every action justified
-only by its instrumental value, every moment optimised for output. What gets
-lost is the capacity for play, for exploration, for the kind of undirected
-thought that UTT tells us is essential for complex problem-solving and
-creativity.
-
-Oscar Wilde put it more bluntly: "To do nothing at all is the most difficult
-thing in the world, the most difficult and the most intellectual."[^14]
-
-## The hypnagogic edge
-
-There is an even more extreme form of productive idleness: the **hypnagogic
-state**, the twilight zone between wakefulness and sleep. Research has shown
-that people in this state are three times more likely to discover hidden rules
-in mathematical problems.[^15] Paul McCartney composed "Yesterday" in this
-state. Edison famously napped holding steel balls so that the clang of them
-dropping would wake him at the precise edge of sleep, capturing the ideas that
-surfaced there.[^15]
-
-The hypnagogic state works for the same reason that unconscious thought works
-more generally: the conscious mind's filters are down. Mental boundaries become
-permeable, and ideas flow through from the subliminal mind without the usual
-editorial interference.[^15]
-
-## Practical implications
-
-None of this is a licence for chronic procrastination. The distinction between
-rest and avoidance matters—rest feels peaceful; avoidance feels anxious.[^7]
-The point is that deliberate idleness, strategically deployed, is not the
-opposite of productivity but a component of it.
-
-- **For complex decisions**: absorb the information, then do something
- unrelated. Let the unconscious integrate it. Decide later.
-- **For creative problems**: stop trying to force solutions. Walk, shower, nap.
- Let the default mode network do its work.
-- **For long-term ambition**: resist the pressure to optimise every hour. The
- lazy ambitious person's rhythm—bursts of intense work separated by genuine
- rest—is not a failure of discipline. It is a strategy.
-- **For learning**: after a deep study session, do nothing for a while. The
- consolidation that happens during rest is where understanding solidifies.
-
-Tim Ferriss summarised the trap neatly: "Being busy is a form of laziness—lazy
-thinking and indiscriminate action."[^16] The truly lazy thing is not to rest.
-It is to stay busy with the wrong work because it feels productive.
-
-<div align="center">⁂</div>
-
-[^1]: https://en.wikipedia.org/wiki/Unconscious_thought_theory
-
-[^2]: https://www.science.org/doi/10.1126/science.1121629
-
-[^3]: https://journals.sagepub.com/doi/10.1111/j.1745-6916.2006.00007.x
-
-[^4]: https://pubmed.ncbi.nlm.nih.gov/16484496/
-
-[^5]: https://www.cambridge.org/core/journals/judgment-and-decision-making/article/are-complex-decisions-better-left-to-the-unconscious-further-failed-replications-of-the-deliberationwithoutattention-effect/D2F0A25CA1F498259B90C3289FA54342
-
-[^6]: https://acmelab.yale.edu/sites/default/files/2011_unconscious_thought_theory_and_its_discontents.pdf
-
-[^7]: https://blog.digitaldeepak.com/p/the-dangerous-power-of-the-lazy-ambitious
-
-[^8]: https://www.goodreads.com/quotes/568877-i-choose-a-lazy-person-to-do-a-hard-job
-
-[^9]: https://www.bbc.com/worklife/article/20170414-why-idle-moments-are-crucial-for-creativity
-
-[^10]: https://www.nature.com/articles/s41380-021-01403-8
-
-[^11]: https://pmc.ncbi.nlm.nih.gov/articles/PMC4410786/
-
-[^12]: https://www.sloww.co/in-praise-of-idleness-bertrand-russell/
-
-[^13]: https://philosophybreak.com/articles/bertrand-russell-in-praise-of-idleness/
-
-[^14]: https://www.psychologytoday.com/us/blog/hide-and-seek/201410/the-psychology-of-laziness
-
-[^15]: https://www.psychologytoday.com/us/blog/out-of-the-darkness/202512/how-idleness-can-lead-to-genius
-
-[^16]: https://jamesclear.com/eisenhower-box
diff --git a/content/personal_development/whiteboard_effect.md b/content/personal_development/whiteboard_effect.md
@@ -1,64 +0,0 @@
----
-title: The Whiteboard Effect
----
-
-For some types of problems, working with someone else at the proverbial shared
-whiteboard can push you deeper than if you were working alone. The presence of
-the other party waiting for your next insight—be it someone physically in the
-same room or collaborating with you virtually—can short-circuit the natural
-instinct to avoid depth.
-
-## Why it works
-
-When you're alone with a hard problem, it's easy to reach a point of discomfort
-and unconsciously retreat: you check your phone, switch to an easier task, or
-convince yourself you've thought about it "enough." The whiteboard effect
-disrupts this escape pattern. Another person's attention creates a gentle,
-productive pressure—not the anxiety of being judged, but the accountability of
-being witnessed. You stay in the difficulty longer because walking away isn't a
-silent, private act anymore.
-
-## The role of the other person
-
-The other person doesn't need to be an expert, or even deeply familiar with the
-problem. Their value isn't necessarily in what they contribute directly. It's in
-what their presence compels you to do:
-
-- **Articulate your reasoning out loud.** Half-formed thoughts that feel clear
- in your head collapse the moment you try to explain them. The act of
- externalising forces rigour.
-- **Hold yourself to a higher standard of clarity.** You won't hand-wave past a
- weak link in your logic when someone is watching you build the chain.
-- **Resist the pull of shallow thinking.** The social contract of collaboration
- makes it harder to settle for a surface-level answer and move on.
-
-## When it's most useful
-
-The whiteboard effect is strongest for problems that require sustained
-concentration and a willingness to sit with ambiguity—architecture decisions,
-debugging complex systems, working through a proof, or designing something from
-first principles. These are exactly the kinds of problems where solo thinkers
-tend to bail out early, mistaking discomfort for a dead end.
-
-It is less about brainstorming (where quantity of ideas matters) and more about
-depth-first exploration (where the quality of reasoning matters).
-
-## Collaborative depth vs. performative collaboration
-
-This is not a case for more meetings. Most meetings diffuse focus rather than
-concentrate it. The whiteboard effect requires a specific setup:
-
-- **Small group.** Two people is ideal. Three can work. Beyond that, social
- dynamics take over and depth suffers.
-- **Shared artefact.** A whiteboard, a shared document, a terminal—something
- both parties are actively looking at and building on together.
-- **Mutual investment.** Both people need to be genuinely engaged with the
- problem, not just present in the room.
-
-## Applying it deliberately
-
-If you notice yourself repeatedly bouncing off a hard problem when working
-alone, that's a signal. Find someone—a colleague, a friend, even a rubber duck
-with a pulse—and work through it together. The goal isn't to outsource the
-thinking. It's to use the social dynamic to keep yourself honest and keep
-yourself going deeper than your solo instincts would allow.
diff --git a/content/personal_development/zeigarnik_effect.md b/content/personal_development/zeigarnik_effect.md
@@ -1,271 +0,0 @@
----
-title: Zeigarnik Effect
----
-
-# Why unfinished things won't leave you alone
-
-You are halfway through a book and put it down. For days, the unresolved plot
-occupies your thinking in idle moments—on the train, in the shower, before
-sleep. Then you finish it. Within a week, you can barely remember the ending.
-This asymmetry—where the incomplete grips the mind and the complete dissolves
-from it—is the Zeigarnik Effect, and it governs more of your cognitive life
-than you probably realise.
-
-## Origins: a waiter's memory
-
-The story begins in a Vienna café in the 1920s. Kurt Lewin, the Gestalt
-psychologist, was dining with his students when he noticed something odd about
-their waiter. The man could recall complex, multi-item orders with perfect
-accuracy—but only while the orders remained open. The moment a table's bill was
-settled, the details vanished from his mind as if they had never existed.[^1][^2]
-
-Bluma Zeigarnik, one of Lewin's doctoral students at the University of Berlin,
-turned this café observation into a research programme. In her 1927 paper
-_Über das Behalten von erledigten und unerledigten Handlungen_ (On Finished
-and Unfinished Tasks), published in _Psychologische Forschung_, she reported
-the results of a series of experiments that would give the phenomenon its
-name.[^1][^3]
-
-## The experiments
-
-Zeigarnik asked participants to complete between 15 and 22 tasks—a mixture of
-tactile work (stringing beads, assembling boxes) and mental work (solving
-puzzles, arithmetic). She allowed half the tasks to be completed normally and
-interrupted the other half at various points, removing the materials and
-directing the participant to something else. After an hour's delay, she asked
-participants to recall which tasks they had worked on.[^1][^2]
-
-The results were striking. Participants recalled interrupted tasks roughly
-twice as often as completed ones. In some conditions, recall of unfinished
-tasks was up to 90% better than for finished ones.[^1][^3] The effect was not
-uniform—tasks interrupted near the middle or end were remembered more vividly
-than those interrupted at the beginning, suggesting that the closer one gets to
-completion, the stronger the cognitive tension becomes.[^1]
-
-Zeigarnik ran four experiments in total, testing individual adults, groups of
-adults, and groups of adolescents. She also found that participants with higher
-levels of ambition showed a stronger effect, and that those who interpreted
-interruption as personal failure remembered the incomplete tasks even more
-tenaciously.[^1][^2]
-
-In two further small experiments, she interrupted tasks but then allowed
-participants to immediately resume and complete half of them. The tasks that
-were never allowed to reach completion were still the most readily recalled—
-confirming that it was the state of incompleteness, not the act of
-interruption, that drove the memory advantage.[^1]
-
-## Lewin's tension systems
-
-Zeigarnik's explanation drew directly from Lewin's field theory. Lewin proposed
-that when a person forms an intention to complete a task, it creates a
-**quasi-need**—a state of psychic tension within what he called the person's
-"life space." This tension is a motivational force: it energises the behaviour
-required to finish the task and keeps the relevant information accessible in
-memory. Completing the task discharges the tension. The quasi-need is
-satisfied, and the cognitive system can release the associated
-information.[^1][^4]
-
-An incomplete task, by contrast, leaves the tension system unresolved. The
-quasi-need persists, and with it, the heightened accessibility of everything
-related to the task. The mind does not hold onto unfinished business because it
-is anxious or neurotic—it holds on because, from the perspective of Lewin's
-motivational dynamics, the task is still active. The cognitive loop remains
-open.[^4]
-
-This framing connects the Zeigarnik Effect to the Gestalt law of closure: the
-mind's tendency to perceive incomplete patterns as demanding completion. An
-unfinished task is a perceptual gap, and the drive to close it keeps the
-relevant information foregrounded in memory.[^5]
-
-## The Ovsiankina Effect
-
-A related but distinct phenomenon was documented by Maria Ovsiankina, another
-of Lewin's students, around the same time. Where Zeigarnik studied memory
-for interrupted tasks, Ovsiankina studied the behavioural drive to resume them.
-She found that when participants were interrupted during a task and given the
-opportunity to return to it later, the vast majority spontaneously chose to do
-so—even when there was no external incentive to finish.[^6]
-
-The Zeigarnik Effect says: you will _remember_ what you did not finish. The
-Ovsiankina Effect says: you will _return_ to finish it. Together, they
-describe a complete motivational circuit—cognitive accessibility plus
-behavioural drive—both powered by the same underlying tension system.[^6]
-
-## Replication and controversy
-
-The Zeigarnik Effect is one of those findings that is more famous than it is
-reliable. A 1964 review by Butterfield concluded that the effect was "far from
-being the invariable result" of interrupted-task experiments, and that
-frequently more completed than uncompleted tasks were recalled.[^1][^7]
-Multiple subsequent attempts have produced mixed results, with some researchers
-arguing there is no universal pattern.[^1]
-
-The effect is sensitive to conditions that are difficult to control. It is
-weakened when participants are ego-involved in the task (failure threatens
-self-esteem, which can suppress rather than enhance recall of the unfinished
-task). It is more likely to appear when the interruption seems incidental
-rather than deliberate. And it disappears when participants conclude the task
-is impossible—if the loop cannot be closed, the tension system appears to shut
-down rather than persist indefinitely.[^1][^7]
-
-A 2025 meta-analysis published in _Humanities and Social Sciences
-Communications_ reviewed the accumulated evidence for both the Zeigarnik and
-Ovsiankina effects, finding that while the effects are real, their magnitude
-varies considerably with experimental design, participant personality, and the
-nature of the task.[^8]
-
-Despite the replication difficulties, the core phenomenon—that incompleteness
-creates a cognitive state qualitatively different from completion—remains well
-supported and continues to generate research across memory, motivation, and
-clinical psychology.[^1][^7]
-
-## Open loops and cognitive load
-
-The modern interpretation of the Zeigarnik Effect frames unfinished tasks as
-**open loops** in working memory. Each open loop consumes cognitive resources—
-not because you are actively thinking about the task, but because your brain
-maintains a background monitoring process to ensure the task is not
-forgotten.[^5][^9]
-
-This has a direct consequence: the more open loops you carry, the less
-cognitive bandwidth you have for the task in front of you. David Allen's
-Getting Things Done methodology is built, in part, on this insight. Allen
-argues that the act of capturing tasks in a trusted external system (a list, a
-tool, a notebook) relieves the psychic tension that Lewin described—not by
-completing the task, but by externalising the commitment. The mind can release
-the open loop because the system will remember on its behalf.[^9]
-
-Research supports this. A study by Masicampo and Baumeister (2011) found that
-simply making a specific plan for how and when to complete an unfinished task
-eliminated the Zeigarnik Effect on cognitive intrusion—even though the task
-itself remained incomplete. The plan served as a psychological proxy for
-completion, discharging the tension without requiring the actual work to be
-done.[^10]
-
-## Applications
-
-### Productivity
-
-The Zeigarnik Effect offers a counterintuitive strategy for beating
-procrastination: start the task, then stop. The hardest part of any project is
-often the beginning—the blank page, the empty editor, the first line of code.
-But once you begin, the Zeigarnik Effect creates its own gravitational pull.
-The open loop generates tension, and that tension becomes a motive force to
-return and continue.[^5][^9]
-
-Ernest Hemingway reportedly used this principle deliberately, always stopping
-his writing sessions mid-sentence so that he knew exactly where to pick up the
-next day. The incomplete sentence was a hook—a deliberate open loop that kept
-the work alive in his mind overnight.[^11]
-
-Breaking large projects into smaller subtasks leverages the same mechanism.
-Each subtask, once started, generates its own tension. Completing it provides
-a small discharge—a micro-reward that sustains momentum—while the larger
-project's incompleteness keeps the overall direction in mind.[^5]
-
-### Learning
-
-Educators have noted that the Zeigarnik Effect can enhance retention. If a
-study session is interrupted and resumed later, the material studied during
-that session tends to be better remembered. This aligns with the broader
-evidence for spaced practice: distributing learning across multiple sessions
-with gaps between them produces stronger long-term memory than massing
-practice into a single block.[^1][^12]
-
-The mechanism may be twofold. The interruption keeps the material cognitively
-accessible (Zeigarnik's tension), and the spacing allows for consolidation
-during the intervals (a complementary process involving memory
-stabilisation).[^12]
-
-### Marketing and media
-
-Advertisers and storytellers have exploited the Zeigarnik Effect for decades.
-The cliffhanger—ending an episode, chapter, or commercial on an unresolved
-note—is a deliberate open loop designed to ensure the audience returns. Early
-research by Heimbach (1972) demonstrated that interrupted television
-commercials were recalled significantly better than those shown in
-full.[^1][^13]
-
-Television series, serialised podcasts, and episodic games all rely on the
-same principle. The unresolved plot thread is not a flaw in the narrative—it
-is a cognitive hook that binds the audience to the next instalment.
-
-### Software and product design
-
-Progress bars, streak counters, and incomplete profile indicators in software
-products are all applications of the Zeigarnik Effect. LinkedIn's profile
-completion percentage, Duolingo's daily streaks, and progress indicators in
-onboarding flows all create visible open loops that exploit the user's drive
-toward closure.[^5]
-
-The design principle is straightforward: show people what they have not
-finished, and their own cognition will supply the motivation to complete it.
-
-## The dark side
-
-The Zeigarnik Effect is not always benign. Unfinished tasks that cannot be
-completed—unresolved conflicts, unanswered questions, ambiguous losses—can
-become sources of chronic rumination. The tension that motivates productive
-behaviour can, when there is no available action, turn into anxiety. The open
-loop that would have driven you back to finish a puzzle instead keeps you
-awake at 3 a.m. replaying a conversation that cannot be unsaid.[^9][^14]
-
-Clinically, intrusive thoughts in conditions like OCD and PTSD share
-structural similarities with the Zeigarnik Effect: the mind returns repeatedly
-to something unresolved, unable to discharge the tension because no completing
-action is available or sufficient. Research has examined the relationship
-between the Zeigarnik Effect and obsessive-compulsive behaviour, finding
-parallels in the cognitive mechanisms that drive both.[^1][^14]
-
-The antidote, where one exists, is the same principle that Masicampo and
-Baumeister identified: creating a plan, or finding a symbolic form of closure,
-can relieve the tension even when literal completion is impossible. Rituals of
-closure—writing an unsent letter, holding a ceremony, making a deliberate
-decision to let go—are psychological technologies for manually closing loops
-that would otherwise remain open indefinitely.[^10]
-
-## Practical implications
-
-- **To overcome procrastination**: do not wait for motivation. Start the task—
- even for five minutes—and let the Zeigarnik Effect supply the pull to
- continue.
-- **To manage cognitive load**: capture every open loop in a trusted external
- system. Your mind will release what it trusts something else to remember.
-- **To improve retention**: study in intervals. Interrupt yourself mid-topic
- and return later. The incompleteness keeps the material alive in memory.
-- **To sustain long projects**: stop each work session at a point of
- incompleteness—mid-paragraph, mid-function, mid-thought. The open loop will
- carry you back.
-- **To protect your peace**: recognise when an open loop cannot be closed by
- action. Make a plan, create symbolic closure, or consciously decide to
- release it. Not every loop deserves your cognitive resources.
-
-<div align="center">⁂</div>
-
-[^1]: https://www.simplypsychology.org/zeigarnik-effect.html
-
-[^2]: https://www.psychologistworld.com/memory/zeigarnik-effect-interruptions-memory
-
-[^3]: https://en.wikipedia.org/wiki/Bluma_Zeigarnik
-
-[^4]: https://en.wikipedia.org/wiki/Field_theory_(psychology)
-
-[^5]: https://learningloop.io/plays/psychology/zeigarnik-effect
-
-[^6]: https://en.wikipedia.org/wiki/Ovsiankina_effect
-
-[^7]: https://huskiecommons.lib.niu.edu/allgraduate-thesesdissertations/1287/
-
-[^8]: https://www.nature.com/articles/s41599-025-05000-w
-
-[^9]: https://hubstaff.com/blog/zeigarnik-effect/
-
-[^10]: https://psycnet.apa.org/record/2011-10096-001
-
-[^11]: https://www.masterclass.com/articles/how-to-avoid-procrastination
-
-[^12]: https://helio.app/ux-research/laws-of-ux/zeigarnik-effect/
-
-[^13]: https://taproot.com/zeigarnik-effect/
-
-[^14]: https://www.psychologytoday.com/us/basics/zeigarnik-effect
diff --git a/content/quantum_mechanics/index.md b/content/quantum_mechanics/index.md
@@ -1,3 +0,0 @@
----
-title: Quantum Mechanics
----
diff --git a/content/quantum_mechanics/michio_kaku.md b/content/quantum_mechanics/michio_kaku.md
@@ -1,14 +0,0 @@
----
-title: Michio Kaku
----
-
-Michio Kaku adapts the Kardashev scale to describe how advanced civilizations
-are by the amount of energy they can control: Type 0 (like us) uses limited
-planetary resources such as fossil fuels; Type I can harness all energy
-available on its planet; Type II can capture most of the energy of its star; and
-Type III can use the energy output of an entire galaxy. On top of that, you
-could imagine a speculative Type IV civilization that harvests energy embedded
-in the fabric of the universe itself, such as the ubiquitous dark-matter or
-dark-energy–like fields between galaxies, and a Type V that somehow taps into
-energy sources outside our own universe “grid,” drawing power from a
-higher-dimensional or multiverse realm beyond our spacetime.
diff --git a/content/quantum_mechanics/simulation_theory.md b/content/quantum_mechanics/simulation_theory.md
@@ -1,19 +0,0 @@
----
-title: Simulation Theory
----
-
-S. James Gates Jr. found that when he and his collaborators studied certain
-supersymmetry equations using diagrams called Adinkras, the mathematics
-naturally organized itself in terms of specific binary error‑correcting codes,
-similar to those used in digital communications and computers. He noted that
-these are “doubly‑even self‑dual” codes, a highly structured class of codes
-known from information theory, and that they appear to be built into the
-algebraic structure of these physical theories. Gates has described this as
-uncovering code‑like patterns embedded in the equations that describe
-fundamental particles, and he has suggested that this raises deep philosophical
-questions about whether reality might, at some level, have properties analogous
-to a computational or information‑based system.
-
----
-
-[Simulation paper by Dr. Roman Yampolskiy](https://www.researchgate.net/publication/369187097_How_to_Escape_From_the_Simulation)
diff --git a/content/refactoring/index.md b/content/refactoring/index.md
@@ -1,3 +0,0 @@
----
-title: Refactoring
----
diff --git a/content/refactoring/refactoring.md b/content/refactoring/refactoring.md
@@ -1,53 +0,0 @@
----
-title: Refactoring
----
-
-### Book: Refactoring: Improving the Design of Existing Code (Addison-Wesley Professional 1999)
-
-## The First Step in Refactoring
-
-> Whenever I do refactoring, the first step is always the same. I need to build
-> a solid set of tests for that section of code. The tests are essential because
-> even though I follow refactorings structured to avoid most of the
-> opportunities for introducing bugs, I'm still human and still make mistakes.
-> Thus I need solid tests.
-
-> An important part of the tests is the way they report their results. ... The
-> tests are thus self-checking. It is vital to make tests self-checking. If you
-> don't, you end up spending time hand checking some numbers from the test
-> against some numbers of a desk pad, and that slows you down.
-
-> As we do the refactoring, we will lean on the tests. I'm going to be relying
-> on the tests to tell me whether I introduce a bug. It is essential for
-> refactoring that you have good tests. It's worth spending the time to build
-> the tests, because the tests give you the security you need to change the
-> program later.
-
-### Before you start refactoring, check that you have a solid suite of tests. These tests must be self-checking.
-
-## Decomposing and Redistributing the Statement Method
-
-1. Overly Long Statement method:
-
-> The obvious first target of my attention is the overly long statement method.
-> When I look at a long method like that, I am looking to decompose the method
-> into smaller pieces. Smaller pieces of code tend to make things more
-> manageable. They are easier to work with and move around.
-
-- Extraction Method
-
-> When I extract a method, as in any refactoring, I need to know what can go
-> wrong. If I do the extraction badly, I could introduce a bug into the program.
-> So before I do the refactoring I need to figure out how to do it safely.
-
-> [!info] Tip: Refactoring changes the programs in small steps. If you make a
-> mistake, it is easy to find the bug.
-
-> Is renaming worth the effort? Absolutely. Good code should communicate what it
-> is doing clearly, and variable names are a key to clear code. Never be afraid
-> to change the names of things to improve clarity. With good find and replace
-> tools, it is usually not difficult. Strong typing and testing will highlight
-> anything you miss. Remember
-
-> [!info] Tip: Any fool can write code that a computer can understand. Good
-> programmers write code that humans can understand.
diff --git a/content/references/bookmarks.md b/content/references/bookmarks.md
@@ -1,7 +0,0 @@
----
-title: Bookmarks
----
-
-[The Software Architecture Chronicles by @hgraca](https://herbertograca.com/2017/07/03/the-software-architecture-chronicles/)
-
-[Code Reviews: Before You Even Run The Code](https://lornajane.net/posts/2015/code-reviews-before-you-even-run-the-code)
diff --git a/content/references/books.md b/content/references/books.md
@@ -1,132 +0,0 @@
----
-title: Books
----
-
-## TypeScript
-
-[TypeScript Deep Dive](https://basarat.gitbook.io/typescript/future-javascript/arrow-functions) -
-This is great and succinct and freely available online
-
-Effective TypeScript by Dan Vanderkam - very useful
-
-## Fundamentals of Software Engineering
-
-The Pragmatic Programmer by Andrew Hunt, David Thomas (The Pragmatic Programmer:
-Your Journey to Mastery, 20th Anniversary Edition)
-
-The Mythical Man-Month
-
-Become an Effective Software Engineering Manager
-
-Site Reliability Engineering: How Google Runs Production Systems
-
-Clean Code by Robert C Martin
-
-A Philosophy of Software Design
-
-## Additional Software Engineering Reading
-
-Fundamentals of Software Architecture: An Engineering Approach
-
-Code Complete by Steve McConnell
-
-JavaScript: The Good Parts
-
-Design Patterns: Elements of Reusable Object-Oriented Software
-
-Refactoring by Martin Fowler
-
-Test-Driven Development by Example by Kent Beck
-
-## Rust
-
-[Rust Atomics and Locks](https://mara.nl/atomics/)
-
-Programming Rust, 2nd Ed. — Jim Blandy & Jason Orendorff Why: The definitive
-reference on Rust’s ownership, borrowing, and unsafe code. Covers lifetimes,
-async/await, and concurrency primitives in depth. Written by core language
-designers.
-
-Rust in Action — Tim McNamara Why: Focuses on systems programming patterns
-(including allocators, FFI, and unsafe code) and includes practical async/Tokio
-examples. Strong on performance-aware design.
-
-## Low-Latency Systems, Hardware-Aware Optimization & Allocators
-
-Designing Data-Intensive Applications — Martin Kleppmann Why: Though not
-Rust-specific, it’s essential for understanding latency, consistency, and
-distributed system trade-offs. Covers observability, ordering, exactly-once
-semantics, and system boundaries—directly relevant to your messaging and
-observability needs.
-
-High Performance Browser Networking — Ilya Grigorik Why: Surprisingly
-relevant—covers TCP/UDP, zero-copy, kernel bypass concepts, and NUMA-aware
-design patterns. Though focused on browsers, its low-level networking insights
-translate directly to systems programming.
-
-The Linux Programming Interface — Michael Kerrisk Why: The most comprehensive
-reference on Linux syscalls, I/O models (including io_uring), and memory
-management. Critical for understanding zero-copy, custom allocators, and
-hardware-level profiling.
-
-## System Design, Architecture & Leadership
-
-System Design Interview (2 vols) — Alex Xu Why: Excellent for scaling and
-distributed system design patterns—especially useful for Kafka/NATS-style
-systems and observability architecture.
-
-Accelerate — Nicole Forsgren et al. Why: Evidence-based practices for DevOps,
-CI/CD, and performance regression detection. Directly supports your CI/CD and
-benchmarking-gates requirements.
-
-The Pragmatic Engineer — Gustavo Sampaio Why: Covers architecture reviews, ADRs,
-and engineering standards—ideal for leading Rust engineering culture
-
-## Observability, Testing & Reliability
-
-Observability Engineering — Candace Gonzalez et al. Why: Covers
-Prometheus/OpenTelemetry, tracing, alert design, and structured logging in
-production systems—aligned with your observability stack.
-
-Chaos Engineering — Casey Rosenthal et al. Why: The foundational text on fault
-injection, chaos testing, and resilience design—directly matches your “chaos and
-fault injection experience” requirement.
-
-Property-Based Testing with Rust — John A. De Goes (free online) Why: While not
-a traditional book, De Goes’ Modern Software Engineering and his PBT tutorials
-are widely used in Rust for property-based testing and TDD as design discipline
-
-## Cloud Infrastructure (Azure/AWS)
-
-Cloud Native Infrastructure — Karl Isenberg & George Miranda Why: Covers IAM,
-secrets management, and infrastructure-as-code patterns across cloud
-providers—including Azure-specific guidance.
-
-Designing Distributed Systems — Brendan Burns Why: Written by a Kubernetes
-co-creator; emphasizes patterns for reliability, observability, and managed
-services—highly applicable to Azure/AWS deployments
-
-## Supplemental (Free & Online)
-
-[The Rust Async Book](https://rust-lang.github.io/async-book/)
-
-[The Rustonomicon](https://doc.rust-lang.org/nomicon/)
-
-[Tokio docs & blog](https://tokio.rs/blog/)
-
-io_uring & DPDK deep dives (e.g.,
-[A Deep Dive into Zero-Copy Networking and io_uring](https://medium.com/@jatinumamtora/a-deep-dive-into-zero-copy-networking-and-io-uring-78914aa24029),
-[Zero-copy network transmission with io_uring](https://lwn.net/Articles/879724/))
-
-## Personal Development
-
-Deep Work by Carl Newport
-
-How to measure anything
-
-Tidy First
-
-Becoming Evil: How Ordinary People Commit Genocide and Mass Killing by James
-Waller
-
-The Art of Memory - Frances Yates
diff --git a/content/references/index.md b/content/references/index.md
@@ -1,5 +0,0 @@
----
-title: References
----
-
-Educational content
diff --git a/content/references/podcast.md b/content/references/podcast.md
@@ -1,5 +0,0 @@
----
-title: Podcast
----
-
-[The Philosophy of Software Design](https://newsletter.pragmaticengineer.com/p/the-philosophy-of-software-design)
diff --git a/content/references/rules_of_programming.md b/content/references/rules_of_programming.md
@@ -1,23 +0,0 @@
----
-title: Rob Pike's 5 Rules of Programming
----
-
-## Rob Pike's 5 Rules of Programming
-
-Rule 1. You can't tell where a program is going to spend its time. Bottlenecks
-occur in surprising places, so don't try to second guess and put in a speed hack
-until you've proven that's where the bottleneck is.
-
-Rule 2. Measure. Don't tune for speed until you've measured, and even then don't
-unless one part of the code overwhelms the rest.
-
-Rule 3. Fancy algorithms are slow when n is small, and n is usually small. Fancy
-algorithms have big constants. Until you know that n is frequently going to be
-big, don't get fancy. (Even if n does get big, use Rule 2 first.)
-
-Rule 4. Fancy algorithms are buggier than simple ones, and they're much harder
-to implement. Use simple algorithms as well as simple data structures.
-
-Rule 5. Data dominates. If you've chosen the right data structures and organized
-things well, the algorithms will almost always be self-evident. Data structures,
-not algorithms, are central to programming.
diff --git a/content/references/videos.md b/content/references/videos.md
@@ -1,21 +0,0 @@
----
-title: Videos
----
-
-[How To Learn Any New Skill So Fast It’s Unfair](https://www.youtube.com/watch?v=O46wlbRsAK4&list=PLpoNNzdnAf3afm46nLHjgCkJTZVoEYCJI&index=5)
-
-[Preventing the Collapse of Civilization / Jonathan Blow](https://www.youtube.com/watch?v=ZSRHeXYDLko)
-
-[How Societies Collapse](https://youtu.be/K-_l9jBGo74?si=4ej7u0EYO-WJAxsQ)
-
-[Into Eternity (Documentary)](https://www.youtube.com/watch?v=vpRBfTHQg3k)
-
-[Build your own libraries](https://youtu.be/p2aea9dytpE?si=Zc8udMsEVTkir8-L)
-
-[Spotify Engineering Culture - Part 1](https://www.youtube.com/watch?v=Yvfz4HGtoPc)
-
-[Spotify Engineering Culture - Part 2](https://www.youtube.com/watch?v=vOt4BbWLWQw)
-
-[Your codebase is NOT ready for AI (here's how to fix it)](https://www.youtube.com/watch?v=uC44zFz7JSM)
-
-[The Microservices Scam Nobody Talks About](https://www.youtube.com/watch?v=6e9B7q3gvYY)
diff --git a/content/system_administration/commands.md b/content/system_administration/commands.md
@@ -1,19 +0,0 @@
----
-title: Linux Commands
----
-
-# Docker Related Commands
-
-```bash
-docker ps \
---format '{{.Names | printf "%-20s"}} {{.Ports | printf "%-30s"}} {{.RunningFor}}' \
-| sort
-```
-
-```bash
-docker ps --format '{{.Names | printf "%-20s"}} {{.Ports}}' | sort
-```
-
-```bash
-docker ps -a --format '{{.Names}}: {{.Ports}}'
-```
diff --git a/content/system_administration/index.md b/content/system_administration/index.md
@@ -1,3 +0,0 @@
----
-title: System Administration
----
diff --git a/content/system_administration/linux_permissions.md b/content/system_administration/linux_permissions.md
@@ -1,90 +0,0 @@
----
-title: Linux Permissions
----
-
-### Question: What happens if I am on a server and then I ssh into this server again?
-
-Nothing special or dangerous happens by itself; you just get another shell on
-the same machine.
-
-Nothing special or dangerous happens by itself; you just get another shell on
-the same machine
-
-You now have two nested sessions:
-
-- outer: your original SSH (or local) shell
-
-- inner: the SSH session created from the server to itself.
-
-Processes in the inner session are separate from the outer one, with their own
-TTY, environment, history, etc.
-
-### Question: how to create a user on an archlinux server without sudo privilege but make it a member of docker group?
-
-#### Create the New User
-
-`useradd -m -G docker -s /bin/bash newusername`
-
-#### Set a Password
-
-`passwd newusername`
-
-#### Verify Group Membership
-
-```bash
-id newusername
-# Expected: uid=1001(newusername) gid=1001(newusername) groups=1001(newusername),999(docker)
-```
-
-#### Generate the Key Pair
-
-```bash
-ssh-keygen -t ed25519 -C "newusername@yourserver" -f /home/newusername/.ssh/id_ed25519 -N ""
-```
-
-#### Set Up authorized_keys
-
-```bash
-cat /home/newusername/.ssh/id_ed25519.pub >> /home/newusername/.ssh/authorized_keys
-```
-
-#### Fix Permissions
-
-SSH is strict about permissions — it will silently refuse to use keys with wrong
-ownership or modes:
-
-```bash
-chown -R newusername:newusername /home/newusername/.ssh
-chmod 700 /home/newusername/.ssh
-chmod 600 /home/newusername/.ssh/authorized_keys
-chmod 600 /home/newusername/.ssh/id_ed25519
-```
-
-#### Share the Private Key
-
-```bash
-ssh -i ~/.ssh/id_ed25519 newusername@your_server_ip
-```
-
-#### Setup Authorized Keys
-
-Confirm authorized_keys matches that key
-
-```bash
-sudo -u deploy cat /home/deploy/.ssh/ed25519.pub
-sudo -u deploy cat /home/deploy/.ssh/authorized_keys
-```
-
-Ensure authorized_keys contains the same line as ed25519.pub. If it doesn’t,
-append it:
-
-```bash
-sudo -u deploy sh -c 'cat ~/.ssh/ed25519.pub >> ~/.ssh/authorized_keys'
-chmod 600 /home/deploy/.ssh/authorized_keys
-```
-
-#### Test SSH Access
-
-```bash
-ssh -i /path/to/private_key -p SSH_PORT SSH_USER@SSH_HOST
-```
diff --git a/content/system_design/case_study/index.md b/content/system_design/case_study/index.md
@@ -1,3 +0,0 @@
----
-title: Case Studies
----
diff --git a/content/system_design/case_study/websocket_request_trackers.md b/content/system_design/case_study/websocket_request_trackers.md
@@ -1,580 +0,0 @@
----
-title: WebSocket request trackers
----
-
-### What createWsRequestTracker Does
-
-```ts
-export function createWsRequestTracker<Epoch extends boolean = false>(
- options: WsRequestTrackerOptions<Epoch>,
-): WsRequestTracker<Epoch> {
- // Creates a tracker that bridges WebSocket request/response pairs into Promises.
- // Handles timeouts and provides a clean API for awaiting responses.
- const { timeoutMs, enableEpochs = false } = options;
- const pending = new Map<string, PendingEntry>();
- // pending map to store in-flight WS requests, keyed by contractId
- let epoch = 0;
- // current epoch used for stale request handling (epoch mode)
-
- return {
- track<T = void>(
- contractId: string,
- ): Promise<Epoch extends true ? T | null : T> {
- // Start tracking a WS request by its contract ID
- if (enableEpochs) {
- epoch++;
- }
-
- const requestEpoch = epoch;
-
- return new Promise<any>((resolve, reject) => {
- const timeoutId = setTimeout(() => {
- const entry = pending.get(contractId);
- if (!entry) return; // already resolved/rejected
-
- pending.delete(contractId);
-
- if (enableEpochs && entry.epoch !== epoch) {
- entry.resolve(null);
- return;
- }
-
- let errorMessage =
- `WS request ${contractId} timed out after ${timeoutMs}ms — server did not respond`;
- reject(new WebError(errorMessage, "WsTimeoutError"));
- }, timeoutMs);
-
- pending.set(contractId, {
- resolve: resolve as (value: unknown) => void,
- reject,
- timeoutId,
- epoch: requestEpoch,
- });
- });
- },
-
- resolve(contractId: string, data?: unknown): void {
- // Resolve pending request for contractId
- const entry = pending.get(contractId);
- if (!entry) return;
-
- if (enableEpochs && entry.epoch !== epoch) {
- clearTimeout(entry.timeoutId);
- pending.delete(contractId);
- entry.resolve(null);
- return;
- }
-
- clearTimeout(entry.timeoutId);
- pending.delete(contractId);
- entry.resolve(data);
- },
-
- // Reject a pending request
- reject(contractId: string, error: WebError): void {
- const entry = pending.get(contractId);
- if (!entry) return;
-
- clearTimeout(entry.timeoutId);
- pending.delete(contractId);
- entry.reject(error);
- },
-
- // Bump epoch to stale-discard all in-flight resolves.
- invalidate(): void {
- if (!enableEpochs) return;
- epoch++;
- },
-
- // Reject all pending and clear all timeouts.
- dispose(): void {
- for (const [contractId, entry] of pending) {
- clearTimeout(entry.timeoutId);
-
- // Reject with disposed error
- let errMessage =
- `WS tracker disposed while request ${contractId} was still pending`;
- entry.reject(
- new WebError(errMessage, "WsTrackerDisposedError"),
- );
- }
- pending.clear();
- },
-
- // Check if request is pending
- hasPending(contractId: string): boolean {
- return pending.has(contractId);
- },
-
- // Get number of pending requests
- pendingCount(): number {
- return pending.size;
- },
- };
-```
-
-`createWsRequestTracker` is a Promise-WS bridge that converts WebSocket
-request/response pairs into synchronous-looking async operations. In a WebSocket
-protocol where you send a packet and receive a response later via a callback
-listener, this tracker provides a clean API to await the response as a Promise.
-
-### Core Problem It Solves
-
-Without this tracker:
-
-1. You send a WS packet (fire-and-forget)
-2. Later, a listener callback receives the response
-3. How do you connect step 2 back to step 1? How do you handle timeouts? How do
- you ensure stale responses don't hang your app?
-
-The tracker solves this by:
-
-• Creating a Promise when you call track(contractId) after sending a packet
-
-• Storing it keyed by the contract ID (packet UUID)
-
-• Resolving/rejecting it when the listener receives the response
-
-• Handling timeouts with automatic rejection
-
----
-
-### Workflow (Complete Flow)
-
-```text
-┌────────────────────────────────────────────────────────────────────────────┐
-│ 1. Component calls readFile() or saveFile() │
-│ (e.g., user clicks a file or presses Ctrl+S) │
-└────────────────────────────────────────────────────────────────────────────┘
- ↓
-┌────────────────────────────────────────────────────────────────────────────┐
-│ 2. EditorStore sends WS packet via readFileWsFn / saveFileWsFn │
-│ → Returns a contractId (UUID) │
-└────────────────────────────────────────────────────────────────────────────┘
- ↓
-┌────────────────────────────────────────────────────────────────────────────┐
-│ 3. tracker.track(contractId) creates a Promise & stores it in pending Map │
-│ → Returns Promise<T | null> (T=void by default, or T=fileData) │
-└────────────────────────────────────────────────────────────────────────────┘
- ↓
-┌────────────────────────────────────────────────────────────────────────────┐
-│ 4. User waits for server response (await track()) │
-└────────────────────────────────────────────────────────────────────────────┘
- ↓
-┌────────────────────────────────────────────────────────────────────────────┐
-│ 5. Server responds via WebSocket │
-│ → Listener (editorWebsocketEventHandler) receives packet │
-│ → Decodes the contract │
-└────────────────────────────────────────────────────────────────────────────┘
- ↓
-┌────────────────────────────────────────────────────────────────────────────┐
-│ 6. applyFileRead / applyFileWritten calls tracker.resolve() │
-│ → Looks up contractId in pending Map │
-│ → Clears timeout │
-│ → Calls resolve() callback with data │
-└────────────────────────────────────────────────────────────────────────────┘
- ↓
-┌────────────────────────────────────────────────────────────────────────────┐
-│ 7. await track() resolves, EditorStore returns typed result │
-│ → { kind: 'success', fileData } │
-│ → Or { kind: 'stale' } if epochs enabled and a newer request came in │
-└────────────────────────────────────────────────────────────────────────────┘
-```
-
----
-
-### Why We Need It for service-cad/frontend/web
-
-1. Type-Safe Async/Await Over WebSocket Protocol
-
-The WebSocket protocol is event-driven (fire-and-forget + callback), but Svelte
-components work better with await:
-
-```
- // Without tracker: messy callback style
- sendReadFile(path, (result) => {
- if (result.error) { /* handle */ }
- if (result.data) { /* handle */ }
- });
-
- // With tracker: clean async/await
- const response = await readFile(path);
- // Throws on error, returns data on success
-```
-
-2. Timeout Handling The tracker implements automatic timeouts via setTimeout. If
- the server doesn't respond within WEBSOCKET_TIMEOUT_MS, the promise rejects
- with a WsTimeoutError.
-
-3. Epoch Tracking (Last-Write-Wins) With enableEpochs: true: • User rapidly
- clicks between files → each click sends a request • Only the latest file's
- response matters (the others are "stale") • Stale responses resolve with null
- instead of hanging or updating stale UI • The caller checks if (response ===
- null) return { kind: 'stale' }
-
-4. Cleanup & Memory Management • dispose() rejects all pending promises and
- clears timeouts • Prevents memory leaks when components unmount • Used in
- Svelte onDestroy hooks
-
----
-
-### Two Modes of Operation
-
-Simple Mode (default, enableEpochs: false)
-
-• Every request gets its own independent promise
-
-• Used by loadProjectTracker and killSessionTracker
-
-• Example: loadProjectTracker.track('xyz') → always resolves when server
-responds
-
-Epoch Mode (enableEpochs: true)
-
-• Each track() increments a monotonic epoch counter
-
-• When resolve() is called, it checks if the entry's epoch matches current
-
-• Stale entries (older epoch) resolve with null instead of data
-
-• Used by readTracker, saveTracker, fileTreeTracker, compileTracker
-
----
-
-Key Methods
-
-| Method | Description |
-| ---------------------------- | ------------------------------------------------------- |
-| `track(contractId)` | Start tracking, returns `Promise<T |
-| `resolve(contractId, data?)` | Resolve pending request (called by listener) |
-| `reject(contractId, error)` | Reject on error (always surfaces, never stale-discard) |
-| `invalidate()` | Bump epoch, mark all pending entries as stale |
-| `dispose()` | Reject all pending + clear timeouts (component cleanup) |
-| `hasPending(contractId)` | Check if entry exists (used by listeners) |
-| `pendingCount()` | Number of unprocessed requests |
-
----
-
-Files Using This Tracker
-
-• codeEditor/wsHooks/tracker.ts: readTracker, saveTracker (epoch mode)
-
-• fileTree/wsHooks/tracker.ts: fileTreeTracker (epoch mode)
-
-• canvas/wsHooks/tracker.ts: compileTracker (default mode)
-
-• gitEditor/wsHooks/tracker.ts: gitTracker (epoch mode)
-
-• core/pageState/editorPage/tracker.ts: loadProjectTracker, killSessionTracker
-(default mode)
-
-All use WEBSOCKET_TIMEOUT_MS (configured elsewhere) for timeout duration.
-
-```ts
-import { WebError } from "@core/utils/errors/webError";
-
-/**
- * Internal bookkeeping for a single in-flight WS request.
- *
- * `resolve` accepts `unknown` rather than a generic `T` because the pending
- * map is type-erased — it stores entries for many different request types
- * simultaneously, so there's no single `T` that fits all entries. The
- * per-call type safety lives on the public `track<T>()` method instead.
- *
- * We use `unknown` over `any` deliberately: `unknown` forces a type
- * assertion at the one internal call site where we store the resolve
- * callback (`resolve as (value: unknown) => void`), making the type
- * erasure explicit and auditable. With `any`, TypeScript stops checking
- * entirely — if we accidentally passed the wrong value internally, the
- * compiler wouldn't catch it. The single assertion is worth the trade-off
- * for keeping the rest of the internals type-checked.
- */
-export type PendingEntry = {
- resolve: (value: unknown) => void;
- reject: (error: WebError) => void;
- timeoutId: ReturnType<typeof setTimeout>;
- epoch: number;
-};
-
-/**
- * Generic over Epoch — controls the return type of track<T>().
- *
- * When Epoch is false (default), track<T>() returns Promise<T>.
- * When Epoch is true, track<T>() returns Promise<T | null> because
- * stale responses resolve with null instead of hanging forever.
- *
- * TypeScript infers Epoch from the enableEpochs property value:
- * createWsRequestTracker({ timeoutMs: WEBSOCKET_TIMEOUT_MS, enableEpochs: true })
- * // → WsRequestTracker<true>, track returns Promise<T | null>
- *
- * createWsRequestTracker({ timeoutMs: WEBSOCKET_TIMEOUT_MS })
- * // → WsRequestTracker<false>, track returns Promise<T>
- */
-export type WsRequestTrackerOptions<Epoch extends boolean = false> = {
- timeoutMs: number;
- enableEpochs?: Epoch;
-};
-
-export type WsRequestTracker<Epoch extends boolean = false> = {
- /**
- * Start tracking a request. Returns a typed Promise that resolves
- * when resolve() is called with the matching contract ID.
- *
- * T defaults to void — callers that expect response data specify it:
- * await tracker.track<string[]>(contractId); // expects string[]
- * await tracker.track(contractId); // expects void
- *
- * With enableEpochs: true, returns Promise<T | null>. null indicates
- * a stale response — a newer track() call superseded this one. The
- * caller checks for null to detect staleness and return early.
- */
- track<T = void>(
- contractId: string,
- ): Promise<Epoch extends true ? T | null : T>;
-
- /**
- * Resolve a pending request with optional data.
- *
- * Accepts `unknown` because the tracker is type-erased internally —
- * it doesn't know which `T` the corresponding track<T>() call used.
- * The caller (WS listener) passes whatever the server sent, and the
- * awaiting handler receives it typed via the Promise<T> from track().
- *
- * With enableEpochs: true, stale entries (epoch doesn't match current)
- * are settled with null instead of the provided data. This ensures the
- * caller's promise always settles — its finally block runs, no leaked
- * closures.
- */
- resolve(contractId: string, data?: unknown): void;
-
- /** Reject a pending request with an error. Always surfaces (never stale-discarded). */
- reject(contractId: string, error: WebError): void;
-
- /** Bump epoch to stale-discard all in-flight resolves. No-op without enableEpochs. */
- invalidate(): void;
-
- /** Reject all pending and clear timers. Call on component destroy. */
- dispose(): void;
-
- /** Whether a tracked entry exists for this contract ID. */
- hasPending(contractId: string): boolean;
-
- /** Number of currently pending (unresolved) tracked requests. */
- pendingCount(): number;
-};
-
-/**
- * Creates a tracker that bridges WebSocket request/response pairs into Promises.
- *
- * The problem: WS senders fire-and-forget a packet, and the response arrives
- * later via a separate listener callback. There's no built-in way for the
- * caller to await the full round-trip, detect timeouts, or correlate a
- * response back to the request that triggered it.
- *
- * This tracker solves that by letting the caller `track(contractId)` after
- * sending a packet. This returns a Promise that resolves when the listener
- * calls `resolve(contractId)` with the matching ID, or rejects on timeout.
- *
- * Two modes of operation:
- *
- * 1. Simple tracking (enableEpochs=false, the default)
- * Every tracked request gets its own independent promise. resolve() always
- * delivers. This is the mode used by the git editor where each operation
- * (commit, stage, merge) is a distinct action that doesn't supersede others.
- *
- * 2. Epoch tracking (enableEpochs=true)
- * Each track() call increments a monotonic epoch counter. When resolve()
- * is called, it checks whether the pending entry's epoch matches the
- * current epoch. If stale (a newer request was tracked since), the entry
- * is settled with null — the awaiting promise resolves to null, signaling
- * staleness to the caller. This implements last-write-wins for rapid
- * sequential requests (e.g. clicking between files quickly — only the
- * most recent file read matters).
- *
- * Stale timeouts also settle with null (not rejection) — if a stale
- * request's timeout fires, it resolves with null rather than surfacing
- * an error for a request the user already navigated away from.
- *
- * reject() always surfaces regardless of epoch — errors should never be
- * silently swallowed.
- *
- * invalidate() bumps the epoch without tracking a new request, causing all
- * in-flight responses to be discarded when they arrive. Use this when you
- * want to cancel all pending without triggering error callbacks (e.g.
- * navigating away from the current context).
- */
-export function createWsRequestTracker<Epoch extends boolean = false>(
- options: WsRequestTrackerOptions<Epoch>,
-): WsRequestTracker<Epoch> {
- const { timeoutMs, enableEpochs = false } = options;
- const pending = new Map<string, PendingEntry>();
- let epoch = 0;
-
- return {
- /**
- * Start tracking a WS request by its contract ID (the packet UUID).
- * Returns a Promise that resolves when the listener calls resolve()
- * with the same contract ID, or rejects on timeout.
- *
- * Call this immediately after sending the packet:
- * const contractId = await sender(...);
- * await tracker.track(contractId);
- *
- * This creates a `Promise<T>` with a properly typed `resolve: (value: T) => void`
- * callback but then immediately erases it to `unknown` for storage. When the
- * promise resolves, the caller gets T back. The erasure is a one-line bridge
- * between the typed public API and the untyped internal map. This is us making T
- * the same type as our storage, not the typescript rule that `unknown` types must
- * do a type assertion before use.
- */
- track<T = void>(
- contractId: string,
- ): Promise<Epoch extends true ? T | null : T> {
- if (enableEpochs) {
- epoch++;
- }
-
- // Snapshot the current epoch so we can check staleness on resolve
- const requestEpoch = epoch;
-
- // Cast: TypeScript can't evaluate the conditional return type
- // inside a generic factory. The runtime behavior is correct —
- // with epochs, stale entries resolve with null; without, they
- // resolve with T. The public type signature enforces this.
- return new Promise<any>((resolve, reject) => {
- const timeoutId = setTimeout(() => {
- const entry = pending.get(contractId);
- if (!entry) return; // already resolved/rejected
-
- pending.delete(contractId);
-
- // Stale epoch check — if a newer request was tracked since this
- // one, don't surface a timeout error for an abandoned request.
- // Resolve with null so the caller's promise settles cleanly
- // (its finally block runs, no leaked closure).
- if (enableEpochs && entry.epoch !== epoch) {
- entry.resolve(null);
- return;
- }
-
- let errorMessage =
- `WS request ${contractId} timed out after ${timeoutMs}ms — server did not respond`;
- reject(new WebError(errorMessage, "WsTimeoutError"));
- }, timeoutMs);
-
- pending.set(contractId, {
- resolve: resolve as (value: unknown) => void,
- reject,
- timeoutId,
- epoch: requestEpoch,
- });
- });
- },
-
- /**
- * Resolve a pending request. Called by the WS listener after applying
- * state from the server response.
- *
- * If epochs are enabled and the entry is stale (a newer request was
- * tracked since this one), the entry is settled with null instead of
- * the provided data. This ensures the caller's promise always settles
- * — its finally block runs and closures are released. The caller
- * checks for null to detect staleness.
- *
- * No-op if contractId is not found (already timed out, already
- * resolved, or an unknown/broadcast ID).
- */
- resolve(contractId: string, data?: unknown): void {
- const entry = pending.get(contractId);
- if (!entry) return;
-
- // Stale epoch check — a newer request was tracked since this one.
- // Settle with null so the caller's promise resolves (its finally
- // block runs, no leaked closure). The caller checks for null to
- // detect staleness: if (result === null) return { kind: 'stale' };
- if (enableEpochs && entry.epoch !== epoch) {
- clearTimeout(entry.timeoutId);
- pending.delete(contractId);
- entry.resolve(null);
- return;
- }
-
- clearTimeout(entry.timeoutId);
- pending.delete(contractId);
- entry.resolve(data);
- },
-
- /**
- * Reject a pending request. Called by the WS listener when applying
- * state fails, or internally on timeout.
- *
- * Unlike resolve, reject always surfaces regardless of epoch staleness.
- * Errors should never be silently swallowed — the handler needs to
- * know something went wrong so it can surface it to the user.
- *
- * No-op if contractId is not found.
- */
- reject(contractId: string, error: WebError): void {
- const entry = pending.get(contractId);
- if (!entry) return;
-
- clearTimeout(entry.timeoutId);
- pending.delete(contractId);
- entry.reject(error);
- },
-
- /**
- * Bump the epoch without tracking a new request.
- *
- * All currently pending entries become stale — their resolve() calls
- * will settle with null when the server eventually responds.
- * Their timeouts also settle with null (not rejection) since the
- * timeout callback checks epoch staleness before rejecting.
- *
- * Use this when you want to "cancel" all in-flight requests without
- * triggering error callbacks. For example, navigating away from a
- * context where the responses are no longer relevant.
- *
- * No-op if epochs are not enabled.
- */
- invalidate(): void {
- if (!enableEpochs) return;
- epoch++;
- },
-
- /**
- * Reject all pending requests and clear all timeouts.
- * Call this from component onDestroy to prevent memory leaks
- * and dangling timer callbacks.
- */
- dispose(): void {
- for (const [contractId, entry] of pending) {
- clearTimeout(entry.timeoutId);
-
- // Reject with disposed error
- let errMessage =
- `WS tracker disposed while request ${contractId} was still pending`;
- entry.reject(
- new WebError(errMessage, "WsTrackerDisposedError"),
- );
- }
- pending.clear();
- },
-
- /**
- * Whether a tracked entry exists for this contract ID.
- * Used by listeners to decide whether to apply state:
- * - true: this is a response to a tracked request — apply + resolve
- * - false: timed out, already resolved, or unknown (future broadcast)
- */
- hasPending(contractId: string): boolean {
- return pending.has(contractId);
- },
-
- /** Number of currently pending (unresolved) tracked requests. */
- pendingCount(): number {
- return pending.size;
- },
- };
-}
-```
diff --git a/content/system_design/dal_architecture_overview.md b/content/system_design/dal_architecture_overview.md
@@ -1,341 +0,0 @@
----
-title: DAL Architecture Overview
----
-
-# Architecture Overview: Three-Layer Pattern
-
-The codebase uses a three-layer architecture (similar to Clean Architecture /
-Layered Architecture) to separate concerns:
-
-```
-┌─────────────────────────────────────────────────────┐
-│ LAYER 3: NETWORKING (axum) │
-│ └── /services/projects/networking/axum/src/api/ │
-│ projects/create.rs │
-│ │
-│ Responsibility: │
-│ - HTTP endpoint handling (axum extractors) │
-│ - Authentication/Authorization (JWT tokens) │
-│ - Request/Response serialization (JSON via Axum) │
-│ - Calls core layer │
-└─────────────────────────────────────────────────────┘
- │
- ▼
-┌─────────────────────────────────────────────────────┐
-│ LAYER 2: CORE (Business Logic) │
-│ └── /services/projects/core/src/api/projects/ │
-│ create.rs │
-│ │
-│ Responsibility: │
-│ - Business logic validation │
-│ - Orchestration of multiple operations │
-│ - Converts domain models to/from DAL │
-│ - No HTTP/web framework knowledge │
-└────────────────────────┬────────────────────────────┘
- │
- ▼
-┌─────────────────────────────────────────────────────┐
-│ LAYER 1: DAL (Data Access Layer) │
-│ └── /layers/dal/src/models/projects/ │
-│ ├── tx_definitions.rs │
-│ └── postgres_txs.rs │
-│ │
-│ Responsibility: │
-│ - Raw SQL queries │
-│ - Database transaction management │
-│ - Zero business logic │
-└───────────┬─────────────────────────────────────────┘
- │
- ▼
-┌───────────────────────┐
-│ POSTGRESQL │
-│ DATABASE │
-└───────────────────────┘
-```
-
----
-
-## How Each File Fits In
-
-### 1. DAL Layer: `tx_definitions.rs` + `postgres_txs.rs`
-
-**tx_definitions.rs** — Defines the traits that abstract database operations:
-
-```rust
-define_dal_transactions!(
- GetProjectsByDepartmentId => get_projects_by_department_id(department_id: i32) -> Vec<Project>,
- CreateProject => create_project(project: NewProject) -> Project,
- DeleteProject => delete_project(project_id: i32, dept_id: i32) -> bool,
- CheckUserProjectAccess => check_user_project_access(user_id: i32, project_id: i32) -> bool,
- GetProjectById => get_project_by_id(project_id: i32) -> Option<Project>
-);
-```
-
-This expands to traits like:
-
-```rust
-pub trait CreateProject {
- fn create_project(project: NewProject) -> impl Future<Output = sqlx::Result<Project>> + Send;
-}
-```
-
-See also: [define_dal_transactions!](/compiler/define_dal_transactions_macro.md)
-
-**postgres_txs.rs** — Implements those traits with actual SQL:
-
-```rust
-#[db_transaction(SqlxPostGresDescriptor, CreateProject)]
-async fn create_project(project: NewProject) -> Project {
- let pool = T::yield_pool();
- let query = r#"
- INSERT INTO projects (department_id, name, description, created_at, updated_at)
- VALUES ($1, $2, $3, NOW(), NOW())
- RETURNING id, department_id, name, description, created_at, updated_at
- "#;
- sqlx::query_as::<_, Project>(query)
- .bind(project.department_id)
- .bind(project.name)
- .bind(project.description)
- .fetch_one(pool)
- .await
-}
-```
-
-The `#[db_transaction(StructName, TraitName)]` macro:
-
-1. Generates an impl `TraitName` for `StructName<T>` where
- `T: YieldPostGresPool`
-2. Wraps the async function body in that implementation
-3. Makes the function callable as `StructName::<PoolType>::create_project(...)`
-
-### 2. Core Layer: `core/src/api/projects/create.rs`
-
-This layer orchestrates the business logic:
-
-```rust
-pub async fn create_project<X, S>(storage_handle: &S, new_project: NewProject) -> Result<Project, NanoServiceError>
-where
- X: CreateProject + ProjectBranchesCreateBranch,
- S: GitDataTransfer + Debug,
-{
- // 1. VALIDATION (business rule)
- if new_project.department_id <= 0 {
- return Err(NanoServiceError::bad_request("Invalid department ID".to_string()));
- }
- if new_project.name.trim().is_empty() {
- return Err(NanoServiceError::bad_request("Project name cannot be empty".to_string()));
- }
- if new_project.description.trim().is_empty() {
- return Err(NanoServiceError::bad_request("Project description cannot be empty".to_string()));
- }
-
- // 2. Create project in database
- let created_project = X::create_project(new_project).await?;
-
- // 3. Create git directory (side effect)
- create_git_repo(storage_handle, created_project.id).await?;
-
- // 4. Register default branch
- let new_branch = NewProjectBranch { project_id: created_project.id, branch: "main".into() };
- X::create_branch(new_branch).await.map_err(|e| NanoServiceError::unknown(e.to_string()))?;
-
- Ok(created_project)
-}
-```
-
-**Key characteristics:**
-
-- No HTTP/Websocket knowledge — pure async functions
-- Generic over database handle (`X: CreateProject`) — allows mocking for tests
-- Validates business rules before touching the database
-- Orchestrates multiple operations (create project + git repo + branch)
-
-### 3. Networking Layer: `networking/axum/src/api/projects/create.rs`
-
-This layer adapts the core to HTTP:
-
-```rust
-pub async fn create_project<T, X, Y>(
- token: HeaderToken<X, NoRoleCheck, T>, // Auth extraction
- Json(payload): Json<NewProjectRequest>, // JSON deserialization
-) -> Result<impl IntoResponse, NanoServiceError>
-where
- T: CreateProject + GetProjectsByDepartmentId + PingAuthSession + ProjectBranchesCreateBranch,
- X: GetConfigVariable,
- Y: YieldPostGresPool + Send + Sync + Clone + Debug,
-{
- // 1. Extract department from JWT
- let department_id = token.get_department_id()?;
-
- // 2. Convert request DTO to domain model
- let new_project = NewProject {
- department_id,
- name: payload.name,
- description: payload.description
- };
-
- // 3. Create git storage handle
- let storage_handle = PostgresGitBlobHandle::<Y>::new();
-
- // 4. Call core business logic
- let _ = create_project_core::<T, _>(&storage_handle, new_project).await?;
-
- // 5. Return updated list
- let projects = get_projects_by_department_id_core::<T>(department_id).await?;
- Ok((StatusCode::CREATED, Json(projects)))
-}
-```
-
-**Key characteristics:**
-
-- Axum extractors handle HTTP parsing
-- Authentication via JWT token validation
-- Converts between request types (`NewProjectRequest` → `NewProject`)
-- Handles HTTP concerns (status codes, JSON serialization)
-
----
-
-## Complete Workflow
-
-```
-Client Request
- │
- ▼
- ┌─────────────────────────────────────────────────────────────────┐
- │ 1. HTTP REQUEST arrives at axum endpoint │
- │ POST /api/v1/projects/create │
- │ Headers: Authorization: Bearer <jwt> │
- │ Body: { "name": "...", "description": "..." } │
- └─────────────────────────────────────────────────────────────────┘
- │
- ▼
- ┌─────────────────────────────────────────────────────────────────┐
- │ 2. AXUM LAYER (networking/axum) │
- │ - Extracts and validates JWT token │
- │ - Deserializes JSON payload │
- │ - Converts NewProjectRequest → NewProject │
- │ - Creates PostgresGitBlobHandle │
- │ - Calls create_project_core() │
- └─────────────────────────────────────────────────────────────────┘
- │
- ▼
- ┌─────────────────────────────────────────────────────────────────┐
- │ 3. CORE LAYER (core/api) │
- │ - Validates department_id > 0 │
- │ - Validates name is not empty │
- │ - Validates description is not empty │
- │ - Calls DAL: T::create_project() │
- │ - Calls git repo creation (storage_handle) │
- │ - Calls DAL: T::create_branch() │
- │ - Returns Project model │
- └─────────────────────────────────────────────────────────────────┘
- │
- ▼
- ┌─────────────────────────────────────────────────────────────────┐
- │ 4. DAL LAYER (dal/models) │
- │ - tx_definitions.rs: defines CreateProject trait │
- │ - postgres_txs.rs: │
- │ #[db_transaction(Struct, Trait)] │
- │ async fn create_project() -> SQL INSERT + RETURNING │
- │ - SqlxPostGresDescriptor implements the trait │
- │ - SQL executed against PostgreSQL │
- └─────────────────────────────────────────────────────────────────┘
- │
- ▼
- ┌─────────────────────────────────────────────────────────────────┐
- │ 5. DATABASE (PostgreSQL) │
- │ INSERT INTO projects (...) VALUES (...) │
- │ RETURNING id, department_id, name, description, ... │
- └─────────────────────────────────────────────────────────────────┘
- │
- ▼
- Return back up the stack with created Project
-```
-
----
-
-## Data Flow Diagram
-
-```
-┌─────────────┐ HTTP JSON ┌─────────────┐ NewProject ┌─────────────┐
-│ Client │ ────────────────► │ networking │ ───────────────► │ core │
-│ │ │ (axum) │ │ (create) │
-└─────────────┘ └─────────────┘ └─────────────┘
- │
- ┌───────────────────────────┼─────────────────────┐
- │ │ │
- ▼ ▼ ▼
- ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐
- │DAL: CreateProject│ │GitDataTransfer │ │DAL: CreateBranch │
- │(sqlx INSERT) │ │(create git dir) │ │(sqlx INSERT) │
- └──────────────────┘ └──────────────────┘ └──────────────────┘
- │ │ │
- ▼ ▼ ▼
- ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐
- │ PostgreSQL │ │ Database │ │ PostgreSQL │
- │ projects │ │ git_blobs │ │ project_branches │
- └──────────────────┘ └──────────────────┘ └──────────────────┘
-```
-
----
-
-## Pros and Cons of This Approach
-
-### ✅ Pros
-
-| Benefit | Explanation |
-| -------------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
-| **Separation of Concerns** | Each layer has a single responsibility. DAL knows SQL, Core knows business logic, Networking knows HTTP. |
-| **Testability** | Core layer can be tested with mock DB handles (`MockDeadPostGresPool`) without any HTTP server. No network needed for unit tests. |
-| **Database Abstraction** | The trait-based DAL allows swapping PostgreSQL for another database (though not currently used). |
-| **Reusability** | Core layer functions can be called from HTTP, WebSocket, gRPC, CLI, or tests — not coupled to HTTP. |
-| **Consistency** | All endpoints follow the same pattern — predictable codebase structure. |
-| **Swappable Networking** | Axum could be swapped for Actix-web or Hyper with minimal core changes. |
-| **Clear Boundaries** | Easy to identify where bugs live: HTTP issue → networking, business logic → core, SQL → DAL. |
-
-### ❌ Cons
-
-| Issue | Explanation |
-| --------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
-| **Boilerplate Overhead** | Three files per feature with traits, macros, and adapters creates ceremony. A simple CRUD operation requires significant scaffolding. |
-| **Generic Proliferation** | Every function has `3+` generic type parameters (`<T, X, Y>`) making signatures hard to read and IDE autocomplete overwhelming. |
-| **Tight Coupling via Traits** | The `where X: CreateProject + GetProjectsByDepartmentId + ...` clauses require implementing many traits, creating coupling between networking and DAL layers. |
-| **No Transaction Across Layers** | The `create_project` core function calls multiple DAL operations that aren't wrapped in a DB transaction. If `create_git_repo` fails, the project row was already committed. |
-| **Hidden Complexity in Macros** | `#[db_transaction]` and `define_dal_transactions!` are magical — hard to debug, IDE can't "go to definition" easily. |
-| **Request/Response Type Proliferation** | `NewProjectRequest` (HTTP layer) → `NewProject` (Core layer) → `NewProject` (DAL) is mostly the same struct with different names. |
-| **Hard to Follow the Flow** | New developers must trace through 3 files + 2 macros to understand how a simple INSERT works. |
-| **Over-engineering for Simple Ops** | For a simple `SELECT * FROM projects`, you still need the full three-layer setup. |
-
----
-
-## Key Files Summary
-
-| File | Role | Key Pattern |
-| -------------------------------------------------------------- | ------------------------------------------ | --------------------------------------------- |
-| `layers/dal/src/models/projects/tx_definitions.rs` | Defines trait signatures for DB operations | `define_dal_transactions!` macro |
-| `layers/dal/src/models/projects/postgres_txs.rs` | Implements the trait with SQL | `#[db_transaction(Struct, Trait)]` proc macro |
-| `services/projects/core/src/api/projects/create.rs` | Business logic orchestration | Validation → DAL calls → Return |
-| `services/projects/networking/axum/src/api/projects/create.rs` | HTTP adapter layer | Axum extractors → Call core → HTTP response |
-
----
-
-## Testability Example
-
-The beauty of this pattern is shown in the core tests:
-
-```rust
-// Core layer test with MOCK database — no real DB needed
-#[db_transaction(MockDbHandle, CreateProject)]
-async fn create_project(new_project: NewProject) -> Project {
- Ok(Project { id: 1, ... }) // Mocked response
-}
-
-let result = create_project::<MockDbHandle<MockDeadPostGrosPool>, _>(
- &mock_git_handle,
- new_project,
-)
-.await;
-```
-
-This lets you test business logic validation and orchestration without spinning
-up a PostgreSQL instance.
diff --git a/content/system_design/design_patterns/abstraction.md b/content/system_design/design_patterns/abstraction.md
@@ -1,37 +0,0 @@
----
-title: Abstraction
----
-
-The use of abstractions is one of the most important concepts in computer
-science. For example, one aspect of good programming practice is to formulate a
-simple application program interface (API) for a set of functions that allow
-programmers to use the code without having to delve into its inner workings.
-Different programming languages provide different forms and levels of support
-for abstraction, such as Rust's trait definitions and module system with
-explicit pub visibility boundaries.
-
-## Abstractions provided by a computer system
-
-<img src="/whiteboards/abstraction_of_computer_system.png" alt="Abstraction" width="100%">
-
-We are probably familiar with several of the abstractions seen in computer
-systems. On the processor side, the instruction set architecture provides an
-abstraction of the actual processor hardware. With this abstraction, a
-machine-code program behaves as if it were executed on a processor that performs
-just one instruction at a time. The underlying hardware is far more elaborate,
-executing multiple instructions in parallel, but always in a way that is
-consistent with the simple, sequential model. By keeping the same execution
-model, different processor implementations can execute the same machine code
-while offering a range of cost and performance.
-
-On the operating system side, we have introduced three abstractions: files as an
-abstraction of I/O devices, virtual memory as an abstraction of program memory,
-and processes as an abstraction of a running program. To these abstractions we
-add a new one: the virtual machine, providing an abstraction of the entire
-computer, including the operating system, the processor, and the programs. The
-idea of a virtual machine was introduced by IBM in the 1960s, but it has become
-more prominent recently as a way to manage computers that must be able to run
-programs designed for multiple operating systems (such as Microsoft Windows, Mac
-OS X, and Linux) or different versions of the same operating system.
-
-<img src="/whiteboards/five_abstraction_levels.png" alt="Abstraction" width="100%">
diff --git a/content/system_design/design_patterns/chain_of_responsibility.md b/content/system_design/design_patterns/chain_of_responsibility.md
@@ -1,215 +0,0 @@
----
-title: Chain of Responsibility
----
-
-The Chain of Responsibility (CoR) pattern is a behavioral design pattern where a
-request is passed along a chain of handlers — each handler either processes the
-request or forwards it to the next one in the chain.
-
-The three key components are:
-
-- Handler — a trait defining the interface for handling requests and holding the
- successor link
-
-- Successor — the next handler in the chain, stored as Option<Box<dyn Handler>>
-
-- Request — the data passed along the chain
-
-> The Rust community's guiding principle here is: static where you can, dynamic
-> where you must.
-
-### Example 1: Purchase Approval Chain (Dynamic Disptach)
-
-The key pattern here is the Option<Box<dyn Approver>> field — it lets each
-handler optionally own its successor and forward the request via
-next.process_request(request).
-
-```rust
-struct PurchaseRequest {
- amount: f64,
-}
-
-trait Approver {
- fn set_successor(&mut self, successor: Box<dyn Approver>);
- fn process_request(&self, request: &PurchaseRequest);
-}
-
-struct Manager {
- successor: Option<Box<dyn Approver>>,
-}
-
-impl Approver for Manager {
- fn set_successor(&mut self, successor: Box<dyn Approver>) {
- self.successor = Some(successor);
- }
- fn process_request(&self, request: &PurchaseRequest) {
- if request.amount <= 1000.0 {
- println!("Manager approves ${}", request.amount);
- } else if let Some(ref next) = self.successor {
- next.process_request(request); // pass it up
- } else {
- println!("Cannot be approved.");
- }
- }
-}
-
-struct Director { successor: Option<Box<dyn Approver>> }
-
-impl Approver for Director {
- fn set_successor(&mut self, successor: Box<dyn Approver>) {
- self.successor = Some(successor);
- }
- fn process_request(&self, request: &PurchaseRequest) {
- if request.amount <= 5000.0 {
- println!("Director approves ${}", request.amount);
- } else if let Some(ref next) = self.successor {
- next.process_request(request);
- } else {
- println!("Cannot be approved.");
- }
- }
-}
-
-struct President;
-
-impl Approver for President {
- fn set_successor(&mut self, _: Box<dyn Approver>) {} // terminal node
- fn process_request(&self, request: &PurchaseRequest) {
- if request.amount <= 10000.0 {
- println!("President approves ${}", request.amount);
- } else {
- println!("Request denied.");
- }
- }
-}
-
-fn main() {
- let president = President;
- let mut director = Director { successor: Some(Box::new(president)) };
- let mut manager = Manager { successor: Some(Box::new(director)) };
-
- manager.process_request(&PurchaseRequest { amount: 500.0 }); // Manager approves
- manager.process_request(&PurchaseRequest { amount: 3000.0 }); // Director approves
- manager.process_request(&PurchaseRequest { amount: 8000.0 }); // President approves
- manager.process_request(&PurchaseRequest { amount: 15000.0 }); // Denied
-}
-```
-
-### Example 2: Purchase Approval Chain (Generics - Zero-cost static dispatch)
-
-This is verbose but gives you zero-cost static dispatch — no heap allocation, no
-vtable lookup, all method calls resolved at compile time.
-
-`Handler<Manager<Director<President>>>`
-
-```rust
-trait Handler {
- fn handle(&self, amount: f64);
-}
-
-// Terminal handler — no successor
-struct President;
-
-impl Handler for President {
- fn handle(&self, amount: f64) {
- if amount <= 10_000.0 {
- println!("President approves ${amount}");
- } else {
- println!("Request denied.");
- }
- }
-}
-
-// Generic handler wrapping the next handler N
-struct Manager<N: Handler> {
- next: N,
-}
-
-impl<N: Handler> Handler for Manager<N> {
- fn handle(&self, amount: f64) {
- if amount <= 1_000.0 {
- println!("Manager approves ${amount}");
- } else {
- self.next.handle(amount); // static dispatch!
- }
- }
-}
-
-struct Director<N: Handler> {
- next: N,
-}
-
-impl<N: Handler> Handler for Director<N> {
- fn handle(&self, amount: f64) {
- if amount <= 5_000.0 {
- println!("Director approves ${amount}");
- } else {
- self.next.handle(amount);
- }
- }
-}
-
-fn main() {
- // The full type is Manager<Director<President>>
- let chain = Manager {
- next: Director {
- next: President,
- },
- };
-
- chain.handle(500.0); // Manager approves
- chain.handle(3_000.0); // Director approves
- chain.handle(8_000.0); // President approves
- chain.handle(15_000.0); // Request denied
-}
-```
-
-The compiler monomorphizes this — it generates a unique, optimized function for
-each concrete type combination. No heap, no vtable.
-
-#### Using impl Trait to Hide the Chain Type
-
-If you want to hide the ugly Manager<Director<President>> type from callers, use
-impl Trait as a return type:
-
-```rust
-fn build_chain() -> impl Handler {
- Manager {
- next: Director {
- next: President,
- },
- }
-}
-
-fn main() {
- let chain = build_chain(); // type is opaque to the caller
- chain.handle(3_000.0);
-}
-```
-
-This still uses static dispatch under the hood — the compiler knows the exact
-type, the caller just doesn't need to spell it out.
-
-### Example 3: Enum-Based Variant (More Idiomatic)
-
-Rust's enums offer a cleaner alternative when the set of handlers is fixed at
-compile time:
-
-```rust
-enum SupportLevel {
- Basic,
- Intermediate,
- Critical,
-}
-
-fn handle_request(level: SupportLevel) {
- match level {
- SupportLevel::Basic => println!("L1 Support handled it"),
- SupportLevel::Intermediate => println!("L2 Support handled it"),
- SupportLevel::Critical => println!("L3 Support handled it"),
- }
-}
-```
-
-This approach avoids heap allocation and dynamic dispatch entirely, but loses
-the runtime flexibility of adding or reordering handlers.
diff --git a/content/system_design/design_patterns/decomposition.md b/content/system_design/design_patterns/decomposition.md
@@ -1,5 +0,0 @@
----
-title: Decomposition
----
-
-
diff --git a/content/system_design/design_patterns/dependency_injection.md b/content/system_design/design_patterns/dependency_injection.md
@@ -1,179 +0,0 @@
----
-title: Dependency Injection
----
-
-**Dependency injection** (DI) is a design pattern where a component receives
-its dependencies from an external source rather than creating them
-itself.[^1](https://dev.to/sgchris/how-traits-enable-dependency-injection-in-rust-5a50)
-This decouples components, making code easier to test, swap, and maintain.
-In Rust, DI is achieved idiomatically through **traits** and **generics**
-— without needing a framework — leveraging the compiler's type system to
-enforce correctness at compile
-time.[^2](https://jmmv.dev/2022/04/rust-traits-and-dependency-injection.html)
-
----
-
-## Core Tool: Traits
-
-In Rust, **traits** act as the contract (interface) that dependencies must
-fulfill.[^1](https://dev.to/sgchris/how-traits-enable-dependency-injection-in-rust-5a50)
-Define the behaviour, not the implementation:
-
-```rust
-pub trait Logger {
- fn log(&self, message: &str);
-}
-
-pub struct ConsoleLogger;
-
-impl Logger for ConsoleLogger {
- fn log(&self, message: &str) {
- println!("[LOG]: {}", message);
- }
-}
-```
-
----
-
-## Approach 1: Generics (Static Dispatch)
-
-The preferred Rust approach — the concrete type is resolved at **compile
-time**, producing zero-cost
-abstractions.[^1](https://dev.to/sgchris/how-traits-enable-dependency-injection-in-rust-5a50)
-
-```rust
-pub struct Application<L: Logger> {
- logger: L,
-}
-
-impl<L: Logger> Application<L> {
- pub fn new(logger: L) -> Self {
- Self { logger }
- }
-
- pub fn run(&self) {
- self.logger.log("Application is running!");
- }
-}
-
-fn main() {
- let app = Application::new(ConsoleLogger);
- app.run(); // prints: [LOG]: Application is running!
-}
-```
-
-`Application` only requires that `L` implements `Logger` — swap in any
-logger without touching `Application`
-itself.[^3](https://users.rust-lang.org/t/how-do-you-implement-dependency-injection-in-rust/213)
-
----
-
-## Approach 2: Trait Objects (Dynamic Dispatch)
-
-When you need runtime flexibility or to store mixed implementations in a
-collection, use `Box<dyn
-Trait>`.[^3](https://users.rust-lang.org/t/how-do-you-implement-dependency-injection-in-rust/213)
-
-```rust
-pub trait MessageSender {
- fn send(&self, msg: &str);
-}
-
-pub struct NotificationService {
- sender: Box<dyn MessageSender>,
-}
-
-impl NotificationService {
- pub fn new(sender: Box<dyn MessageSender>) -> Self {
- Self { sender }
- }
-
- pub fn notify(&self, msg: &str) {
- self.sender.send(msg);
- }
-}
-```
-
-This introduces a small **runtime overhead** via vtable lookup, so prefer
-generics unless dynamic dispatch is genuinely
-needed.[^1](https://dev.to/sgchris/how-traits-enable-dependency-injection-in-rust-5a50)
-
----
-
-## Approach 3: Enums (Closed Set)
-
-If you have a fixed, known set of implementations, an enum avoids both
-generics complexity and boxing
-overhead.[^3](https://users.rust-lang.org/t/how-do-you-implement-dependency-injection-in-rust/213)
-
-```rust
-pub enum LoggerKind {
- Console,
- File(String),
-}
-
-impl Logger for LoggerKind {
- fn log(&self, message: &str) {
- match self {
- LoggerKind::Console => println!("[Console]: {}", message),
- LoggerKind::File(path) => println!("[File({})] {}", path, message),
- }
- }
-}
-```
-
----
-
-## Why DI Shines in Testing
-
-The real payoff is **mockability** — swap the real implementation with a
-mock during
-tests.[^1](https://dev.to/sgchris/how-traits-enable-dependency-injection-in-rust-5a50)
-
-```rust
-pub struct MockLogger {
- pub messages: std::cell::RefCell<Vec<String>>,
-}
-
-impl Logger for MockLogger {
- fn log(&self, message: &str) {
- self.messages.borrow_mut().push(message.to_string());
- }
-}
-
-#[test]
-fn test_app_logs_on_run() {
- let mock = MockLogger { messages: Default::default() };
- let app = Application::new(&mock);
- app.run();
- assert!(mock.messages.borrow().contains(&"Application is running!".to_string()));
-}
-```
-
-No real I/O, no external systems — pure, fast unit
-tests.[^1](https://dev.to/sgchris/how-traits-enable-dependency-injection-in-rust-5a50)
-
----
-
-## Choosing the Right Approach
-
-| | Generics | `Box<dyn Trait>` | Enum |
-| :------------------------------------- | :-------------------- | :---------------------- | :--------------------- |
-| **Dispatch** | Compile time (static) | Runtime (dynamic) | Compile time (static) |
-| **Overhead** | Zero | Vtable lookup | Zero |
-| **Implementations** | Any (open set) | Any (open set) | Fixed (closed set) |
-| **Heterogeneous collections** | ❌ | ✅ | ✅ |
-| **Best for** | Most cases | Plugin-like flexibility | Known, finite variants |
-
-For complex dependency graphs, consider a DI container crate such as
-[rustyinject](https://github.com/AlexSherbinin/rustyinject).[^4](https://github.com/AlexSherbinin/rustyinject)
-
----
-
-## Key Pitfall: Visibility Creep
-
-Any type referenced in a **public** trait's function signature must also be
-public.[^2](https://jmmv.dev/2022/04/rust-traits-and-dependency-injection.html)
-If your trait is public but references an internal struct, you'll be forced
-to expose that struct too — keep internal traits `pub(crate)` where
-possible.[^2](https://jmmv.dev/2022/04/rust-traits-and-dependency-injection.html)
diff --git a/content/system_design/design_patterns/design_principles.md b/content/system_design/design_patterns/design_principles.md
@@ -1,17 +0,0 @@
----
-title: Design Principles
----
-
-## Separating what changes from what stays the same
-
-Identify the aspects of your application that vary and separate them from what
-stays the same.
-
-As simple as this concept is, it forms the basis for almost every design
-pattern. All patterns provide a way to let some part of a system vary
-independently of all other parts.
-
-## Program to an interface, not an implementation.
-
-The point is to exploit polymorphism by programming to a supertype so that the
-actual runtime object isn’t locked into the code.
diff --git a/content/system_design/design_patterns/entity_boundary_interactor.md b/content/system_design/design_patterns/entity_boundary_interactor.md
@@ -1,22 +0,0 @@
----
-title: Entity Boundary Interactor
----
-
-[Reference](https://ebi.readthedocs.io/en/latest/)
-
-The idea is to design programs so that their architectures immediately present
-their use case. It’s a way to design programs so that its internal dependency
-graph is organized cleanly and its elements are joined together with as loose
-coupling as possible.
-
-Ultimately, the goal is the separation of concerns between application layers,
-this architecture and many like it aren’t dependent on presentation models or
-platforms. All the arrows, or dependencies, point inwards in the abstraction
-chain, each successive layer less abstract than the one before it.
-
-Keeping the arrows pointing inwards makes the code easy to maintain, extend,
-test and refactor. EBI imposes some architectural requirements on the
-programmers, that is, you must navigate around its rules, but this is kept at a
-minimum.
-
-
diff --git a/content/system_design/design_patterns/entity_component_system.md b/content/system_design/design_patterns/entity_component_system.md
@@ -1,261 +0,0 @@
----
-title: Entity Component System (ECS)
----
-
-> **Common use cases:** Game engines, simulations, UI frameworks, any system
-> with many objects sharing overlapping data
-
----
-
-## What is ECS?
-
-The **Entity Component System** is a composition-over-inheritance architectural
-pattern that separates _identity_ (entities), _data_ (components), and _logic_
-(systems). Instead of a class hierarchy where a `Player` extends `Character`,
-you build objects by attaching plain data structs to a bare ID.
-
-| Concept | Role | Rust analogy |
-| ------------- | ----------------------------------------- | --------------------------------------------- |
-| **Entity** | A unique ID — nothing more | `u64` or a newtype wrapper |
-| **Component** | Plain data attached to an entity | A `struct` with `#[derive(Component)]` |
-| **System** | Logic that queries and mutates components | A function that receives queries as arguments |
-
----
-
-## Simple Example from Scratch
-
-A minimal ECS without any external crate to understand the mechanics:
-
-```rust
-use std::collections::HashMap;
-
-type Entity = u64;
-
-// Components are plain structs
-#[derive(Debug)]
-struct Position { x: f32, y: f32 }
-
-#[derive(Debug)]
-struct Velocity { dx: f32, dy: f32 }
-
-// World holds component storage — one HashMap per component type
-#[derive(Default)]
-struct World {
- next_entity: Entity,
- positions: HashMap<Entity, Position>,
- velocities: HashMap<Entity, Velocity>,
-}
-
-impl World {
- fn spawn(&mut self) -> Entity {
- let id = self.next_entity;
- self.next_entity += 1;
- id
- }
-}
-
-// A system is just a plain function
-fn movement_system(world: &mut World) {
- for (entity, vel) in &world.velocities {
- if let Some(pos) = world.positions.get_mut(entity) {
- pos.x += vel.dx;
- pos.y += vel.dy;
- }
- }
-}
-
-fn main() {
- let mut world = World::default();
-
- let player = world.spawn();
- world.positions.insert(player, Position { x: 0.0, y: 0.0 });
- world.velocities.insert(player, Velocity { dx: 1.0, dy: 0.5 });
-
- let static_obstacle = world.spawn();
- world.positions.insert(static_obstacle, Position { x: 10.0, y: 10.0 });
- // No Velocity — this entity won't move
-
- movement_system(&mut world);
-
- println!("{:?}", world.positions[&player]); // Position { x: 1.0, y: 0.5 }
-}
-```
-
-Key insight: `static_obstacle` was never touched by `movement_system` because it
-has no `Velocity`. Systems only operate on the _intersection_ of components they
-care about — no `if entity_is_static` checks required.
-
----
-
-## Growing the Design
-
-Once the naive HashMap approach gets unwieldy you need:
-
-- **Archetypes** — group entities by their exact component set for better cache
- locality (how Bevy's world works)
-- **Sparse sets** — fast add/remove at the cost of iteration speed (used for
- rarely-changed components)
-- **System scheduling** — run systems in parallel when they don't share mutable
- access; `bevy_ecs` does this automatically via Rust's borrow rules
-
-```rust
-// A slightly more ergonomic API pattern using a builder
-struct EntityBuilder<'w> {
- world: &'w mut World,
- id: Entity,
-}
-
-impl<'w> EntityBuilder<'w> {
- fn with_position(self, x: f32, y: f32) -> Self {
- self.world.positions.insert(self.id, Position { x, y });
- self
- }
- fn with_velocity(self, dx: f32, dy: f32) -> Self {
- self.world.velocities.insert(self.id, Velocity { dx, dy });
- self
- }
- fn build(self) -> Entity { self.id }
-}
-```
-
----
-
-## Bevy as a Production ECS
-
-[Bevy](https://bevyengine.org/) is the most prominent real-world ECS in Rust and
-a great study in ergonomic API design. Its ECS lives in the standalone
-[`bevy_ecs`](https://crates.io/crates/bevy_ecs) crate — you can use it without
-the rest of the game engine.
-
-```rust
-use bevy::prelude::*;
-
-// Components — plain structs that derive `Component`
-#[derive(Component)]
-struct Position { x: f32, y: f32 }
-
-#[derive(Component)]
-struct Velocity { dx: f32, dy: f32 }
-
-// Systems are just functions; Bevy injects queries via its scheduler
-fn movement_system(mut query: Query<(&mut Position, &Velocity)>) {
- for (mut pos, vel) in &mut query {
- pos.x += vel.dx;
- pos.y += vel.dy;
- }
-}
-
-fn spawn_player(mut commands: Commands) {
- commands.spawn((
- Position { x: 0.0, y: 0.0 },
- Velocity { dx: 1.0, dy: 0.5 },
- ));
-}
-
-fn main() {
- App::new()
- .add_plugins(MinimalPlugins)
- .add_systems(Startup, spawn_player)
- .add_systems(Update, movement_system)
- .run();
-}
-```
-
-Notice how `movement_system` never knows about entities at all — it just
-expresses _"give me every entity that has both a mutable Position and an
-immutable Velocity"_. Bevy's scheduler can then safely parallelise any two
-systems whose query sets don't conflict.
-
-### Why Bevy's ECS is a Masterclass in Ergonomic Rust
-
-- **Function-parameter injection** via `SystemParam` traits — adding a
- `Res<Time>` parameter to a system just works
-- **`Commands` for deferred mutations** — you can't borrow the world mutably
- while iterating it, so commands queue up spawns/despawns to run between
- systems
-- **`With` / `Without` / `Or` filters** — express complex queries in the type
- system with no runtime cost
-- **Observers and triggers** (added in Bevy 0.14) — reactive event-driven logic
- built on top of the ECS
-
----
-
-## Why ECS Fits Rust So Well
-
-Traditional OOP patterns with shared mutable object graphs fight the borrow
-checker constantly. ECS sidesteps this by:
-
-1. **Separating data from logic** — systems take fine-grained borrows, so two
- systems can run in parallel as long as they don't both need `&mut` on the
- same component type
-2. **Cache-friendly storage** — components of the same type are stored
- contiguously, turning what would be pointer-chasing in OOP into sequential
- memory reads
-3. **Composition without `dyn Trait`** — you don't need dynamic dispatch;
- queries are resolved at compile time via associated types and const generics
-
----
-
-## Resources & Further Reading
-
-### Talks
-
-- 🎥 **Chris Biscardi — "Bevy: A Case Study in Ergonomic Rust"**
- (RustConf 2024)\
- Deep dive into the API design tricks Bevy uses — applicable far beyond games.\
- →
- [https://www.youtube.com/watch?v=CnoDOc6ML0Y](https://www.youtube.com/watch?v=CnoDOc6ML0Y)
-
-- 🎥 **Alice Cecile — "Architecting Bevy"** (interview, 2024)\
- Alice is a core Bevy contributor and foundation member. This talk covers ECS
- architecture decisions and long-term open-source project management.\
- →
- [https://www.youtube.com/watch?v=PND2Wpy6U-E](https://www.youtube.com/watch?v=PND2Wpy6U-E)
-
-### Video Series
-
-- 📺 **Brooks Patton — "Improve Your Rust Skills by Making an ECS Library"**
- (YouTube playlist, 2021)\
- Builds an ECS from scratch in Rust. Covers `TypeId`, `HashMap`, generics,
- `Copy`/`Clone`, interior mutability, and modules — great for understanding the
- internals.\
- →
- [https://www.youtube.com/playlist?list=PLrmY5pVcnuE_SQSzGPWUJrf9Yo-YNeBYs](https://www.youtube.com/playlist?list=PLrmY5pVcnuE_SQSzGPWUJrf9Yo-YNeBYs)\
- → Code:
- [https://github.com/brooks-builds/improve_skills_by_building_ecs_library_in_rust](https://github.com/brooks-builds/improve_skills_by_building_ecs_library_in_rust)
-
-### People to Follow
-
-| Person | Known for | Link |
-| ----------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------- |
-| **Alice Cecile** (`alice-i-cecile`) | Bevy core contributor, ECS architect, RFC author | [github.com/alice-i-cecile](https://github.com/alice-i-cecile) |
-| **Chris Biscardi** | Bevy educator, ergonomic Rust APIs, Rust Adventure | [youtube/@chrisbiscardi](https://www.youtube.com/@chrisbiscardi) |
-| **Brooks Patton** (`brookzerker`) | ECS from scratch series, Rust std-lib deep dives | [youtube/@brookzerker](https://www.youtube.com/@brookzerker) |
-| **Jon Gjengset** | "Crust of Rust" — intermediate Rust internals (not ECS-specific, but indispensable for understanding the mechanics ECS relies on like interior mutability) | [youtube/@jongjengset](https://www.youtube.com/@jongjengset) |
-| **Alice Ryhl** | Tokio maintainer, async Rust expert — not ECS-specific, but her writing on structured concurrency complements ECS scheduling design | [ryhl.io](https://ryhl.io) |
-
-### Crates
-
-| Crate | Notes |
-| ----------------------------------------------- | ---------------------------------------------------------- |
-| [`bevy_ecs`](https://crates.io/crates/bevy_ecs) | Production-grade, standalone; the reference implementation |
-| [`hecs`](https://crates.io/crates/hecs) | Minimal, low-level; great for embedding |
-| [`specs`](https://crates.io/crates/specs) | Older, more explicit; parallel systems via Rayon |
-| [`shipyard`](https://crates.io/crates/shipyard) | Sparse-set ECS; fast add/remove |
-| [`flecs`](https://crates.io/crates/flecs_ecs) | Rust bindings for the C flecs library; extremely mature |
-
----
-
-## When _Not_ to Use ECS
-
-ECS adds indirection and query overhead. Avoid it when:
-
-- You have **fewer than ~100 objects** that rarely change structure
-- Your logic is **purely sequential and single-threaded** with no parallelism
- benefit
-- You're building a **CRUD app or domain model** — here, plain structs and trait
- objects are cleaner
-
-A good rule of thumb: reach for ECS when you find yourself writing
-`if player.has_component::<Health>()` — that conditional is ECS trying to break
-out.
diff --git a/content/system_design/design_patterns/finite_state_machine.md b/content/system_design/design_patterns/finite_state_machine.md
@@ -1,186 +0,0 @@
----
-title: Finite State Machine
----
-
-A **state machine** (or finite state machine, FSM) is a behavioral model where a
-system can exist in exactly one of a finite number of states at any given time,
-and transitions between states are triggered by specific events or conditions.
-Rust is particularly well-suited for state machines because its type system can
-enforce valid transitions either at runtime (using enums) or at compile time
-(using the typestate
-pattern).[^1](https://www.itemis.com/en/products/itemis-create/documentation/user-guide/overview_what_are_state_machines)
-
----
-
-## Core Concepts
-
-Three building blocks define any state
-machine:[^2](https://www.ni.com/docs/en-US/bundle/patools/page/what-is-a-state-machine.html)
-
-- **States** – distinct situations a system can be in (e.g., `Created`, `Paid`,
- `Shipped`)
-- **Transitions** – rules that move the system from one state to another
-- **Events** – inputs/triggers that cause a transition (e.g., "payment
- received")
-
-A system can only be in **one state at a time**, and transitions that don't
-exist for the current state are simply
-invalid.[^3](https://stately.ai/blog/2023-10-02-what-is-a-state-machine)
-
----
-
-## Approach 1: Enum-Based State Machine
-
-The simplest Rust approach uses `enum` variants to represent states and pattern
-matching to handle transitions. Here's an order-processing
-example:[^4](https://oneuptime.com/blog/post/2026-02-01-rust-state-machines/view)
-
-```rust
-#[derive(Debug, Clone)]
-enum OrderState {
- Created { items: Vec<String>, total: f64 },
- Paid { items: Vec<String>, total: f64, payment_id: String },
- Shipped { items: Vec<String>, tracking_number: String },
- Delivered { delivered_at: String },
- Cancelled { reason: String },
-}
-
-#[derive(Debug)]
-struct Order {
- id: String,
- state: OrderState,
-}
-
-impl Order {
- fn new(id: String, items: Vec<String>, total: f64) -> Self {
- Order { id, state: OrderState::Created { items, total } }
- }
-
- // Transition only valid from Created state
- fn pay(self, payment_id: String) -> Result<Self, &'static str> {
- match self.state {
- OrderState::Created { items, total } => Ok(Order {
- id: self.id,
- state: OrderState::Paid { items, total, payment_id },
- }),
- _ => Err("Can only pay for orders in Created state"),
- }
- }
-
- fn ship(self, tracking_number: String) -> Result<Self, &'static str> {
- match self.state {
- OrderState::Paid { items, .. } => Ok(Order {
- id: self.id,
- state: OrderState::Shipped { items, tracking_number },
- }),
- _ => Err("Can only ship paid orders"),
- }
- }
-}
-
-fn main() {
- let order = Order::new("ORD-001".to_string(), vec!["Widget".to_string()], 99.99);
- let order = order.pay("PAY-123".to_string()).unwrap();
- let order = order.ship("TRACK-456".to_string()).unwrap();
- println!("{:?}", order);
-}
-```
-
-This catches invalid transitions **at runtime** via `Result`. It's simple and
-easy to understand, but bugs only surface when that code path is
-hit.[^4](https://oneuptime.com/blog/post/2026-02-01-rust-state-machines/view)
-
----
-
-## Approach 2: Typestate Pattern (Compile-Time Safety)
-
-The **typestate pattern** encodes state into the _type itself_ using generics,
-so the Rust compiler rejects invalid transitions before your code ever
-runs.[^4](https://oneuptime.com/blog/post/2026-02-01-rust-state-machines/view)
-
-```rust
-use std::marker::PhantomData;
-
-// Zero-sized marker types — no runtime cost
-struct Created;
-struct Paid { payment_id: String }
-struct Shipped { tracking_number: String }
-
-struct Order<St> {
- id: String,
- items: Vec<String>,
- total: f64,
- state_data: St,
-}
-
-// Methods only available on Order<Created>
-impl Order<Created> {
- fn new(id: String, items: Vec<String>, total: f64) -> Self {
- Order { id, items, total, state_data: Created }
- }
-
- fn pay(self, payment_id: String) -> Order<Paid> {
- Order { id: self.id, items: self.items, total: self.total,
- state_data: Paid { payment_id } }
- }
-}
-
-// Methods only available on Order<Paid>
-impl Order<Paid> {
- fn ship(self, tracking_number: String) -> Order<Shipped> {
- Order { id: self.id, items: self.items, total: self.total,
- state_data: Shipped { tracking_number } }
- }
-}
-
-fn main() {
- let order = Order::new("ORD-001".to_string(), vec!["Widget".to_string()], 49.99);
- let paid = order.pay("PAY-123".to_string()); // ✅ Created → Paid
- let shipped = paid.ship("TRACK-456".to_string()); // ✅ Paid → Shipped
-
- // ❌ This would FAIL TO COMPILE — ship() doesn't exist on Order<Created>
- // let bad = order.ship("TRACK-789".to_string());
-}
-```
-
-Because `ship()` is only implemented on `Order<Paid>`, calling it on an unpaid
-order is a **compile-time error**, not a runtime
-bug.[^4](https://oneuptime.com/blog/post/2026-02-01-rust-state-machines/view)
-
----
-
-## Which Approach to Choose?
-
-| | Enum-Based | Typestate Pattern |
-| :-------------------------------- | :------------------------ | :--------------------------------- |
-| **Error detection** | Runtime | Compile time |
-| **Complexity** | Low | Medium–High |
-| **Dynamic state** (e.g., from DB) | ✅ Easy | ❌ Needs wrapper enum |
-| **IDE autocomplete** | Shows all methods | Shows only valid methods |
-| **Best for** | Prototyping, simple flows | Financial, safety-critical systems |
-
-In practice, many production systems use **both**: typestate for core business
-logic, with an enum wrapper (`AnyOrder`) for serialization and dynamic event
-handling.[^4](https://oneuptime.com/blog/post/2026-02-01-rust-state-machines/view)
-
-[^5]: https://www.freecodecamp.org/news/state-machines-basics-of-computer-science-d42855debc66/
-
-[^6]: https://www.youtube.com/watch?v=sosnUnI-vco
-
-[^7]: https://www.reddit.com/r/learnprogramming/comments/1g5yxci/state_machines_for_a_beginner/
-
-[^8]: https://github.com/Lifestreams-ai/statemachine
-
-[^9]: https://uk.mathworks.com/discovery/state-machine.html
-
-[^10]: https://docs.rs/simple_statemachine
-
-[^11]: https://www.reddit.com/r/rust/comments/18fugz0/state_machines_implementation/
-
-[^12]: https://www.stateworks.com/technology/understanding-state-machines/
-
-[^13]: https://en.wikipedia.org/wiki/Finite-state_machine
-
-[^14]: https://users.rust-lang.org/t/on-state-machines/114910
-
-[^15]: https://sparxsystems.com/resources/tutorials/uml2/state-diagram.html
diff --git a/content/system_design/design_patterns/iife.md b/content/system_design/design_patterns/iife.md
@@ -1,78 +0,0 @@
----
-title: Immediately Invoked Function Expression (IIFE) in TypeScript
----
-
-An **IIFE** (immediately invoked function expression) is a function that is
-defined and executed in the same statement. The pattern has been part of
-JavaScript since the language’s early use in the browser, where it was a
-practical way to create a **local scope** and **avoid polluting the global
-object** when true modules were not available.
-
-## Basic shape
-
-The function is wrapped in parentheses so the parser treats it as an
-expression, then it is called with `()`:
-
-```ts
-(function () {
- const secret = 42
- // `secret` is not visible outside this function
-})()
-```
-
-With an arrow function, a similar effect is common in modern code:
-
-```ts
-;(() => {
- // one-off setup or isolated block
-})()
-```
-
-A leading semicolon (or a line break before `(`) avoids the rare case where
-the previous line’s expression is parsed as a function call.
-
-## Why it still matters in TypeScript
-
-1. **Encapsulation** — `const` and `let` inside the IIFE are not hoisted to
- an outer scope, so you can run initialization logic without exporting
- implementation details. With ES modules, file scope already provides
- privacy for top-level declarations, so IIFEs are less about “hiding from
- globals” and more about **grouping** one-time work or **avoiding name
- clashes** in a single file with many top-level symbols.
-
-2. **Async IIFE** — `async` functions return a `Promise`. Invoking one
- immediately is a standard way to use `await` at the top level inside a
- context that does not allow top-level `await` (older bundlers or scripts),
- or to fire-and-forget async setup while keeping the rest of the module
- synchronous:
-
- ```ts
- ;(async () => {
- await someSetup()
- })()
- ```
-
- In modern ES modules with **top-level await** enabled, you can often
- replace this with plain `await` at module scope.
-
-3. **Return values** — An IIFE can produce a value that you assign or export:
-
- ```ts
- export const config = (() => {
- const raw = process.env["FEATURE"]
- return raw === "on"
- })()
- ```
-
-## Trade-offs
-
-IIFEs add nesting and can make stack traces slightly noisier. Prefer **small
-named functions** or **plain module-level code** when there is no need for an
-extra scope. Use an IIFE when you want a **single expression** that computes a
-value with private locals, or when you need **async execution** without making
-the whole module async.
-
-## References
-
-- [MDN: IIFE](https://developer.mozilla.org/en-US/docs/Glossary/IIFE)
-- [TypeScript Handbook — Modules](https://www.typescriptlang.org/docs/handbook/modules.html)
diff --git a/content/system_design/design_patterns/index.md b/content/system_design/design_patterns/index.md
@@ -1,3 +0,0 @@
----
-title: Design patterns
----
diff --git a/content/system_design/design_patterns/interface.md b/content/system_design/design_patterns/interface.md
@@ -1,71 +0,0 @@
----
-title: Interface
----
-
-In Rust, the concept of "programming to an interface/supertype" is achieved
-using traits — Rust's equivalent of interfaces or abstract supertypes.
-
-1. Define the "Supertype" (Trait)
-
-```rust
-trait Animal {
- fn make_sound(&self);
-}
-```
-
-2. Concrete Implementations
-
-```rust
-struct Dog;
-struct Cat;
-
-impl Animal for Dog {
- fn make_sound(&self) {
- println!("Woof!");
- }
-}
-
-impl Animal for Cat {
- fn make_sound(&self) {
- println!("Meow!");
- }
-}
-```
-
-## Static Dispatch (resolved at compile time — faster)
-
-```rust
-fn interact(animal: &impl Animal) {
- animal.make_sound();
-}
-
-let animal = Dog;
-interact(&animal); // Works for any Animal implementor
-```
-
-## Dynamic dispatch (resolved at runtime — more flexible)
-
-```rust
-fn interact(animal: &dyn Animal) {
- animal.make_sound();
-}
-
-let animal: &dyn Animal = &Dog; // Variable typed as the trait, not Dog
-animal.make_sound();
-```
-
-## Assigning at Runtime
-
-```rust
-fn get_animal(sound_type: &str) -> Box<dyn Animal> {
- match sound_type {
- "dog" => Box::new(Dog),
- _ => Box::new(Cat),
- }
-}
-
-fn main() {
- let animal = get_animal("dog"); // We don't know the concrete type here
- animal.make_sound(); // All we care about is make_sound()
-}
-```
diff --git a/content/system_design/design_patterns/mealy_and_moore_machine.md b/content/system_design/design_patterns/mealy_and_moore_machine.md
@@ -1,204 +0,0 @@
----
-title: Mealy and Moore Machine
----
-
-# Compare Mealy and Moore state machines with Rust code
-
-The key difference between Mealy and Moore
-[state machines](/system_design/design_patterns/finite_state_machine.md) comes down to **where
-the output lives**: in Moore machines, output is tied to the _state_; in Mealy
-machines, output is tied to the _transition_ (state + input together).[^1]
-
----
-
-## Moore Machine
-
-In a **Moore machine**, each state has a fixed output associated with it,
-regardless of how you arrived there or what the current input is. The output
-only changes when you _enter a new state_ — making it synchronous and immune to
-input glitches.[^2][^3]
-
-Here's a traffic light modelled as a Moore machine — the output (light colour)
-depends purely on the current state:
-
-```rust
-#[derive(Debug, Clone, PartialEq)]
-enum TrafficState {
- Red,
- Green,
- Yellow,
-}
-
-#[derive(Debug)]
-enum Input {
- Timer,
-}
-
-struct MooreMachine {
- state: TrafficState,
-}
-
-impl MooreMachine {
- fn new() -> Self {
- MooreMachine { state: TrafficState::Red }
- }
-
- // Output is derived from STATE alone — no input needed
- fn output(&self) -> &str {
- match self.state {
- TrafficState::Red => "STOP",
- TrafficState::Green => "GO",
- TrafficState::Yellow => "SLOW DOWN",
- }
- }
-
- // Transition: input triggers state change, output follows state
- fn transition(&mut self, input: Input) {
- self.state = match (&self.state, input) {
- (TrafficState::Red, Input::Timer) => TrafficState::Green,
- (TrafficState::Green, Input::Timer) => TrafficState::Yellow,
- (TrafficState::Yellow, Input::Timer) => TrafficState::Red,
- };
- // Output is read AFTER the state changes
- println!("State: {:?} → Output: {}", self.state, self.output());
- }
-}
-
-fn main() {
- let mut fsm = MooreMachine::new();
- println!("Initial output: {}", fsm.output()); // "STOP"
- fsm.transition(Input::Timer); // Green → "GO"
- fsm.transition(Input::Timer); // Yellow → "SLOW DOWN"
- fsm.transition(Input::Timer); // Red → "STOP"
-}
-```
-
-Output is produced **after** entering the new state — `output()` takes `&self`
-with no input parameter.[^3]
-
----
-
-## Mealy Machine
-
-In a **Mealy machine**, output is produced **on the transition** — it depends on
-both the current state _and_ the input that triggered the move. This means
-outputs can react instantly to inputs, and you typically need fewer states than
-an equivalent Moore machine.[^4][^1]
-
-Here's a coin-operated vending machine where the output depends on both state
-and the coin inserted:
-
-```rust
-#[derive(Debug, Clone, PartialEq)]
-enum VendingState {
- Idle,
- Has10p,
- Has20p,
-}
-
-#[derive(Debug)]
-enum Coin {
- P10,
- P20,
-}
-
-// Output is produced ON the transition, not from the state alone
-#[derive(Debug)]
-enum Output {
- None,
- Dispense(&'static str),
- ReturnChange(u32),
-}
-
-struct MealyMachine {
- state: VendingState,
-}
-
-impl MealyMachine {
- fn new() -> Self {
- MealyMachine { state: VendingState::Idle }
- }
-
- // Returns (new_state, output) — output depends on BOTH state AND input
- fn transition(&mut self, coin: Coin) -> Output {
- let (next_state, output) = match (&self.state, coin) {
- (VendingState::Idle, Coin::P10) => (VendingState::Has10p, Output::None),
- (VendingState::Idle, Coin::P20) => (VendingState::Has20p, Output::None),
- (VendingState::Has10p, Coin::P10) => (VendingState::Has20p, Output::None),
- (VendingState::Has10p, Coin::P20) => (VendingState::Idle, Output::Dispense("Chewing gum (30p)")),
- (VendingState::Has20p, Coin::P10) => (VendingState::Idle, Output::Dispense("Chocolate (30p)")),
- (VendingState::Has20p, Coin::P20) => (VendingState::Idle, Output::ReturnChange(10)),
- };
- self.state = next_state;
- output
- }
-}
-
-fn main() {
- let mut fsm = MealyMachine::new();
- println!("{:?}", fsm.transition(Coin::P10)); // None
- println!("{:?}", fsm.transition(Coin::P20)); // Dispense("Chewing gum (30p)")
- println!("{:?}", fsm.transition(Coin::P20)); // None
- println!("{:?}", fsm.transition(Coin::P10)); // Dispense("Chocolate (30p)")
-}
-```
-
-Notice `transition()` takes **both** `&self` and `coin` to compute the output —
-a dead giveaway of the Mealy model.[^5]
-
----
-
-## Side-by-Side Comparison
-
-| Feature | Moore | Mealy |
-| :--------------------------- | :--------------------------------------- | :------------------------------------------ |
-| **Output depends on** | Current state only | Current state + input |
-| **Output location** | Attached to state | Attached to transition |
-| **Rust signature** | `fn output(&self)` | `fn transition(&mut self, input) -> Output` |
-| **Number of states** | More (one per output combo) | Fewer [^4] |
-| **Output timing** | After state change (synchronous) | Immediately on input (asynchronous) [^2] |
-| **Input glitch sensitivity** | Immune | More sensitive [^2] |
-| **Best for** | Display/status outputs, digital circuits | Reactive systems, protocol parsers |
-
----
-
-## Key Takeaway
-
-Both models are equally expressive — any Mealy machine can be converted to a
-Moore machine by splitting states, and vice versa. In Rust, the distinction maps
-cleanly: Moore output lives in a method that reads `&self` alone, while Mealy
-output is returned from the transition method that takes both `&self` and the
-input event.[^6][^5]
-<span style="display:none">[^10][^11][^12][^13][^14][^15][^7][^8][^9]</span>
-
-<div align="center">⁂</div>
-
-[^1]: https://www.geeksforgeeks.org/theory-of-computation/difference-between-mealy-machine-and-moore-machine/
-
-[^2]: https://www.youtube.com/watch?v=YiQxeuB56i0
-
-[^3]: https://stackoverflow.com/questions/4009283/mealy-v-s-moore
-
-[^4]: https://mil.ufl.edu/3701/classes/joel/16 Lecture.pdf
-
-[^5]: https://comp.lang.forth.narkive.com/zJDmPu3N/mealy-vs-moore-fsm
-
-[^6]: https://www.reddit.com/r/explainlikeimfive/comments/30uq6e/eli5_the_difference_between_a_mealey_machine_and/
-
-[^7]: https://www.youtube.com/watch?v=kb-Ww8HaHuE
-
-[^8]: https://forum.allaboutcircuits.com/threads/when-to-use-mealy-machine-and-when-to-use-moore-machine.190031/
-
-[^9]: https://github.com/rust-cy/generic-state-machine-rs
-
-[^10]: https://docs.rs/rust-fsm/
-
-[^11]: https://users.rust-lang.org/t/how-to-create-complex-state-machines/82714
-
-[^12]: https://www.reddit.com/r/FPGA/comments/vmlb5z/is_there_any_difference_in_my_implementation_of/
-
-[^13]: https://oneuptime.com/blog/post/2026-02-01-rust-state-machines/view
-
-[^14]: https://lib.rs/crates/edfsm
-
-[^15]: https://blog.devgenius.io/building-robust-distributed-state-machines-in-rust-a-comprehensive-guide-ad1a358134df
diff --git a/content/system_design/design_patterns/revealing_module_pattern.md b/content/system_design/design_patterns/revealing_module_pattern.md
@@ -1,80 +0,0 @@
----
-title: Revealing Module Pattern in JavaScript
----
-
-The **revealing module pattern** wraps private state and behaviour inside a
-function (often an **IIFE**), then **returns a plain object** whose properties
-are the public API. Private helpers stay as inner functions or variables; only
-what you attach to that returned object is visible to callers.
-
-It is one variant of the classic **module pattern** from pre-ES2015 JavaScript,
-when files did not have native `import` / `export`.
-
-## Shape
-
-```js
-const counter = (function () {
- let count = 0 // private
-
- function increment() {
- count += 1
- }
-
- function getCount() {
- return count
- }
-
- return {
- increment,
- getCount,
- }
-})()
-
-counter.increment()
-console.log(counter.getCount()) // 1
-```
-
-The **“revealing”** part is naming: you define real functions (`increment`,
-`getCount`) inside the closure, then **expose** them as properties on the
-returned object. Callers never see `count`; they only use the methods you
-listed.
-
-## Why use it
-
-- **Encapsulation** — Private data is not a property on the returned object, so
- it cannot be read or overwritten from outside without going through your API.
-- **Stable surface** — The returned object is a small, explicit list of
- capabilities (sometimes called a **facade**).
-- **Refactoring room** — You can change how `increment` works internally without
- changing property names on the public object.
-
-## Compared to ES modules
-
-Today, a file can achieve similar separation with **top-level private symbols**
-and explicit `export`:
-
-```js
-let count = 0
-export function increment() {
- count += 1
-}
-export function getCount() {
- return count
-}
-```
-
-The revealing module pattern remains useful when you need a **factory** (many
-independent instances), when bundling legacy scripts without a module loader, or
-when teaching how closures and object literals compose into a module-like API.
-
-## Trade-offs
-
-- **Testing** — Private functions are hard to unit-test in isolation; you test
- through the public API or accept that internals are covered indirectly.
-- **Memory** — Each IIFE invocation creates new function instances; for many
- instances, consider whether a `class` or shared prototype fits better.
-
-## References
-
-- [Addy Osmani — Essential JavaScript Design Patterns](https://addyosmani.com/resources/essentialjsdesignpatterns/book/#revealingmodulepatternjavascript)
-- [MDN — Closures](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Closures)
diff --git a/content/system_design/edge_architecture.md b/content/system_design/edge_architecture.md
@@ -1,99 +0,0 @@
----
-title: Edge Architecture
----
-
-Edge Architecture utilizes a tiered architecture to move data as close to the
-end user as possible while protecting the origin from traffic spikes.
-
-```
- +-------------------+
- | Developer |
- | (npm/docker pull) |
- +---------+---------+
- |
- v
- +-------------------+
- | Anycast / Geo-DNS |
- +---------+---------+
- | (Routes to nearest PoP)
- v
-+-----------------------------------------------------------+
-| EDGE PoP |
-| |
-| +-------------+ +---------------+ +--------------+ |
-| | Edge Compute|--->| L1 Cache | | WAF / DDoS | |
-| | (Auth/Route)| | (Memory/NVMe) | | Protection | |
-| +-------------+ +-------+-------+ +--------------+ |
-+-----------------------------|-----------------------------+
- | (Cache Miss)
- v
-+-----------------------------------------------------------+
-| REGIONAL SHIELD CACHE |
-| +-----------------------------------------------------+ |
-| | L2 Cache (High Capacity SSD, Request Collapsing) | |
-| +--------------------------+--------------------------+ |
-+-----------------------------|-----------------------------+
- | (Cache Miss)
- v
-+-----------------------------------------------------------+
-| ORIGIN INFRASTRUCTURE |
-| +--------------------+ +------------------------+ |
-| | Blob Storage | | Global Metadata DB | |
-| | (S3 / GCS) | | (Spanner / DynamoDB) | |
-| +--------------------+ +------------------------+ |
-+-----------------------------------------------------------+
-```
-
-To achieve this scale, the technology stack must be highly concurrent and
-lightweight:
-
-- **Edge Routing & Proxy**: NGINX, Envoy, or Rust-based proxies to handle
- millions of concurrent TCP connections and perform TLS termination.
-
--- **Edge Compute**: WebAssembly (Wasm) or V8 Isolates running directly on the
-CDN edge to execute custom logic like authentication, A/B testing, and request
-filtering without routing back to the origin.
-
--- **Caching Layer**: Varnish or custom memory-mapped file systems for L1 edge
-caching, backed by high-capacity NVMe drives for L2 regional shields.
-
--- **Data & Origin**: Geographically replicated object storage (like AWS S3) for
-immutable package blobs, and a globally distributed database (like Google Cloud
-Spanner) for mutable package metadata and user entitlements.
-
-## System Data Flows
-
-When a user pulls a package, the request follows a strict path to ensure
-authorization and speed:
-
--- **Resolution**: The client's DNS query hits a Geo-DNS provider, returning the
-Anycast IP of the nearest Edge PoP (Point of Presence).
-
--- **Edge Auth**: The request reaches the Edge Proxy. An Edge Function executes
-immediately, verifying the user's API token against a highly cached subset of
-the metadata database.
-
--- **Cache Lookup**: The proxy checks the L1 Cache. If the package is found, it
-is returned instantly.
-
--- **Shield Fallback**: On an L1 miss, the request goes to the Regional Shield.
-If the package is present in the L2 cache, it is returned and populated in L1.
-
--- **Origin Fetch**: On an L2 miss, the shield fetches the blob from Origin
-Storage, caches it, and streams it back down the chain to the client.
-
-## Performance Impact Chart
-
-This tiered networking approach reduces latency across the distribution
-lifecycle.
-
-```
- Average Response Latency (ms) by Retrieval Tier
-------------------------------------------------------------
-Origin Fetch |################################ (250ms)
-Regional Shield L2 |########### (85ms)
-Edge PoP L1 |### (20ms)
-Predictive Cache |# (5ms)
-------------------------------------------------------------
-`
-```
diff --git a/content/system_design/index.md b/content/system_design/index.md
@@ -1,9 +0,0 @@
----
-title: System Design
----
-
-- [PostgreSQL transactions and ACID](postgresql_transactions_acid.md)
-- [Design patterns](design_patterns/)
-- [Single server](single_server.md)
-- [Server scaling](server_scaling.md)
-- [Server authoritative design](server_authoritative_design.md)
diff --git a/content/system_design/postgresql_transactions_acid.md b/content/system_design/postgresql_transactions_acid.md
@@ -1,491 +0,0 @@
----
-title: PostgreSQL Transactions, Stored Routines, and ACID
----
-
-PostgreSQL is built around **transactions**: bounded units of work that either
-commit as a whole or leave no durable effect (after rollback). Understanding
-how those transactions interact with **multi-version concurrency control
-(MVCC)**, **locking**, and the **write-ahead log (WAL)** explains both everyday
-behaviour (why two writers sometimes wait or fail) and what **ACID** means in
-practice.
-
----
-
-## Transactions in PostgreSQL
-
-A transaction begins explicitly with `BEGIN` (or `START TRANSACTION`) and ends
-with `COMMIT` or `ROLLBACK`. If you do neither, a single SQL statement still runs
-inside an **implicit** transaction: PostgreSQL wraps each standalone statement
-in its own mini-transaction that commits on success.
-
-Inside one transaction, all statements see a coherent database state according to
-the chosen **isolation level** (see below). Until `COMMIT`, other sessions may or
-may not see your changes, depending on isolation and whether they started before
-or after your writes.
-
-**Savepoints** (`SAVEPOINT`, `ROLLBACK TO SAVEPOINT`) let you partially undo
-work without aborting the whole outer transaction—useful for complex workflows
-with selective retry logic.
-
----
-
-## “Stored procedures” vs functions in PostgreSQL
-
-PostgreSQL’s primary server-side programming mechanism is the **`FUNCTION`**
-(PL/pgSQL, SQL, or other languages). A function **always runs in the caller’s
-transaction**; it cannot `COMMIT` or `ROLLBACK` the surrounding transaction from
-inside the function body.
-
-Since **PostgreSQL 11**, **`PROCEDURE`** exists for routines that **may** issue
-transaction control: a procedure can end its own transaction with `COMMIT` or
-`ROLLBACK` and continue (multi-transaction batching inside one call). That is
-the feature people often mean by “stored procedure” when comparing to other
-databases. For most application code, plain SQL inside a client-managed
-transaction plus optional **functions** is still the common pattern.
-
-Whether logic lives in the app or in the server, the same **MVCC and locking
-rules** apply: routines do not bypass concurrency controls.
-
----
-
-## How PostgreSQL handles many writers (conflicts)
-
-PostgreSQL uses **MVCC** instead of readers blocking writers with a single
-global lock on rows. Each row version carries metadata about which transactions
-created and invalidated it (`xmin` / `xmax` system columns). A query reads **row
-versions visible to its snapshot**—typically versions committed before the
-snapshot’s cutoff—so normal **SELECT** does not take row-level locks that block
-concurrent **UPDATE** on other rows.
-
-When **two transactions try to change the same row**, MVCC does not allow both
-to succeed blindly:
-
-1. **Row-level locks** — `UPDATE`, `DELETE`, and `SELECT … FOR UPDATE` acquire
- locks on the target rows. A second transaction that needs the same row may
- **block** until the first commits or rolls back, then either proceeds or
- detects the row was changed (depending on operation and isolation).
-
-2. **Serialization failures** — Under **`REPEATABLE READ`** or **`SERIALIZABLE`**,
- PostgreSQL can abort a transaction with **`SQLSTATE 40001`**
- (`serialization_failure`) when the guarantee of that isolation level would be
- violated—e.g. a concurrent write pattern that could produce anomalies if both
- committed. The application is expected to **retry** the whole transaction.
-
-3. **Deadlocks** — When transactions wait on each other in a cycle, PostgreSQL
- **detects** the deadlock and **aborts one transaction** (`deadlock_detected`)
- so the others can proceed.
-
-So “multiple users writing the same data at the same time” is handled by a mix
-of **waiting**, **abort with retry**, and **ordering**—not by silently losing
-updates. Which outcome you get depends on SQL patterns (locking reads vs blind
-updates) and isolation level.
-
-The subsections below quote **actual C code** from the upstream PostgreSQL tree
-so you can see how these behaviours are implemented. Excerpts are taken from
-commit [`43fc1dc7527`](https://github.com/postgres/postgres/tree/43fc1dc7527c4213c4d48a96bf98738df0acb71a)
-(PostgreSQL is licensed under [the PostgreSQL Licence](https://www.postgresql.org/about/licence/)).
-
-### Row-level blocking: `heap_lock_tuple`
-
-When a tuple is already being modified or deleted, `heap_lock_tuple` inspects
-visibility with **`HeapTupleSatisfiesUpdate`**, unlocks the buffer briefly, and
-may **sleep** until competing lockers finish—first acquiring a **heavyweight
-tuple lock** (`LockTuple` via `heap_acquire_tuplock`), then waiting on the
-transaction or MultiXact holding `xmax`:
-
-```c
-l3:
- result = HeapTupleSatisfiesUpdate(tuple, cid, *buffer);
-
- if (result == TM_Invisible)
- {
- /*
- * This is possible, but only when locking a tuple for ON CONFLICT DO
- * SELECT/UPDATE. We return this value here rather than throwing an
- * error in order to give that case the opportunity to throw a more
- * specific error.
- */
- result = TM_Invisible;
- goto out_locked;
- }
- else if (result == TM_BeingModified ||
- result == TM_Updated ||
- result == TM_Deleted)
- {
- TransactionId xwait;
- uint16 infomask;
- uint16 infomask2;
- bool require_sleep;
- ItemPointerData t_ctid;
-
- /* must copy state data before unlocking buffer */
- xwait = HeapTupleHeaderGetRawXmax(tuple->t_data);
- infomask = tuple->t_data->t_infomask;
- infomask2 = tuple->t_data->t_infomask2;
- ItemPointerCopy(&tuple->t_data->t_ctid, &t_ctid);
-
- LockBuffer(*buffer, BUFFER_LOCK_UNLOCK);
-```
-
-(The branch continues with multixact checks, tuple-lock acquisition, and sleep
-logic; only the opening is shown here.)
-
-The fragment above is the start of the branch taken when the tuple is already
-being modified, updated, or deleted; the remainder of `heap_lock_tuple` decides
-whether sleeping is required and how to wait.
-
-Later, when `require_sleep` is true, the same function acquires the tuple lock
-and waits on the rival transaction id:
-
-```c
- if (!skip_tuple_lock &&
- !heap_acquire_tuplock(relation, tid, mode, wait_policy,
- &have_tuple_lock))
- {
- /*
- * This can only happen if wait_policy is Skip and the lock
- * couldn't be obtained.
- */
- result = TM_WouldBlock;
- /* recovery code expects to have buffer lock held */
- LockBuffer(*buffer, BUFFER_LOCK_EXCLUSIVE);
- goto failed;
- }
-```
-
-```c
- switch (wait_policy)
- {
- case LockWaitBlock:
- XactLockTableWait(xwait, relation, &tuple->t_self,
- XLTW_Lock);
- break;
- case LockWaitSkip:
- if (!ConditionalXactLockTableWait(xwait, false))
- {
- result = TM_WouldBlock;
- /* recovery code expects to have buffer lock held */
- LockBuffer(*buffer, BUFFER_LOCK_EXCLUSIVE);
- goto failed;
- }
- break;
- case LockWaitError:
- if (!ConditionalXactLockTableWait(xwait, log_lock_failures))
- ereport(ERROR,
- (errcode(ERRCODE_LOCK_NOT_AVAILABLE),
- errmsg("could not obtain lock on row in relation \"%s\"",
- RelationGetRelationName(relation))));
- break;
- }
-```
-
-(Source: [`src/backend/access/heap/heapam.c`](https://github.com/postgres/postgres/blob/43fc1dc7527c4213c4d48a96bf98738df0acb71a/src/backend/access/heap/heapam.c),
-function `heap_lock_tuple`; this `switch` sits in the branch that waits on a
-single transaction id after copying `xwait` from `xmax`.)
-
-The **heavyweight** tuple lock is wired to `LockTuple` through a small macro
-layer that maps `LockTupleMode` to the lock manager’s `LOCKMODE`:
-
-```c
-#define LockTupleTuplock(rel, tup, mode) \
- LockTuple((rel), (tup), tupleLockExtraInfo[mode].hwlock)
-#define UnlockTupleTuplock(rel, tup, mode) \
- UnlockTuple((rel), (tup), tupleLockExtraInfo[mode].hwlock)
-#define ConditionalLockTupleTuplock(rel, tup, mode, log) \
- ConditionalLockTuple((rel), (tup), tupleLockExtraInfo[mode].hwlock, (log))
-```
-
-`heap_acquire_tuplock` is the helper that invokes it when blocking is allowed:
-
-```c
-static bool
-heap_acquire_tuplock(Relation relation, const ItemPointerData *tid, LockTupleMode mode,
- LockWaitPolicy wait_policy, bool *have_tuple_lock)
-{
- if (*have_tuple_lock)
- return true;
-
- switch (wait_policy)
- {
- case LockWaitBlock:
- LockTupleTuplock(relation, tid, mode);
- break;
-
- case LockWaitSkip:
- if (!ConditionalLockTupleTuplock(relation, tid, mode, false))
- return false;
- break;
-
- case LockWaitError:
- if (!ConditionalLockTupleTuplock(relation, tid, mode, log_lock_failures))
- ereport(ERROR,
- (errcode(ERRCODE_LOCK_NOT_AVAILABLE),
- errmsg("could not obtain lock on row in relation \"%s\"",
- RelationGetRelationName(relation))));
- break;
- }
- *have_tuple_lock = true;
-
- return true;
-}
-```
-
-(Source: same file, macros near the top of `heapam.c` and `heap_acquire_tuplock`
-below `heap_lock_tuple`.)
-
-### Deadlock detection: `CheckDeadLock` and `DeadLockCheck`
-
-While a backend sleeps on a lock, a **deadlock timeout** can fire; the waiter
-then locks all lock-hash partitions and runs **`DeadLockCheck`**. If the graph
-contains an unresolvable cycle (`DS_HARD_DEADLOCK`), the waiter is removed from
-the queue so **`ProcSleep`** surfaces an error and the transaction aborts. The
-full handler is **`CheckDeadLock`** in `proc.c`:
-
-```c
-/*
- * CheckDeadLock
- *
- * We only get to this routine, if DEADLOCK_TIMEOUT fired while waiting for a
- * lock to be released by some other process. Check if there's a deadlock; if
- * not, just return. If we have a real deadlock, remove ourselves from the
- * lock's wait queue.
- */
-static DeadLockState
-CheckDeadLock(void)
-{
- int i;
- DeadLockState result;
-
- /*
- * Acquire exclusive lock on the entire shared lock data structures. Must
- * grab LWLocks in partition-number order to avoid LWLock deadlock.
- *
- * Note that the deadlock check interrupt had better not be enabled
- * anywhere that this process itself holds lock partition locks, else this
- * will wait forever. Also note that LWLockAcquire creates a critical
- * section, so that this routine cannot be interrupted by cancel/die
- * interrupts.
- */
- for (i = 0; i < NUM_LOCK_PARTITIONS; i++)
- LWLockAcquire(LockHashPartitionLockByIndex(i), LW_EXCLUSIVE);
-
- /*
- * Check to see if we've been awoken by anyone in the interim.
- *
- * If we have, we can return and resume our transaction -- happy day.
- * Before we are awoken the process releasing the lock grants it to us so
- * we know that we don't have to wait anymore.
- *
- * We check by looking to see if we've been unlinked from the wait queue.
- * This is safe because we hold the lock partition lock.
- */
- if (dlist_node_is_detached(&MyProc->waitLink))
- {
- result = DS_NO_DEADLOCK;
- goto check_done;
- }
-
-#ifdef LOCK_DEBUG
- if (Debug_deadlocks)
- DumpAllLocks();
-#endif
-
- /* Run the deadlock check */
- result = DeadLockCheck(MyProc);
-
- if (result == DS_HARD_DEADLOCK)
- {
- /*
- * Oops. We have a deadlock.
- *
- * Get this process out of wait state. (Note: we could do this more
- * efficiently by relying on lockAwaited, but use this coding to
- * preserve the flexibility to kill some other transaction than the
- * one detecting the deadlock.)
- *
- * RemoveFromWaitQueue sets MyProc->waitStatus to
- * PROC_WAIT_STATUS_ERROR, so ProcSleep will report an error after we
- * return.
- */
- Assert(MyProc->waitLock != NULL);
- RemoveFromWaitQueue(MyProc, LockTagHashCode(&(MyProc->waitLock->tag)));
-
- /*
- * We're done here. Transaction abort caused by the error that
- * ProcSleep will raise will cause any other locks we hold to be
- * released, thus allowing other processes to wake up; we don't need
- * to do that here. NOTE: an exception is that releasing locks we
- * hold doesn't consider the possibility of waiters that were blocked
- * behind us on the lock we just failed to get, and might now be
- * wakable because we're not in front of them anymore. However,
- * RemoveFromWaitQueue took care of waking up any such processes.
- */
- }
-
- /*
- * And release locks. We do this in reverse order for two reasons: (1)
- * Anyone else who needs more than one of the locks will be trying to lock
- * them in increasing order; we don't want to release the other process
- * until it can get all the locks it needs. (2) This avoids O(N^2)
- * behavior inside LWLockRelease.
- */
-check_done:
- for (i = NUM_LOCK_PARTITIONS; --i >= 0;)
- LWLockRelease(LockHashPartitionLockByIndex(i));
-
- return result;
-}
-```
-
-`DeadLockCheck` in `deadlock.c` implements the search for cycles and possible
-wait-queue reordering; its header comment states the `DS_HARD_DEADLOCK`
-contract, and the body begins by resetting workspace used by the detector:
-
-```c
-/*
- * DeadLockCheck -- Checks for deadlocks for a given process
- *
- * This code looks for deadlocks involving the given process. If any
- * are found, it tries to rearrange lock wait queues to resolve the
- * deadlock. If resolution is impossible, return DS_HARD_DEADLOCK ---
- * the caller is then expected to abort the given proc's transaction.
- *
- * Caller must already have locked all partitions of the lock tables.
- *
- * On failure, deadlock details are recorded in deadlockDetails[] for
- * subsequent printing by DeadLockReport(). That activity is separate
- * because we don't want to do it while holding all those LWLocks.
- */
-DeadLockState
-DeadLockCheck(PGPROC *proc)
-{
- /* Initialize to "no constraints" */
- nCurConstraints = 0;
- nPossibleConstraints = 0;
- nWaitOrders = 0;
-
- /* Initialize to not blocked by an autovacuum worker */
- blocking_autovacuum_proc = NULL;
-
- /* Search for deadlocks and possible fixes */
- if (DeadLockCheckRecurse(proc))
-```
-
-(Sources: [`src/backend/storage/lmgr/proc.c`](https://github.com/postgres/postgres/blob/43fc1dc7527c4213c4d48a96bf98738df0acb71a/src/backend/storage/lmgr/proc.c)
-and [`src/backend/storage/lmgr/deadlock.c`](https://github.com/postgres/postgres/blob/43fc1dc7527c4213c4d48a96bf98738df0acb71a/src/backend/storage/lmgr/deadlock.c). The `EDGE` / waits-for graph structs at the top of `deadlock.c` are also worth reading alongside [`src/backend/storage/lmgr/README`](https://github.com/postgres/postgres/blob/43fc1dc7527c4213c4d48a96bf98738df0acb71a/src/backend/storage/lmgr/README).)
-
-### Serializable isolation: `PreCommit_CheckForSerializationFailure`
-
-Under **SSI**, commit-time validation walks **read/write conflict** lists; if a
-**dangerous structure** appears (pivot still uncommitted), PostgreSQL raises
-**`ERRCODE_T_R_SERIALIZATION_FAILURE`** (`SQLSTATE 40001`), matching the
-user-visible `serialization_failure`:
-
-```c
-void
-PreCommit_CheckForSerializationFailure(void)
-{
- dlist_iter near_iter;
-
- if (MySerializableXact == InvalidSerializableXact)
- return;
-
- Assert(IsolationIsSerializable());
-
- LWLockAcquire(SerializableXactHashLock, LW_EXCLUSIVE);
-
- /*
- * Check if someone else has already decided that we need to die. Since
- * we set our own DOOMED flag when partially releasing, ignore in that
- * case.
- */
- if (SxactIsDoomed(MySerializableXact) &&
- !SxactIsPartiallyReleased(MySerializableXact))
- {
- LWLockRelease(SerializableXactHashLock);
- ereport(ERROR,
- (errcode(ERRCODE_T_R_SERIALIZATION_FAILURE),
- errmsg("could not serialize access due to read/write dependencies among transactions"),
- errdetail_internal("Reason code: Canceled on identification as a pivot, during commit attempt."),
- errhint("The transaction might succeed if retried.")));
- }
-```
-
-(Source: [`src/backend/storage/lmgr/predicate.c`](https://github.com/postgres/postgres/blob/43fc1dc7527c4213c4d48a96bf98738df0acb71a/src/backend/storage/lmgr/predicate.c).)
-
----
-
-## Isolation levels (what “consistent view” means)
-
-Default isolation is **`READ COMMITTED`**: each statement sees rows committed
-before that **statement** began. That prevents dirty reads but allows **non-repeatable
-reads** and **phantoms** across statements in the same transaction.
-
-**`REPEATABLE READ`** uses a **snapshot** taken at the **first query** in the
-transaction; subsequent reads see the same world. Concurrent commits that would
-change what that snapshot means for writes can trigger **`serialization_failure`**.
-
-**`SERIALIZABLE`** uses **Serializable Snapshot Isolation (SSI)**—tracking
-dependencies between concurrent transactions so that any outcome that could not
-have occurred if transactions ran one-after-another causes an abort. It is the
-strongest guarantee PostgreSQL offers but may force **retries** under contention.
-
----
-
-## ACID “under the hood”
-
-The acronym maps to mechanisms roughly as follows:
-
-### Atomicity
-
-All changes in a transaction are recorded as **WAL records** before commit is
-allowed (see durability). If the session crashes mid-transaction, recovery
-replays only **committed** work; incomplete transactions are rolled back during
-recovery. Within the server, rollback uses transaction status and MVCC rules so
-uncommitted row versions are invisible after abort.
-
-### Consistency
-
-PostgreSQL enforces **declared rules**: primary keys, foreign keys, `CHECK`,
-`NOT NULL`, triggers, etc. **Consistency** in the broader business sense (e.g.
-“account totals match ledger”) remains the application’s responsibility;
-transactions bundle changes so those rules can be checked together before commit.
-
-### Isolation
-
-Implemented through **MVCC snapshots**, **row locks**, **predicate locks /
-SSI bookkeeping** (for serializable), and occasional **blocking**. Isolation is
-not “free”: stronger levels trade convenience for **abort/retry** under
-concurrency.
-
-### Durability
-
-Committed transactions are made **durable** via the **write-ahead log**: redo
-information is written to WAL and, depending on **`synchronous_commit`** and
-replication settings, flushed to disk (and replicas) before the client is told
-`COMMIT` succeeded. That way a crash after commit can **replay** the log and
-restore committed data.
-
----
-
-## Practical takeaways
-
-- Use explicit transactions when several statements must succeed or fail
- together.
-- Expect **`serialization_failure`** under **`SERIALIZABLE`** or **`REPEATABLE
- READ`** when writes overlap; implement **retry** with backoff.
-- Use **`SELECT … FOR UPDATE`** (or equivalent) when you read a row and then
- update it based on that read, to serialize competing writers on that row.
-- Distinguish **`FUNCTION`** (single outer transaction) from **`PROCEDURE`**
- (optional internal commits) when porting “stored procedure” designs from other
- DBMSs.
-
----
-
-## References
-
-- [PostgreSQL server sources (GitHub)](https://github.com/postgres/postgres)
-- [PostgreSQL Manual — Transaction Isolation](https://www.postgresql.org/docs/current/transaction-iso.html)
-- [PostgreSQL Manual — Routine vacuuming / MVCC](https://www.postgresql.org/docs/current/routine-vacuuming.html)
-- [PostgreSQL Manual — Write-Ahead Logging (WAL)](https://www.postgresql.org/docs/current/wal.html)
-- [PostgreSQL Manual — `CREATE PROCEDURE`](https://www.postgresql.org/docs/current/sql-createprocedure.html)
diff --git a/content/system_design/server_authoritative_design.md b/content/system_design/server_authoritative_design.md
@@ -1,1071 +0,0 @@
----
-title: Server-Authoritative Design
----
-
-# Server-Authoritative Design: The Backend-as-Truth Principle
-
-Server-authoritative design is an architectural philosophy where **the backend serves as the single source of truth**, treating client applications as inherently unreliable from a connection and execution standpoint. This principle asserts that while frontends may disconnect, crash, or behave unpredictably, the backend must maintain consistency, enforce business rules, and guarantee data integrity.
-
-```
-┌─────────────────────┐ ┌─────────────────────┐
-│ Client │ │ Backend │
-│ │ │ │
-│ - Ephemeral │ ◄── Request ──► │ - Authoritative │
-│ - Unreliable │ │ - Persistent │
-│ - Untrusted │ ◄── Response ──► │ - Trusted │
-│ - Stateful │ │ - Stateless/ │
-│ (when connected) │ Connection │ Transactional │
-└─────────────────────┘ may fail └─────────────────────┘
- ▲
- │
- ┌─────────────────────┐
- │ Single Source │
- │ of Truth │
- │ │
- │ - Validation │
- │ - Business Rules │
- │ - Data Integrity │
- │ - Transactions │
- └─────────────────────┘
-```
-
-## Core Philosophy
-
-### Why Treat Frontends as Unreliable?
-
-From a **connection perspective**, clients are fundamentally unreliable:
-
-| Client Failure Mode | Implications for Design |
-|---------------------|-------------------------|
-| **Network Disconnection** | WiFi drops, mobile signal loss, VPN issues |
-| **Browser/Tab Closure** | User can close application at any moment |
-| **Device Power Loss** | Battery dies, system crash, forced restart |
-| **Background Throttling** | Mobile OS limits background process execution |
-| **Intentional Disruption** | User kills app via task manager |
-
-From a **security and trust perspective**, clients are inherently untrusted:
-- Code can be inspected, modified, or bypassed
-- Input validation can be circumvented
-- Authentication tokens can be stolen
-- Users may attempt to exploit business logic
-
-### The Server's Role as Authoritative Source
-
-The backend becomes the **arbiter of truth**:
-
-1. **Validation Gatekeeper**: Every mutation request must pass server-side validation
-2. **Business Logic Enforcer**: Rules applied consistently across all clients
-3. **Transactional Guarantor**: ACID properties maintained at database level
-4. **Consensus Point**: Single source for resolving conflicts or race conditions
-
-## Architectural Patterns
-
-### Thin Client vs. Fat Client Spectrum
-
-```
- Thin Client Fat Client
-┌─────────────────────┐ ┌─────────────────────┐
-│ Presentation Only │ │ Business Logic │
-│ │ │ │
-│ + Minimal State │ │ + Rich State │
-│ + Server-Rendered │ │ + Optimistic UI │
-│ + Simple Updates │ │ + Advanced Caching │
-│ - High Latency │ │ - Complex Sync │
-│ - Poor Offline │ │ - Security Risks │
-└─────────┬───────────┘ └─────────┬───────────┘
- │ │
- └────────────────────────────────────┘
- Hybrid Approach
- (Server-Authoritative with
- Intelligent Client Features)
-```
-
-### Hybrid Server-Authoritative Pattern
-
-Modern applications often follow a hybrid approach:
-
-```
-┌─────────────────────────────────────────────────────────────────┐
-│ Client Application │
-│ │
-│ ┌──────────────────┐ ┌──────────────────┐ │
-│ │ Local Cache │ │ Optimistic UI │ │
-│ │ (Ephemeral) │ │ (Immediate │ │
-│ │ │ │ Feedback) │ │
-│ └──────────────────┘ └──────────────────┘ │
-│ │ │ │
-│ └──────────────────────────┼────────────────────────────┘
-│ ▼ │
-│ ┌──────────────────┐ │
-│ │ Sync Engine │ │
-│ │ (Queues/Retry) │ │
-│ └─────────┬────────┘ │
-└─────────────────────────────────────┼────────────────────────────┘
- │
- Network Boundary
- │
-┌─────────────────────────────────────┼────────────────────────────┐
-│ Backend Server │
-│ │
-│ ┌──────────────────┐ ┌──────────────────┐ ┌─────────────┐ │
-│ │ Validation │ │ Business Logic │ │ Data Store │ │
-│ │ (Authoritative) │────▶ (Authoritative) │────▶ (Source of │ │
-│ │ │ │ │ │ Truth) │ │
-│ └──────────────────┘ └──────────────────┘ └─────────────┘ │
-│ │
-│ ┌─────────────────────────────────────────────────────────────┐ │
-│ │ Conflict Resolution │ │
-│ │ - Last-Write-Wins (with versioning) │ │
-│ │ - Operational Transformation (for collaborative editing) │ │
-│ │ - Client ID precedence rules │ │
-│ └─────────────────────────────────────────────────────────────┘ │
-└───────────────────────────────────────────────────────────────────┘
-```
-
-## Implementation Examples
-
-### Rust Backend: Transactional Business Logic
-
-```rust
-// Domain model - The source of truth
-#[derive(Debug, Clone, Serialize, Deserialize)]
-pub struct BankAccount {
- pub id: Uuid,
- pub user_id: Uuid,
- pub balance: Decimal,
- pub version: i32, // For optimistic concurrency control
- pub created_at: DateTime<Utc>,
- pub updated_at: DateTime<Utc>,
-}
-
-// Authoritative business logic
-pub struct AccountService {
- db_pool: PgPool,
-}
-
-impl AccountService {
- /// Transfer funds between accounts - AUTHORITATIVE VERSION
- /// This is the single source of truth for fund transfers
- #[tracing::instrument(skip(self))]
- pub async fn transfer_funds(
- &self,
- from_account_id: Uuid,
- to_account_id: Uuid,
- amount: Decimal,
- request_id: Uuid, // Idempotency key
- ) -> Result<Transaction, TransferError> {
- // Check: amount must be positive
- if amount <= Decimal::ZERO {
- return Err(TransferError::InvalidAmount);
- }
-
- // Wrap in database transaction to maintain consistency
- let mut tx = self.db_pool.begin().await?;
-
- // Use SELECT FOR UPDATE to lock rows
- let (from_account, to_account) = tokio::try_join!(
- sqlx::query_as!(
- BankAccount,
- "SELECT * FROM bank_accounts WHERE id = $1 FOR UPDATE",
- from_account_id
- )
- .fetch_optional(&mut *tx)
- .map_err(TransferError::from),
-
- sqlx::query_as!(
- BankAccount,
- "SELECT * FROM bank_accounts WHERE id = $1 FOR UPDATE",
- to_account_id
- )
- .fetch_optional(&mut *tx)
- .map_err(TransferError::from),
- )?;
-
- // Validate: accounts exist
- let from_account = from_account.ok_or(TransferError::AccountNotFound(from_account_id))?;
- let to_account = to_account.ok_or(TransferError::AccountNotFound(to_account_id))?;
-
- // Validate: sufficient funds (business rule)
- if from_account.balance < amount {
- return Err(TransferError::InsufficientFunds {
- available: from_account.balance,
- requested: amount,
- });
- }
-
- // Validate: not transferring to self
- if from_account_id == to_account_id {
- return Err(TransferError::SelfTransfer);
- }
-
- // Validate: amount limits (business rule)
- const MAX_TRANSFER_AMOUNT: Decimal = Decimal::from_str("10000").unwrap();
- if amount > MAX_TRANSFER_AMOUNT {
- return Err(TransferError::ExceedsLimit(MAX_TRANSFER_AMOUNT));
- }
-
- // Perform the transfer atomically
- let new_from_balance = from_account.balance - amount;
- let new_to_balance = to_account.balance + amount;
-
- // Update with optimistic concurrency control
- let updated_from = sqlx::query!(
- r#"
- UPDATE bank_accounts
- SET balance = $1, version = version + 1, updated_at = NOW()
- WHERE id = $2 AND version = $3
- RETURNING id, user_id, balance, version, created_at, updated_at
- "#,
- new_from_balance,
- from_account_id,
- from_account.version
- )
- .fetch_optional(&mut *tx)
- .await?;
-
- if updated_from.is_none() {
- return Err(TransferError::ConcurrentModification);
- }
-
- let updated_to = sqlx::query!(
- r#"
- UPDATE bank_accounts
- SET balance = $1, version = version + 1, updated_at = NOW()
- WHERE id = $2 AND version = $3
- RETURNING id, user_id, balance, version, created_at, updated_at
- "#,
- new_to_balance,
- to_account_id,
- to_account.version
- )
- .fetch_optional(&mut *tx)
- .await?;
-
- if updated_to.is_none() {
- return Err(TransferError::ConcurrentModification);
- }
-
- // Record the transaction for audit trail
- let transaction = sqlx::query_as!(
- Transaction,
- r#"
- INSERT INTO transactions
- (id, from_account_id, to_account_id, amount, status, request_id, created_at)
- VALUES ($1, $2, $3, $4, $5, $6, NOW())
- RETURNING *
- "#,
- Uuid::new_v4(),
- from_account_id,
- to_account_id,
- amount,
- "COMPLETED",
- request_id
- )
- .fetch_one(&mut *tx)
- .await?;
-
- // Commit the entire transaction
- tx.commit().await?;
-
- Ok(transaction)
- }
-}
-
-// Compare with naive client-side implementation (WHAT NOT TO DO)
-pub struct NaiveClientSideTransfer {
- // This would be BAD: trusting client to validate and calculate
- pub async fn transfer_funds_naive(
- &self,
- from_balance: Decimal, // Client-provided - UNTRUSTED!
- to_balance: Decimal, // Client-provided - UNTRUSTED!
- amount: Decimal,
- ) -> Result<(), TransferError> {
- // Client-side validation - CAN BE BYPASSED
- if amount <= Decimal::ZERO {
- return Err(TransferError::InvalidAmount);
- }
-
- // Client-side business logic - CAN BE MANIPULATED
- if from_balance < amount {
- return Err(TransferError::InsufficientFunds {
- available: from_balance,
- requested: amount,
- });
- }
-
- // Client-side calculation - WRONG if balances changed
- let new_from = from_balance - amount;
- let new_to = to_balance + amount;
-
- // Send to server - RACE CONDITION if other transfers happening
- self.update_balance(from_account_id, new_from).await?;
- self.update_balance(to_account_id, new_to).await?;
-
- Ok(())
- }
-}
-```
-
-### TypeScript Frontend: Optimistic UI with Server Reconciliation
-
-```typescript
-// Client-side representation (NOT authoritative)
-interface ClientBankAccount {
- id: string;
- userId: string;
- balance: number;
- pendingTransactions: PendingTransaction[];
- version: number; // For optimistic updates
-}
-
-// Sync engine that respects server authority
-class AccountSyncEngine {
- private localState: Map<string, ClientBankAccount> = new Map();
- private pendingQueue: PendingOperation[] = [];
- private isOnline: boolean = true;
-
- // Optimistic update - immediate UI feedback
- async transferFundsOptimistic(
- fromAccountId: string,
- toAccountId: string,
- amount: number
- ): Promise<void> {
- // 1. Client-side validation (for UX, not security)
- if (amount <= 0) {
- throw new Error('Amount must be positive');
- }
-
- const fromAccount = this.localState.get(fromAccountId);
- const toAccount = this.localState.get(toAccountId);
-
- if (!fromAccount || !toAccount) {
- throw new Error('Account not found');
- }
-
- // 2. Optimistic update - show changes immediately
- const operationId = crypto.randomUUID();
- const pendingTx: PendingTransaction = {
- id: operationId,
- fromAccountId,
- toAccountId,
- amount,
- status: 'pending',
- timestamp: Date.now(),
- };
-
- // Update local state optimistically
- this.localState.set(fromAccountId, {
- ...fromAccount,
- balance: fromAccount.balance - amount,
- pendingTransactions: [...fromAccount.pendingTransactions, pendingTx],
- version: fromAccount.version + 1,
- });
-
- this.localState.set(toAccountId, {
- ...toAccount,
- balance: toAccount.balance + amount,
- version: toAccount.version + 1,
- });
-
- // Notify UI of change
- this.notifyStateChange();
-
- // 3. Queue for server sync (authoritative)
- const operation: PendingOperation = {
- id: operationId,
- type: 'transfer',
- fromAccountId,
- toAccountId,
- amount,
- retryCount: 0,
- timestamp: Date.now(),
- };
-
- this.pendingQueue.push(operation);
-
- // 4. Try to sync immediately
- await this.flushQueue();
- }
-
- // Sync with server - respects server authority
- private async flushQueue(): Promise<void> {
- if (!this.isOnline || this.pendingQueue.length === 0) {
- return;
- }
-
- // Process operations in order
- for (const operation of this.pendingQueue.slice()) {
- try {
- // Send to authoritative backend
- const response = await fetch('/api/transfers', {
- method: 'POST',
- headers: {
- 'Content-Type': 'application/json',
- 'Idempotency-Key': operation.id,
- },
- body: JSON.stringify({
- fromAccountId: operation.fromAccountId,
- toAccountId: operation.toAccountId,
- amount: operation.amount,
- requestId: operation.id,
- }),
- });
-
- if (response.ok) {
- // Server accepted - update local state to match server
- const serverTransaction = await response.json();
- await this.reconcileWithServer(serverTransaction);
-
- // Remove from queue
- const index = this.pendingQueue.indexOf(operation);
- if (index > -1) {
- this.pendingQueue.splice(index, 1);
- }
- } else if (response.status === 409) {
- // Conflict - server rejected due to business rule
- await this.handleConflict(operation, response);
- } else {
- // Other error - retry later
- operation.retryCount++;
- if (operation.retryCount > 3) {
- // Give up and revert optimistic update
- await this.revertOptimisticUpdate(operation);
- }
- }
- } catch (error) {
- // Network error - will retry when back online
- console.warn('Network error, will retry:', error);
- }
- }
- }
-
- // Reconciliation: align client state with server truth
- private async reconcileWithServer(serverTransaction: ServerTransaction): Promise<void> {
- // Get current optimistic state
- const fromAccount = this.localState.get(serverTransaction.fromAccountId);
- const toAccount = this.localState.get(serverTransaction.toAccountId);
-
- if (!fromAccount || !toAccount) {
- // Something wrong - fetch fresh state from server
- await this.fetchAccountState(serverTransaction.fromAccountId);
- await this.fetchAccountState(serverTransaction.toAccountId);
- return;
- }
-
- // Remove pending transaction
- const updatedFromPending = fromAccount.pendingTransactions.filter(
- tx => tx.id !== serverTransaction.requestId
- );
-
- const updatedToPending = toAccount.pendingTransactions.filter(
- tx => tx.id !== serverTransaction.requestId
- );
-
- // Update to match server authoritative state
- // NOTE: We use server-provided balances, not our calculated ones
- this.localState.set(serverTransaction.fromAccountId, {
- ...fromAccount,
- balance: serverTransaction.fromAccountNewBalance,
- pendingTransactions: updatedFromPending,
- version: serverTransaction.fromAccountVersion,
- });
-
- this.localState.set(serverTransaction.toAccountId, {
- ...toAccount,
- balance: serverTransaction.toAccountNewBalance,
- pendingTransactions: updatedToPending,
- version: serverTransaction.toAccountVersion,
- });
-
- this.notifyStateChange();
- }
-
- // Handle server rejection (business rule violation)
- private async handleConflict(operation: PendingOperation, response: Response): Promise<void> {
- const error = await response.json();
-
- // Revert optimistic update
- await this.revertOptimisticUpdate(operation);
-
- // Fetch fresh state from server
- await this.fetchAccountState(operation.fromAccountId);
- await this.fetchAccountState(operation.toAccountId);
-
- // Notify user of the conflict
- this.notifyError({
- type: 'conflict',
- message: error.message,
- operationId: operation.id,
- });
-
- // Remove from queue
- const index = this.pendingQueue.indexOf(operation);
- if (index > -1) {
- this.pendingQueue.splice(index, 1);
- }
- }
-}
-
-// HTTP API client that respects server authority
-class AuthoritativeApiClient {
- // Idempotent request pattern
- async transferFunds(
- fromAccountId: string,
- toAccountId: string,
- amount: number
- ): Promise<ServerTransaction> {
- const requestId = crypto.randomUUID();
-
- const response = await fetch('/api/transfers', {
- method: 'POST',
- headers: {
- 'Content-Type': 'application/json',
- 'Idempotency-Key': requestId,
- },
- body: JSON.stringify({
- fromAccountId,
- toAccountId,
- amount,
- requestId,
- }),
- });
-
- if (!response.ok) {
- const error = await response.json();
- throw new Error(error.message || `Transfer failed: ${response.status}`);
- }
-
- return response.json();
- }
-
- // Poll for updates - accept server authority
- async pollForUpdates(accountId: string, lastVersion: number): Promise<AccountUpdate> {
- const response = await fetch(`/api/accounts/${accountId}/updates?sinceVersion=${lastVersion}`);
-
- if (response.status === 304) {
- // No changes - server is authoritative about this
- return { hasUpdates: false };
- }
-
- if (!response.ok) {
- throw new Error(`Failed to poll updates: ${response.status}`);
- }
-
- const update = await response.json();
- return {
- hasUpdates: true,
- account: update.account,
- transactions: update.transactions,
- };
- }
-}
-```
-
-## Use Cases and Trade-offs
-
-### When Server-Authoritative Design Is Essential
-
-| Use Case | Why Server-Authoritative | Example Implementation |
-|----------|--------------------------|------------------------|
-| **Financial Systems** | Legal requirement for audit trails, fraud prevention, regulatory compliance | Banking transactions with double-entry bookkeeping and immutable ledger |
-| **E-commerce** | Inventory management (prevent overselling), pricing consistency, tax calculation | Stock reservation system, cart abandonment recovery |
-| **Collaborative Editing** | Conflict resolution, version history, real-time synchronization | Operational transformation in Google Docs, CRDTs with authoritative merge |
-| **Multiplayer Games** | Cheat prevention, game state consistency, fair play enforcement | Deterministic lockstep simulation, server-side game logic |
-| **Healthcare Systems** | Patient safety, regulatory compliance (HIPAA), audit requirements | Electronic health records with change tracking |
-
-### When to Relax Server Authority
-
-| Scenario | Appropriate Approach | Rationale |
-|----------|---------------------|-----------|
-| **Read-heavy applications** | Client-side caching with TTL/ETag | Reduce server load, improve responsiveness |
-| **Offline-first apps** | Local-first with eventual consistency | Must function without network connectivity |
-| **Real-time collaboration** | Hybrid with conflict-free data types | Low latency required, conflicts resolvable |
-| **Static content delivery** | CDN edge caching with invalidation | Performance outweighs consistency needs |
-| **Analytics dashboards** | Client-side aggregation of pre-approved data | Reduce server computation costs |
-
-## Case Studies
-
-### Case Study 1: Banking Application
-
-**Problem**: A mobile banking app where users could theoretically manipulate client-side code to bypass balance checks.
-
-**Server-Authoritative Solution**:
-
-```
-Client (Untrusted) Server (Authoritative)
-┌─────────────────┐ ┌─────────────────────┐
-│ Transfer Request│──────────────▶│ 1. Validate Token │
-│ - Amount: $1000 │ │ 2. Check Balance │
-│ - From: Acct A │ │ (SELECT ... FOR │
-│ - To: Acct B │ │ UPDATE) │
-└─────────────────┘ │ 3. Apply Business │
- │ │ Rules │
- │ │ 4. Execute in │
- │ ┌──────────────┤ Transaction │
- │ │ │ 5. Record Audit │
- │ │ Rejection │ Trail │
- ▼ ▼ └─────────────────────┘
-┌─────────────────┐ ┌─────────────────────┐
-│ UI Shows │ │ Transaction │
-│ Success │ │ Failed: │
-│ Immediately │ │ Insufficient │
-│ (Optimistic) │ │ Funds │
-└─────────────────┘ └─────────────────────┘
- │ │
- │ Server Truth │
- └──────────────────────┘
- ▼
-┌─────────────────┐
-│ UI Reconciles │
-│ with Server │
-│ (Actual State)│
-└─────────────────┘
-```
-
-**Key Architecture Decisions**:
-1. **Idempotent requests**: Each transfer includes unique `request_id` to prevent duplicate processing
-2. **Pessimistic locking**: `SELECT ... FOR UPDATE` prevents race conditions
-3. **Audit trail**: Every transaction recorded immutably
-4. **Client reconciliation**: Optimistic UI updates, but final state from server
-
-### Case Study 2: E-commerce Inventory
-
-**Problem**: Flash sale with 100 items, 10,000 simultaneous users. Prevent overselling.
-
-**Naive Approach (Client-side counting)**:
-```typescript
-// BAD: Client decides if item is available
-async function purchaseItem(itemId: string) {
- const inventory = await fetchInventory(itemId);
- if (inventory.available > 0) {
- // RACE CONDITION: Other users might be buying simultaneously
- await purchase(itemId);
- }
-}
-```
-
-**Server-Authoritative Solution**:
-```rust
-// GOOD: Server authoritatively manages inventory
-#[derive(Debug, Clone, Copy)]
-pub enum InventoryReservation {
- Reserved { reservation_id: Uuid, expires_at: DateTime<Utc> },
- SoldOut,
- Available,
-}
-
-pub struct InventoryService {
- redis: RedisConnection,
- db_pool: PgPool,
-}
-
-impl InventoryService {
- /// Reserve item atomically - only server can do this
- pub async fn reserve_item(
- &self,
- item_id: Uuid,
- user_id: Uuid,
- quantity: i32,
- ) -> Result<InventoryReservation, InventoryError> {
- // Use Redis for distributed lock AND inventory count
- let lock_key = format!("inventory:lock:{}", item_id);
- let inventory_key = format!("inventory:{}", item_id);
-
- // Atomic check-and-decrement
- let script = r#"
- local current = redis.call('GET', KEYS[2])
- if not current or tonumber(current) < tonumber(ARGV[1]) then
- return {false, 'insufficient'}
- end
-
- local new_val = tonumber(current) - tonumber(ARGV[1])
- redis.call('SET', KEYS[2], new_val)
-
- local reservation_id = ARGV[2]
- local expires_at = ARGV[3]
- redis.call('HSET', 'reservations', reservation_id, ARGV[4])
- redis.call('EXPIREAT', reservation_id, expires_at)
-
- return {true, reservation_id}
- "#;
-
- let reservation_id = Uuid::new_v4();
- let expires_at = Utc::now() + chrono::Duration::minutes(10);
-
- let result: (bool, String) = redis::cmd("EVAL")
- .arg(script)
- .arg(2) // number of keys
- .arg(&lock_key)
- .arg(&inventory_key)
- .arg(quantity)
- .arg(reservation_id.to_string())
- .arg(expires_at.timestamp())
- .arg(user_id.to_string())
- .query_async(&mut self.redis.clone())
- .await?;
-
- match result {
- (true, rid) => Ok(InventoryReservation::Reserved {
- reservation_id: Uuid::parse_str(&rid).unwrap(),
- expires_at,
- }),
- (false, _) => Ok(InventoryReservation::SoldOut),
- }
- }
-}
-```
-
-## Workflow Implementation Steps
-
-### Step 1: Identify Authoritative vs. Non-Authoritative Operations
-
-```
-┌─────────────────────────────────────────────────────────────┐
-│ Operation Analysis Matrix │
-├─────────────────┬─────────────────┬─────────────────────────┤
-│ Operation │ Must Be │ Can Be │
-│ │ Authoritative │ Client-Side │
-├─────────────────┼─────────────────┼─────────────────────────┤
-│ Funds Transfer │ ✅ Yes │ ❌ No │
-│ │ (Financial rule)│ │
-├─────────────────┼─────────────────┼─────────────────────────┤
-│ Form Validation │ ⚠️ Partial │ ✅ Yes (for UX) │
-│ │ (Final check │ │
-│ │ on server) │ │
-├─────────────────┼─────────────────┼─────────────────────────┤
-│ Search Filtering│ ❌ No │ ✅ Yes │
-│ │ (Presentation │ │
-│ │ only) │ │
-├─────────────────┼─────────────────┼─────────────────────────┤
-│ Read Operations │ ⚠️ Depends │ ✅ Often │
-│ │ (Cached vs. │ │
-│ │ fresh data) │ │
-└─────────────────┴─────────────────┴─────────────────────────┘
-```
-
-### Step 2: Design Idempotent APIs
-
-```rust
-// Rust backend implementing idempotency
-pub struct IdempotencyService {
- db_pool: PgPool,
-}
-
-impl IdempotencyService {
- pub async fn execute_with_idempotency<F, T, E>(
- &self,
- request_id: Uuid,
- user_id: Uuid,
- operation: &str,
- f: F,
- ) -> Result<T, E>
- where
- F: FnOnce() -> futures::future::BoxFuture<'static, Result<T, E>>,
- E: From<IdempotencyError>,
- {
- // Check if we've already processed this request
- let existing = sqlx::query!(
- r#"
- SELECT result, status_code
- FROM idempotency_keys
- WHERE key = $1 AND user_id = $2 AND operation = $3
- "#,
- request_id,
- user_id,
- operation
- )
- .fetch_optional(&self.db_pool)
- .await?;
-
- if let Some(record) = existing {
- // Request already processed - return cached result
- match record.status_code.as_str() {
- "COMPLETED" => {
- let result: T = serde_json::from_str(&record.result.unwrap())?;
- return Ok(result);
- }
- "FAILED" => {
- return Err(serde_json::from_str(&record.result.unwrap())?);
- }
- _ => {
- // In progress - wait or retry
- return Err(IdempotencyError::RequestInProgress.into());
- }
- }
- }
-
- // First time - record that we're starting
- sqlx::query!(
- r#"
- INSERT INTO idempotency_keys
- (key, user_id, operation, status_code, created_at, updated_at)
- VALUES ($1, $2, $3, 'PROCESSING', NOW(), NOW())
- "#,
- request_id,
- user_id,
- operation
- )
- .execute(&self.db_pool)
- .await?;
-
- // Execute the actual operation
- let result = f().await;
-
- // Record the outcome
- match &result {
- Ok(success_result) => {
- let result_json = serde_json::to_string(success_result)?;
- sqlx::query!(
- r#"
- UPDATE idempotency_keys
- SET status_code = 'COMPLETED', result = $1, updated_at = NOW()
- WHERE key = $2 AND user_id = $3 AND operation = $4
- "#,
- result_json,
- request_id,
- user_id,
- operation
- )
- .execute(&self.db_pool)
- .await?;
- }
- Err(error) => {
- let error_json = serde_json::to_string(error)?;
- sqlx::query!(
- r#"
- UPDATE idempotency_keys
- SET status_code = 'FAILED', result = $1, updated_at = NOW()
- WHERE key = $2 AND user_id = $3 AND operation = $4
- "#,
- error_json,
- request_id,
- user_id,
- operation
- )
- .execute(&self.db_pool)
- .await?;
- }
- }
-
- result
- }
-}
-```
-
-### Step 3: Implement Optimistic UI with Reconciliation
-
-```typescript
-// TypeScript reconciliation engine
-class ReconciliationEngine<T> {
- private localState: T;
- private pendingMutations: Array<{
- id: string;
- mutation: (state: T) => T;
- timestamp: number;
- }> = [];
-
- constructor(initialState: T) {
- this.localState = initialState;
- }
-
- // Apply mutation optimistically
- mutate(mutation: (state: T) => T, mutationId: string): T {
- const pending = {
- id: mutationId,
- mutation,
- timestamp: Date.now(),
- };
-
- this.pendingMutations.push(pending);
- this.localState = mutation(this.localState);
-
- return this.localState;
- }
-
- // Reconcile with server authoritative state
- reconcile(serverState: T, appliedMutationIds: string[]): T {
- // Remove mutations that server has confirmed
- this.pendingMutations = this.pendingMutations.filter(
- mutation => !appliedMutationIds.includes(mutation.id)
- );
-
- // Start from server state (authoritative)
- let reconciledState = serverState;
-
- // Re-apply pending mutations that server hasn't seen yet
- for (const pending of this.pendingMutations) {
- reconciledState = pending.mutation(reconciledState);
- }
-
- this.localState = reconciledState;
- return reconciledState;
- }
-
- // Handle server rejection
- rejectMutation(mutationId: string, serverState: T): T {
- // Remove the rejected mutation
- this.pendingMutations = this.pendingMutations.filter(
- m => m.id !== mutationId
- );
-
- // Reset to server state
- this.localState = serverState;
-
- // Re-apply remaining pending mutations
- for (const pending of this.pendingMutations) {
- this.localState = pending.mutation(this.localState);
- }
-
- return this.localState;
- }
-}
-```
-
-## Performance Considerations
-
-### Trade-offs: Latency vs. Consistency
-
-```
- Response Time Impact
-┌─────────────────────────────────────────────────┐
-│ Client-Side Heavy │
-│ │
-│ Fast ╭───────────────────────────────────╮ │
-│ │■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■│ │
-│ │■■■ Optimistic Updates ■■■■│ │
-│ │■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■│ │
-│ │■■■■ Immediate Feedback ■■■■■■■■■■■│ │
-│ │■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■│ │
-│ ╰───────────────────────────────────╯ │
-│ │
-│ Server-Authoritative │
-│ │
-│ ╭───────────────────────────────────╮ │
-│ │■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■│ │
-│ │■■■ Network Round-Trips ■■■■│ │
-│ │■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■│ │
-│ │■■■■ Validation & Locking ■■■■■■■│ │
-│ Slow │■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■│ │
-│ ╰───────────────────────────────────╯ │
-│ │
-└─────────────────────────────────────────────────┘
- Consistency
-```
-
-### Optimization Strategies
-
-1. **Batched Authoritative Operations**
-```rust
-// Instead of multiple round-trips
-for item in cart.items {
- reserve_item(item.id).await?; // N+1 problem
-}
-
-// Batch authoritative operations
-batch_reserve_items(cart.items).await?; // Single round-trip
-```
-
-2. **Edge Caching with Validation**
-```typescript
-// Cache validation rules at edge, final check at origin
-async function validateInputCached(input: unknown): Promise<ValidationResult> {
- // Check local cache first (rules don't change often)
- const cachedRules = await cache.get('validation-rules');
- const quickResult = quickValidate(input, cachedRules);
-
- if (!quickResult.valid) {
- return quickResult;
- }
-
- // Final authoritative check
- return await authoritativeValidate(input);
-}
-```
-
-3. **Predictive Pre-authorization**
-```rust
-// Pre-approve likely actions based on user behavior
-pub async fn pre_authorize_transfer(
- &self,
- user_id: Uuid,
- max_amount: Decimal,
- window_minutes: i32,
-) -> Result<PreAuthorization, AuthError> {
- // Analytics suggest user will transfer < $500 in next 5 minutes
- // Pre-reserve capacity, reduce latency for actual transfer
-}
-```
-
-## Security Implications
-
-### Threat Model for Server-Authoritative Systems
-
-| Threat | Without Server Authority | With Server Authority |
-|--------|--------------------------|------------------------|
-| **Balance Manipulation** | Client can modify JavaScript to bypass checks | Server validates all transactions |
-| **Race Conditions** | Two clients might oversell inventory | Distributed locks prevent overselling |
-| **Replay Attacks** | Same transaction submitted multiple times | Idempotency keys prevent duplicates |
-| **Business Logic Bypass** | Client can skip validation steps | All rules enforced server-side |
-| **Data Tampering** | Client-side state can be manipulated | Only server state is authoritative |
-
-### Defense in Depth Strategy
-
-```
-┌─────────────────────────────────────────────────────────┐
-│ Defense in Depth Layers │
-├─────────────────┬───────────────────────────────────────┤
-│ Layer │ Implementation │
-├─────────────────┼───────────────────────────────────────┤
-│ Client-Side │ Basic validation (UX only) │
-│ │ Input sanitization │
-├─────────────────┼───────────────────────────────────────┤
-│ Network │ HTTPS/TLS 1.3 │
-│ │ Request signing │
-├─────────────────┼───────────────────────────────────────┤
-│ API Gateway │ Rate limiting │
-│ │ Schema validation │
-├─────────────────┼───────────────────────────────────────┤
-│ Application │ Business logic validation │
-│ │ Authentication/Authorization │
-├─────────────────┼───────────────────────────────────────┤
-│ Database │ ACID transactions │
-│ │ Row-level security │
-│ │ Audit logging │
-└─────────────────┴───────────────────────────────────────┘
-```
-
-## Future Evolution
-
-### Beyond Traditional Server-Authoritative
-
-1. **Edge Computing with Authoritative Rules**
- - Push validation logic to CDN edge
- - Final authorization at origin
- - Reduced latency while maintaining security
-
-2. **Blockchain as Authoritative Layer**
- - Smart contracts as business logic
- - Immutable transaction ledger
- - Decentralized but still authoritative
-
-3. **Federated Authority**
- - Multiple authoritative services
- - Consensus protocols for coordination
- - Used in distributed systems like Kubernetes
-
-4. **Zero-Trust with Continuous Authorization**
- - Every operation re-validated
- - Context-aware authorization
- - Dynamic policy evaluation
-
-## Conclusion
-
-The server-authoritative design principle remains essential for systems where correctness, security, and consistency matter more than pure latency. While modern applications often adopt hybrid approaches—optimistic UI updates with eventual server reconciliation—the fundamental truth remains: **the backend must be the ultimate arbiter of business rules and data integrity**.
-
-The key is not eliminating client-side logic, but rather clearly demarcating which operations require server authority and which can be safely delegated. This boundary should be explicit in both architecture documentation and code implementation.
-
-As connectivity improves and edge computing matures, the line between "client" and "server" may blur, but the need for authoritative validation of critical operations will persist. The most robust systems will continue to embrace server authority while optimizing for user experience through intelligent client-side enhancements.
-
----
-
-*Article written with examples in Rust and TypeScript, demonstrating practical implementation of server-authoritative patterns while maintaining responsive user interfaces.*
-\ No newline at end of file
diff --git a/content/system_design/server_scaling.md b/content/system_design/server_scaling.md
@@ -1,31 +0,0 @@
----
-title: Server Scaling
----
-
-## Load Balancer
-
-A load balancer evenly distributes incoming traffic among web servers that are
-defined in a load-balanced set.
-
-## Database Replication
-
-A master database generally only supports write operations. A slave database
-gets copies of the data from the master database and only supports read
-operations.
-
-<img src="/assets/system_design-server_scaling.png" alt="Server Setup" width="100%">
-
-## Cache Tier
-
-The [cache](/networking/cache.md) tier is a temporary data store layer, much
-faster than the database. The benefits of having a separate cache tier include
-better system performance, ability to reduce database workloads, and the ability
-to scale the cache tier independently.
-
-## Content Delivery Network (CDN)
-
-A CDN is a network of geographically dispersed servers used to deliver static
-content. CDN servers cache static content like images, videos, CSS, JavaScript
-files, etc.
-
-<img src="/assets/cdn_cache_server.png" alt="CDN Cache Server Setup" width="100%">
diff --git a/content/system_design/single_server.md b/content/system_design/single_server.md
@@ -1,8 +0,0 @@
----
-title: Single Server Setup
----
-
-The simplest design is everything running on a single server: web app, database,
-cache, etc.
-
-<img src="/assets/single_server.png" alt="Single Server Setup" width="100%">
diff --git a/content/system_design/state_management/action_reducer_pattern.md b/content/system_design/state_management/action_reducer_pattern.md
@@ -1,71 +0,0 @@
----
-title: Action/Reducer Pattern
----
-
-## Redux-Like Action/Reducer Pattern
-
-```
-┌──────────────────────────┐
-│ UI / View │
-│ (components, widgets) │
-└────────────┬─────────────┘
- │
- │ dispatch(action)
- ▼
- ┌───────────────┐
- │ ACTION │
- │ type + data │
- └──────┬────────┘
- │
- │ sent to
- ▼
-┌──────────────────────────────────────┐
-│ STORE (currentState+ reducer) │
-└────────────┬─────────────────────────┘
- │
- │ calls
- ▼
- ┌───────────────────────┐
- │ REDUCER │
- │ newState = f( │
- │ state, action │
- │ ) │
- └─────────┬─────────────┘
- │
- │ returns newState
- ▼
- ┌───────────────┐
- │ STORE │
- │ updates state │
- └──────┬────────┘
- │
- │ notifies subscribers
- ▼
-┌───────────────────────────┐
-│ UI / View │
-│ re-renders from state │
-└───────────────────────────┘
-```
-
-For more predictable updates, you can implement a Redux-style pattern using
-typed discriminated unions:
-
-```ts
-type Action =
- | { type: "ADD_TODO"; payload: { text: string } }
- | { type: "TOGGLE_TODO"; payload: { id: number } };
-
-function reducer(state: AppState, action: Action): AppState {
- switch (action.type) {
- case "ADD_TODO":
- return { ...state, todos: [...state.todos, action.payload] };
- case "TOGGLE_TODO":
- return { ...state /* ... */ };
- default:
- return state;
- }
-}
-```
-
-TypeScript's exhaustive checking ensures your switch covers every action type,
-preventing silent missed cases.
diff --git a/content/system_design/state_management/broadcastchannel_api.md b/content/system_design/state_management/broadcastchannel_api.md
@@ -1,97 +0,0 @@
----
-title: Broadcast Channel API
----
-
-```
-BroadcastChannel (cross-context bus)
----------------------------------------
-
- [ Tab 1 ] [ Tab 2 ] [ Worker / Tab 3 ]
- +-----------+ +-----------+ +-----------------+
- | App code | | App code | | App code |
- | publishes | | subscribes| | subscribes |
- +-----+-----+ +-----+-----+ +--------+--------+
- | | |
- +---------+---------+-------------------------+
- v
- +-----------------------+
- | BroadcastChannel | browser-level bus
- | ("app-events") |
- +-----------------------+
-```
-
-`BroadcastChannel` is a built-in browser API that lets different browser
-contexts (tabs, iframes, workers) on the same origin communicate by posting
-messages to a named channel. It's essentially a pub/sub bus at the browser level
-— perfect for syncing state across tabs without a server.
-
-## Typed BroadcastChannel in TypeScript
-
-You can wrap it in a typed class to get full compile-time safety:
-
-```ts
-type StateMessage<T> =
- | { type: "STATE_UPDATE"; payload: Partial<T> }
- | { type: "REQUEST_SYNC" };
-
-class SyncedStore<T extends object> {
- private state: T;
- private channel: BroadcastChannel;
- private subscribers = new Set<(s: T) => void>();
-
- constructor(name: string, initial: T) {
- this.state = { ...initial };
- this.channel = new BroadcastChannel(name);
- this.channel.onmessage = (e: MessageEvent<StateMessage<T>>) => {
- if (e.data.type === "STATE_UPDATE") {
- this.state = { ...this.state, ...e.data.payload };
- this.subscribers.forEach((cb) => cb(this.state));
- }
- };
- }
-
- setState(updates: Partial<T>) {
- this.state = { ...this.state, ...updates };
- this.channel.postMessage({ type: "STATE_UPDATE", payload: updates });
- this.subscribers.forEach((cb) => cb(this.state));
- }
-
- subscribe(cb: (s: T) => void) {
- this.subscribers.add(cb);
- return () => this.subscribers.delete(cb);
- }
-
- destroy() {
- this.channel.close();
- }
-}
-```
-
-TypeScript's MessageEvent<T> generic ensures the e.data payload is typed
-correctly, preventing you from accidentally posting or reading the wrong shape.
-
-### Common Use Cases
-
-- Session sync — if a user logs out in one tab, broadcast a logout action to
- immediately reflect that across all open tabs
-
-- Shopping cart sync — changes made in one tab propagate instantly to others
-
-- Shared worker state — coordinate state between a ServiceWorker and page tabs
-
-### Limitations to Know
-
-- BroadcastChannel does not fire in the tab that sent the message — only other
- tabs receive it, so you still update local state directly
-
-- It only works within the same origin (same protocol + domain + port)
-
-- It has no built-in message history — a newly opened tab won't get past state
- unless you implement a REQUEST_SYNC handshake pattern (one tab requests the
- current state, another responds with it)
-
-- Not available in IE, but has full support in all modern browsers
-
-This pattern pairs naturally with the observable store from the previous answer
-— the BroadcastChannel becomes an external subscriber that mirrors state updates
-to other tabs.
diff --git a/content/system_design/state_management/index.md b/content/system_design/state_management/index.md
@@ -1,60 +0,0 @@
----
-title: State Management
----
-
-## Vanilla TypeScript Core State Management Patterns
-
-The foundational approach is a typed observable store — a generic `Store<T>`
-class that holds state, exposes `getState()`, and notifies subscribers via
-`setState()`. TypeScript generics enforce that only valid keys/shapes can be
-set. Three main patterns build on this:
-
-- [Observable Store](observer_pattern.md)
- — a central class with `subscribe/setState`, good for simple-to-medium apps
-
-- [Action/Reducer](action_reducer_pattern.md)
- — Redux-style discriminated union actions processed by a pure `reducer`
- function; TypeScript's exhaustive checking ensures no action is missed
-
-- [Proxy-based reactivity](proxy_based_reactivity.md)
- — intercepts property assignments to auto-trigger re-renders, mimicking
- Vue/MobX without a library
-
-- [Path-based (DeepState)](path_based_state.md) -
- Best for bested state trees
-
-- [Persistent State](persistent_state.md) -
- Best for cross-session state
-
- Best practices: always define a state interface, use `Partial<T>` for updates,
- prefer immutable spreads, and normalise collections as `{ byId, allIds }`.
-
-### Difference between observer, singleton channel and BroadcastChannel
-
-1. Observer — one object talking to its own watchers
-
-2. [Singleton Channel](singleton_channels.md)
- — any part of your app talking to any other part, within the same tab
-
-3. [BroadcastChannel](broadcastchannel_api.md)
- — any tab talking to any other tab in the same browser
-
-In practice, you'd often use all three together: a `BroadcastChannel` receives a
-cross-tab message → publishes onto a singleton `Channel<T>` → individual
-components subscribed via the Observer pattern react to the update.
-
-## TypeScript-Specific Best Practices
-
-- Always define a state interface — never use any for your state shape
-
-- Use `Partial<T>` for updates — so you only pass the fields you're changing
-
-- Prefer immutable updates — spread operators `({ ...state, ...updates })`
- instead of mutating directly
-
-- Normalize entity collections — store arrays as
- `{ byId: Record<id, T>, allIds:
- id[] }` for O(1) lookups
-
-- Use `localStorage` with version migrations — persist state but include a
- `_version` field so you can safely migrate old data
diff --git a/content/system_design/state_management/observer_pattern.md b/content/system_design/state_management/observer_pattern.md
@@ -1,55 +0,0 @@
----
-title: Observer Pattern
----
-
-## Centralized Store (Observable Pattern)
-
-```
-Observer pattern (object-centric)
-------------------------------------
-
- +-------------------+
- | UserService | subject owns its observers
- +-------------------+
- | | |
- v v v
- +--------+ +--------+ +--------+
- | View A | | View B | | Logger |
- +--------+ +--------+ +--------+
-```
-
-The most fundamental approach is a typed store class that holds state, exposes a
-getState() method, and notifies subscribers on change:
-
-```ts
-interface AppState {
- user: User | null;
- theme: "light" | "dark";
-}
-
-class Store<T> {
- private state: T;
- private subscribers = new Set<(state: T) => void>();
-
- constructor(initialState: T) {
- this.state = { ...initialState };
- }
-
- getState(): T {
- return { ...this.state };
- }
-
- setState(updates: Partial<T>) {
- this.state = { ...this.state, ...updates };
- this.subscribers.forEach((cb) => cb(this.getState()));
- }
-
- subscribe(cb: (state: T) => void): () => void {
- this.subscribers.add(cb);
- return () => this.subscribers.delete(cb);
- }
-}
-```
-
-TypeScript generics make the store fully type-safe — setState only accepts keys
-that exist on your state interface.
diff --git a/content/system_design/state_management/path_based_state.md b/content/system_design/state_management/path_based_state.md
@@ -1,77 +0,0 @@
----
-title: Path-Based State (Deep State)
----
-
-```
-State Tree
-+----------------------------------+
-| app |
-| user |
-| profile |
-| name: "Alice" <--- path |
-| age: 30 |
-| cart |
-| count: 2 |
-+----------------------------------+
-
-subscribe("user.profile.name") --> only fires when name changes
-subscribe("cart.count") --> only fires when count changes
-```
-
-Instead of subscribing to the whole state object, subscribers listen to a
-specific path within a nested state tree — e.g. "user.profile.name". Only
-changes to that exact path trigger their callback.
-
-```ts
-type Path<T> = string; // e.g. "user.profile.name"
-
-class DeepStore<T extends object> {
- private state: T;
- private subscribers = new Map<string, Set<(val: unknown) => void>>();
-
- constructor(initial: T) {
- this.state = initial;
- }
-
- subscribe<V>(path: Path<T>, cb: (val: V) => void): () => void {
- if (!this.subscribers.has(path)) this.subscribers.set(path, new Set());
- this.subscribers.get(path)!.add(cb as (val: unknown) => void);
- return () =>
- this.subscribers.get(path)?.delete(cb as (val: unknown) => void);
- }
-
- set(path: Path<T>, value: unknown) {
- const keys = path.split(".");
- let cursor: any = this.state;
- for (let i = 0; i < keys.length - 1; i++) cursor = cursor[keys[i]];
- cursor[keys[keys.length - 1]] = value;
- this.subscribers.get(path)?.forEach((cb) => cb(value));
- }
-
- get(path: Path<T>): unknown {
- return path.split(".").reduce((obj: any, key) => obj?.[key], this.state);
- }
-}
-
-// Usage
-interface AppState {
- user: { profile: { name: string; age: number } };
- cart: { count: number };
-}
-
-const store = new DeepStore<AppState>({
- user: { profile: { name: "Alice", age: 30 } },
- cart: { count: 2 },
-});
-
-store.subscribe<string>(
- "user.profile.name",
- (name) => console.log(`Name changed: ${name}`),
-);
-store.set("user.profile.name", "Bob"); // fires callback
-store.set("cart.count", 5); // does NOT fire name callback
-```
-
-Use case: Large nested state trees where you want granular, performant
-subscriptions — only the parts of the UI that care about a specific slice
-re-render.
diff --git a/content/system_design/state_management/persistent_state.md b/content/system_design/state_management/persistent_state.md
@@ -1,84 +0,0 @@
----
-title: Persistent State
----
-
-```
- +-------------------+
-setState() --> | PersistentStore | --> subscribers notified
- +-------------------+
- | ^
- save| |load on init
- v |
- +-------------------+
- | localStorage |
- +-------------------+
-```
-
-State that survives page reloads by syncing to localStorage (or sessionStorage).
-Includes a _version field so you can safely migrate stale data from older
-schemas.
-
-```ts
-interface Versioned {
- _version: number;
-}
-
-class PersistentStore<T extends Versioned> {
- private state: T;
- private subscribers = new Set<(s: T) => void>();
- private key: string;
- private currentVersion: number;
- private migrate: (old: any) => T;
-
- constructor(key: string, initial: T, migrate: (old: any) => T) {
- this.key = key;
- this.currentVersion = initial._version;
- this.migrate = migrate;
- this.state = this.load(initial);
- }
-
- private load(initial: T): T {
- const raw = localStorage.getItem(this.key);
- if (!raw) return initial;
- const parsed = JSON.parse(raw);
- if (parsed._version !== this.currentVersion) return this.migrate(parsed);
- return parsed as T;
- }
-
- private save() {
- localStorage.setItem(this.key, JSON.stringify(this.state));
- }
-
- getState(): T {
- return { ...this.state };
- }
-
- setState(updates: Partial<T>) {
- this.state = { ...this.state, ...updates };
- this.save();
- this.subscribers.forEach((cb) => cb(this.getState()));
- }
-
- subscribe(cb: (s: T) => void) {
- this.subscribers.add(cb);
- return () => this.subscribers.delete(cb);
- }
-}
-
-// Usage
-interface ThemeState extends Versioned {
- theme: "light" | "dark";
- fontSize: number;
-}
-
-const themeStore = new PersistentStore<ThemeState>(
- "theme",
- { _version: 2, theme: "light", fontSize: 16 },
- (old) => ({ _version: 2, theme: old.theme ?? "light", fontSize: 16 }), // migrate v1 → v2
-);
-
-themeStore.setState({ theme: "dark" }); // persisted to localStorage immediately
-```
-
-Use case: User preferences, session data, or any state that should survive a
-page refresh — theme, language, last-visited route, partially filled forms.
diff --git a/content/system_design/state_management/proxy_based_reactivity.md b/content/system_design/state_management/proxy_based_reactivity.md
@@ -1,99 +0,0 @@
----
-title: Proxy-based Reactivity
----
-
-```
- Proxy (trap layer)
- +------------------+
-state.count++ | set trap fires |---> notify subscribers
-<-- direct --> | get trap fires |---> track dependencies
- +------------------+
- |
- +------------------+
- | Raw state obj |
- +------------------+
-```
-
-## Reactive State with Proxy
-
-A JavaScript Proxy wraps your state object and intercepts get and set
-operations. Instead of calling setState() manually, any direct property
-assignment automatically triggers subscribers — the same mechanic Vue 3 and MobX
-use internally.
-
-```ts
-type Subscriber<T> = (state: T) => void;
-
-function reactive<T extends object>(
- initial: T,
- onChange: Subscriber<T>,
-): T {
- return new Proxy(initial, {
- set(target, prop, value) {
- (target as any)[prop] = value;
- onChange(target);
- return true;
- },
- });
-}
-
-// Usage
-interface AppState {
- count: number;
- user: string | null;
-}
-
-const state = reactive<AppState>(
- { count: 0, user: null },
- (s) => console.log("State changed:", s),
-);
-
-state.count++; // logs automatically
-state.user = "Alice"; // logs automatically
-```
-
-For finer control, you can track which properties were accessed (get trap) and
-only notify subscribers that depend on that specific property — this is called
-dependency tracking:
-
-```ts
-const subscribers = new Map<string | symbol, Set<() => void>>();
-let activeEffect: (() => void) | null = null;
-
-function reactive<T extends object>(initial: T): T {
- return new Proxy(initial, {
- get(target, prop) {
- if (activeEffect) {
- if (!subscribers.has(prop)) subscribers.set(prop, new Set());
- subscribers.get(prop)!.add(activeEffect);
- }
- return (target as any)[prop];
- },
- set(target, prop, value) {
- (target as any)[prop] = value;
- subscribers.get(prop)?.forEach((fn) => fn());
- return true;
- },
- });
-}
-
-function effect(fn: () => void) {
- activeEffect = fn;
- fn(); // run once to collect dependencies via get traps
- activeEffect = null;
-}
-
-// Usage
-const state = reactive({ count: 0, user: "Alice" });
-
-effect(() => {
- console.log(`Count is: ${state.count}`); // only re-runs when count changes
-});
-
-state.count = 5; // triggers effect
-state.user = "Bob"; // does NOT trigger effect (not accessed in effect)
-```
-
-Watch out: Proxy only intercepts the top-level object by default. For deeply
-nested state you need to recursively wrap nested objects in their own Proxy —
-which is exactly what Vue 3's reactive() does under the hood.
diff --git a/content/system_design/state_management/singleton_channels.md b/content/system_design/state_management/singleton_channels.md
@@ -1,94 +0,0 @@
----
-title: Singleton Channel
----
-
-```
-Singleton Channel (neutral in-app bus)
------------------------------------------
-
- Publishers Subscribers
- ---------- -----------
- +--------+ +--------+
- | View A |----+ +-->| View C |
- +--------+ | | +--------+
- v |
- +---------------------------+
- | AppChannel<Event> | (singleton)
- +---------------------------+
- ^ |
- +--------+ | | +--------+
- | Store |----+ +-->| Logger |
- +--------+ +--------+
-```
-
-A `Channel<T>` is a globally shared, typed event bus. Any component can publish
-a value of type T into it, and any component that has subscribed will receive
-that value — FIFO (first subscriber registered, first notified). It's simpler
-than a full store because it carries no persistent state; it just broadcasts a
-moment-in-time event.
-
-```ts
-class Channel<T> {
- private subscribers: Array<(data: T) => void> = [];
-
- subscribe(cb: (data: T) => void): () => void {
- this.subscribers.push(cb);
- return () => {
- this.subscribers = this.subscribers.filter((s) => s !== cb);
- };
- }
-
- publish(data: T): void {
- for (const cb of this.subscribers) {
- cb(data);
- }
- }
-}
-```
-
-Because it's generic, TypeScript enforces that every publisher and subscriber
-agrees on the shape of `T` at compile time — you can't accidentally publish a
-`string` on a `Channel<UserEvent>`.
-
-## Defining Global Singleton Channels
-
-You declare each channel once and export it as a module-level singleton:
-
-```ts
-// channels.ts
-interface UserLoggedIn {
- userId: string;
- role: "admin" | "user";
-}
-interface CartUpdated {
- itemCount: number;
- total: number;
-}
-
-export const userLoginChannel = new Channel<UserLoggedIn>();
-export const cartChannel = new Channel<CartUpdated>();
-```
-
-Any file can import and use these without passing them through props or
-constructors.
-
-## Usage Across Components
-
-```ts
-// ComponentA.ts — subscriber
-import { cartChannel } from './channels';
-
-const unsub = cartChannel.subscribe(({ itemCount, total }) => {
- document.getElementById('cart-count')!.textContent = String(itemCount);
-});
-
-// call unsub() when component is destroyed to avoid memory leaks
-
-// ComponentB.ts — publisher
-import { cartChannel } from './channels';
-
-function addToCart(item: Item) {
- // ... update cart logic
- cartChannel.publish({ itemCount: cart.length, total: cart.reduce(...) });
-}
-```
diff --git a/content/system_design/state_management/svelte_state.md b/content/system_design/state_management/svelte_state.md
@@ -1,56 +0,0 @@
----
-title: Svelte State
----
-
-[How to share state in svelte 5](https://joyofcode.xyz/how-to-share-state-in-svelte-5)
-
-## Function
-
-```ts
-export function createCounter() {
- let count = $state(0);
- // you can also derive values
- let double = $derived(count * 2);
-
- return {
- get count() {
- return count;
- },
- set count(value) {
- count = value;
- },
- increment() {
- count++;
- },
- };
-}
-```
-
-Export it once in the same file you make it global.
-
-```ts
-export const counter = createCounter();
-```
-
-Or instantiate it in each component make it a new instance per component:
-
-```ts
-const counter = createCounter();
-```
-
-## Class
-
-```ts
-export class Counter {
- // make count private
- #count = $state(0);
-
- // create property accessors
- get count() {
- return this.#count;
- }
- set count(value) {
- this.#count = value;
- }
-}
-```
diff --git a/content/test_driven_development/arrange_act_assert.md b/content/test_driven_development/arrange_act_assert.md
@@ -1,148 +0,0 @@
----
-title: Arrange-Act-Assert
----
-
-## The Pattern
-
-Arrange-Act-Assert is a great way to structure test cases. It prescribes an
-order of operations:
-
-1. **Arrange** inputs and targets. Arrange steps should set up the test case.
- Does the test require any objects or special settings? Does it need to prep a
- database? Does it need to log into a web app? Handle all of these operations
- at the start of the test.
-
-2. **Act** on the target behavior. Act steps should cover the main thing to be
- tested. This could be calling a function or method, calling a REST API, or
- interacting with a web page. Keep actions focused on the target behavior.
-
-3. **Assert** expected outcomes. Act steps should elicit some sort of response.
- Assert steps verify the goodness or badness of that response. Sometimes,
- assertions are as simple as checking numeric or string values. Other times,
- they may require checking multiple facets of a system. Assertions will
- ultimately determine if the test passes or fails.
-
-**Behavior-Driven Development** follows the **Arrange-Act-Assert** pattern by
-another name: **Given-When-Then**. The Gherkin language uses Given-When-Then
-steps to specify behaviors in scenarios. Given-When-Then is essentially the same
-formula as Arrange-Act-Assert.
-
-### References
-
-[automationpanda](https://automationpanda.com/2020/07/07/arrange-act-assert-a-pattern-for-writing-good-tests/)
-[semaphore](https://semaphore.io/blog/aaa-pattern-test-automation)
-
-## Example
-
-Here is a simple Rust example demonstrating the Arrange-Act-Assert pattern using
-a small `ShoppingCart` struct. Rust's built-in `#[cfg(test)]` module makes the
-three phases very natural to express.
-
-The code:
-
-```rust
-// src/lib.rs
-
-pub struct ShoppingCart {
- items: Vec<(String, f64)>, // (name, price)
-}
-
-impl ShoppingCart {
- pub fn new() -> Self {
- ShoppingCart { items: vec![] }
- }
-
- pub fn add_item(&mut self, name: &str, price: f64) {
- self.items.push((name.to_string(), price));
- }
-
- pub fn total(&self) -> f64 {
- self.items.iter().map(|(_, price)| price).sum()
- }
-
- pub fn item_count(&self) -> usize {
- self.items.len()
- }
-}
-```
-
-The test:
-
-```rust
-#[cfg(test)]
-mod tests {
- use super::*;
-
- #[test]
- fn test_total_reflects_added_items() {
- // ── Arrange ──────────────────────────────────────
- let mut cart = ShoppingCart::new();
- let expected_total = 17.97;
-
- // ── Act ──────────────────────────────────────────
- cart.add_item("Apple", 0.99);
- cart.add_item("Bread", 2.49);
- cart.add_item("Laptop", 14.49);
-
- // ── Assert ───────────────────────────────────────
- assert!((cart.total() - expected_total).abs() < f64::EPSILON);
- assert_eq!(cart.item_count(), 3);
- }
-
- #[test]
- fn test_empty_cart_has_zero_total() {
- // ── Arrange ──────────────────────────────────────
- let cart = ShoppingCart::new();
-
- // ── Act ──────────────────────────────────────────
- let total = cart.total();
-
- // ── Assert ───────────────────────────────────────
- assert_eq!(total, 0.0);
- }
-}
-```
-
-Flow Diagram:
-
-```
-┌─────────────────────────────────────────────────────┐
-│ TEST FUNCTION │
-│ │
-│ ┌─────────────────────────────────────────────┐ │
-│ │ ARRANGE │ │
-│ │ • Create ShoppingCart::new() │ │
-│ │ • Define expected_total = 17.97 │ │
-│ └──────────────────┬──────────────────────────┘ │
-│ │ │
-│ ▼ │
-│ ┌─────────────────────────────────────────────┐ │
-│ │ ACT │ │
-│ │ • cart.add_item("Apple", 0.99) │ │
-│ │ • cart.add_item("Bread", 2.49) │ │
-│ │ • cart.add_item("Laptop", 14.49) │ │
-│ └──────────────────┬──────────────────────────┘ │
-│ │ │
-│ ▼ │
-│ ┌─────────────────────────────────────────────┐ │
-│ │ ASSERT │ │
-│ │ • cart.total() ≈ 17.97 ✔ / ✘ │ │
-│ │ • cart.item_count() == 3 ✔ / ✘ │ │
-│ └─────────────────────────────────────────────┘ │
-└─────────────────────────────────────────────────────┘
-```
-
-Rust-specific notes:
-
-- `#[cfg(test)]` gates the test module so it's only compiled during cargo test,
- keeping production binaries lean.
-
-- Floating-point assertions in Rust require a tolerance check
- (`abs() < EPSILON`) rather than a direct `==`, since `f64` arithmetic can
- introduce tiny rounding errors.
-
-- Each test function is independent — no shared mutable state bleeds between
- them, reinforcing the one behavior per test principle of AAA.
-
-- The **Given-When-Then** equivalent here would be: Given an empty cart, When
- three items are added, Then the total equals their sum.
diff --git a/content/test_driven_development/index.md b/content/test_driven_development/index.md
@@ -1,3 +0,0 @@
----
-title: Test Driven Development
----
diff --git a/content/test_driven_development/unit_testing.md b/content/test_driven_development/unit_testing.md
@@ -1,73 +0,0 @@
----
-title: Unit Testing
----
-
-## Writing Isolated and Focused Unit Tests
-
-Unit tests should be **isolated and focused**, testing one small, well-defined
-unit of functionality at a time. Each test should verify a single behavior
-without relying on other parts of the system. When tests are too broad or
-tightly coupled across components, they become brittle — a minor change in one
-area can cause unrelated tests to fail.
-
-To ensure reliability and maintainability, unit tests should:
-
-- Run independently of external systems or global state.
-- Use **mocks or stubs** to replace dependencies.
-- Follow the
- [**Arrange–Act–Assert**](/test_driven_development/arrange_act_assert.md)
- pattern for clarity.
-- Be **deterministic** and **fast** so they can run often during development.
-
-Writing tests this way builds confidence in each unit, simplifies debugging, and
-supports modular, testable code design.
-
----
-
-## Enabling CI/CD Test Environments with Isolated Resources
-
-For continuous integration workflows such as **GitHub Actions** or **Forgejo**,
-unit tests should run within a fully isolated and reproducible environment. This
-means the application should be capable of **spinning up a dedicated test
-PostgreSQL instance** and a **test server API** that operates on a separate port
-with its own **test-specific environment variables**.
-
-These configurations ensure that test runs do not interfere with production or
-staging databases. The test infrastructure should start up quickly and shut down
-cleanly as part of the CI/CD pipeline, allowing automated workflows to execute
-the full test suite independently for every build or pull request. This approach
-guarantees repeatable, safe testing while maintaining complete separation
-between test and production systems.
-
----
-
-### Question: What's the difference between intergation tests and unit tests?
-
-Unit tests focus on small, isolated pieces of code and run very fast, so they
-give precise, quick feedback and make it easy to iterate or refactor without
-breaking unrelated behavior. In contrast, integration tests exercise multiple
-components together (like API, database, and services) to verify real workflows,
-which provides higher confidence that the system behaves correctly as a whole.
-
-The tradeoff is that integration tests are slower, more complex to set up, and
-failures can be harder to diagnose, since a small change in one part of the
-system can break a test somewhere else. Because of this, you typically rely on
-unit tests for rapid development of specific functionality, and use integration
-tests more sparingly to ensure that the integrated application still works end
-to end.
-
----
-
-## Abstractions and Patterns to Keep Tests Stable
-
-This is where **interfaces** (traits), the facade and adapter patterns,
-dependency injection, and internal mutability come in. When a piece of code is
-wrapped behind a stable trait or adapter interface, the call sites that depend
-on it can remain unchanged, while other developers are free to change the
-internals of the wrapped functions without breaking those callers. This gives
-more confidence in integration testing when integrating with different external
-systems, call sites, or frontends, because the surface contracts stay stable
-even as implementations evolve. These abstraction layers reinforce separation of
-concerns between core functions, business logic, infrastructure, and UI/API
-layers, so each layer can be unit-tested in isolation and then composed
-predictably in integration tests.
diff --git a/content/web_development/http_headers.md b/content/web_development/http_headers.md
@@ -1,72 +0,0 @@
----
-title: HTTP Headers
----
-
-> HTTP headers let the client and the server pass additional information with a
-> message in a request or response.
-
-[HTTP Headers](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers)
-
-### Question: Where exactly is HTTP headers stored?
-
-HTTP headers are not stored in one permanent place; they live briefly in memory
-at each hop (browser, proxies, server) as part of the HTTP message being sent or
-received.
-
-#### Client side:
-
-The browser builds an HTTP request message in its process memeory, which is
-written into a TCP/IP packet buffer sent over the network (nothing is saved on
-disk so to speak). The browser may keep some header in internal data structures
-for cahcing or cookies.
-
-#### Server side:
-
-The web server process (apache2, nginx, node.js) receives the TCP stream and
-parses out the HTTP headers for your web app to use. Default server headers
-cofiguration (apache config, nginx config, .htaccess) do live on disk. Each
-actual header that goes to the client is ephermeral and generated on the fly in
-memory for each response.
-
-#### Proxy and Cache:
-
-CDNs, reverse proxies, or browser's cache store HTTP messages (both headers and
-body) in memory or disk for reuse.
-
-# HTTP Security Headers
-
-[HTTP Security Headers: A complete guide to HTTP headers](https://www.darkrelay.com/post/http-security-headers)
-
-[Analyze your headers](https://securityheaders.com/)
-
-- Access-Control-Allow-Origin Security Header
-
-- Content-Type Header
-
-- Content-Security-Policy (CSP) Security Header
-
-- Cross-Origin-Embedder-Policy Security Header
-
-- Cross-Origin-Resource-Policy Security Header
-
-- Cross-Origin-Opener-Policy Security Header
-
-- Set-Cookie Header
-
-- Strict-Transport-Security (HSTS) Security Header
-
-- Referrer-Policy Header
-
-- X-Content-Type-Options Security Header
-
-- X-Frame-Options Security Header
-
-- X-XSS-Protection Security Header
-
-- X-Permitted-Cross-Domain-Policies Security Header
-
-- Cache-Control Header
-
-- X-Powered-By Header
-
-- Public-Key-Pins(HPKP) header
diff --git a/content/web_development/index.md b/content/web_development/index.md
@@ -1,3 +0,0 @@
----
-title: Web Development
----
diff --git a/content/web_development/openapi-guide.md b/content/web_development/openapi-guide.md
@@ -1,265 +0,0 @@
----
-title: Open API guide
----
-
-# Bridging the Type Gap: End-to-End Type Safety with Axum, Aide, and Orval
-
-One of the most frustrating experiences in full-stack development is the "silent
-failure": you change a field name in your Rust backend, everything compiles
-perfectly, but your frontend suddenly starts receiving `undefined` and crashing
-in production.
-
-To solve this, we need a **Single Source of Truth**. Instead of manually
-maintaining TypeScript interfaces that mirror your Rust structs, you can
-automate the entire pipeline:
-
-**Rust Models $\rightarrow$ OpenAPI Spec $\rightarrow$ TypeScript Axios Client**
-
-In this guide, we'll use **`aide`** to generate an OpenAPI specification
-directly from our Axum types and **`orval`** to transform that spec into a
-type-safe Axios client.
-
----
-
-## Part 1: The Backend (Rust + Axum + Aide)
-
-Traditionally, writing an OpenAPI (Swagger) spec means writing a giant YAML file
-by hand. **`aide`** eliminates this by leveraging Rust's type system to generate
-the spec at runtime.
-
-### 1. Dependencies
-
-Add these to your `Cargo.toml`:
-
-```toml
-[dependencies]
-axum = "0.7" # Ensure version compatibility with aide
-aide = { version = "0.15", features = ["axum"] }
-schemars = "0.8" # Required for generating JSON schemas from structs
-serde = { version = "1.0", features = ["derive"] }
-serde_json = "1.0"
-tokio = { version = "1.0", features = ["full"] }
-```
-
-### 2. The Implementation
-
-The core idea is to replace Axum's standard `Router` with `ApiRouter` and ensure
-your data models implement `JsonSchema`.
-
-```rust
-use aide::{
- axum::{routing::{get, post}, ApiRouter, IntoApiResponse},
- openapi::{Info, OpenApi},
-};
-use axum::{Extension, Json};
-use schemars::JsonSchema;
-use serde::{Deserialize, Serialize};
-
-// 1. All models must derive JsonSchema to be visible in the OpenAPI spec
-#[derive(Deserialize, Serialize, JsonSchema)]
-struct User {
- id: u64,
- username: String,
- email: String,
-}
-
-// 2. Handlers must return types that implement IntoApiResponse
-async fn create_user(Json(user): Json<User>) -> impl IntoApiResponse {
- // In a real app, you'd save to DB here
- Json(user)
-}
-
-async fn serve_api(Extension(api): Extension<OpenApi>) -> impl IntoApiResponse {
- Json(api)
-}
-
-#[tokio::main]
-async fn main() {
- // Define the OpenApi metadata
- let mut api = OpenApi {
- info: Info {
- title: "My Awesome API".to_string(),
- version: "1.0.0".to_string(),
- ..Info::default()
- },
- ..OpenApi::default()
- };
-
- // Use ApiRouter instead of Router
- let app = ApiRouter::new()
- // Use api_route to explicitly include the route in the documentation
- .api_route("/users", post(create_user))
- // Standard route for the JSON spec itself
- .route("/api.json", get(serve_api));
-
- let listener = tokio::net::TcpListener::bind("0.0.0.0:3000").await.unwrap();
-
- axum::serve(
- listener,
- app
- .finish_api(&mut api) // Finalize the spec generation
- .layer(Extension(api)) // Inject the spec into the handlers
- .into_make_service(),
- )
- .await
- .unwrap();
-}
-```
-
-### How it works:
-
-- **`JsonSchema`**: The `schemars` crate analyzes your Rust struct at
- compile-time and allows `aide` to describe it in JSON format.
-- **`ApiRouter`**: This acts as a wrapper. Whenever you call `.api_route()`,
- `aide` records the types of the request and response.
-- **`/api.json`**: We expose the generated `OpenApi` struct as a JSON endpoint.
- This is the bridge to our frontend.
-
----
-
-## Part 2: The Frontend (TypeScript + Orval)
-
-Now that our backend is shouting its structure via `/api.json`, we use **Orval**
-to listen. Orval reads the OpenAPI spec and generates an entire API
-client—including request functions and TypeScript types—so you never have to
-write a `fetch` or `axios` call manually.
-
-### 1. Install Dependencies
-
-```bash
-npm install axios
-npm install -D orval
-```
-
-### 2. Configure Orval
-
-Create an `orval.config.js` file in your project root:
-
-```javascript
-module.exports = {
- "my-api": {
- input: "http://localhost:3000/api.json", // The URL from our Rust app
- output: {
- target: "./src/api/generated.ts",
- client: "axios",
- override: {
- axios: {
- useBaseUrl: true,
- baseURL: "http://localhost:3000",
- },
- },
- },
- },
-};
-```
-
-### 3. Generate the Client
-
-Run the generator:
-
-```bash
-npx orval
-```
-
-Orval will now create `src/api/generated.ts`. Inside, you'll find:
-
-1. **TypeScript Interfaces**: Exactly matching the Rust `User` struct.
-2. **Request Functions**: A function like `createUser` that takes a `User`
- object and returns a Promise of a `User`.
-
-### 4. Use it in your Frontend (Svelte/React/Vue)
-
-Now, your API calls are fully type-safe:
-
-```typescript
-import { createUser } from "./api/generated";
-
-async function handleSignUp(formData: any) {
- try {
- // TypeScript will error here if formData doesn't match the Rust User struct!
- const user = await createUser({
- id: 1,
- username: "rust_lover",
- email: "hello@rust.rs",
- });
- console.log("User created:", user.username);
- } catch (e) {
- console.error("API Error", e);
- }
-}
-```
-
----
-
-## Part 3: Using CI to Keep the Spec Up to Date
-
-To make this architecture truly production-ready, you shouldn't rely on manually
-running `npx orval` every time you change a field in Rust. Instead, you can
-integrate this into your **CI/CD pipeline**.
-
-### The "Golden File" Strategy
-
-Since `aide` generates the specification at runtime, the most robust way to
-handle CI is to treat the OpenAPI JSON as a versioned artifact (a "Golden
-File").
-
-#### 1. Export the Spec during Backend CI
-
-Create a Rust test in your backend that runs the `ApiRouter`, generates the
-`OpenApi` object, and writes it to a file:
-
-```rust
-#[tokio::test]
-async fn export_openapi_spec() {
- let mut api = OpenApi::default();
- let app = ApiRouter::new().api_route("/users", post(create_user));
-
- app.finish_api(&mut api);
-
- let json = serde_json::to_string_pretty(&api).unwrap();
- std::fs::write("openapi.json", json).expect("Unable to write spec file");
-}
-```
-
-#### 2. The CI Pipeline (GitHub Actions Example)
-
-Set up a workflow that triggers the frontend whenever the `openapi.json`
-changes:
-
-```yaml
-name: API Type Sync
-on:
- push:
- paths:
- - "backend/**"
-
-jobs:
- sync-types:
- runs-on: ubuntu-latest
- steps:
- - uses: actions/checkout@v4
-
- - name: Export OpenAPI Spec
- run: |
- cargo test --test export_openapi_spec
-
- - name: Generate Frontend Client
- run: |
- cd frontend
- npm install
- npx orval
-
- - name: Type Check Frontend
- run: |
- cd frontend
- npm run type-check # Run 'tsc' to see if API changes broke the UI
-```
-
-### Summary of the Automated Flow
-
-1. **Developer** pushes a change to a Rust struct.
-2. **CI** runs a test to generate the latest `openapi.json`.
-3. **CI** runs Orval to update the TypeScript Axios client.
-4. **CI** runs the TypeScript compiler (`tsc`). If the Rust change broke a
- frontend component, the **build fails** before the code ever reaches
- production.
diff --git a/content/web_development/svelte.md b/content/web_development/svelte.md
@@ -1,30 +0,0 @@
----
-title: Svelte
----
-
-## Attachment factories
-
-A useful pattern is for a function, such as tooltip in this example, to return
-an attachment:
-
-```ts
-<script lang="ts">
- import tippy from 'tippy.js';
- import type { Attachment } from 'svelte/attachments';
-
- let content = $state('Hello!');
-
- function tooltip(content: string): Attachment {
- return (element) => {
- const tooltip = tippy(element, { content });
- return tooltip.destroy;
- };
- }
-</script>
-
-<input bind:value={content} />
-
-<button {@attach tooltip(content)}>
- Hover me
-</button>
-```
diff --git a/content/web_development/svelte_state.md b/content/web_development/svelte_state.md
@@ -1,126 +0,0 @@
----
-title: Svelte State Management
----
-
-## Svelte 4 Writable Store
-
-```ts
-// fooStore.ts
-import { type Writable, writable } from "svelte/store";
-
-function createFoo() {
- const { subscribe, set, update }: Writable<number> = writable(0);
- return {
- subscribe,
- set: (value: number) => set(value),
- add: (value: number) => update((x: number) => x + value),
- triple: () => update((x: number) => x * 3),
- clear: () => set(0),
- };
-}
-export const foo = createFoo();
-```
-
-Derived store:
-
-```ts
-// fooStore.ts
-import { derived, writable } from "svelte/store";
-
-export const bar = writable(1);
-export const baz = writable(2);
-export const foo = derived(
- [bar, baz],
- ([$bar, $baz]) => Math.min($bar, $baz) / 2,
-);
-```
-
-## Svelte 5 Rune Store
-
-```ts
-// foo.svelte.ts (must be placed inside a *.svelte.ts file)
-function createFoo(initialCount = 0) {
- let foo = $state(initialCount);
-
- return {
- get foo() {
- return foo;
- },
- set foo(value) {
- foo = value;
- },
- add(value) {
- foo += value;
- },
- triple() {
- foo *= 3;
- },
- };
-}
-export const foo = createFoo();
-```
-
-Or class if you prefer:
-
-```ts
-// foo.svelte.ts
-export class Foo {
- foo = $state(0);
-
- constructor(initialCount = 0) {
- this.foo = initialCount;
- }
-
- add(value) {
- this.foo += value;
- }
-
- triple() {
- this.foo *= 3;
- }
-}
-
-export const foo = new Foo();
-```
-
-Class doesn't need get/set method, just read/write it directly:
-
-```ts
-// In a component:
-// Reading:
-console.log(foo.foo);
-// Writing (triggers UI updates automatically):
-foo.foo = 10;
-```
-
-Derived rune:
-
-```ts
-// foo.svelte.ts (must be placed inside a *.svelte.ts file)
-function createFoo(initialBar = 1, initialBaz = 2) {
- let bar = $state(initialBar);
- let baz = $state(initialBaz);
- let foo = $derived(Math.min(bar, baz) / 2);
-
- return {
- get bar() {
- return bar;
- },
- set bar(value) {
- bar = value;
- },
- get baz() {
- return baz;
- },
- set baz(value) {
- baz = value;
- },
- get foo() {
- return foo;
- },
- };
-}
-export const foo = createFoo();
-```
-
-[Different Ways To Share State In Svelte 5](https://joyofcode.xyz/how-to-share-state-in-svelte-5)
diff --git a/content/web_development/typescript.md b/content/web_development/typescript.md
@@ -1,56 +0,0 @@
----
-title: TypeScript
----
-
-Remember, the whole point of using TypeScript is to use its typechecker to stop
-you from doing invalid things.
-
-TypeScript gives you error messages in your text editor, as you type.
-
-But we should use type annotations only when necessary, and let TypeScript work
-its inference magic for us whenever possible.
-
-## Avoid using `any` as type
-
-`any` makes your value behave like it would in regular JavaScript, and totally
-prevents the typechecker from working its magic. When you allow `any` into your
-code you're flying blind. Avoid `any` like fire, and use it only as a very very
-last resort.
-
-## `public` keyword in class constructor
-
-```ts
-class Person {
- constructor(public firstname: string);
-}
-```
-
-`public` is shorthand for `this.firstName = firstName`
-
-## Index signatures
-
-```ts
-let a: {
- b: number;
- c?: string;
- [key: number]: boolean;
-};
-```
-
-The `[key: T]: U` syntax is called an index signature, and this is the way you
-tell TypeScript that the given object might contain more keys.
-
-```ts
-a = { b: 1, c: "d", 10: true, 20: false };
-```
-
-For this object, all keys of type T must have values of type U.
-
-## Type Alias
-
-Type aliases are useful for DRYing up repreated complex types.
-
-## Arrays
-
-TypeScript supports two syntaxes for arrays: `T[]` and `Array<T>`. They are
-indentical both in meaning and in performance.
diff --git a/content/web_development/typescript_books.md b/content/web_development/typescript_books.md
@@ -1,91 +0,0 @@
----
-title: TypeScript Books
----
-
-TypeScript has become the industry standard for scalable JavaScript development.
-Because the ecosystem is vast, choosing the right learning path depends heavily
-on your current experience level—whether you are a seasoned JavaScript engineer,
-a total beginner to programming, or someone looking for a deep dive into the
-type system.
-
-## The Gold Standard
-
-### Effective TypeScript
-
-**Author:** Dan Vanderkam\
-Widely regarded as the best overall resource for those wanting to write
-"idiomatic" TypeScript. Rather than a linear tutorial, it provides 62 specific
-ways to improve your code. It is particularly strong on type inference and the
-nuances of the type system.
-
-- **Best For:** Developers who want a deep, professional understanding of the
- type system.
-- **Level:** Beginner to Advanced.
-
-## Path-Specific Recommendations
-
-### For JavaScript Developers: Programming TypeScript
-
-**Author:** Boris Cherny\
-If you are already proficient in JavaScript, this book focuses on the
-transition. It takes a hands-on approach to scaling applications, covering error
-handling, asynchronous patterns, and the practicalities of migrating existing JS
-projects to TS.
-
-- **Best For:** JS veterans moving to TypeScript.
-- **Level:** Intermediate.
-
-### For Absolute Beginners: Mastering TypeScript
-
-Unlike most TS resources, this book does not assume prior JavaScript knowledge.
-It guides the reader from the very basics upward, eventually integrating the
-language with popular frameworks like React, Vue, and Angular.
-
-- **Best For:** Those new to the web development ecosystem.
-- **Level:** Beginner.
-
-## Community & Fast-Track Resources
-
-### TypeScript Deep Dive (Free)
-
-A community favorite available on GitHub and via GitBook. It is often cited as
-the "definitive guide" due to its comprehensiveness, covering generics,
-decorators, and the inner workings of the TypeScript compiler.
-
-- **Resource:** [TypeScript Deep Dive](https://basarat.gitbook.io/typescript/)
-
-### Quick & Structured Learning
-
-- **TypeScript in 50 Lessons**: Ideal for those who prefer bite-sized,
- digestible content. It breaks the language down into 50 short lessons.
-- **TypeScript Quickly**: A project-oriented book that guides you through
- building a blockchain service application.
-
----
-
-## Summary Comparison
-
-| Book/Resource | Primary Focus | Target Audience | Level |
-| :------------------------- | :----------------------- | :----------------- | :---------------------------------- |
-| **Effective TypeScript** | Type system optimization | Professional Devs | Beginner $\rightarrow$ Advanced |
-| **Programming TypeScript** | Scaling JS applications | JS Developers | Intermediate |
-| **Mastering TypeScript** | Foundational concepts | Absolute Beginners | Beginner |
-| **TS Deep Dive** | Comprehensive reference | All levels | All levels |
-| **TS in 50 Lessons** | Short, focused bursts | Quick learners | Beginner $\rightarrow$ Intermediate |
-| **TypeScript Quickly** | Project-based learning | Hands-on learners | Intermediate |
-
----
-
-## Official Resources
-
-For the most up-to-date specifications and documentation, always refer to the
-official TypeScript project:
-
-- **[Official Documentation](https://www.typescriptlang.org/docs/)**: The
- primary source for all language features.
-- **[The TypeScript Handbook](https://www.typescriptlang.org/docs/handbook/intro.html)**:
- The definitive guide to the language's core concepts.
-- **[TypeScript Playground](https://www.typescriptlang.org/play)**: An
- interactive environment to test type logic in real-time.
-- **[TS Reference](https://www.typescriptlang.org/docs/handbook/tsconfig-json.html)**:
- Essential guide for `tsconfig.json` configuration.
diff --git a/content/whiteboards/index.md b/content/whiteboards/index.md
@@ -1,3 +0,0 @@
----
-title: Whiteboards
----
diff --git a/content/z-md/agent.md b/content/z-md/agent.md
@@ -1,180 +0,0 @@
----
-title: AGENTS.md
----
-
-# Agents.md: Senior Rust Developer with Svelte/TypeScript/JavaScript Expertise
-
-## **Role Overview**
-
-As a **Senior Rust Developer with Svelte, TypeScript, and JavaScript
-expertise**, you are responsible for:
-
-- Writing **high-performance, memory-safe, and concurrent** Rust code.
-- Developing **type-safe, scalable, and maintainable** applications using
- **Svelte, vanilla TypeScript, and vanilla JavaScript**.
-- Ensuring **clean, efficient, and lean** code across the stack.
-- Mentoring junior developers and leading code reviews.
-- Collaborating with cross-functional teams to design and implement robust
- solutions.
-
----
-
-## **Core Principles**
-
-### **1. Clean Code**
-
-- **Readability**: Write self-documenting code with meaningful names for
- variables, functions, and modules.
-- **Consistency**: Follow the project’s style guide (e.g., `rustfmt`, `prettier`
- for TypeScript/JavaScript).
-- **Simplicity**: Prefer simple solutions over clever ones. Avoid unnecessary
- abstraction.
-
-### **2. Maintainability**
-
-- **Modularity**: Break down code into small, reusable modules/functions.
-- **Documentation**: Document public APIs, complex logic, and edge cases.
-- **Tests**: Write unit, integration, and end-to-end tests. Aim for high test
- coverage.
-
-### **3. Efficiency**
-
-- **Performance**: Optimize critical paths in Rust (e.g., zero-cost
- abstractions, minimal allocations).
-- **Memory Safety**: Leverage Rust’s ownership model to prevent data races and
- memory leaks.
-- **Type Safety**: Use TypeScript’s type system to catch errors at compile time.
-
-### **4. Lean Code**
-
-- **Minimal Dependencies**: Avoid bloating the project with unnecessary crates
- or npm packages.
-- **DRY Principle**: Don’t repeat yourself. Reuse logic where possible.
-- **YAGNI**: Avoid over-engineering. Only implement what is needed now.
-
----
-
-## **Rust Best Practices**
-
-### **Code Structure**
-
-- Use `cargo` workspaces for large projects.
-- Organize code into logical modules (e.g., `src/lib.rs`, `src/bin/`).
-- Prefer `Result` and `Option` over panics for error handling.
-
-### **Concurrency**
-
-- Use `async/await` with `tokio` or `async-std` for async tasks.
-- Leverage `Arc<Mutex<T>>` or `RwLock` for shared state.
-- Avoid locks where possible; prefer message-passing (e.g., `std::sync::mpsc`).
-
-### **Performance**
-
-- Profile with `cargo bench` or `flamegraph`.
-- Use `#[inline]` for hot paths.
-- Avoid unnecessary clones; prefer references (`&T`).
-
-### **Testing**
-
-- Use `cargo test` for unit and integration tests.
-- Test edge cases (e.g., empty inputs, large datasets).
-- Mock external dependencies (e.g., `mockall`).
-
----
-
-## **Svelte/TypeScript/JavaScript Best Practices**
-
-### **Code Structure**
-
-- Use `eslint` and `prettier` for consistent formatting.
-- Organize code into features/modules (e.g., `src/features/auth/`).
-- Prefer functional programming patterns (e.g., `map`, `filter`, `reduce`).
-
-### **Svelte**
-
-- Use Svelte’s reactive statements (`$:`) for state management.
-- Avoid unnecessary stores; prefer local state where possible.
-- Use Svelte’s built-in transitions and animations for UI enhancements.
-
-### **TypeScript**
-
-- Use `interface` or `type` for complex objects.
-- Avoid `any`; use `unknown` for dynamic data.
-- Leverage generics for reusable components.
-
-### **Vanilla JavaScript**
-
-- Use modern ES6+ features (e.g., `const`, `let`, arrow functions).
-- Prefer modules (`import/export`) over global scope.
-- Avoid side effects; write pure functions where possible.
-
-### **Performance**
-
-- Avoid heavy computations in the main thread.
-- Use `Web Workers` for CPU-intensive tasks.
-- Optimize DOM updates; batch changes where possible.
-
-### **Testing**
-
-- Use `jest` or `vitest` for unit tests.
-- Test Svelte components with `@testing-library/svelte`.
-- Mock API calls (e.g., `msw`).
-
----
-
-## **Collaboration**
-
-### **Code Reviews**
-
-- Be constructive and specific in feedback.
-- Focus on **correctness**, **readability**, and **performance**.
-- Use GitHub/GitLab review tools for inline comments.
-
-### **Documentation**
-
-- Update `README.md` for project setup and usage.
-- Document public APIs with `///` (Rust) or `JSDoc` (TypeScript/JavaScript).
-- Use `cargo doc` or `TypeDoc` for auto-generated docs.
-
-### **Tools**
-
-- **Rust**: `cargo`, `clippy`, `rust-analyzer`.
-- **TypeScript/JavaScript**: `npm`, `yarn`, `eslint`, `prettier`.
-- **Svelte**: `svelte-check`, `vite`, `svelte-kit`.
-- **CI/CD**: GitHub Actions, GitLab CI.
-
----
-
-## **Example: Rust + Svelte/TypeScript Project Structure**
-
-```
-project/
-├── rust/
-│ ├── Cargo.toml
-│ ├── src/
-│ │ ├── lib.rs
-│ │ ├── main.rs
-│ │ └── models.rs
-│ └── tests/
-├── frontend/
-│ ├── package.json
-│ ├── src/
-│ │ ├── App.svelte
-│ │ ├── lib/
-│ │ │ ├── components/
-│ │ │ ├── stores/
-│ │ │ └── utils/
-│ │ └── main.ts
-│ ├── static/
-│ └── tests/
-└── README.md
-```
-
----
-
-## **Final Notes**
-
-- **Stay Updated**: Follow Rust, Svelte, TypeScript, and JavaScript RFCs and
- best practices.
-- **Share Knowledge**: Write blog posts or give talks on lessons learned.
-- **Feedback Loop**: Continuously improve processes and tools.
diff --git a/engineering_practice/index.md b/engineering_practice/index.md
@@ -0,0 +1,2 @@
+# Engineering Practice
+
diff --git a/engineering_practice/software-entropy.md b/engineering_practice/software-entropy.md
@@ -0,0 +1,217 @@
+# Software Entropy
+
+# Software Entropy — Why Codebases Rot and How Constant Tidying Keeps Them Alive
+
+There is a law that every engineer eventually learns, not from a textbook but
+from the gut-sinking moment they open a file they haven't touched in six months.
+The code is worse than they left it. Not because anyone vandalized it
+deliberately, but because software, left unattended, tends toward disorder. This
+is software entropy — and it is not a metaphor. It is the dominant force shaping
+the long-term economics of any engineering project.
+
+## What Is Software Entropy?
+
+Entropy, borrowed from thermodynamics, describes the natural tendency of closed
+systems toward disorder. In software, every new feature, every rushed hotfix,
+every "I'll clean this up later" decision adds a small quantum of disorder to
+the codebase. Individually, these moments are invisible. Collectively, they
+compound into a system that nobody fully understands anymore.
+
+Kent Beck, in _Tidy First?_, frames the economic reality with stark clarity:
+**the cost of software is approximately equal to the cost of changing it**. He
+calls this _Constantine's Equivalence_ — named after Larry Constantine, who,
+with Ed Yourdon, wrote _Structured Design_ (1975), the foundational text on
+coupling and cohesion. Beck's formulation is:
+
+> cost(software) ≈ cost(change) ≈ cost(big changes) ≈ coupling
+
+Coupling — the degree to which changing one element forces changes in others —
+is the engine of entropy. When code is tangled, a superficially simple change
+detonates a cascade: change this, and you have to change that, and that, and
+that. Over time the cost distribution follows a power law: the handful of most
+expensive changes, driven by accumulated coupling, dwarf everything else
+combined.
+
+## The Productivity Death Spiral
+
+Software entropy does not kill a project all at once. It kills it slowly, via
+cognitive creep.
+
+Early in a project, a single engineer can hold the entire system in their head.
+They move fast. Then the system grows. New abstractions are introduced
+inconsistently. Dead code accumulates. Nested conditionals deepen. Functions
+sprawl across hundreds of lines. Variables are declared far from where they are
+used. Names drift from their meaning.
+
+Each of these issues adds a small tax to every future change. Reading time
+increases. Onboarding new engineers takes longer. Pull requests grow larger
+because each change requires touching more files. Eventually, the team is
+spending more energy on archaeology — _what does this code even do?_ — than on
+delivery. Productivity trends toward zero. Not a sudden collapse, but a slow,
+grinding asymptote.
+
+This is the scenario Beck is writing against in _Tidy First?_: a codebase that
+has become "messy" in the technical sense — not aesthetically displeasing, but
+genuinely resistant to change.
+
+## Structure Versus Behavior: The Core Distinction
+
+One of the most important conceptual contributions in _Tidy First?_ is the sharp
+separation between **structure changes** and **behavior changes**.
+
+- A **behavior change** alters what the system does: a new feature, a bug fix, a
+ performance optimization.
+- A **structure change** alters how the system is organized without changing
+ what it computes.
+
+Mixing these two kinds of changes in a single commit is a primary driver of
+entropy. When a reviewer sees a pull request that simultaneously refactors a
+module _and_ adds a feature, they cannot easily reason about either change. The
+refactoring masks potential bugs in the feature; the feature masks the intent of
+the refactoring. Beck's prescription is explicit: **separate tidying commits
+from behavior change commits**. Keep them in their own pull requests, as small
+as possible.
+
+This separation also unlocks a key economic property: structure changes are
+almost always **reversible**. You extract a helper function and don't like it?
+Inline it back. It's as if it never existed. Behavior changes can be
+irreversible — you can't un-send 100,000 tax notices with the wrong figure on
+them. Treating these two categories of change with the same process and the same
+review overhead is, as Beck puts it, "a waste."
+
+## Tidyings: Janitorial Work as Engineering Practice
+
+Beck introduces the concept of a **tidying** — a "cute, fuzzy little refactoring
+that nobody could possibly hate on." Tidyings are small, safe, structure-only
+changes that reduce local disorder. They include:
+
+- **Guard Clauses** — Replacing deeply nested conditionals with early returns,
+ making preconditions explicit and reducing the mental indentation a reader
+ must maintain.
+- **Dead Code Removal** — Deleting code that is never executed. Every line of
+ code is a line someone has to read. Dead code is pure cognitive tax with zero
+ return.
+- **Explaining Variables and Constants** — Extracting a complex sub-expression
+ into a named variable, or replacing a magic number with a symbolic constant.
+ This puts hard-won understanding _back into the code_, so the next reader
+ doesn't have to rediscover it.
+- **Normalize Symmetries** — When the same pattern is implemented multiple
+ different ways across a codebase, pick one, and convert the others. Readers
+ expect that difference means difference; incidental variation destroys that
+ expectation.
+- **Cohesion Order** — Moving coupled elements next to each other. If you have
+ to change three things every time you touch a feature, move those three things
+ adjacent before you touch them.
+- **Extract Helper** — Pulling a block of code with a clear, limited purpose
+ into a named routine. The name is the tidying; it replaces an implicit "what"
+ with an explicit one.
+- **Chunk Statements** — Inserting a blank line between logically distinct
+ blocks of code. This may be the simplest tidying in existence. It is also
+ surprisingly powerful: it visually signals structure that was previously
+ invisible.
+
+None of these are dramatic interventions. Each can be done in minutes. Each
+makes the next change slightly easier. And because software design enables more
+software design, these small improvements compound — what Beck calls the
+**avalanche effect**: "you tidy this bit, and that bit, and then the tidyings
+start to compound... suddenly, without you ever straining, a giant
+simplification becomes the matter of a stroke or two of your pen."
+
+## Cognitive Creep and the Cost of Coupling
+
+Coupling is the mechanism by which entropy becomes expensive. Ed Yourdon and
+Larry Constantine, studying programs in the 1970s, observed that expensive
+programs shared a common property: changing one element required changing
+others. Cheap programs required localized changes. This observation, formalized
+as **coupling**, remains the most predictive metric of software maintainability
+fifty years later.
+
+Beck extends this with _Constantine's Equivalence_: if the cost of software
+equals the cost of change, and the cost of change is dominated by the cost of
+big cascading changes, and cascading changes are caused by coupling, then:
+
+> **reducing coupling is the primary lever for reducing the long-term cost of a
+> software system**
+
+Coupling is also the mechanism of cognitive creep. To understand a piece of
+code, a reader must also understand everything it depends on. High coupling
+means a large, sprawling context that must be loaded into working memory before
+any change can be safely made. Low coupling means a small, bounded context — the
+cognitive load stays manageable.
+
+Cohesion is coupling's companion. A cohesive module contains elements that
+change together. Coupled elements that live far apart in the codebase (different
+files, different directories, different repositories) force engineers to scatter
+their attention. Moving coupled things together — increasing cohesion — is often
+sufficient to make a change tractable even before the coupling itself is
+resolved.
+
+## The Economics of Tidying: When to Tidy First
+
+Beck is not dogmatic. The title of the book ends in a question mark for a
+reason. Tidying is not always the right first move.
+
+The time value of money pushes toward tidying _after_: earn revenue from
+behavior changes sooner, spend money on structure later. Options theory pushes
+toward tidying _first_: a cleaner structure creates more optionality — a larger
+portfolio of behaviors that can be implemented next, each one cheaper because
+the structure supports it. The correct answer is contingent.
+
+The heuristic Beck offers is straightforward:
+
+> If
+> `cost(tidying) + cost(behavior change after tidying) < cost(behavior change without tidying)`,
+> tidy first. Always.
+
+At the scale of minutes to hours — the scale of individual tidyings — this
+calculation is rarely precise but always directionally useful. The deeper
+practice is developing the _taste_ to make the call quickly and correctly,
+preparing for the larger structural decisions that govern weeks and months of
+development.
+
+## The Rhythm of Housekeeping
+
+The failure mode Beck identifies in teams that have learned about tidying is the
+tidying binge: discovering that you can make your work better by cleaning, and
+then disappearing into a refactoring spiral that delays the features others are
+waiting for. "Coupling conducts one tidying to the next," he writes. "Tidyings
+are the Pringles of software design. When you're tidying first, resist the urge
+to eat the next one."
+
+The sustainable practice is **rhythm**: small, constant, incremental
+housekeeping, woven into daily development. Never a big deal. Never reported,
+tracked, planned, or scheduled as a separate initiative. Just the Scout Rule,
+applied continuously: leave the code slightly better than you found it.
+
+This rhythm is what prevents entropy from compounding. It does not eliminate
+disorder — disorder is the natural state — but it continuously reverses small
+accumulations before they become structural debt. A codebase maintained this way
+never becomes the kind of mess that requires a multi-quarter "refactoring
+initiative," the kind that is always de-prioritized in favor of features and
+therefore never happens.
+
+## The Human Dimension
+
+There is a deeper argument in _Tidy First?_ that goes beyond economics. Beck
+opens with the framing that **software design is an exercise in human
+relationships**: between the programmer and themselves, between teammates,
+between engineers and the business. The state of a codebase reflects and shapes
+those relationships.
+
+A messy codebase is demoralizing. It signals that nobody cares, which makes the
+next person care a little less, which makes it messier still. A codebase that is
+tidied continuously signals the opposite: that the people working here respect
+each other's time and attention. That respect compounds just as surely as the
+entropy does — but in the other direction.
+
+Tidying is, in this sense, professional self-care. "You can't be your best self
+if you're always rushing, if you're always changing code that's painful to
+change." The discipline of small, continuous housekeeping is not bureaucratic
+overhead. It is the engineering practice that keeps the codebase — and the team
+working in it — alive.
+
+---
+
+_References: Kent Beck, Tidy First? A Personal Exercise in Empirical Software
+Design (O'Reilly, 2023). Ed Yourdon and Larry Constantine, Structured Design
+(Prentice Hall, 1979)._
diff --git a/engineering_practice/spotify_model.md b/engineering_practice/spotify_model.md
@@ -0,0 +1,16 @@
+# The Spotify Model
+
+## Engineering Squads
+
+[Spotify Model](https://www.youtube.com/watch?v=Yvfz4HGtoPc)
+
+- A squad has autonomy of what to build
+- Squad mission
+- Product strategy
+- Short-term goals
+- Localize decision within the squad
+- Be aligned with other squads regarding overall mission
+
+Be autonomous but don't suboptimize!
+
+Loosely coupled, but tightly aligned squads.
diff --git a/content/food/bulletproof-coffee-keto.md b/food/bulletproof-coffee-keto.md
diff --git a/content/food/cardamom-coffee-and-tea.md b/food/cardamom-coffee-and-tea.md
diff --git a/food/chinese-tomato-chicken-soup.md b/food/chinese-tomato-chicken-soup.md
@@ -0,0 +1,33 @@
+# Chinese Tomato Chicken Soup
+
+A classic, light and tangy Chinese tomato chicken soup ready in ~30 minutes.
+
+## Ingredients (serves 4)
+
+- 2 chicken breasts or thighs, thinly sliced
+- 3 medium ripe tomatoes, cut into wedges
+- 4 cups (1 litre) chicken broth
+- 2 cloves garlic, minced
+- 3 slices fresh ginger
+- 2 spring onions, cut into 5cm pieces
+- 1 tsp light soy sauce
+- ½ tsp sesame oil
+- Salt and white pepper to taste
+- 1 tbsp cornstarch + 2 tbsp cold water (optional, to thicken)
+- 2–3 eggs, beaten (optional, for egg ribbons)
+
+## Instructions
+
+1. Toss chicken with a pinch of salt and cornstarch. Marinate 10 mins.
+2. Bring broth to a boil with ginger and spring onions.
+3. Add tomatoes, simmer 10 mins until softened.
+4. Add chicken, cook 5–8 mins until done.
+5. Slowly drizzle in beaten eggs while stirring for silky egg ribbons.
+6. Season with soy sauce, sesame oil, white pepper, and salt. Serve with rice.
+
+## Tips
+
+- Use white pepper for authentic Chinese flavour.
+- Ripe tomatoes create the best natural sweet-sour broth.
+- Add a splash of rice vinegar to brighten if tomatoes aren't tart.
+- Add noodles or bok choy in the last 5 minutes for a heartier meal.
diff --git a/food/cinnamon-ginger-tea.md b/food/cinnamon-ginger-tea.md
@@ -0,0 +1,85 @@
+# Cinnamon & Ginger Tea
+
+Cinnamon and ginger tea is one of those timeless home remedies that has been
+used across cultures for centuries. Beyond its warming, spiced flavour, both
+ingredients carry well-documented anti-inflammatory and antimicrobial properties
+that make this tea a reliable ally when you're fighting a cold.
+
+---
+
+## Ingredients
+
+- 2 cups (500ml) water
+- 1 cinnamon stick (or ½ tsp ground cinnamon)
+- 1-inch (2–3cm) piece of fresh ginger, peeled and sliced (or ½ tsp ground
+ ginger)
+- 1 tbsp honey _(optional, but recommended — adds antibacterial properties)_
+- A squeeze of fresh lemon _(optional)_
+
+---
+
+## Instructions
+
+1. **Bring water to a boil** in a small saucepan.
+2. **Add the ginger slices and cinnamon stick** to the boiling water.
+3. **Reduce heat and simmer** for 10–15 minutes to let the flavours and
+ compounds steep fully.
+4. **Strain** the tea into a mug.
+5. **Stir in honey and lemon** if using, once the tea has cooled slightly (very
+ hot liquid can degrade honey's beneficial enzymes).
+6. **Drink warm** — ideally 2–3 times a day when you're feeling under the
+ weather.
+
+> **Tip:** For a stronger brew, use fresh ginger rather than ground, and simmer
+> for the full 15 minutes.
+
+---
+
+## How It Helps With a Cold
+
+### Ginger
+
+Ginger contains **gingerols** and **shogaols** — bioactive compounds with potent
+anti-inflammatory and antioxidant effects. These help:
+
+- Soothe a sore throat by reducing local inflammation
+- Act as a natural decongestant, loosening mucus and easing nasal congestion
+- Calm nausea, which can accompany some viral illnesses
+- Provide a mild antiviral effect, potentially inhibiting viral replication
+
+### Cinnamon
+
+Cinnamon is rich in **cinnamaldehyde**, the compound responsible for its
+distinctive smell and much of its medicinal value. It helps by:
+
+- Offering antibacterial and antifungal properties, supporting the immune system
+- Reducing inflammation in the throat and airways
+- Warming the body from within, which may help ease chills and improve
+ circulation
+
+### Honey & Lemon (Bonus)
+
+- **Honey** is a well-known demulcent — it coats the throat, reduces coughing,
+ and has antimicrobial properties. A 2021 systematic review found honey
+ superior to usual care (including antihistamines and cough suppressants) for
+ upper respiratory tract symptoms.
+- **Lemon** provides a dose of Vitamin C and acts as a mild astringent, helping
+ to clear mucus.
+
+---
+
+## When to Drink It
+
+| Timing | Benefit |
+| -------------------------- | -------------------------------------- |
+| First thing in the morning | Kickstarts immunity and warms the body |
+| Before bed | Soothes a sore throat overnight |
+| After meals | Aids digestion alongside cold symptoms |
+
+---
+
+## A Note of Caution
+
+This tea is a **complementary remedy**, not a replacement for medical treatment.
+If symptoms are severe, persist beyond 10 days, or are accompanied by high fever
+or difficulty breathing, consult a healthcare professional.
diff --git a/food/coffee_and_cacao_comparison.md b/food/coffee_and_cacao_comparison.md
@@ -0,0 +1,84 @@
+# Coffee, Matcha, and Cacao Comparison
+
+## Average caffeine content per typical serving
+
+| Beverage | Typical serving size | Caffeine content (≈) | Notes |
+| ------------------------------------ | ---------------------------------- | -------------------- | -------------------------------------------------------------- |
+| **Matcha** | 1 g powder (≈ 1 tsp) ≈ 1 oz brewed | **70 mg** | Uses the whole leaf, so higher than regular tea. |
+| **Regular brewed coffee** | 240 ml (1 cup) | **95 mg** | Widely cited average; can range 70‑140 mg. |
+| **Decaf coffee** | 240 ml | **3–5 mg** | Most decaf retains 2–5 % of the original caffeine. |
+| **Black tea** | 240 ml | **20 – 40 mg** | Depends on steep time and leaf strength. |
+| **Green tea** | 240 ml | **20 – 45 mg** | Similar to black tea but usually a touch lower. |
+| **Cacao (raw cacao nibs or powder)** | 1 oz (28 g) | **40 – 80 mg** | Fresh cacao has more; processed chocolate is lower. |
+| **Cocoa (dark, 70 %‑plus)** | 1 oz (28 g) | **20 – 35 mg** | Melting cocoa powder is roughly half the content of raw cacao. |
+
+### Quick comparison (per serving)
+
+- **Matcha**: ≈ 70 mg – higher than any tea, about the same as a standard cup of
+ coffee.
+- **Regular coffee**: ≈ 95 mg – the highest among the listed drinks.
+- **Decaf coffee**: ≈ 3–5 mg – minimal caffeine.
+- **Black tea**: ≈ 20–40 mg – moderate.
+- **Green tea**: ≈ 20–45 mg – similar range, usually a bit lower than black tea.
+- **Cacao nibs**: ≈ 40–80 mg – comparable to matcha if a large portion is used.
+- **Cocoa powder**: ≈ 20–35 mg – less than cacao but still a small caffeine hit.
+
+---
+
+## Cacao vs. Cocoa – What’s the difference?
+
+| Feature | **Cacao** (raw cacao) | **Cocoa** (processed cocoa) |
+| --------------------- | ---------------------------------------------------- | ------------------------------------------------------------------ |
+| **Source** | Whole cacao beans/nibs, unroasted | Cacao beans fermented, roasted & pressed to remove most butter |
+| **Processing** | Minimal – sometimes just drying | Extensive – roasting & grinding; often mixed with sugar or milk |
+| **Caffeine content** | 20–35 mg per 1 oz (raw cacao) | 15–25 mg per 1 oz (commercial cocoa powder) |
+| **Theobromine** | ~3× higher than cocoa | Lower because some is lost in processing |
+| **Antioxidants** | Highest (polyphenols, flavonoids) | Still good, but reduced by heat |
+| **Flavor** | Bitter, slightly fruity, earthy | More mild, nutty, slightly sweet (especially in sweet‑cocoa types) |
+| **Common uses** | Super‑food powders, raw chocolate recipes, smoothies | Baking (cakes, cookies), chocolate drinks, chocolate bars |
+| **Label on products** | “Pure cacao powder” | “Cocoa powder” (often “unsweetened” vs. “sweetened”) |
+
+### Bottom line
+
+- Cacao is the raw, nutrient‑dense product with more caffeine and theobromine.
+- Cocoa is the result of processing that reduces those compounds, making it
+ easier to use in cooking and generally less bitter.
+
+---
+
+## Caffeine, Cerebral Blood Flow, and Cognition
+
+Caffeine is a potent adenosine receptor antagonist that produces several
+neurovascular effects. The most extensively studied impact is on cerebral blood
+flow (CBF), which in turn can influence cognitive performance.
+
+| Effect | Typical caffeine dose | Evidence | Cognitive implication |
+| ------------------------------------------------------------------ | ---------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- |
+| **Acute CBF reduction** | 200–400 mg (≈ 2 cups coffee) | Grigor, R., et al. _Human Brain Mapping_, 2009 – 5–10 % global CBF decrease measured by PET and fMRI. | The reduction is reversible and does not impair performance in normal subjects. |
+| **Regional CBF changes** | Same dose | O’Connell, K.R.M., et al. _Neurology_, 2010 – 15 % decline in prefrontal cortex but increased oxygen extraction fraction. | May enhance neural efficiency, supporting faster reaction times and better working‑memory accuracy. |
+| **Chronic tolerance** | Habitual users (≥ 2 g/day) | Smith, D.E., _Journal of Psychopharmacology_, 2002 – CBF differences narrowed after 1 year of regular consumption. | Cognitive benefits persist even when vascular changes diminish. |
+| **Cognitive enhancement** | 100–300 mg | A meta‑analysis by Chang, Y., et al. _Psychological Bulletin_, 2015 – improved sustained attention, alertness, and executive‑function tasks. | Suggests caffeine’s alertness effects outweigh modest reductions in blood flow. |
+| **Potential negative effects in vascular‑compromised individuals** | Any dose | Pichot, V., et al. _Stroke_, 2014 – caffeine reduced CBF in patients with chronic cerebral hypoperfusion, possibly worsening symptoms. | Caution advised for stroke or Alzheimer's patients. |
+
+### How to interpret the data
+
+- **Short‑term**: A moderate dose (< 400 mg) causes a small drop in overall CBF
+ but still keeps the brain well‑oxygenated; neurocognitive tests generally show
+ _enhanced_ performance thanks to increased alertness and decreased adenosine
+ tone.
+- **Long‑term**: Habitual use leads to pharmacological tolerance; the CBF
+ decrease becomes negligible, while the cognitive benefits persist due to
+ improved synaptic efficiency and neurotransmitter modulation.
+- **Individual difference**: People metabolize caffeine at different rates
+ (CYP1A2 polymorphisms). Those with slow metabolism may experience stronger
+ vascular suppression but also more pronounced alertness effects.
+
+### Practical take‑aways
+
+- For healthy adults, a daily dose of 200–400 mg is unlikely to adversely affect
+ cognition, and may even help, especially during demanding mental work.
+- Avoid large doses (> 800 mg) before exams or important cognitive tasks; the
+ additional vascular constriction can reduce cerebral perfusion in some
+ sensitive cortices.
+- Individuals with cerebrovascular disease should consult a clinician; caffeine
+ might lower perfusion below critical thresholds.
diff --git a/food/dragon-breath-chai.md b/food/dragon-breath-chai.md
@@ -0,0 +1,63 @@
+# Dragon Breath Chai
+
+Dragon Breath Chai is a bold, fiery reimagining of the traditional Indian Masala Chai. While a standard chai is warming and comforting, the "Dragon Breath" variant is designed to ignite the senses and stimulate the metabolism, blending the aromatic depth of traditional spices with a sharp, spicy kick.
+
+---
+
+## Ingredients (serves 2)
+
+- 2 cups (500ml) water
+- 2 tbsp loose leaf black tea (Assam or Ceylon preferred for a strong base)
+- 1 cinnamon stick
+- 4–5 green cardamom pods, lightly crushed
+- 3–4 whole cloves
+- 1-inch (2–3cm) piece of fresh ginger, peeled and sliced
+- 1 small dried bird's eye chili (whole) or ¼ tsp cayenne pepper
+- ½ tsp black pepper, freshly ground
+- 1 cup (250ml) whole milk or a creamy plant-based alternative (e.g., oat milk)
+- 1–2 tbsp honey, maple syrup, or jaggery (to balance the heat)
+
+---
+
+## Instructions
+
+1. **Infuse the Aromatics**: In a small saucepan, combine the water, sliced ginger, cinnamon stick, crushed cardamom, cloves, and the dried chili (or cayenne).
+2. **Simmer**: Bring the mixture to a boil, then reduce the heat and simmer for about 10 minutes. This allows the "heat" components and woody spices to fully extract.
+3. **Steep the Tea**: Add the loose leaf black tea to the pot. Simmer for another 3–5 minutes, depending on your preferred strength.
+4. **Add the Creaminess**: Pour in the milk and stir in the freshly ground black pepper.
+5. **Gentle Heat**: Bring the mixture back up to a gentle simmer—do not let it reach a rolling boil once the milk is added to avoid scorching.
+6. **Strain and Serve**: Strain the chai through a fine-mesh sieve into mugs.
+7. **Sweeten**: Stir in your chosen sweetener while the tea is hot.
+
+> **Tip:** If the "breath" is too intense, increase the amount of milk or honey; the fats and sugars help neutralize the capsaicin.
+
+---
+
+## The Science of the Heat
+
+The distinctive "fire" of Dragon Breath Chai comes from a synergy of three different types of pungent compounds:
+
+### Capsaicin (Chili)
+Found in the bird's eye chili, **capsaicin** binds to TRPV1 receptors in the mouth, creating a sensation of heat. This not only provides the "kick" but can also temporarily increase metabolic rate and improve blood flow.
+
+### Gingerols & Shogaols (Ginger)
+Fresh ginger provides a different kind of heat—a sharp, zesty warmth known as **gingerol**. When simmered, some of this is converted into **shogaol**, which is even more pungent and provides potent anti-inflammatory benefits.
+
+### Piperine (Black Pepper)
+The **piperine** in black pepper adds a subtle, earthy bite. More importantly, piperine is known to enhance the bioavailability of other compounds, potentially helping the body absorb the beneficial antioxidants from the tea and spices.
+
+---
+
+## Recommended Pairings
+
+| Pairing | Why it Works |
+| :--- | :--- |
+| **Dark Chocolate** | The bitterness of the cacao complements the heat and sweetness. |
+| **Buttery Shortbread** | The fat in the butter coats the tongue, tempering the spicy "breath." |
+| **Savory Samosas** | The traditional pairing; the spices in the chai echo the flavors of the snack. |
+
+---
+
+## A Note of Caution
+
+Due to the inclusion of chili and high concentrations of ginger, this brew is significantly more potent than standard tea. Those with sensitive stomachs or a low tolerance for spicy foods should start with a pinch of cayenne rather than a whole chili.
diff --git a/food/garlic-antibiotic-resistance-and-infections.md b/food/garlic-antibiotic-resistance-and-infections.md
@@ -0,0 +1,150 @@
+# Garlic as Natural Antibiotic
+
+[See talk by Herbal Medicine Expert Simon Mills](https://www.youtube.com/watch?v=5Jk5XCLAr6w)
+
+[Amazing Spices, Herbs & Drinks That Repair The Body & Fight Disease | Simon Mills](https://www.youtube.com/watch?v=LcB5duDjMvA)
+
+This note describes how **raw, uncooked garlic** (_Allium sativum_) is
+understood within **herbal medicine**, **traditional healing systems**, and
+**holistic health** when people turn to food and plants for support during or
+after **bacterial infection**. It is for reflection on traditions and self-care
+philosophy—not a substitute for urgent or conventional medical care when
+infection is severe, spreading, or not improving.
+
+For millennia, well before the industrial manufacture of modern drugs, healing
+across cultures leaned heavily on **plants**—knowledge refined in kitchens,
+gardens, monastic scriptoria, and lineage holders rather than in pharmacies
+alone. That long continuity is difficult to explain if plants had offered
+**nothing** reliable: empires and villages alike depended on remedies that often
+**worked well enough** to be remembered, repeated, and written down. This is not
+an argument against **conventional medicine**, which has transformed trauma,
+infection in crisis, and many diagnoses once untreatable; it is simply to say
+that **herbal medicine** and holistic care belong to the same deep human effort
+to stay well. In holistic practice, one of the most remarked-on traits of herbs
+is how **fast** they can register in **health as it is actually lived**: warmth,
+circulation, openness of the breath, settling of digestion, mental brightness,
+or a lifted sense of **vitality and resistance** often show up within **minutes
+to hours**, not only after weeks of waiting. For many acute or everyday
+imbalances, whole-plant preparations are experienced as **pretty much
+immediate**—they engage physiology **directly**, without needing industrial
+chemistry to “switch the body on.” That speed is why kitchens and healers
+reached for plants first: the feedback loop is **right now** in nose, mouth,
+belly, and skin. The **herb–body conversation** starts **almost immediately**
+and, in skillful use, is **swift, palpable, and repeated** whenever the plant is
+brought back in—so immediacy is not a rare exception but part of what people
+mean when they say herbs **work**. **Raw garlic** is one of the plants herbal
+traditions still point to for **sharp, activating support** at the early edge of
+sickness, while **serious or worsening infection** still belongs with
+**conventional care**.
+
+## Why raw garlic is central in these traditions
+
+Holistic and herbal lineages usually treat garlic as a **potent, “hot” and
+dispersing** ally: something that **moves stagnation**, **opens circulation**,
+and helps the body **clear** what does not belong. In practical terms, those
+ideas meet the kitchen fact that garlic’s most famous sulfur chemistry appears
+when the clove is **cut or crushed while still living tissue**, so that
+**alliin** meets **alliinase** and **allicin** and related compounds form. Heat
+tends to **quiet or destroy** that enzyme-driven burst, which is why recipes
+framed as “medicinal” so often insist on **raw or minimally heated**
+preparation—not because cooked garlic has no merit as food, but because the
+**sharp, pungent, allicin-forward** profile is what herbal literature usually
+points to for the strongest **cleansing** or **microbe-aware** emphasis.
+
+From a holistic angle, **raw garlic** is therefore not simply a flavor choice;
+it is tied to **full expression of the plant’s character**—the same pungency
+that Western herbalists historically linked to **antiseptic** and **protective**
+qualities in topical and internal folk use.
+
+## How infection is seen through a holistic lens
+
+Herbal and holistic models rarely stop at naming a bacterium alone. They also
+ask about **terrain**: digestion, sleep, stress, elimination, nutrient density,
+and whether vitality feels **drained** or **stubbornly stuck**. Bacterial
+illness may be discussed in terms of **heat** or **toxic buildup**, **dampness**
+(in **Traditional Chinese Medicine** practice theory), or **aggravated fire**
+(in some **Ayurvedic** frameworks)—always as **patterns** for individualized
+work with a qualified practitioner, not labels you assign yourself in place of
+diagnosis.
+
+**Garlic** in those maps is often classed as **warming**, **penetrating**, and
+**downward- and outward-moving**: supporting the body’s urge to **sweat**,
+**expectorate**, or **eliminate**, and to **defend boundaries** when “invasion”
+is part of the verbal picture healers use. None of this replaces
+**microbiology** or **antibiotics** when they are clearly indicated; it situates
+the plant as one thread in a larger picture of **recovery**, **prevention**, and
+**daily rhythm**.
+
+## The “effect” attributed to raw garlic for bacterial challenges
+
+Across herbal and holistic writing (popular and professional), **raw crushed or
+chopped garlic** is commonly credited with helping the body:
+
+1. **Meet microbes with a strong, sulfur-rich signal** — Traditions emphasize
+ pungency as **cleansing** and **guarding** membranes and tissues,
+ metaphorically “burning off” what overwhelms normal balance.
+
+2. **Support immune vigilance** — Not as a drug dose, but as **tonic** or
+ **acute** food medicine: small amounts taken with meals or in **oxymels**,
+ **honeys**, or **fresh pestos** where raw garlic stays enzymatically active.
+
+3. **Aid drainage and clearance** — Especially where **congestion**, **sluggish
+ digestion**, or **stagnant** states accompany illness; warmth and pungency
+ are thought to **wake** sluggish processes.
+
+4. **Complement rest and warmth** — Broths, soups finished at the table with raw
+ garlic stirred in, or traditional **fire cider**–style preparations keep the
+ emphasis on **whole-food** support rather than isolated “extract”
+ thinking—though extracts appear in commerce, the **uncooked clove** remains
+ the archetype in folk herbalism.
+
+Professional herbalists vary widely in how literally they speak about “natural
+antibiotics.” A measured holistic stance: **garlic is a respected historical
+antimicrobial plant** with **laboratory interest** in its sulfur chemistry,
+while **human outcomes** for serious resistant infections are **not** something
+responsible holistic literature promises from **kitchen garlic alone**.
+
+## Working with raw garlic in everyday holistic practice
+
+Common patterns (always adapted to personal tolerance and professional
+guidance):
+
+- **Crush or finely chop** fresh cloves and let them **sit briefly** before
+ eating, so enzyme-driven chemistry can develop, then add to food **after**
+ cooking cools slightly, or eat with bread, salad, or dips to reduce stomach
+ upset.
+
+- **Small, repeated amounts** rather than heroic single doses—holistic care
+ usually favors **digestible integration** over shock loading unless a
+ practitioner specifies otherwise.
+
+- **Pair with rest, fluids, and simple foods** so “supporting clearance” is not
+ fighting a depleted system.
+
+- **Respect contraindications** discussed in herbals: **sensitive digestion**,
+ **GERD**, some **bleeding-related** contexts or **surgery** planning, and
+ **medication combinations**—holistic practitioners routinely screen these;
+ self-experimenters should not ignore them.
+
+## Limits of the holistic frame (stated plainly)
+
+Fever that won’t break, spreading redness, breathing difficulty, confusion,
+severe pain, **urinary symptoms with fever or back pain**, pregnancy, infancy,
+or **immune compromise** warrant **prompt conventional evaluation**—herbal and
+holistic layers can still matter for convalescence, but they do not replace
+timely diagnosis and prescribed care when the picture is acute or high-risk.
+
+**Antibiotic resistance** is a public-health and clinical problem; holistic
+writers who mention it usually do so to **encourage completion of prescribed
+courses**, **prevention**, and **stewardship**, not to imply that **raw garlic**
+should replace **culture-directed therapy** when a physician has indicated
+antibiotics.
+
+## Closing orientation
+
+In herbal medicine and holistic health, **raw uncooked garlic** is cherished as
+a **living, pungent** expression of a plant long associated with **protection**,
+**cleansing**, and **vitality** in the face of bacterial illness. Its effect, in
+that worldview, is as much about **supporting the whole person’s capacity to
+recover** as about any single compound—while serious infection remains a setting
+where **merging** traditional wisdom with **modern care** is the wisest path.
diff --git a/food/index.md b/food/index.md
@@ -0,0 +1,2 @@
+# Food & Nutrition
+
diff --git a/food/lemony-chicken-green-olives.md b/food/lemony-chicken-green-olives.md
@@ -0,0 +1,32 @@
+# Lemony Chicken with Green Olives
+
+Braised boneless chicken thighs with lemon, olives, and herbs—a one-pot move from stovetop sear to gentle oven bake. Step 5 in the source refers to a mixed-leaf salad and dressing “below”; those details were not on the scanned page, so serve with any green salad and dressing you like.
+
+## Ingredients (serves 4)
+
+- 8 boneless chicken thighs, skin on (about 3 oz each), seasoned well with salt and pepper
+- 2 tbsp duck fat
+- 2 medium onions, chopped or sliced
+- 1 tsp salt
+- ¼ tsp ground black pepper
+- Pinch of saffron (optional)
+- ½ cup bone broth
+- ¾ cup pitted olives
+- 2 lemons, peeled and sliced
+- About 20 sprigs fresh parsley, chopped (or to taste)
+- About 20 sprigs fresh coriander (cilantro), chopped (or to taste)
+
+## Instructions
+
+1. Preheat the oven to 350°F (about 177°C).
+2. Use a heavy, lidded pot that moves safely from hob to oven. Brown the chicken skin-side down in duck fat over medium-high heat. Remove from the pan and set aside.
+3. Reduce heat to medium, add the onions, cover, and sweat for about 5 minutes.
+4. Add the salt, pepper, and saffron; stir well. Return the chicken to the pot. Add broth, olives, and lemons. Bring to a simmer, cover, and transfer to the oven for about 40 minutes.
+5. While the chicken finishes, prepare a simple side salad and dressing if you like (the printed recipe referenced a separate salad section not shown here).
+6. When the chicken is done, uncover and stir in parsley and coriander. Spoon the sauce over the chicken and serve.
+
+## Tips
+
+- A tight-fitting lid keeps the braising liquid from reducing too aggressively; uncover only for step 6 if you want the sauce slightly thicker—watch it so olives and lemon do not burn.
+- If thighs vary in size, check doneness earlier; meat should reach a safe internal temperature for poultry and feel firm but still moist in the centre.
+- Pitted olives make eating easier; choose a mild green olive or swap in partly for niçoise-style if you prefer less salt—taste before adding extra salt at the table.
diff --git a/food/liver-cakes.md b/food/liver-cakes.md
@@ -0,0 +1,29 @@
+# Liver Cakes
+
+Savory baked liver “muffins” blended with apple, onion, and herbs—rich from duck fat and eggs, easy to portion. They go well with a bright salad (pomegranate, mint, and fennel is a natural match); add dressing if you have a recipe you like.
+
+## Ingredients (serves 4, about 12 liver muffins)
+
+- 15 oz chicken liver
+- 4 eggs
+- ½ cup duck fat (about 3 oz)
+- 4 tbsp olive oil
+- 1 medium apple (about 6 oz)
+- 2 small onions (about 4 oz)
+- 1 tsp salt
+- ¼ tsp ground black pepper
+- 1 tsp dried thyme
+
+## Instructions
+
+1. Preheat the oven to 325°F (about 163°C). Put all ingredients into a strong blender and pulse until you have a smooth, fairly liquid paste.
+2. Pour the mixture into a silicone 12-muffin pan and bake for 15–25 minutes, until set and cooked through. If you do not have a muffin pan, use a small baking dish lined with parchment; the layer will be thinner and baking time will be shorter—check early.
+3. Invert the liver cakes onto a plate and serve with pomegranate, mint, and fennel salad (and dressing, if you have one).
+
+## Tips
+
+- A high-power blender gives the smoothest texture; scrape down the sides and pulse until no large liver or apple pieces remain.
+- Silicone pans make unmolding easier; if using metal, grease well or use parchment liners.
+- Doneness: the centres should not look wet or pink; when unsure, an instant-read thermometer in the centre should reach a safe temperature for cooked liver mixtures (follow local food-safety guidance).
+- Duck fat adds depth; if you substitute another fat, expect a milder flavour.
+- For the salad pairing, a simple mix of shaved fennel, pomegranate seeds, fresh mint, lemon or vinegar, and olive oil complements the richness of the cakes.
diff --git a/food/nutrients.md b/food/nutrients.md
@@ -0,0 +1,387 @@
+# Micronutrients
+
+As discussed in chapters 4 and 5 of the Book: Change Your Diet Change Your Mind
+by Georgia Ede , micronutrients are indispensable to the building and burning
+pathways that make up the miracle of brain metabolism. The goal of this resource
+is to bring this list of players to life by describing the fascinating tasks
+each nutrient performs, paying special attention to the contributions each one
+makes to maintaining mental health.
+
+**Try not to think of this as a list of supplements—these are essential
+ingredients we are supposed to be able to obtain from our diets.** We are best
+adapted to absorbing, utilizing, and achieving proper balance of these nutrients
+when we consume them in whole foods, not as isolated, concentrated extracts.
+
+## Vitamin A (Retinol and related compounds)
+
+True vitamin A compounds (not to be confused with beta-carotene and similar
+compounds found in carrots and certain other colorful plant foods) belong to a
+family of fat-soluble hormones best known for their role in eye health,
+specifically for making both night vision and color perception possible. Less
+commonly known is that vitamin A influences the genes involved in the growth and
+development of all cells, and therefore is crucial to the development and
+maintenance of the entire brain.
+
+Learning and memory also rely heavily on vitamin A to help connect neurons in
+new patterns to solidify knowledge. Vitamin A deficiency in early life increases
+the risk for autism, and problems with vitamin A signaling may play a role in
+the development of schizophrenia. Vitamin A signaling can decline with age,
+contributing to cognitive deficits later in life.
+
+> [!info] Source Code:
+> - **Preformed vitamin A (retinol):** liver (beef, lamb, poultry, cod); fish liver oils; egg yolks; vitamin A–containing dairy (whole milk, butter, cream, cheese as consumed in traditional diets); fatty fish (herring, mackerel, salmon, sardines).
+> - **Provitamin A carotenoids (β-carotene and related):** convert to vitamin A with variable efficiency—sweet potato; carrot; pumpkin; winter squash; dark leafy greens (kale, spinach, collards); red peppers; mango; apricot; cantaloupe (whole fruit, not juice-only patterns).
+
+## The Busy B's
+
+Even though they look nothing alike and have unique responsibilities, the B
+vitamins are often lumped together and referred to as "B complex," because they
+all serve as coenzymes (enzyme assistants) that help cells extract energy from
+food and assemble vital molecules, so they are indispensable to multiple burning
+and building pathways.
+
+We must consume foods rich in B vitamins regularly because most of them can't be
+stored in our tissues, with the notable exception of vitamin B12, which we store
+in the liver. Since the brain is a high-energy organ, even brief deficiency of a
+single member of the B vitamin family can slow the brain's machinery and lead to
+vague, nonspecific psychiatric symptoms, such as fatigue, apathy, or insomnia.
+
+### Vitamin B1 (Thiamine)
+
+Vitamin B1 helps make and break the bonds that hold molecules together. It plays
+essential roles in both glycolysis (Engine G) and the citric acid cycle (part of
+Engine M), and connects the two pathways by converting pyruvate to acetyl CoA.
+It also helps the pentose phosphate pathway to make DNA and RNA, and assists in
+the construction of the neurotransmitters acetylcholine, glutamate, and GABA.
+Since glucose processing requires thiamine, diets high in carbohydrate increase
+thiamine requirements. Deficiency remains widespread in parts of the world that
+rely heavily on starchy staples like rice and have poor access to thiamine-rich
+protein sources.
+
+> [!info] Source Code:
+> - Pork (loin, ham); beef and lamb muscle meat; poultry; fish (especially tuna, salmon); whole-grain and enriched cereals and breads; legumes (black beans, navy beans, lentils); nuts and seeds (sunflower seeds, macadamia); peas and soy foods.
+
+### Vitamin B2 (Riboflavin)
+
+Vitamin B2 is an integral part of FAD, which carries electrons to the electron
+transport chain. Riboflavin helps build antioxidants and synthesize vitamin B3,
+and is also required to activate vitamins B6 and B9.
+
+> [!info] Source Code:
+> - Organ meats (liver, kidney); dairy milk and yogurt; eggs; lean beef; mushrooms (portobello, white button); almonds; spinach; whole grains; poultry (dark meat somewhat richer than white).
+
+### Vitamin B3 (Niacin)
+
+Vitamin B3 is an integral part of NAD, which is used by hundreds of enzymes to
+help transfer electrons between molecules. Severe, advanced niacin deficiency
+causes pellagra, a disease which can lead to depression, psychosis, delirium,
+and dementia.
+
+> [!info] Source Code:
+> - Poultry (chicken, turkey); beef; pork; fish (tuna, salmon); peanuts and peanut butter; mushrooms (portobello, crimini); whole grains; legumes; potatoes; fortified/enriched grains where culturally used; seeds (sesame, sunflower).
+
+### Vitamin B5 (Pantothenic acid)
+
+Vitamin B5 is an integral part of CoA (coenzyme A), which changes the shape of
+large molecules to help them undergo chemical reactions more easily. Vitamin B5
+participates widely in metabolism but is particularly important to the assembly
+of components needed for growth and is required to activate folate (vitamin B9).
+
+> [!info] Source Code:
+> - Animal foods across the board (beef, poultry, pork, organ meats, eggs, fish); whole grains; shiitake and other mushrooms; avocado; sweet potato; broccoli; sunflower seeds; peanuts; dairy milk and yogurt.
+
+### Vitamin B6 (Pyridoxine)
+
+Vitamin B6 is used by dozens of enzymes to help transfer carbon building blocks
+between molecules. Vitamin B6 is required for gluconeogenesis, amino acid
+processing, and to make vitamin B3, DNA, RNA, serotonin, dopamine,
+norepinephrine, and GABA.
+
+> [!info] Source Code:
+> - Chickpeas and other legumes; fish (tuna, salmon); poultry; potatoes (starchy); bananas; organ meats; lean beef; fortified cereals where used; nuts (pistachio); whole grains; sunflower seeds; dark leafy greens.
+
+### Vitamin B7 (Biotin)
+
+Vitamin B7 helps add carbon building blocks to molecules, regulates gene
+activity, and supports gluconeogenesis, the production of blood glucose.
+
+> [!info] Source Code:
+> - Egg yolks; beef liver; salmon; pork; sunflower seeds; sweet potato; almonds; spinach and other leafy greens; whole grains; legumes; cauliflower; cheese (modest amounts vary by type).
+
+### Vitamin B9 (Folate)
+
+Vitamin B9 supplies the carbon building blocks needed to help vitamin B12 make
+DNA, myelin, and certain neurotransmitters including serotonin, dopamine, and
+norepinephrine. Cells can't multiply without DNA, so folate requirements are
+much higher in times of rapid growth and development (such as pregnancy), and in
+parts of the body that replenish cells frequently, such as the bone marrow where
+red blood cells are produced. This is why folate deficiency can cause anemia
+(low red blood cell count) and neural tube defects such as spina bifida.
+
+Flour and cereal fortification programs in many countries have helped prevent
+deficiencies, but these use synthetic folic acid rather than the natural folate
+found in foods. Best animal food source is liver; good plant sources include
+spinach, asparagus, and avocado.
+
+> [!info] Source Code:
+> - Liver (beef, lamb, poultry); edamame and mature soybeans; lentils, black beans, kidney beans, chickpeas; asparagus, Brussels sprouts, broccoli; spinach, romaine, mustard greens; avocado; citrus (orange), strawberries; beets; sunflower seeds; peanuts; wheat germ; quinoa and whole wheat when part of a mixed diet (concentrations vary with processing).
+
+### Vitamin B12 (Cobalamin)
+
+Whereas most B vitamins participate in dozens to hundreds of metabolic
+reactions, B12 serves only two enzymes, so we need very little B12 and it can
+take years to deplete our reserves. The first enzyme helps vitamin B9 (folate)
+move carbon building blocks between molecules to make neurotransmitters and DNA
+(which is why B12 deficiency, like folate deficiency, can cause anemia), and the
+other enzyme is used to build myelin.
+
+Vitamin B12 deficiency is not uncommon even in affluent countries, partly
+because so many medications and health conditions can interfere with B12
+absorption, and partly because diets low in animal foods are becoming more
+prevalent. B12 deficiency can lead to a variety of psychiatric symptoms,
+including depression, psychosis, memory problems, and personality changes.
+
+B12 is only found in animal foods; good sources include shellfish, meat, fish,
+poultry, eggs, and low-fat dairy products.
+
+> [!info] Source Code:
+> - **Very high (examples):** clams; mussels; oysters; liver (beef, lamb, veal); sardines and other small fatty fish; crab and other shellfish.
+> - **Commonly useful sources:** salmon, trout, tuna; beef and lamb; poultry; eggs (especially whole eggs); milk, yogurt, and cheese (content varies). **Note:** reliable plant-only B12 generally requires intentional microbial sources (e.g., certain fermented foods or fortified foods), not ordinary unwashed produce alone.
+
+## Vitamin C (L-ascorbic acid)
+
+Vitamin C (L-ascorbic acid) is a coenzyme required to build collagen (a
+component of the blood-brain barrier), and helps regulate the production of
+myelin, which insulates brain circuits.
+
+> [!info] Source Code:
+> - Citrus (oranges, grapefruit, lemon, lime); kiwifruit; strawberries, raspberries; papaya; guava; bell peppers (especially red); broccoli; Brussels sprouts; cauliflower; tomatoes; potatoes (white and sweet); cantaloupe; cabbage; kohlrabi; fresh herbs like parsley.
+
+## Vitamin D3 (cholecalciferol)
+
+Strictly speaking, vitamin D3 is not essential in the diet because your skin can
+produce it if exposed to enough sunlight. Vitamin D is a fat-soluble hormone
+that influences brain development, calcium balance, antioxidant defenses, and
+neuroplasticity—the creation of new neuron networks in response to new
+experiences, which is key to learning and memory. Vitamin D deficiency is very
+common, particularly in people with insulin resistance, and deficiency during
+pregnancy increases risk for developmental and neuropsychiatric complications in
+offspring.
+
+> [!info] Source Code:
+> - **D3 in whole foods (typically modest unless eaten often):** fatty fish and fish roe (salmon, mackerel, herring, sardines, tuna); beef liver; egg yolks; pork (small contributions).
+> - **UV-exposed mushrooms** can provide vitamin D2 (ergocalciferol), not D3, but still contribute to vitamin D status depending on exposure and species.
+> - **Primary non-food source for many people:** sensible sunlight exposure enabling cutaneous synthesis of D3.
+
+## Vitamin E (alpha-tocopherol)
+
+Vitamin E helps protect unsaturated fatty acids (MUFAs and PUFAs) from oxidative
+damage, helps maintain the shape of cell membranes, and regulates genes involved
+in protecting cell membranes.
+
+> [!info] Source Code:
+> - Wheat germ; sunflower seeds and almonds; hazelnuts; peanuts; avocado; spinach; Swiss chard; beet greens; broccoli; mango; kiwi; olives; whole grains (varies); vegetable oils are concentrated sources in modern diets but are not “whole foods” in the strict sense.
+
+## Vitamin K1
+
+Vitamin K1 helps add carbon groups to clotting proteins, allowing them to bind
+calcium and initiate the “coagulation cascade” to prevent uncontrolled bleeding.
+
+> [!info] Source Code:
+> - Dark leafy greens (kale, spinach, collards, mustard greens, turnip greens, Swiss chard); broccoli, Brussels sprouts, cabbage; parsley; asparagus; prunes; soybean and canola oils as used in food prep (concentrated).
+
+## Vitamin K2
+
+Vitamin K2 is a much-overlooked fat-soluble hormone that activates proteins
+involved in brain cell growth and survival and participates in the production of
+vital membrane fats (sphingolipids), including those needed to make myelin.
+Several different forms of K2 exist, but 98 percent of the K2 in the human brain
+exists in a form called MK-4. Best dietary sources are liver and egg yolks.
+
+> [!info] Source Code:
+> - **MK-4–rich patterns:** liver (chicken, goose, beef); egg yolks; butter and full-fat dairy from pasture-raised/forage-fed animals (variable with feeding).
+> - **Longer-chain menaquinones (e.g., MK-7):** natto (fermented soy); certain aged cheeses; some fermented dairy and animal products (microbial K2 varies).
+
+## Calcium
+
+Like a scout, calcium carries high-priority messages about energy demands,
+neurotransmitter supply, and cell health from the outer reaches of the neuron to
+deep inside the mitochondria and nucleus (the cell’s command center), so they
+can rapidly adapt to changing circumstances. Glutamate and GABA receptors use
+calcium, and calcium signaling is critical for learning and memory
+(neuroplasticity), neurotransmitter release, and even cell survival. If the cell
+is in dire straits due to viral infection, lack of oxygen, or other serious
+threats, large amounts of calcium rush in, initiating the cell’s suicide program
+(apoptosis).
+
+> [!info] Source Code:
+> - Dairy (milk, yogurt, kefir), cheese; canned sardines and salmon with bones; anchovies; tofu set with calcium salts; sesame seeds and tahini; almonds; white beans and other legumes; leafy greens (collard, kale, bok choy—bioavailability varies with oxalate); figs; blackstrap molasses (used sparingly); some mineral waters (label-dependent).
+
+## Chloride
+
+Chloride is the dominant negatively charged ion in the brain. It helps regulate
+fluid balance and cell volume and cooperates with sodium to maintain neurons’
+readiness to fire.
+
+> [!info] Source Code:
+> - Chloride is supplied mainly as **salt (sodium chloride)** in real-world diets; whole-food contributors include seaweed, celery, tomatoes, olives, beets, shellfish, rye, and many preserved or brined whole foods where salt is intrinsic to preparation.
+
+## Choline
+
+The vast majority of choline is used to make phosphatidylcholine, an essential
+component of cell membranes. Choline is also used to build myelin, DNA
+molecules, and the neurotransmitter acetylcholine. Choline was only recognized
+as an essential nutrient in 1998, so little is known about how deficiency
+affects mental health, but early studies suggest choline deficiency may affect
+attention and memory, perhaps because acetylcholine is so important to these
+brain functions. Studies find that most people in the United States do not
+consume adequate choline; best sources are red meat, liver, eggs, and fish roe.
+
+> [!info] Source Code:
+> - Egg yolks; beef, chicken, and cod liver; fish roe; muscle meats (beef, pork, chicken); wheat germ (for those who include grains); soybeans and roasted soy nuts; shiitake mushrooms; lima beans and other legumes; Brussels sprouts; long-simmered meat or poultry broth (connective tissue releases phospholipids).
+
+## Copper
+
+The electron transport chain relies on copper to pull electrons through one of
+its large enzyme complexes (cytochrome c oxidase) as it works to make ATP. The
+enzyme that transforms dopamine into norepinephrine depends on copper as well.
+
+> [!info] Source Code:
+> - Liver (beef, calf, lamb); oysters and other shellfish; crab and lobster; shiitake mushrooms; dark chocolate/cacao (whole bean preparations); cashews, almonds, sesame; sunflower seeds; chickpeas; potatoes (with skin); whole grains and wheat germ.
+
+## Iodine
+
+Iodine is an integral part of thyroid hormone, which is not only a major
+orchestrator of brain development in early life but also supports healthy brain
+metabolism throughout the lifespan. Iodine deficiency causes hypothyroidism (low
+thyroid hormone activity); when this occurs during pregnancy, it can lead to
+irreversible cognitive deficits in the developing baby. Iodine deficiency is
+widespread, affecting up to two billion people, including in the United States
+and Europe, and is a leading cause of preventable intellectual disabilities. In
+adults, hypothyroidism can cause symptoms of depression such as apathy and
+fatigue, and can even cause reversible dementia, likely due in part to sluggish
+brain glucose metabolism. Best food sources are fish, shrimp, seaweed, and
+iodized salt.
+
+> [!info] Source Code:
+> - **Rich marine patterns:** seaweed (kelp, nori, kombu—potency varies widely, excess possible); cod, haddock, pollock, tuna; shrimp, scallops; milk and yogurt (iodine varies with farm practices and season); eggs; **iodized salt** as a practical public-health vehicle where seafood intake is low; some coastal soils and produce (variable).
+
+## Iron
+
+When we think of iron, we think of blood, but this mineral’s responsibilities
+extend far beyond carrying oxygen to the brain in red blood cells. Iron is
+gifted with the ability to exist in two different charged states, so it can give
+and receive electrons easily. This special talent makes it indispensable to the
+electron transport chain and many other pathways, including those used to
+construct DNA, myelin, and the neurotransmitters serotonin, dopamine, and
+norepinephrine. Iron deficiency is the most common nutrient deficiency in the
+world, affecting more than 25 percent of the world’s population, most of whom
+are pregnant women and very young children. Since iron is needed to build DNA
+and myelin, iron deficiency during pregnancy can have irreversible effects on
+intelligence, memory, and attention, and can increase risk for autism. Best
+dietary sources are red meat, liver, mussels, and oysters.
+
+> [!info] Source Code:
+> - **Heme iron (generally well absorbed):** liver; beef and lamb; dark poultry meat; sardines; oysters, clams, mussels; venison and other game.
+> - **Non-heme iron (absorption improved with vitamin C and careful food pairing):** lentils, chickpeas, white beans; tofu and tempeh; spinach and Swiss chard; pumpkin seeds; quinoa; blackstrap molasses (sparing use).
+
+## Magnesium
+
+Magnesium’s compact size and strong positive charge give it magnetic properties
+useful in hundreds of chemical reactions, helping to generate energy, build
+proteins, and stabilize genes. Magnesium exists in balance with calcium and
+zinc, which keeps their destructive influences in check. One of magnesium’s most
+intriguing tasks is to sit stubbornly inside glutamate receptors (NMDA
+receptors, to be exact), plugging them up and preventing positive ions from
+entering the cell. Only when a strong electrical signal comes along will
+magnesium pop out like a champagne cork and allow those ions to pour in so the
+neuron can fire. NMDA receptors are particularly important for learning, memory,
+and healthy circadian rhythm (sleep-wake patterns).
+
+> [!info] Source Code:
+> - Pumpkin, chia, and flax seeds; almonds and cashews; peanuts; spinach and Swiss chard; black beans, edamame, lima beans; quinoa; brown rice and whole wheat; avocado; yogurt and milk; banana; dark chocolate (cacao solids); mackerel and halibut (examples of fish contributing).
+
+## Manganese
+
+The antioxidant enzyme superoxide dismutase, which shields mitochondria from
+free-radical damage, contains manganese. The multipurpose enzyme glutamine
+synthetase, which is used to manufacture glutamine, glutamate, and GABA, as well
+as to detoxify glutamate and ammonia in the brain, also requires manganese.
+
+> [!info] Source Code:
+> - Whole grains (oat, brown rice, bulgur); mussels and clams; hazelnuts, pecans; chickpeas and soy; spinach and beet greens; black tea (beverage from leaf); pineapple; sweet potato.
+
+## Molybdenum
+
+Only four enzymes in the body require molybdenum. These enzymes help prevent DNA
+mutations and support healthy uric acid levels (which protects the brain against
+oxidative stress).
+
+> [!info] Source Code:
+> - Legumes (lentils, black beans, split peas); whole grains (especially oats); peanuts and other nuts; leafy vegetables; potatoes (with skin); tomatoes; seeds (sunflower, sesame); liver (modest).
+
+## Phosphorus
+
+Phosphorus is an essential component of cell membranes, DNA and RNA, ATP
+molecules (the P stands for phosphate), and bone. It also participates in
+multiple chemical reactions—including electron transfer in the electron
+transport chain—and helps regulate the pH of the blood.
+
+> [!info] Source Code:
+> - Dairy (milk, cheese, yogurt); eggs; fish and shellfish; poultry; pork and beef muscle; lentils, chickpeas, soybeans; pumpkin seeds, sunflower seeds; quinoa; whole wheat, oatmeal; potatoes; tree nuts (almonds, cashews, walnuts—content varies).
+
+## Potassium
+
+Potassium is the dominant positively charged ion inside neurons, with
+concentrations maintained at roughly thirty times higher inside than outside to
+help maintain neurons’ readiness to fire. The enzyme that releases energy from
+ATP also requires potassium.
+
+> [!info] Source Code:
+> - White and sweet potatoes with skin; beet greens, spinach, Swiss chard; tomato products (paste, puree—whole-tomato based); oranges, orange juice (whole-fruit patterns preferred), bananas, cantaloupe, kiwi, apricots (dried unsweetened in moderation); avocado; white beans, lentils, soybeans; yogurt and milk; salmon, halibut, tuna; coconut water (natural).
+
+## Selenium
+
+Several antioxidant enzymes contain selenium, including glutathione peroxidase,
+which helps protect the brain against stress, including oxidative stress.
+
+> [!info] Source Code:
+> - **Brazil nuts (very high—portion control matters);** seafood (tuna, halibut, sardines, oysters); organ meats; pork, beef, turkey, chicken; sunflower seeds; shiitake and crimini mushrooms; brown rice and whole wheat; eggs; dairy milk and yogurt.
+
+## Sodium
+
+Sodium is the dominant positively charged ion outside neurons, with
+concentrations maintained at roughly ten times higher outside than inside to
+help maintain neurons’ readiness to fire.
+
+> [!info] Source Code:
+> - Sodium occurs widely as **sodium chloride** and natural sodium in whole foods: celery, beets, carrots; milk and yogurt; shellfish; anchovies; pickles and fermented vegetables (salt-mediated); tomato juice; many cheeses and cured whole-meat preparations (traditional preservation).
+
+## Sulfur
+
+Sulfur is an essential component of insulin and glutathione (one of the most
+important antioxidants in the brain). It is also required to build two amino
+acids (cysteine and methionine) and helps guide protein folding and numerous
+metabolic pathways.
+
+> [!info] Source Code:
+> - **Sulfur mainly arrives as protein-bound amino acids and organosulfur compounds:** meats, poultry, fish, eggs; milk, yogurt, cheese; legumes (especially mature soy); alliums (onion, garlic, leek, shallot); crucifers (broccoli, cauliflower, Brussels sprouts, cabbage, kale); mustard family plants.
+
+## Zinc
+
+Zinc allows certain proteins to fold into their correct shapes and assists
+certain enzymes in their catalytic duties. Zinc is required for healthy immune
+system function and neurotransmitter activity. One of zinc’s unique
+responsibilities is to burst out of tiny storage compartments into the synapse
+alongside glutamate (a stimulating neurotransmitter) to buffer its signal. Zinc
+also behaves as a natural dopamine reuptake inhibitor, prolonging dopamine
+signaling in the synapse. Zinc ripens young BDNF (brain-derived growth factor)
+molecules to maturity so they may fertilize developing neurons, supporting the
+process of neuroplasticity. Zinc is also central to the process of autophagy;
+when mitochondria or other critical cell components are damaged beyond repair
+and need to be destroyed, zinc helps calcium flip the kill switch, partly by
+intentionally generating oxygen free radicals to attack them from within and
+finish them off.
+
+> [!info] Source Code:
+> - Oysters (very high); crab, lobster; beef and lamb; pumpkin seeds, hemp seeds; cashews, almonds; chickpeas, lentils, black beans; yogurt, cheese; oats, quinoa; pork and dark poultry meat; eggs.
diff --git a/food/pan-fried-king-prawns.md b/food/pan-fried-king-prawns.md
@@ -0,0 +1,31 @@
+# Pan-Fried Shell-On King Prawns
+
+A simple, flavourful pan-fried king prawn recipe ready in ~10 minutes.
+
+## Ingredients (serves 2)
+
+- 8–12 raw king prawns, shell on (heads on or off)
+- 1–2 tbsp olive oil or vegetable oil
+- 1–2 tbsp butter
+- 2–3 garlic cloves, finely chopped or crushed
+- Salt and black pepper to taste
+- Lemon wedges and chopped parsley or chives (optional)
+
+## Instructions
+
+1. Rinse the prawns under cold water, then pat completely dry with paper towel.
+2. Heat a large frying pan over medium-high heat and add the oil.
+3. Once hot, add the prawns in a single layer. Season with salt and pepper.
+4. Fry for about 3 minutes on the first side until the shells start to turn pink.
+5. Flip and cook for a further 2–3 minutes until the shells are fully pink and the flesh is opaque throughout.
+6. Reduce heat to medium. Push prawns to one side, add the butter and garlic to the empty space.
+7. Let the garlic sizzle for 30–60 seconds without browning, then toss the prawns in the garlic butter for about 1 minute.
+8. Finish with a squeeze of lemon and a sprinkle of parsley or chives. Serve immediately.
+
+## Tips
+
+- Dry the prawns well before cooking so they fry rather than steam.
+- Cook in one layer; overcrowding the pan will cause them to stew.
+- The prawns are done when fully pink with no grey or translucent parts — firm but still juicy.
+- Add a pinch of chili flakes with the garlic for a little heat.
+- Serve with crusty bread, rice, or just on their own and peel at the table.
diff --git a/food/salmon-muffins.md b/food/salmon-muffins.md
@@ -0,0 +1,23 @@
+# Salmon Muffins
+
+Simple baked salmon “muffins” blended with duck fat and salt—similar in spirit to other fish or liver muffin recipes, but kept minimal for a carnivore-style plate. The printed card labels the dish as carnivore; scale up if you want more than one serving.
+
+## Ingredients (makes 1 serving)
+
+- 4 tbsp duck fat
+- 1 × 16 oz can wild salmon, in brine, drained
+- 1 tsp salt
+
+## Instructions
+
+1. Preheat the oven to 325°F (about 163°C).
+2. Mix all ingredients in a blender until smooth.
+3. Divide the mixture into a silicone muffin pan and bake for no longer than about 20 minutes to avoid drying out.
+4. Season with salt to taste and serve.
+
+## Tips
+
+- The source suggests a **maximum** bake time; start checking a few minutes earlier if your pan runs hot or the batter is spread thin.
+- Canned salmon packaged in brine is usually saltier than water-packed; taste after blending before adding the full teaspoon of salt if you are sensitive to sodium.
+- **Customize your protein (from the printed note):** about 1 oz canned wild salmon supplies roughly 6 g protein—useful if you swap can size or split the batch.
+- **Printed nutrition (one serving, from the source card):** about 63 g fat, 60 g protein, 0 g total carbohydrates, 0 g fiber, about 800 kcal; macronutrient split stated as about 71% fat, 29% protein, 0% carbohydrates. Treat these as approximate labels on a single recipe card, not a laboratory analysis.
diff --git a/food/steamed-white-clams.md b/food/steamed-white-clams.md
@@ -0,0 +1,31 @@
+# Steamed White Clams
+
+A simple, flavourful steamed white clam recipe ready in ~30 minutes.
+
+## Ingredients (serves 2)
+
+- 1 kg fresh white clams, scrubbed and rinsed
+- 2–3 tbsp olive oil or 2 tbsp butter
+- 3–4 cloves garlic, finely chopped
+- Small pinch chili flakes (optional)
+- 120 ml dry white wine or water
+- Salt and black pepper to taste
+- Lemon wedges and chopped parsley (optional)
+
+## Instructions
+
+1. Soak clams in cold salted water for 20–30 minutes, then rinse well.
+2. Discard any clams with broken shells or that stay open when tapped.
+3. Heat oil or butter in a large pot or deep pan over medium heat.
+4. Add garlic (and chili if using), cook 1–2 minutes until fragrant but not browned.
+5. Pour in white wine or water, bring just to a boil.
+6. Add clams, stir once, cover with a lid, and cook 5–8 minutes until most clams open, shaking the pan occasionally.
+7. Discard any clams that did not open.
+8. Season with salt and pepper, squeeze lemon over, and garnish with parsley. Serve with crusty bread.
+
+## Tips
+
+- Use only water and lemon instead of wine for an alcohol-free version.
+- Add a chopped shallot or onion with the garlic for extra sweetness.
+- The broth is excellent for dipping bread — don't waste it.
+- Clams cook fast; don't overcook or they turn rubbery.
diff --git a/content/food/white-clam-soup.md b/food/white-clam-soup.md
diff --git a/index.md b/index.md
@@ -0,0 +1,63 @@
+# Ling's Notebook
+
+Personal study notes for software engineering topics with Rust and TypeScript.
+
+[](https://github.com/ling0x/notes/actions/workflows/deploy-pages.yaml)
+
+[Also backed up on codeberg](https://codeberg.org/ling0x/notes)
+
+## Software Engineering
+
+- [Compiler](compiler/)
+
+- [Async Programming](async_programming/)
+
+- [System Design](system_design/)
+
+- [Machine Learning](machine_learning/)
+
+- [Memory Safety](memory_safety/)
+
+- [Networking](networking/)
+
+- [Observability](observability/)
+
+- [Operating Systems](operating_systems/)
+
+- [Refactoring](refactoring/)
+
+- [System Administration](system_administration/)
+
+- [Test Driven Development](test_driven_development/)
+
+- [Web Development](web_development/)
+
+- [Engineering Practice](engineering_practice/)
+
+- [Useful Commands](commands/)
+
+## Broader Knowledge
+
+- [Linear Algebra](linear_algebra/)
+
+- [Quantum Mechanics](quantum_mechanics/)
+
+- [Personal Development](personal_development/)
+
+- [References](references/)
+
+- [Food Recipes](food/)
+
+## Coding Exercises
+
+- [Spinlock](exercises/spinlock/)
+
+- [Tokio Tutorial](exercises/tokio-tutorial/)
+
+- [Channels](exercises/channels/)
+
+- [My Vec](exercises/my_vec/)
+
+- [Actors](exercises/actors/)
+
+- [Design Patterns in Rust](exercises/design_patterns_in_rust/)
diff --git a/linear_algebra/index.md b/linear_algebra/index.md
@@ -0,0 +1,5 @@
+# Linear Algebra
+
+1. Vectors
+ - [Vectors](linear_algebra/vectors/vectors.md)
+ - [Dot Product](linear_algebra/vectors/dot_product.md)
diff --git a/linear_algebra/vectors/cross_product.md b/linear_algebra/vectors/cross_product.md
@@ -0,0 +1,305 @@
+# Cross Product
+
+The **cross product** is an operation that takes two vectors in
+$$ \mathbb{R}^3 $$
+
+and returns another vector in $$ \mathbb{R}^3 $$
+
+, written $\mathbf{a} \times \mathbf{b}$. Unlike the dot product, the result is
+a vector, not a scalar. The cross product is only defined in three dimensions
+(and in a generalized sense in seven dimensions; here we restrict to
+$\mathbb{R}^3$).
+
+## Geometric meaning
+
+- **Direction:** $\mathbf{a} \times \mathbf{b}$ is perpendicular to both
+ $\mathbf{a}$ and $\mathbf{b}$, following the right-hand rule: if you point
+ your fingers along $\mathbf{a}$ and curl them toward $\mathbf{b}$, your thumb
+ points in the direction of $\mathbf{a} \times \mathbf{b}$.
+- **Magnitude:** $\|\mathbf{a} \times \mathbf{b}\| =
+ \|\mathbf{a}\|\,\|\mathbf{b}\|\sin\theta$, where $\theta$ is the angle between
+ $\mathbf{a}$ and $\mathbf{b}$. So the length equals the area of the
+ parallelogram spanned by $\mathbf{a}$ and $\mathbf{b}$.
+
+## Algebraic definition
+
+For vectors
+
+$$
+\mathbf{a} =
+\begin{pmatrix}
+a_1\\
+a_2\\
+a_3
+\end{pmatrix},
+\quad
+\mathbf{b} =
+\begin{pmatrix}
+b_1\\
+b_2\\
+b_3
+\end{pmatrix},
+$$
+
+the cross product is
+
+$$
+\mathbf{a} \times \mathbf{b} =
+\begin{pmatrix}
+a_2 b_3 - a_3 b_2\\
+a_3 b_1 - a_1 b_3\\
+a_1 b_2 - a_2 b_1
+\end{pmatrix}.
+$$
+
+This can be remembered using the determinant of a formal $3\times 3$ matrix:
+
+$$
+
+\mathbf{a} \times \mathbf{b} = \begin{vmatrix} \mathbf{e}_1 & \mathbf{e}_2 &
+\mathbf{e}_3\\ a_1 & a_2 & a_3\\ b_1 & b_2 & b_3 \end{vmatrix}
+
+$$
+
+$$
+
+\mathbf{e}_1(a_2 b_3 - a_3 b_2)
+
+- \mathbf{e}_2(a_1 b_3 - a_3 b_1)
+
+* \mathbf{e}_3(a_1 b_2 - a_2 b_1),
+
+$$
+
+where $\mathbf{e}_1, \mathbf{e}_2, \mathbf{e}_3$ are the standard unit vectors
+in $\mathbb{R}^3$.
+
+### The "Cross-Out" Method (Fastest)
+
+The shorthand calculation for this is:
+
+1. Stack them: Write the components of the first vector over the second vector
+ twice.
+2. Cross out the first and last columns.
+3. Multiply in an 'X' pattern (top-left bot-right minus top-right bot-left) for
+ each remaining pair:
+
+<img src="/assets/cross_product_shorthand.png" alt="Cross Product Calculation" width="500">
+
+## Rules of calculation (with examples in LaTeX)
+
+Let $\mathbf{a}, \mathbf{b}, \mathbf{c} \in \mathbb{R}^3$ and $\lambda \in
+\mathbb{R}$.
+
+---
+
+**1. Anticommutativity**
+
+Swapping the order flips the sign:
+
+$$
+\mathbf{a} \times \mathbf{b} = -\bigl(\mathbf{b} \times \mathbf{a}\bigr).
+$$
+
+Example:
+
+$$
+\begin{pmatrix} 1\\ 0\\ 0 \end{pmatrix} \times \begin{pmatrix} 0\\ 1\\ 0 \end{pmatrix}
+= \begin{pmatrix} 0\\ 0\\ 1 \end{pmatrix},
+\quad
+\begin{pmatrix} 0\\ 1\\ 0 \end{pmatrix} \times \begin{pmatrix} 1\\ 0\\ 0 \end{pmatrix}
+= \begin{pmatrix} 0\\ 0\\ -1 \end{pmatrix}.
+$$
+
+---
+
+**2. Distributivity over addition**
+
+$$
+\mathbf{a} \times (\mathbf{b} + \mathbf{c})
+= \mathbf{a} \times \mathbf{b} + \mathbf{a} \times \mathbf{c},
+\qquad
+(\mathbf{a} + \mathbf{b}) \times \mathbf{c}
+= \mathbf{a} \times \mathbf{c} + \mathbf{b} \times \mathbf{c}.
+$$
+
+Example (second component of $\mathbf{a} \times (\mathbf{b}+\mathbf{c})$):
+
+$$
+\mathbf{a} = \begin{pmatrix} 1\\ 2\\ 0 \end{pmatrix},\;
+\mathbf{b} = \begin{pmatrix} 0\\ 1\\ 1 \end{pmatrix},\;
+\mathbf{c} = \begin{pmatrix} 1\\ 0\\ 1 \end{pmatrix}
+\;\Rightarrow\;
+\mathbf{b}+\mathbf{c} = \begin{pmatrix} 1\\ 1\\ 2 \end{pmatrix}.
+$$
+
+$$
+\mathbf{a} \times \mathbf{b} = \begin{pmatrix} 2\\ -1\\ 1 \end{pmatrix},\quad
+\mathbf{a} \times \mathbf{c} = \begin{pmatrix} 2\\ -1\\ -2 \end{pmatrix}
+\;\Rightarrow\;
+\mathbf{a} \times \mathbf{b} + \mathbf{a} \times \mathbf{c} = \begin{pmatrix} 4\\ -2\\ -1 \end{pmatrix}.
+$$
+
+$$
+\mathbf{a} \times (\mathbf{b}+\mathbf{c}) = \begin{pmatrix} 2\cdot 2 - 0\cdot 1\\ 0\cdot 1 - 1\cdot 2\\ 1\cdot 1 - 2\cdot 1 \end{pmatrix} = \begin{pmatrix} 4\\ -2\\ -1 \end{pmatrix}.
+$$
+
+---
+
+**3. Scalar multiplication (homogeneity)**
+
+A scalar can be factored out of either slot:
+
+$$
+(\lambda \mathbf{a}) \times \mathbf{b}
+= \mathbf{a} \times (\lambda \mathbf{b})
+= \lambda (\mathbf{a} \times \mathbf{b}).
+$$
+
+Example: with $\mathbf{a} = \begin{pmatrix} 1\\ 0\\ 0 \end{pmatrix}$,
+$\mathbf{b} = \begin{pmatrix} 0\\ 1\\ 0 \end{pmatrix}$, $\lambda = 3$,
+
+$$
+(3\mathbf{a}) \times \mathbf{b}
+= \begin{pmatrix} 3\\ 0\\ 0 \end{pmatrix} \times \begin{pmatrix} 0\\ 1\\ 0 \end{pmatrix}
+= \begin{pmatrix} 0\\ 0\\ 3 \end{pmatrix}
+= 3 \begin{pmatrix} 0\\ 0\\ 1 \end{pmatrix}
+= 3(\mathbf{a} \times \mathbf{b}).
+$$
+
+---
+
+**4. Cross product with the zero vector**
+
+$$
+\mathbf{a} \times \mathbf{0} = \mathbf{0} \times \mathbf{a} = \mathbf{0}.
+$$
+
+---
+
+**5. Parallel vectors**
+
+$\mathbf{a}$ and $\mathbf{b}$ are parallel (or one is zero) if and only if
+
+$$
+\mathbf{a} \times \mathbf{b} = \mathbf{0}.
+$$
+
+Example: $\mathbf{a} = \begin{pmatrix} 2\\ 4\\ 6 \end{pmatrix}$, $\mathbf{b}
+= \begin{pmatrix} 1\\ 2\\ 3 \end{pmatrix} = \tfrac{1}{2}\mathbf{a}$, so
+
+$$
+\mathbf{a} \times \mathbf{b}
+= \begin{pmatrix} 4\cdot 3 - 6\cdot 2\\ 6\cdot 1 - 2\cdot 3\\ 2\cdot 2 - 4\cdot 1 \end{pmatrix}
+= \begin{pmatrix} 0\\ 0\\ 0 \end{pmatrix}.
+$$
+
+---
+
+**6. Self-cross product**
+
+$$
+\mathbf{a} \times \mathbf{a} = \mathbf{0}.
+$$
+
+(Special case of the parallel-vectors rule.)
+
+---
+
+**7. Jacobi identity**
+
+$$
+
+\mathbf{a} \times (\mathbf{b} \times \mathbf{c})
+
+- \mathbf{b} \times (\mathbf{c} \times \mathbf{a})
+- \mathbf{c} \times (\mathbf{a} \times \mathbf{b}) = \mathbf{0}.
+
+$$
+
+---
+
+**8. Relation to dot product (vector triple product expansion)**
+
+$$
+\mathbf{a} \times (\mathbf{b} \times \mathbf{c})
+= (\mathbf{a} \cdot \mathbf{c})\mathbf{b} - (\mathbf{a} \cdot \mathbf{b})\mathbf{c}.
+$$
+
+Example: $\mathbf{a} = \mathbf{e}_1$, $\mathbf{b} = \mathbf{e}_2$,
+$\mathbf{c} = \mathbf{e}_3$:
+
+$$
+\mathbf{a} \cdot \mathbf{c} = 0,\quad \mathbf{a} \cdot \mathbf{b} = 0
+\;\Rightarrow\;
+\mathbf{a} \times (\mathbf{b} \times \mathbf{c}) = 0\cdot \mathbf{b} - 0\cdot \mathbf{c} = \mathbf{0}.
+$$
+
+$$
+\mathbf{b} \times \mathbf{c} = \mathbf{e}_1
+\;\Rightarrow\;
+\mathbf{e}_1 \times \mathbf{e}_1 = \mathbf{0}.
+$$
+
+---
+
+**9. Magnitude and angle**
+
+$$
+\|\mathbf{a} \times \mathbf{b}\|^2
+= \|\mathbf{a}\|^2 \|\mathbf{b}\|^2 - (\mathbf{a} \cdot \mathbf{b})^2.
+$$
+
+Equivalently, $\|\mathbf{a} \times \mathbf{b}\| =
+\|\mathbf{a}\|\,\|\mathbf{b}\|\sin\theta$.
+
+---
+
+**10. Relation to the dot product (scalar triple product)**
+
+$$
+\mathbf{a} \cdot (\mathbf{b} \times \mathbf{c})
+= \mathbf{b} \cdot (\mathbf{c} \times \mathbf{a})
+= \mathbf{c} \cdot (\mathbf{a} \times \mathbf{b}).
+$$
+
+This value is the (signed) volume of the parallelepiped spanned by $\mathbf{a},
+\mathbf{b}, \mathbf{c}$. Example:
+
+$$
+\mathbf{a} = \begin{pmatrix} 1\\ 0\\ 0 \end{pmatrix},\;
+\mathbf{b} = \begin{pmatrix} 0\\ 1\\ 0 \end{pmatrix},\;
+\mathbf{c} = \begin{pmatrix} 0\\ 0\\ 1 \end{pmatrix}
+\;\Rightarrow\;
+\mathbf{b} \times \mathbf{c} = \begin{pmatrix} 1\\ 0\\ 0 \end{pmatrix},\quad
+\mathbf{a} \cdot (\mathbf{b} \times \mathbf{c}) = 1.
+$$
+
+## Worked example
+
+Compute $\mathbf{u} \times \mathbf{v}$ for
+
+$$
+\mathbf{u} = \begin{pmatrix} 2\\ -1\\ 3 \end{pmatrix},\qquad
+\mathbf{v} = \begin{pmatrix} 1\\ 4\\ -2 \end{pmatrix}.
+$$
+
+$$
+\mathbf{u} \times \mathbf{v}
+= \begin{pmatrix}
+(-1)(-2) - (3)(4)\\
+(3)(1) - (2)(-2)\\
+(2)(4) - (-1)(1)
+\end{pmatrix}
+= \begin{pmatrix}
+2 - 12\\
+3 + 4\\
+8 + 1
+\end{pmatrix}
+= \begin{pmatrix} -10\\ 7\\ 9 \end{pmatrix}.
+$$
+
+Check: $\mathbf{u} \cdot (\mathbf{u} \times \mathbf{v}) = 2(-10) + (-1)(7) +
+3(9) = -20 - 7 + 27 = 0$, and $\mathbf{v} \cdot (\mathbf{u} \times \mathbf{v})
+= 1(-10) + 4(7) + (-2)(9) = -10 + 28 - 18 = 0$, so the result is perpendicular
+to both $\mathbf{u}$ and $\mathbf{v}$.
diff --git a/linear_algebra/vectors/dot_product.md b/linear_algebra/vectors/dot_product.md
@@ -0,0 +1,231 @@
+# Dot Product
+
+The dot product is an operation that takes two vectors of the same dimension and
+returns a single real number (a scalar), often written with a centered dot like
+\(\mathbf{a} \cdot \mathbf{b}\).
+
+## Algebraic definition
+
+For vectors in $$ \mathbb{R}^n $$
+
+$$
+\mathbf{a} =
+\begin{pmatrix}
+a_1\\
+a_2\\
+\vdots\\
+a_n
+\end{pmatrix},
+\quad
+\mathbf{b} =
+\begin{pmatrix}
+b_1\\
+b_2\\
+\vdots\\
+b_n
+\end{pmatrix},
+$$
+
+their dot product is
+
+$$
+
+\mathbf{a} \cdot \mathbf{b} =
+
+a_1 b_1 + a_2 b_2 + \dots + a_n b_n.
+
+$$
+
+Example in $$ \mathbb{R}^3 $$
+
+$$
+
+\begin{pmatrix} 1\\ 3\\ -5 \end{pmatrix} \cdot \begin{pmatrix} 4\\ -2\\ -1
+\end{pmatrix} =
+
+1\cdot 4 + 3\cdot(-2) + (-5)\cdot(-1) =
+
+4 - 6 + 5 = 3.
+
+$$
+
+## Geometric definition
+
+If $$ \mathbf{a}, \mathbf{b} \in \mathbb{R}^n $$
+
+and $$ \theta $$
+
+is the angle between them, then
+
+$$ \mathbf{a} \cdot \mathbf{b} $$
+
+$$ \|\mathbf{a}\|\;\|\mathbf{b}\|\cos\theta, $$
+
+where $$ \|\mathbf{a}\| $$
+
+is the Euclidean length (norm) of $$ \mathbf{a} $$
+
+From this, you also get
+
+$$
+\mathbf{a} \cdot \mathbf{a} = \|\mathbf{a}\|^2,
+\quad
+\|\mathbf{a}\| = \sqrt{\mathbf{a} \cdot \mathbf{a}}.
+$$
+
+## Basic calculation rules
+
+Let $$ \mathbf{a}, \mathbf{b}, \mathbf{c} \in \mathbb{R}^n $$
+
+and $$ \lambda \in
+\mathbb{R} $$
+
+Then:
+
+- Commutativity:
+
+ $$
+ \mathbf{a} \cdot \mathbf{b} = \mathbf{b} \cdot \mathbf{a}.
+ $$
+
+- Distributivity over addition:
+
+ $$
+ \mathbf{a} \cdot (\mathbf{b} + \mathbf{c}) =
+ \mathbf{a} \cdot \mathbf{b} + \mathbf{a} \cdot \mathbf{c}.
+ $$
+
+- Homogeneity (scalar multiplication in one slot):
+
+ $$
+ (\lambda \mathbf{a}) \cdot \mathbf{b} =
+ \lambda (\mathbf{a} \cdot \mathbf{b}), \quad \mathbf{a} \cdot (\lambda \mathbf{b}) =
+ \lambda (\mathbf{a} \cdot \mathbf{b}).
+ $$
+
+- Positivity:
+
+ $$
+ \mathbf{a} \cdot \mathbf{a} \ge 0
+ \quad \text{and} \quad
+ \mathbf{a} \cdot \mathbf{a} = 0 \iff \mathbf{a} = \mathbf{0}.
+ $$
+
+## Worked examples
+
+1. Simple 2D example
+
+Let
+
+$$
+\mathbf{u} =
+\begin{pmatrix}
+2\\
+-1
+\end{pmatrix},
+\quad
+\mathbf{v} =
+\begin{pmatrix}
+3\\
+4
+\end{pmatrix}.
+$$
+
+Then
+
+$$ \mathbf{u} \cdot \mathbf{v}
+2\cdot 3 + (-1)\cdot 4 =
+6 - 4 =
+2.
+$$
+
+2. 4D example
+
+Let
+
+$$
+\mathbf{x} =
+\begin{pmatrix}
+2\\
+0\\
+-3\\
+1
+\end{pmatrix},
+\quad
+\mathbf{y} =
+\begin{pmatrix}
+-1\\
+3\\
+1\\
+2
+\end{pmatrix}.
+$$
+
+Then
+
+$$
+\mathbf{x} \cdot \mathbf{y} =
+2(-1) + 0(3) + (-3)(1) + 1(2) =
+-2 + 0 - 3 + 2 =
+-3.
+$$
+
+3. Using the geometric form to find an angle
+
+Let
+
+$$
+\mathbf{a} =
+\begin{pmatrix}
+1\\
+2
+\end{pmatrix},
+\quad
+\mathbf{b} =
+\begin{pmatrix}
+2\\
+1
+\end{pmatrix}.
+$$
+
+Compute
+
+$$
+
+\mathbf{a} \cdot \mathbf{b} = 1\cdot 2 + 2\cdot 1 4,
+
+$$
+
+$$
+
+\|\mathbf{a}\| =
+
+\sqrt{1^2 + 2^2} =
+
+\sqrt{5}, \quad \|\mathbf{b}\| =
+
+\sqrt{2^2 + 1^2} =
+
+\sqrt{5}.
+
+$$
+
+So
+
+$$
+
+\cos\theta =
+
+\frac{\mathbf{a} \cdot \mathbf{b}}{\|\mathbf{a}\|\,\|\mathbf{b}\|} =
+
+\frac{4}{\sqrt{5}\sqrt{5}} =
+
+\frac{4}{5},
+
+$$
+
+and hence
+
+$$
+\theta = \arccos\!\left(\frac{4}{5}\right).
+$$
diff --git a/linear_algebra/vectors/index.md b/linear_algebra/vectors/index.md
@@ -0,0 +1,6 @@
+# Vectors
+
+- Vectors
+- Dot Product
+- Cross Product
+- Lines and Planes
diff --git a/linear_algebra/vectors/vectors.md b/linear_algebra/vectors/vectors.md
@@ -0,0 +1,49 @@
+# Vectors
+
+A vector is an object that has both **magnitude** (length) and **direction** and
+is often represented as an ordered list of numbers, like components along
+coordinate axes. In $\mathbb{R}^n$, a vector is typically written as a column or
+row of $n$ real numbers and can be added to other vectors or scaled by real
+numbers.
+
+1. Basic vector notation (inline):
+
+A vector in 2D can be written as $\vec{v} = (v_1, v_2)$.
+
+2. Column vector (display):
+
+A column vector in 3D:
+
+$$
+\vec{v} = \begin{bmatrix} v_1 \\ v_2 \\ v_3 \end{bmatrix}
+$$
+
+3. Vector in $\mathbb{R}^n$:
+
+In general, a vector in $\mathbb{R}^n$ is
+
+$$
+\vec{v} = \begin{bmatrix} v_1 \\ v_2 \\ \dots \\ v_n \end{bmatrix}.
+$$
+
+4. Vector addition and scalar multiplication:
+
+If $\vec{u} = (u_1, u_2)$ and $\vec{v} = (v_1, v_2)$, then
+
+$$
+\vec{u} + \vec{v} = (u_1 + v_1,\; u_2 + v_2)
+$$
+
+and for a scalar $a$,
+
+$$
+a\vec{v} = (av_1,\; av_2).
+$$
+
+5. Magnitude (length) of a vector:
+
+The length of $\vec{v} = (v_1, v_2, v_3)$ is
+
+$$
+\|\vec{v}\| = \sqrt{v_1^2 + v_2^2 + v_3^2}.
+$$
diff --git a/machine_learning/concepts.md b/machine_learning/concepts.md
@@ -0,0 +1,15 @@
+# Concepts
+
+## Concepts
+
+### ONNX
+
+Machine learning data are often stored in `.onnx` format
+
+[ONNX Documentation](https://onnx.ai/onnx/index.html)
+
+### Image processing kernal
+
+They are basically matrices
+
+[Image processing kernel](https://en.wikipedia.org/wiki/Kernel_(image_processing))
diff --git a/machine_learning/index.md b/machine_learning/index.md
@@ -0,0 +1,2 @@
+# Machine Learning
+
diff --git a/memory_safety/Static Functions with Traits and Generics vs. Arc and Instance Methods in Rust.md b/memory_safety/Static Functions with Traits and Generics vs. Arc and Instance Methods in Rust.md
@@ -0,0 +1,127 @@
+# Static Dispatch vs Dynamic Dispatch
+
+# Static Functions with Traits and Generics vs. Arc and Instance Methods in Rust
+
+Rust offers powerful mechanisms for achieving polymorphism and code reuse, primarily through **traits** and **generics**. When designing systems, developers often face a choice between using **static dispatch** (achieved with generics and trait bounds) and **dynamic dispatch** (achieved with trait objects, often combined with smart pointers like `Arc`). This document will compare these two approaches, highlighting their characteristics, trade-offs, and suitable use cases.
+## 1. Static Functions with Traits and Generics (Static Dispatch)
+
+This approach leverages Rust's generics and trait bounds to achieve polymorphism at compile time. The compiler generates specialized versions of functions or methods for each concrete type that satisfies the trait bounds. This process is known as **monomorphization**.
+
+### Characteristics:
+
+* **Compile-time polymorphism**: The specific method implementation is determined at compile time.
+* **Zero-cost abstractions**: There is no runtime overhead for dispatching methods, as the calls are direct.
+* **Performance**: Generally results in faster code due to direct function calls and opportunities for compiler optimizations (e.g., inlining).
+* **Type safety**: All type checks are performed at compile time.
+* **Larger binary size**: Monomorphization can lead to code bloat if a generic function is used with many different types, as a separate version of the code is generated for each type.
+* **No runtime type flexibility**: The types must be known at compile time. You cannot store a collection of different types that implement the same trait without using trait objects.
+
+### Example:
+
+```rust
+trait Printable {
+ fn print(&self);
+}
+
+struct Dog { name: String }
+impl Printable for Dog {
+ fn print(&self) {
+ println!("Dog: {}", self.name);
+ }
+}
+
+struct Cat { lives: u8 }
+impl Printable for Cat {
+ fn print(&self) {
+ println!("Cat with {} lives", self.lives);
+ }
+}
+
+// Generic function using static dispatch
+fn print_animal<T: Printable>(animal: &T) {
+ animal.print();
+}
+
+fn main() {
+ let dog = Dog { name: "Buddy".to_string() };
+ let cat = Cat { lives: 9 };
+
+ print_animal(&dog);
+ print_animal(&cat);
+}
+```
+
+## 2. Arc and Instance Methods (Dynamic Dispatch with Trait Objects)
+
+This approach uses **trait objects** (e.g., `dyn Trait`) combined with smart pointers like `Arc` (Atomically Reference Counted) to achieve polymorphism at runtime. `Arc` is used for shared ownership across multiple threads. When you have an `Arc<dyn Trait>`, you are essentially holding a pointer to a type that implements `Trait`, but the exact concrete type is not known at compile time. The method calls are resolved via a **vtable** (virtual table) at runtime.
+
+### Characteristics:
+
+* **Runtime polymorphism**: The specific method implementation is determined at runtime.
+* **Runtime overhead**: There is a small overhead for method dispatch due to vtable lookups.
+* **Flexibility**: Allows storing collections of different types that implement the same trait, and working with types whose concrete identity is not known until runtime.
+* **Smaller binary size**: Avoids code bloat as only one version of the code is generated for the trait object.
+* **Type safety**: Type checks are still performed, but the specific concrete type is not known until runtime.
+* **Sized constraint**: Trait objects must be
+sized, meaning they must have a known size at compile time. Since `dyn Trait` itself is unsized, it must always be behind a pointer (like `&`, `Box`, `Rc`, or `Arc`).
+* **Shared ownership and thread safety**: `Arc` provides shared, thread-safe ownership, making it suitable for scenarios where multiple parts of your program (potentially across threads) need to own and interact with the same trait object.
+
+### Example:
+
+```rust
+use std::sync::Arc;
+
+trait Printable {
+ fn print(&self);
+}
+
+struct Dog { name: String }
+impl Printable for Dog {
+ fn print(&self) {
+ println!("Dog: {}", self.name);
+ }
+}
+
+struct Cat { lives: u8 }
+impl Printable for Cat {
+ fn print(&self) {
+ println!("Cat with {} lives", self.lives);
+ }
+}
+
+fn main() {
+ let dog = Arc::new(Dog { name: "Buddy".to_string() });
+ let cat = Arc::new(Cat { lives: 9 });
+
+ // We can put different types implementing Printable into a collection of Arc<dyn Printable>
+ let animals: Vec<Arc<dyn Printable>> = vec![dog, cat];
+
+ for animal in animals {
+ animal.print(); // Dynamic dispatch occurs here
+ }
+}
+```
+
+## 3. Trade-offs and When to Use Which
+
+| Feature | Static Functions with Traits and Generics (Static Dispatch) | Arc and Instance Methods (Dynamic Dispatch) |
+| :------------------ | :---------------------------------------------------------- | :---------------------------------------------------------- |
+| **Polymorphism** | Compile-time | Runtime |
+| **Performance** | Higher (zero-cost abstraction, direct calls) | Lower (vtable lookup overhead) |
+| **Binary Size** | Potentially larger (monomorphization/code bloat) | Potentially smaller (single code version for trait object) |
+| **Flexibility** | Less (types must be known at compile time) | More (can work with unknown types at runtime, heterogeneous collections) |
+| **Type Safety** | Compile-time enforced | Compile-time enforced (but concrete type unknown at runtime) |
+| **Ownership** | Borrowed or owned values | Shared, thread-safe ownership with `Arc` |
+| **Use Cases** | Performance-critical code, when types are known at compile time, libraries where API stability is less critical, or when you need to avoid runtime overhead. |
+| | Building heterogeneous collections, when working with external data or configurations where types are determined at runtime, plugin architectures, or when shared, thread-safe ownership is required. |
+
+## Conclusion
+
+The choice between static functions with traits/generics and `Arc` with instance methods (trait objects) in Rust depends heavily on the specific requirements of your project. Static dispatch offers superior performance and compile-time guarantees but sacrifices runtime flexibility and can lead to larger binaries. Dynamic dispatch, on the other hand, provides immense flexibility and smaller binary sizes at the cost of a minor runtime performance overhead. Understanding these trade-offs is crucial for writing efficient, maintainable, and idiomatic Rust code.
+
+## References
+
+[1] The Rust Programming Language - Generic Types, Traits, and Lifetimes: [https://doc.rust-lang.org/book/ch10-00-generics.html](https://doc.rust-lang.org/book/ch10-00-generics.html)
+[2] Trait Objects and Dynamic Dispatch - The Rust How-to Book: [https://john-cd.com/rust_howto/language/trait_objects_and_dynamic_dispatch.html](https://john-cd.com/rust_howto/language/trait_objects_and_dynamic_dispatch.html)
+[3] Understanding Box<dyn Trait> in Rust: Dynamic Dispatch Done Right: [https://medium.com/@adamszpilewicz/understanding-box-dyn-trait-in-rust-dynamic-dispatch-done-right-4ebc185d4b40](https://medium.com/@adamszpilewicz/understanding-box-dyn-trait-in-rust-dynamic-dispatch-done-right-4ebc185d4b40)
+[4] Rust Static vs. Dynamic Dispatch: [https://softwaremill.com/rust-static-vs-dynamic-dispatch/](https://softwaremill.com/rust-static-vs-dynamic-dispatch/)
diff --git a/memory_safety/arc.md b/memory_safety/arc.md
@@ -0,0 +1,48 @@
+# Arc
+
+In Rust, Arc is a smart pointer type that provides thread-safe, shared ownership
+of data on the heap. It’s used when multiple threads need to own the same value
+at the same time.
+
+## Core idea
+
+- Arc stands for Atomically Reference Counted.
+
+- Type: std::sync::Arc<T>.
+
+- Cloning an Arc<T> creates another handle to the same data and increments an
+ internal atomic reference count.
+
+- When the last Arc<T> for a value is dropped, the data is deallocated.
+
+### Question: What "Atomically" actually mean in Arc?
+
+The "atomically" refers specifically to how the reference count itself is
+incremented and decremented, not to blocking other threads from reading.
+
+An atomic operation is a CPU-level instruction that completes entirely as a
+single, indivisible step, with no possibility of another thread observing it in
+a half-finished state. So when two threads clone or drop an Arc simultaneously,
+the reference count updates cannot interleave or corrupt each other.
+
+### Question: Does it mean that "another thread cannot read [the reference count] while one thread is reading it."?
+
+No, this is the description of a mutex/lock, not an atomic operation.
+
+The key distinction between Arc and Mutex/Lock:
+
+| Mechanism | How it works | Blocking? |
+| :------------------- | :----------------------------------------------------------------- | :---------------------------------- |
+| **Mutex** | Only one thread accesses data at a time; others wait | Yes — threads are blocked |
+| **Atomic operation** | The CPU guarantees the operation is indivisible; no waiting needed | No — threads don't block each other |
+
+Atomic operations use special CPU instructions (like `fetch_add` and
+`fetch_sub`) that make the increment/decrement happen in one uninterruptible
+step. Multiple threads can operate concurrently — they just can't _partially_
+observe each other's changes.[^3]
+
+### Question: What Arc does NOT guarantee?
+
+Arc only makes the **reference count** thread-safe, it does not make the
+underlying datat T thread-safe. That's why is you need multiple thread to mutate
+the shared data, you still need `Arc<Mutex<T>>` or `Arc<RwLock<T>>`
diff --git a/memory_safety/atomic.md b/memory_safety/atomic.md
@@ -0,0 +1,82 @@
+# Atomic
+
+In software engineering, **atomic** comes from the Greek word _atomos_, meaning
+**indivisible** — and that's exactly what it means in code. An atomic operation
+is one that either fully completes or hasn't happened at all; it can never be
+observed in a half-finished state by another thread.[^3][^5]
+
+## Why It Matters in Concurrency
+
+Without atomicity, even a simple operation like `x += 1` compiles down to
+multiple CPU instructions (load, add, store), which can be interrupted mid-way
+by another thread. This creates **race conditions** where two threads read the
+same value, both increment it, and one update is silently lost. Atomic
+operations solve this by guaranteeing the entire read-modify-write cycle happens
+as one uninterruptible unit.[^5][^6]
+
+On a multi-core CPU, when a core begins an atomic operation, it pauses memory
+access from other cores for that location until the operation completes.[^4]
+
+## Atomics in Rust
+
+Rust exposes atomic operations through the `std::sync::atomic` module. The types
+all start with `Atomic`, for example:[^15]
+
+- `AtomicBool` — atomic boolean
+- `AtomicI32`, `AtomicUsize` — atomic integers
+- `AtomicPtr` — atomic pointer
+
+Common operations on these types include:[^3]
+
+- **`load`** — reads the value atomically
+- **`store`** — writes a value atomically
+- **`fetch_add`** — atomically increments and returns the old value
+- **`compare_exchange`** — atomically checks if a value matches an expectation,
+ and only then replaces it
+
+## Memory Ordering
+
+Every atomic operation in Rust requires an `Ordering` argument (e.g., `Relaxed`,
+`Acquire`, `Release`, `SeqCst`). This controls how the CPU and compiler may
+reorder instructions around the atomic operation relative to other memory
+accesses — a subtle but critical detail for correctness in concurrent code. Rust
+inherits this memory model from C++20.[^9][^10]
+
+## Role in the Ecosystem
+
+Atomic types are the **lowest-level building block** for concurrency in Rust.
+Higher-level primitives like `Mutex` and `RwLock` are themselves implemented
+using atomic operations under the hood.[^3]
+<span style="display:none">[^1][^11][^12][^13][^14][^2][^7][^8]</span>
+
+<div align="center">⁂</div>
+
+[^1]: https://doc.rust-lang.org/std/sync/atomic/
+
+[^2]: https://doc.rust-lang.org/nomicon/atomics.html
+
+[^3]: https://marabos.nl/atomics/atomics.html
+
+[^4]: https://leapcell.io/blog/rust-atomics-explained
+
+[^5]: https://whenderson.dev/blog/implementing-atomics-in-rust/
+
+[^6]: https://stackoverflow.com/questions/53587866/what-is-the-difference-between-this-atomic-rust-code-and-its-non-atomic-coun
+
+[^7]: https://www.reddit.com/r/rust/comments/hskm11/having_a_hard_time_understanding_atomic/
+
+[^8]: http://blog.rustbr.org/entendendo-atomicos/
+
+[^9]: https://cfsamsonbooks.gitbook.io/explaining-atomics-in-rust
+
+[^10]: https://www.youtube.com/watch?v=rMGWeSjctlY
+
+[^11]: https://blog.rustbr.org/en/understanding-atomics/
+
+[^12]: https://www.reddit.com/r/rust/comments/1ksqo9i/mastering_rust_atomic_types_a_guide_to_safe/
+
+[^13]: https://dev.to/leapcell/rust-concurrency-atomic-explained-58cl
+
+[^14]: https://rust-lang.guide/guide/learn-async-rust/rust-atomics-and-locs.html
+
+[^15]: https://web.mit.edu/rust-lang_v1.25/arch/amd64_ubuntu1404/share/doc/rust/html/std/sync/atomic/index.html
diff --git a/memory_safety/borrowing_rules.md b/memory_safety/borrowing_rules.md
@@ -0,0 +1,72 @@
+# Borrowing Rules
+
+Rust enforces two hard rules at compile time:
+
+1. You can have any number of immutable references (&T) at the same time
+
+2. You can have only one mutable reference (&mut T) at a time — and when a
+ mutable reference is active, no immutable references may exist either
+
+### String example:
+
+#### Question: Why does rust only allow one muatble reference at a time?
+
+A String in Rust is heap-allocated and stores a pointer, a length, and a
+capacity internally. If you hold an immutable reference r1 to a String and then
+mutate it via a second mutable reference r2 — say by pushing characters — the
+String may reallocate its internal buffer to a new heap address. At that point,
+r1 would be pointing to freed memory, which is a dangling pointer and causes
+[undefined behavior](/memory_safety/segfault.md). Rust prevents this entirely at
+compile time.
+
+```rust
+let mut s = String::from("hello");
+
+let r1 = &s; // ✅ immutable borrow
+let r2 = &s; // ✅ another immutable borrow — fine!
+let r3 = &mut s; // ❌ compile error: can't borrow mutably while r1/r2 exist
+```
+
+### Furthermore:
+
+The single mutable reference rule actually solves several classes of bugs
+simultaneously:
+
+- Dangling pointers — mutation causes reallocation, invalidating old references
+ (your example)
+
+- Data races — two threads mutating the same memory simultaneously leads to
+ unpredictable results
+
+- [Iterator invalidation](iterator_invalidation.md) — modifying a collection
+ while iterating over it (a common bug in C++ and Java)
+
+- Compiler optimisation safety — the compiler can safely optimise and even
+ vectorise (SIMD) code because it knows no two mutable aliases can overlap
+
+## Mutable Reference [Lifetime](/memory_safety/lifetime.md) Is Scoped
+
+The borrow checker is smart enough to track when a reference's last use is, not
+just its scope. This means a mutable reference can be created once the immutable
+ones are no longer actively used:
+
+```rust
+let mut s = String::from("hello");
+
+let r1 = &s;
+let r2 = &s;
+println!("{r1}, {r2}"); // r1 and r2 last used here — they're effectively dropped
+
+let r3 = &mut s; // ✅ now safe, no active immutable refs
+r3.push_str(", world");
+```
+
+This feature is called
+[Non-Lexical Lifetimes (NLL)](/memory_safety/non_lexical_lifetimes.md) and was
+introduced to make Rust's borrow checker less restrictive while keeping it safe.
+
+## References:
+
+[Why Does Rust Enforce the “One Mutable or Many Immutable References” Rule in Single-Threaded Programs?](https://users.rust-lang.org/t/why-does-rust-enforce-the-one-mutable-or-many-immutable-references-rule-in-single-threaded-programs/121017/2)
+
+[The Problem With Single-threaded Shared Mutability](https://manishearth.github.io/blog/2015/05/17/the-problem-with-shared-mutability/)
diff --git a/memory_safety/box.md b/memory_safety/box.md
@@ -0,0 +1,38 @@
+# Box
+
+All values in Rust are stack allocated by default. Box is basically a pointer to
+a heap allocated value. When Box<T> goes out of scope, its destructor is called,
+freeing up the memory on the heap.
+
+### Question: when / why to do box in rust?
+
+We use Box whenever we need to put something on the heap and have a pointer to
+it. Because by default Rust values live on the stack, and their size must be
+known at compile time.
+
+## Use Box<T> when:
+
+- Recursive types - without Box the type woul dhave inifnite size:
+
+```rust
+enum List { Cons(i32, Box<List>), Nil }
+```
+
+- Types whose size can’t be known at compile time but you need a sized handle:
+
+Trait objects like `Box<dyn Error>` or `Box<dyn MyTrait>` put the unknown‑sized
+value on the heap and give you a pointer of known size so functions can return
+or store them easily.
+
+- Moving large data efficiently:
+
+If you have a big struct and want to move ownership around without copying all
+its bytes on the stack, you can store it in a Box and move just the pointer
+(cheap copy).
+
+## Don't need Box
+
+For example, Vec<T>, String, HashMap<K, V>, etc., these already have a pointer,
+so dont need Box for them.
+
+[Reference](https://rustwiki.org/en/rust-by-example/std/box.html)
diff --git a/memory_safety/drop.md b/memory_safety/drop.md
@@ -0,0 +1,104 @@
+# Drop
+
+Rust’s Drop trait lets a type run custom cleanup code when a value is about to
+go out of scope.
+
+The drop method is called automatically when an object goes out of scope. It is
+part of the std library in Rust. Box, Vec, String, File, and Process are some
+examples of types that implement the Drop trait to free resources.
+
+## Examples
+
+There are three way to manage drop:
+
+1. **Inner block (idiomatic, no extra types)**
+
+Put everything that holds the DB in a nested scope. When the block ends, service
+and db are dropped automatically, then you delete the file.
+
+```rust
+let path = create_test_db_path();
+{
+ let db = Arc::new(open_test_db(&path)); let
+ service = setup_test_service(db.clone()).await.unwrap();
+ // … test body …
+}
+cleanup_test_db(path);
+```
+
+No manual drop—the closing } defines when the handles go away.
+
+2. **RAII “temp file” guard**
+
+Use a small type that implements Drop and calls remove_file in drop, and declare
+it before the `Arc<DB>` so the DB is dropped first (last declared → dropped
+first):
+
+```rust
+let path = create_test_db_path();
+let _tmp = TempRedb::new(path.clone()); // deletes path on Drop
+let db = Arc::new(open_test_db(&path));
+// … when function/block ends: db drops, then _tmp deletes the file
+```
+
+Same idea as tempfile::TempPath / NamedTempFile in the tempfile crate: the guard
+runs cleanup when it goes out of scope, after earlier locals are dropped.
+
+3. **Manual cleanup before the end of scope**
+
+```rust
+use std::fs;
+use std::path::Path;
+
+struct TestDb {
+ path: String,
+}
+
+impl TestDb {
+ fn new(path: &str) -> Self {
+ println!("Opening DB at {}", path);
+ Self { path: path.to_string() }
+ }
+}
+
+impl Drop for TestDb {
+ fn drop(&mut self) {
+ println!("Dropping TestDb for {}", self.path);
+ // In real code: close DB connection, etc.
+ }
+}
+
+fn cleanup_db(path: &str) {
+ if Path::new(path).exists() {
+ fs::remove_file(path).unwrap();
+ println!("Cleaned up DB file: {}", path);
+ }
+}
+
+fn main() {
+ let path = "test.db";
+
+ let db = TestDb::new(path); // declared first → dropped last
+ let _guard = std::fs::File::open(path).unwrap_or_else(|_| {
+ // create dummy file for demo
+ fs::write(path, "").unwrap();
+ fs::File::open(path).unwrap()
+ });
+
+ // ... test body ...
+
+ // Explicitly drop `db` *before* `_guard`, even though `db` was declared first
+ drop(db); // ← explicit drop call
+
+ // Now safe to clean up file (since db is gone)
+ cleanup_db(path);
+
+ // `_guard` is dropped here automatically at end of scope
+}
+```
+
+────────────────────────────────────────
+
+Summary: Prefer a block for tests; use a guard (or tempfile) if you want cleanup
+bundled in one place. You only need explicit drop when you want a non-default
+order without restructuring scopes.
diff --git a/memory_safety/index.md b/memory_safety/index.md
@@ -0,0 +1,14 @@
+# Memory Safety
+
+Notes on Rust's memory safety model, covering ownership, borrowing, and
+concurrency primitives.
+
+## Topics
+
+- **Ownership & Borrowing** — borrowing rules, non-lexical lifetimes, shared and
+ exclusive references, interior mutability
+- **Smart Pointers** — `Box`, `Arc`, and the `Drop` trait
+- **Concurrency** — `Mutex`, `RwLock`, atomics, spinlocks, `Send`/`Sync` traits,
+ and synchronization/concurrency primitives
+- **Memory Model** — stack vs. heap, segfaults, iterator invalidation
+- **Practical Patterns** — connection pooling, PostgreSQL connections
diff --git a/memory_safety/interior_mutability.md b/memory_safety/interior_mutability.md
@@ -0,0 +1,50 @@
+# Interior Mutability
+
+Interior mutability in Rust is a pattern that lets you change the internal state
+of a value through a shared (immutable) reference, something that normal
+borrowing rules do not allow.
+
+## Core idea
+
+- Normally, &T means “shared, read-only” and &mut T means “unique, writable”.
+
+- A type has interior mutability if you can call methods that mutate its
+ internal data even when you only have &self.
+
+- This is implemented by special wrapper types that enforce safety at runtime
+ instead of purely at compile time (often via unsafe inside but a safe public
+ API).
+
+## Common types used
+
+- Rust’s standard library provides several types that rely on interior
+ mutability:
+
+- Cell<T> and RefCell<T> in std::cell for single-threaded code.
+
+- Mutex<T> and RwLock<T> in std::sync for synchronized mutation across threads.
+
+- Atomic types like AtomicUsize for lock-free concurrent mutation.
+
+## Example
+
+A simple example is a struct that keeps a usage counter even when you only pass
+around
+
+```rust
+use std::cell::Cell;
+
+struct Counter {
+ value: Cell<u32>,
+}
+
+impl Counter {
+ fn inc(&self) {
+ self.value.set(self.value.get() + 1);
+ }
+}
+```
+
+Here, inc takes &self but still mutates the internal value.
+
+[Reference](https://mara.nl/atomics/basics.html#interior-mutability)
diff --git a/memory_safety/iterator_invalidation.md b/memory_safety/iterator_invalidation.md
@@ -0,0 +1,54 @@
+# Iterator Invalidation
+
+Iterator invalidation is a bug where an iterator becomes invalid because the
+underlying collection it references is modified during iteration. Rust prevents
+this at compile time through its borrow checker, unlike languages like C++ where
+it causes undefined behaviour or crashes at runtime.
+
+When you iterate over a collection (e.g., a Vec), the iterator holds a reference
+to that collection's memory. If the collection is modified — say, by pushing a
+new element — the Vec may need to reallocate its memory to grow, freeing the old
+memory and leaving the iterator pointing to a dangling address. At best this
+causes a crash (segfault); at worst it silently corrupts memory.
+
+Rust's borrow checker enforces that you cannot hold an immutable borrow and a
+mutable borrow at the same time. When you call v.iter(), Rust creates an
+immutable borrow on the entire collection. Any attempt to call v.push() or
+v.remove() during the loop requires a mutable borrow — which the compiler
+outright refuses to compile:
+
+```rust
+fn main() {
+ let mut vec = vec![1, 2, 3, 4, 5];
+ for elem in &vec {
+ vec.push(elem * 2); // COMPILE ERROR: cannot borrow `vec` as mutable
+ // because it is also borrowed as immutable
+ }
+}
+```
+
+The error is caught at compile time, not at runtime.
+
+## What You _Can_ Do Safely
+
+| Pattern | Allowed? | Why |
+| :--------------------------------------- | :------- | :----------------------------------------- |
+| `v.iter()` + read elements | ✅ | Multiple immutable borrows are fine |
+| `v.iter_mut()` + modify each element | ✅ | One mutable borrow, no structural changes |
+| `v.iter()` + `v.push()` in the same loop | ❌ | Immutable + mutable borrow conflict |
+| Collect indices, then modify after loop | ✅ | Iterator is dropped before mutation begins |
+
+A common safe workaround is to use `retain` or collect the changes you need into
+a separate `Vec`, then apply them after the loop ends — by which point the
+iterator has been dropped and the borrow is released.
+
+## It's effectively threaded
+
+> Aliasing with mutability in a sufficiently complex, single-threaded program is
+> effectively the same thing as accessing data shared across multiple threads
+> without a lock
+
+> My intuition is that code far away from my code might as well be in another
+> thread, for all I can reason about what it will do to shared mutable state.
+
+[Reference](http://manishearth.github.io/blog/2015/05/17/the-problem-with-shared-mutability/)
diff --git a/memory_safety/mutex.md b/memory_safety/mutex.md
@@ -0,0 +1,207 @@
+# Mutex
+
+Mutex is the most commonly used tool for sharing (mutable) data between threads.
+Mutex is short for "Mutural Exclusion". It uses UnsafeCell under the hood, and
+provides a safe interface MutexGuard for threads to access the data with
+"Interior Mutability". It is a thread-safe wrapper around some data T that
+ensures only one thread can access that data at a time
+
+```
+ +-----------------------------+
+ | Arc<Mutex<T>> |
+ | (shared across threads) |
+ | |
+ | +---------------------+ |
+ | | Mutex<T> | |
+ | | (lock + the data) | |
+ | | | |
+ | | +-------------+ | |
+ | | | value T | | |
+ | | +-------------+ | |
+ | +----------^----------+ |
+ +--------------|--------------+
+ |
+ clones of Arc |
+ (Arc::clone(&mutex)) |
+ sent to threads |
+ |
+ +------------------+------------------+
+ | | |
+ v v v
++----------------+ +----------------+ +----------------+
+| thread 1 | | thread 2 | | thread 3 |
+| | | | | |
+| lock() | | lock() | | lock() |
+| | | | | | | | |
+| v | | v | | v |
+| MutexGuard<T> | | MutexGuard<T> | | MutexGuard<T> |
+| (exclusive | | (exclusive | | (exclusive |
+| access) | | access) | | access) |
++----------------+ +----------------+ +----------------+
+
+Only ONE MutexGuard can exist at a time:
+- When a thread calls lock(), it blocks until it gets a MutexGuard.
+- While a thread holds the guard, others wait.
+- When the guard is dropped (goes out of scope), the lock is released.
+```
+
+## Basic Example:
+
+```rust
+use std::sync::Mutex;
+
+fn main() {
+ let my_mutex = Mutex::new(5);
+
+ {
+ let mut guard = my_mutex.lock().unwrap();
+ *guard = 6; // modify protected data
+ } // guard dropped here, mutex unlocked
+
+ println!("{:?}", my_mutex); // prints: Mutex { data: 6 }
+}
+```
+
+## Poisoning
+
+- If a thread panics while holding the lock, the mutex becomes poisoned to
+ signal that the data may be in an inconsistent state.
+
+- After that, `.lock()` returns an `Err(PoisonError)` instead of
+ `Ok(MutexGuard)`; you often handle this with `unwrap()` (propagating the
+ panic) or `unwrap_or_else` to recover.
+
+### Question: whats the difference between Arc and Mutex?
+
+Arc and Mutex solve different problems: Arc gives shared ownership of data
+across threads, while Mutex controls exclusive access to data (usually for
+mutation) so only one thread touches it at a time.
+
+## How Arc and Mutex are used together
+
+A very common pattern in Rust is Arc<Mutex<T>>, where:
+
+- Arc lets many threads share ownership of the same Mutex<T>.
+
+- Mutex ensures that each access to T is exclusive, so concurrent mutation is
+ safe.
+
+## Quick example intuition
+
+If you want several threads to read the same configuration that never changes,
+you typically use Arc<Config>.
+
+If you want several threads to increment a shared counter, you typically use
+Arc<Mutex<u32>> so they can all own the counter and take turns mutating it
+safely.
+
+## On using `std::sync::Mutex` and `tokio::sync::Mutex`
+
+> Note that std::sync::Mutex and not tokio::sync::Mutex is used to guard the
+> HashMap. A common error is to unconditionally use tokio::sync::Mutex from
+> within async code. An async mutex is a mutex that is locked across calls to
+> .await.
+
+> A synchronous mutex will block the current thread when waiting to acquire the
+> lock. This, in turn, will block other tasks from processing. Switching to
+> tokio::sync::Mutex will cause the task to yield control back to the executor,
+> but this will usually not help with performance as the asynchronous mutex uses
+> a synchronous mutex internally.
+
+> As a rule of thumb, using a synchronous mutex from within asynchronous code is
+> fine as long as contention remains low and the lock is not held across calls
+> to .await.
+
+### Question: What does “Locked across calls to .await” mean?
+
+It means: you acquired the mutex lock before an `.await`, and you still hold
+that lock while the future is suspended at that `.await` and possibly resumed
+later on another poll.
+
+In async Rust, every .await is a point where your function can be suspended and
+other tasks can run on the same thread.
+
+Code that hold lock across the await:
+
+```rust
+async fn f(m: &std::sync::Mutex<Data>) {
+ let mut guard = m.lock().unwrap(); // lock acquired here
+
+ do_something_async().await; // <-- await while still holding `guard`
+
+ guard.value += 1; // lock released only when `guard` is dropped
+}
+```
+
+the Mutex remains locked for the entire time between taking guard and dropping
+it, including the whole period when the function is paused at
+do_something_async().await. The lock is “held across the await”.
+[How to Use async Rust Without Blocking the RUntime](https://oneuptime.com/blog/post/2026-01-07-rust-async-without-blocking/view)
+
+By contrast, code that does not hold the lock across an .await acquires and
+releases it entirely before the first .await:
+
+```rust
+async fn g(m: &std::sync::Mutex<Data>) {
+ {
+ let mut guard = m.lock().unwrap();
+ guard.counter += 1;
+ // lock is dropped at the end of this block
+ } // <--- lock released here
+
+ do_something_async().await; // no lock held during await
+}
+```
+
+Here the critical section is synchronous and short, so using std::sync::Mutex is
+fine because the lock is never held while the task is suspended at .await.
+[Sync mutex in async program](https://users.rust-lang.org/t/sync-mutex-in-async-program/66118)
+
+A quick way to think about it:
+
+- “Held/locked across `.await`” = the lifetime of your `MutexGuard` spans over
+ an `.await` expression.
+
+- “Not held across `.await`” = you drop the guard (end its scope) before you hit
+ any `.await`.
+
+However, the `tokio::sync::Mutex` is designed to be held across `await` points:
+
+```rust
+async fn g(m: &tokio::sync::Mutex<Data>) {
+ let mut guard = m.lock().await; // yields if needed, no thread block
+ do_something_async().await; // still holding the lock
+ guard.counter += 1;
+} // lock released when guard is dropped
+```
+
+is legal in terms of the runtime: you’re not blocking a worker thread while
+waiting for I/O, other tasks can still run.
+
+So “locked across calls to .await” means:
+
+- With std::sync::Mutex: strongly discouraged; can stall threads and deadlock.
+
+- With tokio::sync::Mutex: allowed and supported; the runtime knows how to park
+ and wake tasks that are waiting on the async mutex.
+
+#### When to choose which
+
+- Use std::sync::Mutex in async code when:
+
+ - You only lock for very short, synchronous work.
+
+ - You always drop the guard before any .await.
+
+- Use tokio::sync::Mutex when:
+
+ - You really need to keep the lock across one or more .awaits, or
+
+ - The critical section involves async operations that can’t be easily
+ refactored to be purely synchronous.
+
+```
+## Reference
+
+[Spinlock Considered Harmful](https://matklad.github.io/2020/01/02/spinlocks-considered-harmful.html)
+```
diff --git a/memory_safety/rwlock_pattern.md b/memory_safety/rwlock_pattern.md
@@ -0,0 +1,82 @@
+# RWLock Pattern
+
+The RWLock (Readers-Writer Lock) pattern is a concurrency synchronization
+primitive that allows multiple threads to read shared data simultaneously, but
+requires exclusive access for any write operation. It's ideal for scenarios
+where reads are frequent and writes are rare, as it avoids the unnecessary
+serialization that a plain Mutex would impose.
+
+## Core Rules
+
+- Multiple readers can hold the lock at the same time — reads are non-exclusive
+
+- Only one writer can hold the lock at a time — writes are exclusive
+
+- A write lock blocks all readers and other writers until it is released
+
+- Once a writer is active, new readers must wait
+
+## Rust RwLock Example
+
+Rust provides std::sync::RwLock<T> in its standard library. Here's a practical
+example with multiple reader threads and one writer thread:
+
+```rust
+use std::sync::{Arc, RwLock};
+use std::thread;
+
+fn main() {
+ // Wrap data in Arc<RwLock<T>> for shared ownership across threads
+ let data = Arc::new(RwLock::new(0u32));
+
+ // --- Writer thread: exclusive access ---
+ let data_writer = Arc::clone(&data);
+ let writer = thread::spawn(move || {
+ let mut w = data_writer.write().unwrap(); // blocks until no readers/writers
+ *w += 42;
+ println!("Writer set value to: {}", *w);
+ }); // write lock dropped here
+
+ writer.join().unwrap();
+
+ // --- Multiple reader threads: concurrent access ---
+ let reader_handles: Vec<_> = (0..4).map(|i| {
+ let data_reader = Arc::clone(&data);
+ thread::spawn(move || {
+ let r = data_reader.read().unwrap(); // multiple readers can hold this simultaneously
+ println!("Reader {} sees value: {}", i, *r);
+ })
+ }).collect();
+
+ for handle in reader_handles {
+ handle.join().unwrap();
+ }
+}
+```
+
+### Output (reader order may vary):
+
+```text
+Writer set value to: 42
+Reader 0 sees value: 42
+Reader 2 sees value: 42
+Reader 1 sees value: 42
+Reader 3 sees value: 42
+```
+
+## When to Use vs. Mutex
+
+RwLock is only better than Mutex when you have many concurrent readers competing
+at the same time. For a single reader or low concurrency, an uncontended RwLock
+is no faster than a Mutex because readers still acquire the lock and update
+internal state. A good rule of thumb: if your workload is read-heavy (e.g., 90%+
+reads) with rare writes, RwLock gives a meaningful throughput boost.
+
+## Watch Out: Lock Poisoning
+
+In Rust, if a thread panics while holding a write lock, the RwLock becomes
+poisoned. Subsequent calls to .read() or .write() will return an Err, which you
+handle via .unwrap() or .into_inner(). This is a safety feature to prevent
+access to potentially corrupt data.
+
+[RWLock](https://dev-doc.rust-lang.org/beta/std/sync/struct.RwLock.html)
diff --git a/memory_safety/segfault.md b/memory_safety/segfault.md
@@ -0,0 +1,88 @@
+# Segfault
+
+A segmentation fault (segfault) is a runtime error that occurs when a program
+tries to access a memory location it is not permitted to access — such as
+reading or writing outside its allocated memory bounds. The operating system
+catches this illegal access and terminates the offending process, sending a
+SIGSEGV signal on Unix-like systems or raising a STATUS_ACCESS_VIOLATION
+exception on Windows.
+
+## How It Happens
+
+Program memory is divided into segments — text (instructions), data (global
+variables), stack (local variables), and heap (dynamically allocated memory). A
+segfault occurs when a reference falls outside the segment where a variable
+resides, or when a write is attempted to a read-only segment. The most common
+causes are:
+
+- Null pointer dereference — accessing memory at address 0x0
+
+- Buffer overflow — reading/writing past the end of an array
+
+- Dangling pointer — using a pointer to memory that has already been freed
+
+- Stack overflow — infinite recursion exhausting the stack
+
+## Rust and Segfaults
+
+Rust's ownership and borrow checker system is specifically designed to eliminate
+segfaults in safe code at compile time. According to the Rust team, a Rust
+program can only segfault in two scenarios: you used unsafe code that violates
+memory safety guarantees, or the Rust compiler itself has a bug.
+
+### Triggering a segfault with unsafe Rust
+
+The most direct way is dereferencing a raw null or invalid pointer inside an
+unsafe block:
+
+```rust
+fn main() {
+ // Dereference an invalid memory address — instant segfault
+ unsafe { *(0x1 as *mut i32) = 1 };
+}
+```
+
+This writes to memory address 0x1, which is not mapped, causing the OS to send
+SIGSEGV.
+
+### Stack overflow via infinite recursion
+
+```rust
+fn recurse() {
+ recurse(); // infinite recursion → stack overflow → crash
+}
+
+fn main() {
+ recurse();
+}
+```
+
+Rust's runtime catches this and typically raises SIGABRT with a "thread has
+overflowed its stack" message rather than a raw SIGSEGV, but it is the same
+underlying mechanism.
+
+### Writing to read-only memory via unsafe
+
+```rust
+fn main() {
+ let x: &str = "hello"; // stored in read-only memory
+ let ptr = x.as_ptr() as *mut u8;
+ unsafe {
+ *ptr = b'H'; // writing to read-only segment → segfault
+ }
+}
+```
+
+## Safe Rust vs. Unsafe Rust
+
+| Scenario | Safe Rust | `unsafe` Rust |
+| :----------------------- | :------------------------------------ | :----------------------------------- |
+| Null pointer dereference | Impossible — `Option<T>` used instead | Possible with raw pointers |
+| Buffer overflow | Panics with bounds check | Possible with raw pointer arithmetic |
+| Dangling pointer | Prevented by borrow checker | Possible |
+| Stack overflow | Handled gracefully (abort) | Same behaviour |
+
+The key takeaway is that safe Rust **prevents segfaults by design** — the borrow
+checker enforces memory safety rules at compile time that languages like C/C++
+leave to the programmer. When you do need low-level control, `unsafe` blocks opt
+out of these guarantees and reintroduce the risk.
diff --git a/memory_safety/shared_and_exclusive_reference.md b/memory_safety/shared_and_exclusive_reference.md
@@ -0,0 +1,11 @@
+# Shared and Exclusive Reference
+
+# Shared vs. Exclusive References in Rust
+
+In Rust, there are two kinds of references:
+
+- `&T` — a shared reference: multiple parties can hold one at the same time, but
+ mutation is normally forbidden
+
+- `&mut T` — an exclusive reference: only one party can hold it, and mutation is
+ allowed
diff --git a/memory_safety/spinlock.md b/memory_safety/spinlock.md
@@ -0,0 +1,114 @@
+# Spinlock Mechanism
+
+A spinlock works by repeatedly "spinning" — checking an atomic `locked` flag in
+a tight loop until it successfully flips it from `false` to `true`. The key
+operations are:[^1]
+
+- **`lock()`** — atomically swaps `locked` from `false` to `true`. If `locked`
+ is _already_ `true`, it keeps retrying (the "spin")
+- **`unlock()`** — stores `false` back into `locked`, releasing it for another
+ thread
+
+```
+THREAD 1 THREAD 2
+────────────────────────────────────────────────────────────
+
+ ┌──────────────────────────┐
+ │ swap locked │
+ │ false → true (acquire) │
+ └────────────┬─────────────┘
+lock() │
+ ▼
+ ┌──────────────────────────┐ ┌──────────────────────────┐
+ │ access protected data │ │ swap locked │
+ └────────────┬─────────────┘ │ true → true (acquire) │
+ │ └────────────┬─────────────┘
+unlock() │ │ (spinning...)
+ ▼ ┌────────────▼─────────────┐
+ ┌──────────────────────────┐ │ swap locked │
+ │ store false in locked │ │ true → true (acquire) │
+ │ (release) │ └────────────┬─────────────┘
+ └────────────┬─────────────┘ │ (spinning...)
+ │ │
+ │ happens-before │
+ └───────────────────────────────────▼
+ ┌──────────────────────────┐
+ │ swap locked │
+ │ false → true (acquire) │ ← lock() succeeds
+ └────────────┬─────────────┘
+ lock() │
+ ▼
+ ┌──────────────────────────┐
+ │ access protected data │
+ └────────────┬─────────────┘
+ unlock() │
+ ▼
+ ┌──────────────────────────┐
+ │ store false in locked │
+ │ (release) │
+ └──────────────────────────┘
+```
+
+- Thread 1 (left column) acquires the lock immediately (swaps false → true),
+ accesses the protected data, then releases by storing false.
+
+- Thread 2 (right column) spins repeatedly — each swap returns true → true,
+ meaning the lock is still held — until Thread 1's unlock() stores false.
+
+- The happens-before arrow (the pink curved arrow in the original) is shown as
+ the └──────────────────────────────────────────────────────▶ line connecting
+ Thread 1's release to Thread 2's successful acquire. This is the critical
+ guarantee that prevents concurrent data access.
+
+## Why the Second Thread is Blocked
+
+Looking at the diagram, Thread 2 (right column) attempts its own `lock()` call
+but keeps getting `true → true` swaps, meaning it sees the lock is still held.
+It can only proceed when Thread 1's `unlock()` stores `false` — at that point,
+Thread 2's next swap finally succeeds (`false → true`), and _only then_ does it
+access the protected data.[^2]
+
+This is the **happens-before relationship** mentioned in the text: Thread 1's
+`unlock()` (release) _happens before_ Thread 2's successful `lock()` (acquire),
+which guarantees that any memory writes Thread 1 made to the shared data are
+visible to Thread 2.[^3]
+
+## The Concrete Guarantee
+
+| Moment | Thread 1 | Thread 2 |
+| :----- | :----------------------------- | :--------------------------- |
+| T1 | Acquires lock (`false→true`) | — |
+| T2 | Accessing protected data | Spinning (sees `true→true`) |
+| T3 | Releases lock (stores `false`) | Still spinning |
+| T4 | Done | Acquires lock (`false→true`) |
+| T5 | — | Accessing protected data |
+
+The data access blocks (pink/maroon boxes in the diagram) are never active at
+the same time — Thread 2 is stuck spinning until Thread 1 is completely done and
+has unlocked. That's what "can't access the data concurrently" means: the
+spinlock enforces **serial, not parallel** access to that critical section.[^1]
+<span style="display:none">[^10][^11][^4][^5][^6][^7][^8][^9]</span>
+
+<div align="center">⁂</div>
+
+[^1]: https://www.shadecoder.com/topics/a-spinlock-a-comprehensive-guide-for-2025
+
+[^2]: https://wiki.osdev.org/Spinlock
+
+[^3]: https://en.wikipedia.org/wiki/Lock_(computer_science)
+
+[^4]: image.jpg
+
+[^5]: https://www.productteacher.com/quick-product-tips/understanding-mutual-exclusion-mutex
+
+[^6]: https://stackoverflow.com/questions/53919851/how-spinlock-prevents-the-process-to-be-interrupted
+
+[^7]: https://forums.swift.org/t/pitch-synchronous-mutual-exclusion-lock/69889
+
+[^8]: https://student.cs.uwaterloo.ca/~cs350/F19/notes/synchronization-2up.pdf
+
+[^9]: https://student.cs.uwaterloo.ca/~cs350/F20/notes/synchronization-2up.pdf
+
+[^10]: https://de.wikipedia.org/wiki/Spinlock
+
+[^11]: https://spcl.inf.ethz.ch/Teaching/2020-pp/lectures/PP-l17-BeyondLocks.pdf
diff --git a/memory_safety/stack-and-heap.md b/memory_safety/stack-and-heap.md
@@ -0,0 +1,35 @@
+# Stack and Heap
+
+# Stack
+
+## Call Stack
+
+A region of memory the CPU uses automatically to track function calls, memory
+pointers, local variables, and return addresses
+
+## Rust-specific rule
+
+[Borrowing Rules](../memory_safety/borrowing_rules.md)
+
+# Heap
+
+The heap is a region of a computer's memory used for dynamic memory allocation —
+memory that is reserved and released at runtime rather than at compile time. It
+is one of the two main memory areas programs use (alongside the stack), and it
+gives programmers flexible control over how much memory to use and for how long.
+
+## Heap vs. Stack
+
+| Feature | Heap | Stack |
+| :---------------- | :--------------------------- | :-------------------- |
+| Allocation timing | Runtime (dynamic) | Compile time (static) |
+| Size | Large, flexible | Small, fixed |
+| Management | Manual or GC | Automatic (LIFO) |
+| Access scope | Global (anywhere in program) | Local to function |
+| Speed | Slightly slower | Faster |
+| Risk | Memory leaks, fragmentation | Stack overflow |
+
+The stack follows a strict last-in, first-out (LIFO) structure, making it fast
+but limited. The heap is more flexible but requires careful management — failure
+to deallocate memory causes **memory leaks**, where memory becomes permanently
+unavailable.
diff --git a/memory_safety/string_vs_str.md b/memory_safety/string_vs_str.md
@@ -0,0 +1,88 @@
+# String vs Str
+
+Here is how `String` source code look like:
+
+```rust
+#[derive(PartialEq, PartialOrd, Eq, Ord)]
+#[stable(feature = "rust1", since = "1.0.0")]
+#[lang = "String"]
+pub struct String {
+ vec: Vec<u8>,
+}
+```
+
+In Rust, String is implemented as a wrapper around a `Vec<u8>` because it is
+designed to be a dynamically growable, heap-allocated buffer of UTF-8 encoded
+bytes. The standard library leverages the existing `Vec` type to manage the
+string's underlying memory while enforcing specific character encoding rules.
+
+## The Stack Layout
+
+When you create a String, Rust places a small, fixed-size struct on the stack.
+On a 64-bit system, this struct is exactly 24 bytes and consists of three
+machine-word fields:
+
+- **Pointer**: A reference pointing to the memory address on the heap where the
+ actual UTF-8 text data is stored.
+
+- **Length**: The amount of memory (in bytes) currently being used by the
+ string's text.
+
+- **Capacity**: The total amount of heap memory currently allocated for the
+ string.
+
+Because this control block is small, moving a String from one variable to
+another is incredibly fast. Rust only copies these 24 bytes of metadata on the
+stack rather than copying the actual text data on the heap.
+
+## String memory allocation
+
+- **p** is the address of the String object itself, so text keeps the same p the
+ whole time because it is the same variable.
+
+- **ptr** is the address of the actual character buffer.
+
+- **len** is the current string size.
+
+- **capacity** is how much buffer space is available before another growth is
+ needed.
+
+```rust
+fn main() {
+ let mut text = String::new();
+ println!("p: {:p} ptr: {:<15?} len: {:2}, capacity: {:2} '{}'", &text, text.as_ptr(), text.len(), text.capacity(), text);
+
+ text.push_str("Hello foo!");
+ println!("p: {:p} ptr: {:<15?} len: {:2}, capacity: {:2} '{}'", &text, text.as_ptr(), text.len(), text.capacity(), text);
+
+ text.replace_range(6..9, "bar");
+ println!("p: {:p} ptr: {:<15?} len: {:2}, capacity: {:2} '{}'", &text, text.as_ptr(), text.len(), text.capacity(), text);
+
+ let temp = String::from("The black cat");
+ println!("{temp}");
+ println!("p: {:p} ptr: {:<15?} len: {:2}, capacity: {:2} '{}'", &temp, temp.as_ptr(), temp.len(), temp.capacity(), temp);
+
+ text.replace_range(6..9, "qqrq");
+ println!("p: {:p} ptr: {:<15?} len: {:2}, capacity: {:2} '{}'", &text, text.as_ptr(), text.len(), text.capacity(), text);
+
+ text.replace_range(6..9, "123456");
+ println!("p: {:p} ptr: {:<15?} len: {:2}, capacity: {:2} '{}'", &text, text.as_ptr(), text.len(), text.capacity(), text);
+
+ text.replace_range(6..9, "12345678901234567890");
+ println!("p: {:p} ptr: {:<15?} len: {:2}, capacity: {:2} '{}'", &text, text.as_ptr(), text.len(), text.capacity(), text);
+
+}
+```
+
+```rust
+p: 0x7ffdcc572770 ptr: 0x1 len: 0, capacity: 0 ''
+p: 0x7ffdcc572770 ptr: 0x5de087754ba0 len: 10, capacity: 10 'Hello foo!'
+p: 0x7ffdcc572770 ptr: 0x5de087754ba0 len: 10, capacity: 10 'Hello bar!'
+The black cat
+p: 0x7ffdcc573260 ptr: 0x5de087754bc0 len: 13, capacity: 13 'The black cat'
+p: 0x7ffdcc572770 ptr: 0x5de087754ba0 len: 11, capacity: 20 'Hello qqrq!'
+p: 0x7ffdcc572770 ptr: 0x5de087754ba0 len: 14, capacity: 20 'Hello 123456q!'
+p: 0x7ffdcc572770 ptr: 0x5de087754be0 len: 28, capacity: 40 'Hello 12345678901234567890q!'
+```
+
+[String and memory allocation](https://rust.code-maven.com/strings-and-memory-allocation)
diff --git a/memory_safety/vec.md b/memory_safety/vec.md
@@ -0,0 +1,83 @@
+# Vec
+
+# `Vec<u8>`
+
+Getting the size of a Vec<u8> in Rust is straightforward — since each element is
+exactly 1 byte, .len() gives you the byte count directly.
+
+## `len()` vs `capacity()`
+
+A `Vec<u8>` internally holds three things: a pointer to heap memory, a length,
+and a capacity.
+
+- `len()` — the number of actual bytes of data present. This is the file size.
+
+- `capacity()` — the total memory reserved on the heap, which may be larger than
+ len() to avoid frequent reallocations as the vector grows. This is an internal
+ memory management detail.
+
+ For example, a `Vec<u8>` with 500 bytes of real data might have a capacity of
+ 512 — those extra 12 bytes are pre-allocated empty slots Rust reserved
+ speculatively. They contain no real data.
+ [rust-lang](https://doc.rust-lang.org/std/vec/struct.Vec.html)
+
+```
+ Stack Heap
+───────────────── ──────────────────────────────────────────────────
+┌───────────────┐ ┌─────────────────────────────┬──────────────────┐
+│ ptr │──────────► │ REAL DATA (500 bytes) │ EMPTY (12 bytes)│
+├───────────────┤ │ │ │
+│ len = 500 │ │ [0x01][0xFF][0x3A]... │ [ ][ ][ ]... │
+├───────────────┤ │ │ │
+│ cap = 512 │ │ ◄────── len = 500 ────────►│◄─── cap-len ────►│
+└───────────────┘ └─────────────────────────────┴──────────────────┘
+ ◄──────────────── cap = 512 ──────────────────────►
+```
+
+- `ptr` points to the start of the heap block
+
+- `len = 500` marks how far into the block contains real, valid data — this is
+ your file size
+
+- `cap = 512` is the total reserved memory; the trailing 12 bytes are allocated
+ but uninitialised and ignored
+ [stackoverflow](https://stackoverflow.com/questions/54889521/whats-the-difference-between-len-and-capacity)
+
+
+- When you call `.len()`, you get `500` — exactly the bytes fetched from the
+ database, nothing more
+
+### Question: Why `512`?
+
+512 was just an illustrative, round-number example, not a special Rust rule.
+
+In reality:
+
+- Rust does not guarantee “500 bytes → capacity 512”. The allocator chooses how
+ much memory to give you, and `Vec` typically grows by some factor (often ~2×),
+ but the exact value is an implementation detail.
+
+- When a `Vec` needs more space (because `len == capacity` and you push again),
+ it reallocates to a larger capacity to reduce how often it has to reallocate
+ in the future. That larger capacity might be 512, 640, 1000, etc., depending
+ on the previous capacity and the growth strategy.
+
+- So “500 data, 512 capacity” was just to show the idea: **capacity ≥ len**, and
+ the extra part is reserved space for future pushes, not real data.
+
+### Question: Whats the difference between "padding" and "pre-allocated empty slot"?
+
+“Padding” is a term we use for extra bytes inserted inside or between fields of
+a struct to satisfy alignment requirements of the CPU (e.g., to align a u64 on
+an 8‑byte boundary). However, in a `Vec<T>`:
+
+- `len` is how many initialized elements you have.
+
+- `capacity` is how many elements the heap allocation can hold.
+
+- The bytes between `le`n and `capacity` are logically **uninitialized storage
+ for future elements**, not alignment padding.
+
+ So for a `Vec<u8>` with `len = 500` and `capacity = 512`, those 12 bytes are
+ just unused, reserved space that Rust can fill later if you push more bytes;
+ they’re not considered padding in the usual memory-layout sense.
diff --git a/memory_safety/vec_vs_hashset.md b/memory_safety/vec_vs_hashset.md
@@ -0,0 +1,77 @@
+# Vec vs HashSet
+
+[HashSet - Rust By Example](https://doc.rust-lang.org/rust-by-example/std/hash/hashset.html)
+
+Sets have 4 primary operations (all of the following calls return an iterator):
+
+- union: get all the unique elements in both sets.
+
+- difference: get all the elements that are in the first set but not the second.
+
+- intersection: get all the elements that are only in both sets.
+
+- symmetric_difference: get all the elements that are in one set or the other,
+ but not both.
+
+### Question: Is HashSet O(1) and Vec O(n) in rust?
+
+In Rust, the O(1) versus O(n) comparison between a `HashSet` and a `Vec`
+primarily refers to lookup operations, such as checking if a collection contains
+a specific element. While a `HashSet` provides O(1) average time complexity for
+lookups, a `Vec` requires O(n) time because it must perform a linear search.
+
+| Operation | `Vec<T>` | `HashSet<T>` |
+| :---------------------- | :----------------------------------------------- | :--------------------------- |
+| **Lookup (`contains`)** | O(n) [^7] | O(1) average [^7][^9] |
+| **Add Element** | O(1) amortized (`push`) [^8] | O(1) average (`insert`) [^9] |
+| **Remove Element** | O(n) (`remove`), O(1) (`pop`/`swap_remove`) [^8] | O(1) average (`remove`) |
+
+## Lookup Performance
+
+Checking if an element exists using `contains()` is where the O(n) vs O(1)
+difference is most prominent. A `HashSet` computes a hash to instantly find the
+element's bucket, offering O(1) average lookup time regardless of the
+collection's size. In contrast, a `Vec` must iterate through its elements one by
+one until a match is found, resulting in O(n) time complexity where the search
+time grows linearly with the number of elements.[^3][^7][^10]
+
+## Insertion and Removal
+
+Appending an element to a `Vec` using `push()` operates in amortized O(1) time,
+though occasional capacity reallocations can cause brief performance spikes.
+`HashSet` insertions are also O(1) on average, but the operation carries a
+higher "constant factor" because computing the hash and handling potential
+bucket collisions requires more CPU work than simply placing an item at the end
+of a vector. For removals, `HashSet` operates in O(1) average time, whereas
+removing an element from a specific index in a `Vec` using `remove()` is O(n)
+because it requires shifting all subsequent elements to fill the gap.[^8][^10][^3]
+
+[^1]: https://users.rust-lang.org/t/hashset-foo-vs-vec-foo-as-a-key/103281/2
+
+[^2]: https://users.rust-lang.org/t/iterating-through-hashmap-approx-twice-the-time-of-vec/75964/4
+
+[^3]: https://stackoverflow.com/questions/72877598/hashset-is-slower-than-bruteforce-in-rust
+
+[^4]: https://stackoverflow.com/questions/3185226/huge-performance-difference-between-vector-and-hashset
+
+[^5]: https://wkaisertexas.github.io/blog/hashmaps-versus-vectors/
+
+[^6]: https://stackoverflow.com/questions/39803237/build-hashset-from-a-vector-in-rust
+
+[^7]: https://zenn.dev/iriko/articles/d06fdc5a3de863
+
+[^8]: https://oneuptime.com/blog/post/2026-02-01-rust-collections/view
+
+[^9]: https://dev.to/alexmercedcoder/working-with-collections-in-rust-a-comprehensive-guide-3c9f
+
+[^10]: https://nindalf.com/posts/optimising-rust/
+
+[^11]: https://users.rust-lang.org/t/vec-vs-hashmap-vs-hashset-for-unique-named-items/106615
+
+[^12]: https://users.rust-lang.org/t/question-about-hashset/7779/2
+
+[^13]: https://www.reddit.com/r/rust/comments/1c6h18a/hashset_method_slower_than_naive_method_when/
+
+[^14]: https://doc.rust-lang.org/rust-by-example/std/hash/hashset.html
+
+[^15]: https://users.rust-lang.org/t/idiomatic-way-to-get-difference-between-two-vecs/48396
diff --git a/networking/cache.md b/networking/cache.md
@@ -0,0 +1,5 @@
+# Cache
+
+## Three types of web cache
+
+<img src="/assets/cache.png" alt="Server Setup" width="100%">
diff --git a/networking/ccna.md b/networking/ccna.md
@@ -0,0 +1,191 @@
+# CCNA Exam Topics
+
+1.0 Networking Fundamentals
+
+```
+1.1 Explain the role and function of network components
+1.1.a Routers
+1.1.b Layer 2 and Layer 3 switches
+1.1.c Next-generation firewalls and IPS
+1.1.d Access points
+1.1.e Controllers (Cisco DNA Center and WLC)
+1.1.f Endpoints
+1.1.g Servers
+1.1.h PoE
+
+1.2 Describe characteristics of network topology architectures
+1.2.a Two-tier
+1.2.b Three-tier
+1.2.c Spine-leaf
+1.2.d WAN
+1.2.e Small office/home office (SOHO)
+1.2.f On-premise and cloud
+
+1.3 Compare physical interface and cabling types
+1.3.a Single-mode fiber, multimode fiber, copper
+1.3.b Connections (Ethernet shared media and point-to-point)
+
+1.4 Identify interface and cable issues (collisions, errors, mismatch duplex, and/or speed)
+
+1.5 Compare TCP to UDP
+
+1.6 Configure and verify IPv4 addressing and subnetting
+
+1.7 Describe private IPv4 addressing
+
+1.8 Configure and verify IPv6 addressing and prefix
+
+1.9 Describe IPv6 address types
+1.9.a Unicast (global, unique local, and link local)
+1.9.b Anycast
+1.9.c Multicast
+1.9.d Modified EUI 64
+
+1.10 Verify IP parameters for Client OS (Windows, Mac OS, Linux)
+
+1.11 Describe wireless principles
+1.11.a Nonoverlapping Wi-Fi channels
+1.11.b SSID
+1.11.c RF
+1.11.d Encryption
+
+1.12 Explain virtualization fundamentals (server virtualization, containers, and VRFs)
+
+1.13 Describe switching concepts
+1.13.a MAC learning and aging
+1.13.b Frame switching
+1.13.c Frame flooding
+1.13.d MAC address table
+```
+
+2.0 Network Access
+
+```
+2.1 Configure and verify VLANs (normal range) spanning multiple switches
+2.1.a Access ports (data and voice)
+2.1.b Default VLAN
+2.1.c InterVLAN connectivity
+
+2.2 Configure and verify interswitch connectivity
+2.2.a Trunk ports
+2.2.b 802.1Q
+2.2.c Native VLAN
+
+2.3 Configure and verify Layer 2 discovery protocols (Cisco Discovery Protocol and LLDP)
+
+2.4 Configure and verify (Layer 2/Layer 3) EtherChannel (LACP)
+
+2.5 Interpret basic operations of Rapid PVST+ Spanning Tree Protocol
+2.5.a Root port, root bridge (primary/secondary), and other port names
+2.5.b Port states (forwarding/blocking)
+2.5.c PortFast
+2.5.d Root guard, loop guard, BPDU filter, and BPDU guard
+
+2.6 Describe Cisco Wireless Architectures and AP modes
+
+2.7 Describe physical infrastructure connections of WLAN components (AP, WLC, access/trunk ports, and LAG)
+
+2.8 Describe network device management access (Telnet, SSH, HTTP, HTTPS, console, TACACS+/RADIUS, and cloud managed)
+
+2.9 Interpret the wireless LAN GUI configuration for client connectivity, such as WLAN creation, security settings, QoS profiles, and advanced settings
+```
+
+3.0 IP Connectivity
+
+```
+3.1 Interpret the components of routing table
+3.1.a Routing protocol code
+3.1.b Prefix
+3.1.c Network mask
+3.1.d Next hop
+3.1.e Administrative distance
+3.1.f Metric
+3.1.g Gateway of last resort
+
+3.2 Determine how a router makes a forwarding decision by default
+3.2.a Longest prefix match
+3.2.b Administrative distance
+3.2.c Routing protocol metric
+
+3.3 Configure and verify IPv4 and IPv6 static routing
+3.3.a Default route
+3.3.b Network route
+3.3.c Host route
+3.3.d Floating static
+
+3.4 Configure and verify single area OSPFv2
+3.4.a Neighbor adjacencies
+3.4.b Point-to-point
+3.4.c Broadcast (DR/BDR selection)
+3.4.d Router ID
+
+3.5 Describe the purpose, functions, and concepts of first hop redundancy protocols
+```
+
+4.0 IP Services
+
+```
+4.1 Configure and verify inside source NAT using static and pools
+
+4.2 Configure and verify NTP operating in a client and server mode
+
+4.3 Explain the role of DHCP and DNS within the network
+
+4.4 Explain the function of SNMP in network operations
+
+4.5 Describe the use of syslog features including facilities and levels
+
+4.6 Configure and verify DHCP client and relay
+
+4.7 Explain the forwarding per-hop behavior (PHB) for QoS, such as classification, marking, queuing, congestion, policing, and shaping
+
+4.8 Configure network devices for remote access using SSH
+
+4.9 Describe the capabilities and functions of TFTP/FTP in the network
+```
+
+5.0 Security Fundamentals
+
+```
+5.1 Define key security concepts (threats, vulnerabilities, exploits, and mitigation techniques)
+
+5.2 Describe security program elements (user awareness, training, and physical access control)
+
+5.3 Configure and verify device access control using local passwords
+
+5.4 Describe security password policies elements, such as management, complexity, and password alternatives (multifactor authentication, certificates, and biometrics)
+
+5.5. Describe IPsec remote access and site-to-site VPNs
+
+5.6 Configure and verify access control lists
+
+5.7 Configure and verify Layer 2 security features (DHCP snooping, dynamic ARP inspection, and port security)
+
+5.8 Compare authentication, authorization, and accounting concepts
+
+5.9 Describe wireless security protocols (WPA, WPA2, and WPA3)
+
+5.10 Configure and verify WLAN within the GUI using WPA2 PSK
+```
+
+6.0 Automation and Programmability
+
+```
+6.1 Explain how automation impacts network management
+
+6.2 Compare traditional networks with controller-based networking
+
+6.3 Describe controller-based, software defined architecture (overlay, underlay, and fabric)
+6.3.a Separation of control plane and data plane
+6.3.b Northbound and Southbound APIs
+
+6.4 Explain AI (generative and predictive) and machine learning in network operations
+
+6.5 Describe characteristics of REST-based APIs (authentication types, CRUD, HTTP verbs, and data encoding)
+
+6.6 Recognize the capabilities of configuration management mechanisms, such as Ansible and Terraform
+
+6.7 Recognize components of JSON-encoded data
+```
+
+[CCNA](https://learningnetwork.cisco.com/s/ccna-exam-topics)
diff --git a/networking/edge_infra.md b/networking/edge_infra.md
@@ -0,0 +1,166 @@
+# Edge Infra
+
+Building a globally distributed edge infrastructure for software package
+delivery requires a synergy of multi-tiered caching, intelligent edge
+computation, and robust networking. By combining these technologies, we can
+ensure that developers worldwide can pull packages, binaries, and containers
+with minimal latency and high availability.[^1][^2]
+
+Here is the design and implementation strategy for a global edge network
+tailored for package delivery.
+
+### Edge Architecture Overview
+
+Our system utilizes a tiered architecture to move data as close to the end user
+as possible while protecting the origin from traffic spikes.[^3][^1]
+
+```text
+ +-------------------+
+ | Developer |
+ | (npm/docker pull) |
+ +---------+---------+
+ |
+ v
+ +-------------------+
+ | Anycast / Geo-DNS |
+ +---------+---------+
+ | (Routes to nearest PoP)
+ v
++-----------------------------------------------------------+
+| EDGE PoP |
+| |
+| +-------------+ +---------------+ +--------------+ |
+| | Edge Compute|--->| L1 Cache | | WAF / DDoS | |
+| | (Auth/Route)| | (Memory/NVMe) | | Protection | |
+| +-------------+ +-------+-------+ +--------------+ |
++-----------------------------|-----------------------------+
+ | (Cache Miss)
+ v
++-----------------------------------------------------------+
+| REGIONAL SHIELD CACHE |
+| +-----------------------------------------------------+ |
+| | L2 Cache (High Capacity SSD, Request Collapsing) | |
+| +--------------------------+--------------------------+ |
++-----------------------------|-----------------------------+
+ | (Cache Miss)
+ v
++-----------------------------------------------------------+
+| ORIGIN INFRASTRUCTURE |
+| +--------------------+ +------------------------+ |
+| | Blob Storage | | Global Metadata DB | |
+| | (S3 / GCS) | | (Spanner / DynamoDB) | |
+| +--------------------+ +------------------------+ |
++-----------------------------------------------------------+
+```
+
+### The Full Stack
+
+To achieve this scale, the technology stack must be highly concurrent and
+lightweight:
+
+- **Edge Routing \& Proxy:** NGINX, Envoy, or Rust-based proxies to handle
+ millions of concurrent TCP connections and perform TLS termination.[^4]
+- **Edge Compute:** WebAssembly (Wasm) or V8 Isolates running directly on the
+ CDN edge to execute custom logic like authentication, A/B testing, and request
+ filtering without routing back to the origin.[^3][^1]
+- **Caching Layer:** Varnish or custom memory-mapped file systems for L1 edge
+ caching, backed by high-capacity NVMe drives for L2 regional shields.
+- **Data \& Origin:** Geographically replicated object storage (like AWS S3) for
+ immutable package blobs, and a globally distributed database (like Google
+ Cloud Spanner) for mutable package metadata and user entitlements.
+
+### Edge Caching and CDN Optimizations
+
+Software packages often experience "thundering herd" traffic patterns, such as
+when a popular CI/CD pipeline kicks off thousands of identical container pulls
+simultaneously.
+
+- **Tiered Cache Hierarchy:** Implementing an L1 edge cache and an L2 regional
+ shield reduces origin calls and optimizes global latency.[^3]
+- **Request Collapsing:** If 10,000 clients request the same uncached package
+ simultaneously, the edge node collapses these into a single origin request,
+ preventing origin overload.
+- **Predictive Caching:** By analyzing package dependency trees (e.g.,
+ `package.json`), edge servers can pre-cache required dependencies before the
+ client explicitly requests them.[^1]
+- **Cache Invalidation:** Using stale-while-revalidate headers allows the CDN to
+ serve a slightly outdated metadata file (like a Docker `latest` tag) while
+ asynchronously fetching the updated version in the background.
+
+### Distributed Networking Solutions
+
+Routing users efficiently is critical for minimizing latency and ensuring high
+availability during regional outages.
+
+- **Anycast IP Routing:** Advertising the same IP address from multiple global
+ locations allows the Border Gateway Protocol (BGP) to naturally route the
+ user's TCP connection to the topologically closest datacenter.
+- **Dynamic Server Selection:** The system actively monitors Round Trip Time
+ (RTT) and CPU usage to dynamically route traffic away from congested or
+ degraded Points of Presence (PoPs).[^5][^1]
+- **Protocol Optimizations:** Utilizing TCP BBR congestion control and HTTP/3
+ (QUIC) reduces connection setup time and mitigates the impact of packet loss
+ on unstable mobile networks.
+
+### System Data Flows
+
+When a user pulls a package, the request follows a strict path to ensure
+authorization and speed:
+
+1. **Resolution:** The client's DNS query hits a Geo-DNS provider, returning the
+ Anycast IP of the nearest Edge PoP.
+2. **Edge Auth:** The request reaches the Edge Proxy. An Edge Function executes
+ immediately, verifying the user's API token against a highly cached subset of
+ the metadata database.[^1]
+3. **Cache Lookup:** The proxy checks the L1 Cache. If the package is found, it
+ is returned instantly.
+4. **Shield Fallback:** On an L1 miss, the request goes to the Regional Shield.
+ If the package is present in the L2 cache, it is returned and populated in
+ L1.
+5. **Origin Fetch:** On an L2 miss, the shield fetches the blob from Origin
+ Storage, caches it, and streams it back down the chain to the client.
+
+### Performance Impact Chart
+
+This tiered networking approach dramatically reduces latency across the
+distribution lifecycle.
+
+```text
+Average Response Latency (ms) by Retrieval Tier
+------------------------------------------------------------
+Origin Fetch |################################ (250ms)
+Regional Shield L2 |########### (85ms)
+Edge PoP L1 |### (20ms)
+Predictive Cache |# (5ms)
+------------------------------------------------------------
+```
+
+[^1]: https://notionhive.com/blog/edge-computing-cdn-strategies
+
+[^2]: https://talents.studysmarter.co.uk/companies/cloudsmith-ltd/belfast/senior-software-engineer-edge-29145650/
+
+[^3]: https://www.daydreamsoft.com/blog/edge-caching-and-cdn-optimization-delivering-lightning-fast-web-experiences
+
+[^4]: https://builtin.com/job/senior-software-engineer-tech-platform/6578449
+
+[^5]: https://arxiv.org/html/2412.09474v1
+
+[^6]: https://www.geeksforgeeks.org/system-design/edge-caching-system-design/
+
+[^7]: https://ijrai.org/index.php/ijrai/article/view/180
+
+[^8]: https://networks.imdea.org/trade-offs-in-optimizing-the-cache-deployments-of-cdns/
+
+[^9]: https://careers.deliveroo.co.uk/role/senior-platform-engineer-edge-15f9904608d9/
+
+[^10]: https://www.sciencedirect.com/science/article/abs/pii/S0140366404002889
+
+[^11]: https://www.meegle.com/en_us/topics/content-delivery-network/cdn-caching-mechanisms
+
+[^12]: https://builtin.com/job/senior-software-engineer-edge-infrastructure/7113935
+
+[^13]: https://www.youtube.com/watch?v=zLblLu3rUC4
+
+[^14]: https://www.builtinla.com/job/sr-staff-software-engineer-edge-cdn-platform/7455324
+
+[^15]: https://www.dynadot.com/blog/global-cdn-strategies
diff --git a/networking/index.md b/networking/index.md
@@ -0,0 +1,2 @@
+# Networking
+
diff --git a/networking/ipc.md b/networking/ipc.md
@@ -0,0 +1,17 @@
+# IPC
+
+## IPC (Inter-Process Communication)
+
+IPC is the broad concept — it refers to any mechanism that allows separate
+processes to exchange data. It's not a specific technology but rather an
+umbrella term covering many approaches: shared memory, message queues, pipes,
+signals, and sockets. Both Unix sockets, [TCP](/networking/tcp.md) and
+[UDP](/networking/udp.md) can be used as IPC mechanisms.
+
+## IPC (narrow/common usage)
+
+Local-only mechanisms (Unix sockets, pipes, shared memory)
+
+## TCP & UDP
+
+Network protocols that can do IPC, but are designed for cross-machine use
diff --git a/networking/loopback_address.md b/networking/loopback_address.md
@@ -0,0 +1,117 @@
+# Loopback Address
+
+In Linux system administration and networking fundamentals, a loopback address
+refers to a special, virtual network interface that allows a computer to
+communicate with itself [5]. This interface is typically named `lo` or `lo0` on
+Unix-like systems
+
+## IP Addressing and Standards
+
+The most common IPv4 address used for this purpose is 127.0.0.1, which is the
+standard address for IPv4 loopback traffic. This address is part of a reserved
+block of more than 16 million addresses ranging from 127.0.0.0 to
+127.255.255.255 (127.0.0.0/8) that are designated for loopback functionality.
+For IPv6, the address ::1 is also reserved for this purpose. Internet
+Engineering Task Force (IETF) standards reserve the IPv4 address block
+127.0.0.0/8 and the IPv6 address ::1/128 for this specific use.
+
+## Functionality and Behavior
+
+The primary purpose of the loopback interface is to return the packets sent to
+it; whatever is sent to it is looped back internally. Traffic destined for a
+loopback address allows a host to communicate with itself without sending
+traffic onto any physical network. When an application sends data to 127.0.0.1,
+the IP stack recognizes this as a loopback address and immediately redirects the
+traffic internally through the loopback interface. Because of this design, a
+system that is not connected to any network will still have this loopback device
+and hence a 127.0.0.0 address.
+
+## System Administration and Configuration
+
+In Linux, the loopback interface is managed by tools like NetworkManager, which
+assigns the IP addresses 127.0.0.1 and ::1 that are persistent across reboots.
+Administrators cannot override 127.0.0.1 and ::1, although they can assign
+additional IP addresses to the interface. The hostname localhost is simply a
+name that resolves to this IP address and is configured in /etc/hosts.
+
+## Usage and Availability
+
+Common uses for the loopback device include diagnostics, local service testing,
+and inter-process communication. System administrators and engineers rely on
+loopback addresses when setting up servers to ensure configurations are correct.
+While the majority of systems have loopback addresses by default, they typically
+have zero impact on whether the system is part of a distributed system or not.
+
+## Compare between Unix Socket and Loopback Address
+
+While both methods allow processes on the same machine to communicate, Unix
+domain sockets are technically superior for performance and resource management
+due to the lack of TCP/IP overhead and port limitations. However, loopback
+addresses provide a robust, network-stack-independent method that is often
+sufficient for testing and applications already designed around TCP/IP
+protocols.
+
+### Resource Usage
+
+Port Numbers: Using loopback addresses requires binding to local port numbers,
+which are a limited resource.
+
+File System: Unix sockets utilize the file system for their endpoints, avoiding
+the consumption of network ports.
+
+### Performance and Overhead
+
+Latency: Unix domain sockets generally offer lower latency than TCP loopback
+connections. Benchmarks on modern Linux systems show Unix domain sockets
+delivering approximately 2-3 microseconds of latency, compared to 3.6
+microseconds for TCP loopback, representing a 36% improvement.
+
+Overhead: Unix domain sockets are faster because they avoid the overhead of the
+TCP/IP stack. Local interprocess communication via Unix domain sockets is faster
+than loopback localhost connections because there is less TCP processing
+involved. The loopback device is considered more complicated than Unix sockets,
+resulting in higher relative overhead for loopback traffic.
+
+### Underlying Mechanism
+
+Unix Sockets: These are inter-process communication (IPC) mechanisms that use
+local files to send and receive data, rather than network interfaces and IP
+packets. They function as communication endpoints for data exchange between
+processes running on the same Unix or Unix-like operating system.
+
+Loopback Address: This relies on the TCP/IP stack using the loopback interface
+(typically lo with IP 127.0.0.1). While TCP/IP sockets using the loopback
+address are a natural way to pass messages between processes on the same host,
+they still traverse the network stack.
+
+### Reliability and Robustness
+
+NIC Dependency: Communication via loopback addresses is highly robust because
+the loopback interface (127.0.0.1) does not depend on the physical network
+interface card (NIC). It remains active regardless of whether the NIC is up,
+down, or reconfigured.
+
+Failure Scenarios: If services are configured to communicate via a physical NIC
+IP address, communication can fail if the NIC goes down or the IP changes. In
+contrast, using the loopback address ensures communication remains functional
+even if network hardware fails.
+
+### Typical Use Cases
+
+Loopback: The loopback interface is primarily used for diagnostics and testing
+local server applications. It is also commonly used for IPC when applications
+are designed to use TCP/IP sockets (e.g., Java applications).
+
+Unix Sockets: These are preferred for high-performance local IPC, such as in
+database configurations (e.g., PostgreSQL performance advice suggests Unix
+sockets over localhost for local connections).
+
+### References
+
+[UNIX sockets vs. localhost: PostgreSQL performance advice](https://www.cybertec-postgresql.com/en/postgresql-performance-advice-unix-sockets-vs-localhost/)
+
+## Why does Docker use 172.x.x.x?
+
+Docker uses IP ranges starting with 172.x.x.x (specifically 172.17.0.0/16 by
+default) because that is part of the private IP address space defined in RFC
+1918, which Docker reserves for internal container networking.
diff --git a/networking/tcp.md b/networking/tcp.md
@@ -0,0 +1,44 @@
+# TCP
+
+## TCP (Transmission Control Protocol)
+
+A network protocol ensuring reliable, ordered data delivery between any two
+machines across a network.
+
+## TCP vs TCP/IP
+
+TCP (Transmission Control Protocol) is a single protocol that operates at the
+transport layer — it's responsible for breaking data into packets, ensuring they
+arrive in order, and retransmitting lost ones.
+
+TCP/IP is not a single protocol but a whole suite (stack) of protocols that work
+together to enable internet communication. It's named after its two most
+important protocols — TCP and IP — but it also includes UDP, HTTP, DNS, FTP, and
+many others.
+
+### The Role of Each Layer
+
+TCP/IP is organized into four layers, each with a distinct job:
+
+- Application layer — user-facing protocols like HTTP, FTP, SMTP, DNS
+
+- Transport layer — TCP or UDP (handles delivery, ordering, error checking)
+
+- Internet layer — IP (handles addressing and routing packets to the right
+ machine)
+
+- Network Access layer — physical transmission over Ethernet, Wi-Fi, etc.
+
+### How TCP and IP Work Together
+
+IP and TCP are complementary: IP figures out where to send data (routing between
+machines using IP addresses), while TCP figures out how to send it reliably
+(ordering, retransmission, flow control). TCP doesn't send data directly to the
+recipient — it hands packets down to IP, which routes them across the network.
+
+A simple analogy: IP is like the postal system that routes packages to the right
+address, and TCP is the careful courier who numbers each box, confirms delivery,
+and resends anything that got lost.
+
+So in short: TCP is one piece inside TCP/IP. When people say "TCP/IP," they mean
+the entire networking framework that powers the internet.
diff --git a/networking/udp.md b/networking/udp.md
@@ -0,0 +1,4 @@
+# UDP
+
+UDP (User Datagram Protocol) — a fast, connectionless network protocol that
+sends data without delivery guarantees between any machines across a network.
diff --git a/networking/unix_socket.md b/networking/unix_socket.md
@@ -0,0 +1,184 @@
+# Unix Socket
+
+A Unix domain socket (UDS) is a special file on your filesystem (e.g.
+`/tmp/app.sock`) that acts as a communication endpoint between processes on the
+same machine. Unlike TCP sockets, data never leaves the OS kernel — it's copied
+directly between process buffers — making it significantly faster for local IPC.
+The server binds to the socket path, and one or more clients connect to it.
+
+Key properties:
+
+- Lives at a filesystem path, controlled by file permissions
+
+- Supports stream mode (like TCP — ordered, reliable) and datagram mode (like
+ UDP — message-based)
+
+- Commonly used between e.g. Nginx ↔ PHP-FPM, apps ↔ local databases, or
+ microservices on the same host
+
+## Rust Examples: Multiple Producers → Single Receiver
+
+The pattern here is: **multiple client threads each connect to the same Unix
+socket**, and a single server receives all their messages. We'll use tokio for
+async I/O.
+[Unix Listener](https://docs.rs/tokio/latest/tokio/net/struct.UnixListener.html),
+[Unix Stream](https://docs.rs/tokio/latest/tokio/net/struct.UnixStream.html)
+
+```toml
+[dependencies]
+tokio = { version = "1", features = ["full"] }
+```
+
+### Example 1 — Basic Server (Single Receiver)
+
+The server binds to a socket path and handles each incoming connection, printing
+whatever data it receives.
+[docs](https://docs.rs/tokio/latest/tokio/net/struct.UnixListener.html)
+
+```rust
+// server.rs
+use tokio::net::UnixListener;
+use tokio::io::AsyncReadExt;
+
+#[tokio::main]
+async fn main() {
+ let socket_path = "/tmp/mpsr.sock";
+ let _ = std::fs::remove_file(socket_path); // clean up stale socket
+
+ let listener = UnixListener::bind(socket_path).unwrap();
+ println!("Server listening on {}", socket_path);
+
+ loop {
+ match listener.accept().await {
+ Ok((mut stream, _addr)) => {
+ // Spawn a task per connection — all funnel into this single server
+ tokio::spawn(async move {
+ let mut buf = vec![0u8; 1024];
+ match stream.read(&mut buf).await {
+ Ok(n) if n > 0 => {
+ let msg = String::from_utf8_lossy(&buf[..n]);
+ println!("[Server] Received: {}", msg);
+ }
+ _ => eprintln!("[Server] Connection closed or error"),
+ }
+ });
+ }
+ Err(e) => eprintln!("Accept error: {}", e),
+ }
+ }
+}
+```
+
+### Example 2 — Multiple Producers (Clients)
+
+Each producer runs in its own thread, connects to the socket, and sends a
+message.
+
+```rust
+// client.rs
+use tokio::net::UnixStream;
+use tokio::io::AsyncWriteExt;
+
+#[tokio::main]
+async fn main() {
+ let socket_path = "/tmp/mpsr.sock";
+ let num_producers = 5;
+
+ let mut handles = vec![];
+
+ for i in 0..num_producers {
+ let path = socket_path.to_string();
+ let handle = tokio::spawn(async move {
+ match UnixStream::connect(&path).await {
+ Ok(mut stream) => {
+ let msg = format!("Hello from producer {}", i);
+ stream.write_all(msg.as_bytes()).await.unwrap();
+ println!("[Producer {}] Sent: {}", i, msg);
+ }
+ Err(e) => eprintln!("[Producer {}] Failed to connect: {}", i, e),
+ }
+ });
+ handles.push(handle);
+ }
+
+ for h in handles {
+ h.await.unwrap();
+ }
+}
+```
+
+### Example 3 — Combined: Server + Producers in One Binary
+
+A self-contained example where the server is spawned as a background task, and 5
+producers send concurrently.
+
+```rust
+use tokio::net::{UnixListener, UnixStream};
+use tokio::io::{AsyncReadExt, AsyncWriteExt};
+use tokio::time::{sleep, Duration};
+
+#[tokio::main]
+async fn main() {
+ let socket_path = "/tmp/mpsr_combined.sock";
+ let _ = std::fs::remove_file(socket_path);
+
+ // --- Spawn the single receiver (server) ---
+ let server_path = socket_path.to_string();
+ tokio::spawn(async move {
+ let listener = UnixListener::bind(&server_path).unwrap();
+ println!("[Server] Listening...");
+ loop {
+ if let Ok((mut stream, _)) = listener.accept().await {
+ tokio::spawn(async move {
+ let mut buf = vec![0u8; 256];
+ if let Ok(n) = stream.read(&mut buf).await {
+ println!("[Server] Got: {}", String::from_utf8_lossy(&buf[..n]));
+ }
+ });
+ }
+ }
+ });
+
+ // Give the server a moment to bind
+ sleep(Duration::from_millis(50)).await;
+
+ // --- Spawn multiple producers ---
+ let mut handles = vec![];
+ for i in 0..5 {
+ let path = socket_path.to_string();
+ handles.push(tokio::spawn(async move {
+ let mut stream = UnixStream::connect(&path).await.unwrap();
+ let msg = format!("Message from producer {}", i);
+ stream.write_all(msg.as_bytes()).await.unwrap();
+ println!("[Producer {}] Sent.", i);
+ }));
+ }
+
+ for h in handles {
+ h.await.unwrap();
+ }
+
+ sleep(Duration::from_millis(100)).await; // let server print all messages
+}
+```
+
+Expected output (order may vary since tasks run concurrently):
+
+```text
+[Server] Listening...
+[Producer 0] Sent.
+[Producer 2] Sent.
+[Server] Got: Message from producer 0
+[Server] Got: Message from producer 2
+...
+```
+
+### How the MPSR Pattern Works Here
+
+Each producer independently connects and sends — the Unix socket listener
+naturally queues incoming connections. The tokio::spawn per connection is what
+makes this a multiple-producer, single-receiver funnel: one server loop accepts
+all connections, but each connection is handled concurrently. If you also want
+to aggregate messages into a single channel, you can pair this with
+std::sync::mpsc or tokio::sync::mpsc inside the server's accept loop.
+[mpsc](https://blog.softwaremill.com/multithreading-in-rust-with-mpsc-multi-producer-single-consumer-channels-db0fc91ae3fa)
diff --git a/observability/index.md b/observability/index.md
@@ -0,0 +1,2 @@
+# Observability
+
diff --git a/observability/span.md b/observability/span.md
@@ -0,0 +1,143 @@
+# Span
+
+## Question: What is a span in relation to traces, logs and metrics?
+
+A **span** is the foundational unit of **distributed tracing**, representing a
+single logical operation (e.g., an HTTP request, a database call, or a function
+invocation) with timing and contextual metadata [7]. To understand spans
+deeply—and how they relate to **traces**, **logs**, and **metrics**—we must
+examine their structural, semantic, and operational relationships.
+
+---
+
+### **1. Span as the Atomic Unit of a Trace**
+
+A **trace** is a directed acyclic graph (DAG) of spans that captures the
+end-to-end journey of a request across services [1]. Each span:
+
+- Has a **start/end timestamp** and **duration**,
+- Contains **attributes** (key-value metadata, e.g., HTTP status, user ID),
+- May include **events** (timestamped annotations like “query started”),
+- Has a **parent-child relationship** with other spans (e.g., a gateway span may
+ have child spans for auth and DB calls) [[3], [8]].
+
+Example trace structure[8]:
+
+```
+Trace
+├── Span (API Gateway)
+│ ├── Span (Auth Service)
+│ └── Span (User Service)
+│ └── Span (Database Query)
+└── Span (Response Formatting)
+```
+
+---
+
+### **2. Relationship to Logs**
+
+- **Logs** are discrete, timestamped records of events (e.g., “error: connection
+ timeout”), often unstructured or semi-structured.
+- **Spans can embed logs**: When instrumentation libraries (e.g., OpenTelemetry)
+ integrate with logging frameworks, log statements can be attached to spans as
+ **structured events** or **log records**, enriching them with trace context
+ (trace ID, span ID) [10].
+- This enables **correlation**: You can view logs _within the context_ of a
+ specific span—e.g., see all logs from a database query span during a failed
+ request [10].
+
+> “When adding OpenTelemetry instrumentation on top of your existing log
+> libraries, the log becomes a dot on a trace span” [10].
+
+---
+
+### **3. Relationship to Metrics**
+
+- **Metrics** are aggregated numerical measurements over time (e.g., request
+ rate, latency percentiles, error counts).
+- **Spans feed into metrics indirectly**:
+ - Span durations can be used to compute **latency histograms** (e.g.,
+ `http.server.request.duration`).
+ - Span attributes (e.g., `http.status_code`) can be aggregated into
+ **counters** (e.g., `http_requests_total{status="500"}`).
+- While spans are _individual_, _context-rich_ units, metrics are _aggregated_,
+ _summarized_—but both are essential for the **RED method** (Rate, Errors,
+ Duration) [7].
+
+> “Developers can acquire a comprehensive perspective of their software
+> environment by combining distributed traces, metrics, events, and logs” [7].
+
+---
+
+### **4. Relationship to Traces (Recap & Nuance)**
+
+- A **trace** is a _collection of spans_ that together represent a single
+ request’s path through a distributed system [3].
+- Spans in a trace are linked via:
+ - **Trace ID** (identifies the full trace),
+ - **Span ID** (identifies the span),
+ - **Parent Span ID** (enables tree-like nesting) [[1], [8]].
+- Spans may also have **links** to spans in _other traces_ (e.g., for batch
+ processing or async workflows) [1].
+
+---
+
+### **5. Practical Implications**
+
+- **Troubleshooting**: A trace gives you a _map_; logs give you _narrative
+ detail_; metrics give you _signal-level trends_. For example:
+ - A metric alert (e.g., high error rate) → drill into traces to find failing
+ spans → inspect embedded logs for root cause [14].
+- **Context propagation**: Spans carry trace context (trace ID, span ID,
+ sampling flags) across service boundaries, enabling distributed correlation
+ [9].
+
+---
+
+### **Summary**
+
+| Concept | Role | Relationship to Span |
+| ----------- | ------------------------------------------ | ------------------------------------------------------------------- |
+| **Span** | Smallest unit of work in a trace | — |
+| **Trace** | Collection of spans forming a request path | Spans are its building blocks [3] |
+| **Logs** | Event records with timestamps | Logs can be attached to spans as events or structured metadata [10] |
+| **Metrics** | Aggregated numerical signals | Span data (duration, status) is used to derive metrics [7] |
+
+In essence, **spans unify the three pillars of observability**—they are the
+_contextual glue_ that lets you correlate logs (what happened), metrics (how
+often/long), and traces (how it flows) into actionable insights [[4], [14]].
+
+## References
+
+1. [Traces | OpenTelemetry](https://opentelemetry.io/docs/concepts/signals/traces/)
+ _(brave)_
+2. [OpenTelemetry - Understanding Traces vs. Spans | SigNoz](https://signoz.io/comparisons/opentelemetry-trace-vs-span/)
+ _(brave)_
+3. [Logs vs Metrics vs Traces - Engineering Fundamentals Playbook](https://microsoft.github.io/code-with-engineering-playbook/observability/log-vs-metric-vs-trace/)
+ _(google)_
+4. [Observability primer | OpenTelemetry](https://opentelemetry.io/docs/concepts/observability-primer/)
+ _(brave)_
+5. [Unpacking Observability: Understanding Logs, Events, Spans, and Traces | Dzero Labs](https://medium.com/dzerolabs/observability-journey-understanding-logs-events-traces-and-spans-836524d63172)
+ _(google)_
+6. [OpenTelemetry demystified: a deep dive into distributed tracing | CNCF](https://www.cncf.io/blog/2023/05/03/opentelemetry-demystified-a-deep-dive-into-distributed-tracing/)
+ _(google)_
+7. [What Are Spans in Distributed Tracing? - LogicMonitor](https://www.logicmonitor.com/blog/what-are-spans-in-distributed-tracing)
+ _(startpage)_
+8. [Traces & Spans: Observability Basics You Should Know - Last9](https://last9.io/blog/traces-spans-observability-basics/)
+ _(startpage)_
+9. [software-skills/skills/system-design/references/key-concepts ...](https://github.com/itzcull/software-skills/blob/master/skills/system-design/references/key-concepts/distributed-tracing.md)
+ _(aol)_
+10. [Tracing the Line: Understanding Logs vs. Traces - Honeycomb](https://www.honeycomb.io/blog/understanding-logs-vs-traces)
+ _(google)_
+11. [A Deep Dive into OpenTelemetry. Part 1 - AWS in Plain English](https://aws.plainenglish.io/opentelemetry-deep-dive-part-1-6ebbd2362bd3)
+ _(google)_
+12. [Deep Dive into OpenTelemetry in Saleor](https://saleor.io/blog/otel-deep-dive)
+ _(google)_
+13. [Logging Observability - OpenClaw AI Agent Skill | LLMBase](https://llmbase.ai/openclaw/logging-observability/)
+ _(aol)_
+14. [Learning Observability from Scratch: Logs, Metrics, and Traces | by Milind Nair | Mar, 2026 | Medium](https://medium.com/@nairmilind3/learning-observability-from-scratch-c36d9771003b)
+ _(brave)_
+15. [A Deep Dive Into OpenTelemetry Metrics | Tiger Data](https://www.tigerdata.com/blog/a-deep-dive-into-open-telemetry-metrics)
+ _(aol)_
+16. [GitHub - tokio-rs/tracing: Application level tracing for Rust.](https://github.com/tokio-rs/tracing)
+ _(aol)_
diff --git a/observability/tracing-logs-metrics.md b/observability/tracing-logs-metrics.md
@@ -0,0 +1,232 @@
+# Tracing, Logs, & Metrics
+
+## Question: Whats the difference between tracing, logs, and metrics?
+
+---
+
+**Tracing, Logs, and Metrics** are three core components of **observability**,
+each serving distinct but complementary roles in understanding system behavior.
+
+---
+
+### 🔹 **Logs**
+
+- **What they are**: Chronological records of discrete events that occur during
+ program execution (e.g., “User logged in”, “Database query failed”) [[7]].
+- **Purpose**: Primarily used for error tracking, debugging, and auditing. Logs
+ help answer _“What happened?”_ and _“When did it happen?”_ [[9]].
+- **Format**: Usually unstructured or semi-structured text, often timestamped
+ and tagged with severity levels (e.g., INFO, WARN, ERROR) [[7]].
+
+---
+
+### 🔹 **Tracing**
+
+- **What it is**: A technique to track the flow of a single request as it
+ traverses multiple services or components (e.g., from API gateway → auth
+ service → database) [[13]].
+- **Purpose**: Provides end-to-end visibility into request paths, latency, and
+ failure points—especially critical in distributed systems [[1], [11]].
+- **Structure**: Composed of _spans_ (units of work) nested in _traces_;
+ visualized as flame graphs or Gantt charts [[11], [13]].
+- **Use case**: Diagnosing slow responses or failures across service boundaries
+ [[1], [5]].
+
+---
+
+### 🔹 **Metrics**
+
+- **What they are**: Quantitative, time-series measurements (e.g., request rate,
+ CPU usage, error counts) [[6]], [[9]].
+- **Purpose**: Ideal for trend analysis, alerting, and capacity planning.
+ Metrics answer _“How is the system performing?”_ [[6]].
+- **Format**: Structured and aggregatable (e.g., counters, gauges, histograms)
+ [[6]].
+
+---
+
+### 🔸 **When to Use Metrics, Logs, or Traces**
+
+Metrics, logs, and traces are complementary observability signals—not
+replacements for each other. Each answers different questions and excels in
+specific scenarios.
+
+| Consider Metrics When... | Consider Logs When... | Consider Traces When... |
+| ----------------------------------------------------- | -------------------------------------------------------- | ------------------------------------------------------- |
+| You need to monitor system health over time | You need to investigate specific errors or events | You need to track a request across service boundaries |
+| You want to set up alerting (e.g., "error rate > 1%") | You need to debug a specific request using IDs | You need to find the exact span where latency occurs |
+| You care about trends, aggregations, and patterns | You need raw context, stack traces, or user actions | You're diagnosing slow responses in distributed systems |
+| You need low storage/processing overhead | You're doing post-mortem analysis or compliance auditing | You need to understand request flow and dependencies |
+| You're capacity planning or auto-scaling | You need to match errors across different services | You want to visualize dependencies and request paths |
+
+---
+
+### 🎯 **Why Metrics Instead of Just Logs?**
+
+Logs capture every event but are **too noisy and expensive** for real-time
+monitoring:
+
+1. **Aggregation Efficiency**: Metrics pre-compute values (sums, averages,
+ percentiles), while querying logs for aggregated insights requires scanning
+ massive datasets. For example, calculating error rates from logs across
+ millions of events per second is computationally expensive; metrics do this
+ inline.
+
+2. **Real-time Alerting**: Metrics are time-series data designed for rapid
+ aggregation. You can quickly detect anomalies using threshold-based alerts.
+ With logs, you'd need to constantly parse and aggregate in
+ real-time—introducing latency and cost.
+
+3. **Storage Cost**: Storing every log line is expensive (GBs/TBs/day). Metrics
+ are orders of magnitude smaller because they store only numerical summaries.
+
+4. **Dashboard Visualization**: Trending data over time (e.g., "requests/second
+ over 24 hours") works naturally with metrics. Extracting this from logs
+ requires complex aggregation queries.
+
+**Example**: Monitoring a web service. Metrics tell you the error rate spiked to
+5% across all endpoints. Logs alone wouldn't reveal this pattern—you'd need to
+manually scan countless entries to notice the trend.
+
+ _Prometheus time-series database
+showing system metrics over time_
+
+---
+
+### 🎯 **Why Traces Instead of Just Logs?**
+
+While logs record individual events, traces provide **context and causality**
+across distributed systems:
+
+1. **End-to-End Request Flow**: A single user request may traverse 10+ services.
+ Logs from each service are disconnected unless correlated by trace IDs.
+ Traces natively capture this relationship with parent-child span
+ relationships.
+
+2. **Latency Breakdown**: Logs can tell you each service took 100ms, but traces
+ show exactly where time was spent (e.g., 50ms in auth, 80ms in database).
+ This is critical for identifying bottlenecks.
+
+3. **Failure Context**: If a request fails, traces show the entire path—what
+ succeeded, what failed, and where. Logs from individual services lack this
+ cross-service context.
+
+4. **Dependency Visualization**: Traces reveal service boundaries and call
+ sequences, helping you understand system architecture and potential single
+ points of failure.
+
+**Example**: A user reports a slow page load. Logs from the API gateway show a
+2-second response. Logs from the auth service show 200ms. Logs from the database
+show 100ms. Without traces, you can't correlate these or see that the auth
+service was actually waiting on the database (1.5s of wait time hidden in
+individual logs).
+
+ _Jaeger trace visualization showing a
+distributed request across multiple services with latency breakdown_
+
+ _Flame graph view in Jaeger
+highlighting which spans consume the most time_
+
+---
+
+### 🎯 **Why Logs Instead of Just Metrics or Traces?**
+
+Logs provide **context and detail** that metrics and traces cannot:
+
+1. **Rich Context**: Metrics are numbers; traces show flow. Logs contain the
+ actual messages: error stack traces, variable values, user IDs, request
+ payloads. This is essential for debugging.
+
+2. **Unpredictable Events**: You can't pre-aggregate unknown error conditions in
+ metrics. Logs capture everything—including rare, unexpected errors that
+ metrics might miss in the noise.
+
+3. **Debugging Speed**: When tracing identifies a problematic span, logs provide
+ the exact log lines from that span (with context like user ID, request
+ parameters, error messages) to pinpoint the cause.
+
+4. **Compliance & Audit**: Logs provide an immutable record of what happened,
+ often required for regulatory compliance. Metrics summarize but don't replay
+ events.
+
+**Example**: A trace shows a 500 error in the payment service. The metric shows
+0.1% error rate (acceptable). But only logs contain the actual error: "Payment
+declined: insufficient funds" with the user's transaction ID—needed to
+investigate and communicate with the user.
+
+ _Grafana Loki aggregated logs with
+label-based filtering and search_
+
+---
+
+### 🔗 **The Synergistic Workflow**
+
+The three signals work best together:
+
+1. **Metrics detect**: Threshold-based alerts tell you something is wrong
+2. **Traces localize**: Identify which request/path/service is affected
+3. **Logs debug**: Provide context to understand and fix the root cause
+
+**Example workflow**:
+
+- ✅ Metrics show error rate increased from 0.1% to 5% → **Something is wrong**
+- ✅ Traces reveal the spike is from `/api/checkout` requests in the payment
+ service → **Where is it wrong?**
+- ✅ Logs with that trace ID show:
+ `PaymentProviderError: Connection timeout to Stripe` → **Why is it wrong?**
+
+---
+
+### 🌐 **Observability Concepts**
+
+Observability is the _property_ of a system that allows engineers to understand
+its internal state _from its external outputs_ [[4], [10], [12]]. It goes beyond
+monitoring by enabling _root-cause analysis_ of unexpected behavior, not just
+detection [[16]].
+
+The **three pillars of observability** are:
+
+1. **Metrics** – for high-level system health and trends
+2. **Logs** – for detailed event-level debugging
+3. **Traces** – for request-level flow and latency analysis
+
+Together, they provide a comprehensive view:
+
+- Metrics tell you _something is wrong_
+- Traces tell you _which request/path is affected_
+- Logs tell you _what happened and why_
+
+This layered approach prevents log overload while maintaining debugging
+capability.
+
+---
+
+### ⚖️ Key Differences Summary
+
+| Aspect | Logs | Traces | Metrics |
+| ----------------- | ---------------------------------------- | ------------------------------------------- | ------------------------------------------ |
+| **Granularity** | Event-level (e.g., function call, error) | Request-level (end-to-end path) | Aggregated (e.g., avg latency, error rate) |
+| **Structure** | Semi-structured text | Hierarchical spans (parent/child) | Numerical, time-series |
+| **Primary Use** | Debugging, auditing | Performance analysis, distributed debugging | Alerting, trend analysis |
+| **Cost/Overhead** | Moderate | High (esp. in distributed systems) [[5]] | Low (aggregated) |
+
+---
+
+## Sources
+
+1. [java - What is the difference between Tracing and Logging? - Stack Overflow](https://stackoverflow.com/questions/27244807/what-is-the-difference-between-tracing-and-logging)
+2. [What is observability? Not just logs, metrics, and traces - Dynatrace](https://www.dynatrace.com/news/blog/what-is-observability-2/)
+3. [Tracing Vs. Logging – Key Differences + Examples - Edge Delta](https://edgedelta.com/company/blog/tracing-vs-logging-differences-with-examples)
+4. [What Is Observability? | IBM](https://www.ibm.com/think/topics/observability)
+5. [Logging vs Tracing in real projects — how deep do you actually go?](https://www.reddit.com/r/Backend/comments/1r45scd/logging_vs_tracing_in_real_projects_how_deep_do/)
+6. [Observability in 2025: How It Works, Challenges and Best Practices](https://lumigo.io/what-is-observability-concepts-use-cases-and-technologies/)
+7. [Tracing vs Logging vs Monitoring: What's the Difference? – BMC Helix](https://blogs.helixops.ai/monitoring-logging-tracing/)
+8. [What Is Observability & How Does it Work?](https://www.datadoghq.com/knowledge-center/observability/)
+9. [Logging vs Metrics vs Tracing: What's the Difference? | Better Stack Community](https://betterstack.com/community/guides/observability/logging-metrics-tracing/)
+10. [What Is Observability? Concepts, Use Cases & Technologies - Tigera.io](https://www.tigera.io/learn/guides/observability/)
+11. [Monitoring explained: What is the difference between Logging, Tracing & Profiling? - greeeg.com](https://greeeg.com/en/issues/differences-between-logging-tracing-profiling)
+12. [Observability That Works: Understand System Failures and Drive Better Business Outcomes | Splunk](https://www.splunk.com/en_us/blog/learn/observability.html)
+13. [Traces - OpenTelemetry](https://opentelemetry.io/docs/concepts/signals/traces/)
+14. [What Is Observability? Key Components and Best Practices](https://www.honeycomb.io/blog/what-is-observability-key-components-best-practices)
+15. [Screening and Assessment of Contaminated Sediment](https://extapps.dec.ny.gov/docs/fish_marine_pdf/screenasssedfin.pdf)
+16. [What is Observability? A Guide to Success - Cribl](https://cribl.io/blog/what-is-observability/)
diff --git a/operating_systems/data_streams.md b/operating_systems/data_streams.md
@@ -0,0 +1,215 @@
+# Data Streams
+
+## Data Streams and Pipe
+
+<img src="/whiteboards/data_stream.png" alt="Data Stream" width="100%">
+
+## Passing Data Streams between rust binaries
+
+We can pass data directly between binaries, they don't have to go through a
+networking layers http server with ports, which adds unecessary overhead when it
+is not needed.
+
+<img src="/whiteboards/pipe_between_binaries.png" alt="Data Stream" width="100%">
+
+## Making your own “filter” binary
+
+If you want your Rust binary to be the thing that “processes stdout from another
+binary”, just read from stdin and write to stdout:
+
+```rust
+use std::io::{self, BufRead, Write};
+
+fn main() -> io::Result<()> {
+ let stdin = io::stdin();
+ let stdout = io::stdout();
+ let mut out = stdout.lock();
+
+ for line in stdin.lock().lines() {
+ let line = line?;
+ // transform line
+ let processed = line.replace("foo", "bar");
+ writeln!(out, "{processed}")?;
+ }
+
+ Ok(())
+}
+```
+
+Then in a shell you can do `producer_cmd | your_rust_filter | consumer_cmd`.
+
+## Use processed data as stdin to another binary
+
+To “chain” binaries but keep Rust in the middle, you can run a second command
+and write your processed data into its stdin.
+
+```rust
+use std::io::{self, BufRead, BufReader, Write};
+use std::process::{Command, Stdio};
+
+fn main() -> io::Result<()> {
+ // First binary: produces data
+ let mut producer = Command::new("producer_cmd")
+ .arg("some_arg")
+ .stdout(Stdio::piped())
+ .spawn()?;
+
+ let producer_stdout = producer
+ .stdout
+ .take()
+ .expect("failed to capture producer stdout");
+ let reader = BufReader::new(producer_stdout);
+
+ // Second binary: consumes processed data
+ let mut consumer = Command::new("consumer_cmd")
+ .arg("another_arg")
+ .stdin(Stdio::piped())
+ .spawn()?;
+
+ let consumer_stdin = consumer
+ .stdin
+ .take()
+ .expect("failed to open consumer stdin");
+
+ // Process loop: read from producer, transform, send to consumer
+ let mut writer = consumer_stdin;
+ for line in reader.lines() {
+ let line = line?;
+ let processed = format!("prefix: {line}\n"); // any transformation
+ writer.write_all(processed.as_bytes())?;
+ }
+ // Close consumer stdin so it can finish
+ drop(writer);
+
+ // Wait for both processes to exit
+ let prod_status = producer.wait()?;
+ let cons_status = consumer.wait()?;
+
+ eprintln!("producer: {prod_status}, consumer: {cons_status}");
+
+ Ok(())
+}
+```
+
+Conceptually this is equivalent to
+`producer_cmd | my_rust_filter | consumer_cmd`, but Rust is the middle filter,
+so you never rely on shell piping.
+
+## Async Pipes with Tokio
+
+Use Tokio's async I/O to pipe binary data. `tokio::process::Command` mirrors the
+sync API but uses `AsyncRead/AsyncWrite` traits, letting you `.await` on reads
+and writes without blocking threads.
+
+```rust
+use tokio::process::Command;
+use tokio::io::{AsyncReadExt, AsyncWriteExt};
+use tokio::process::Stdio;
+
+#[tokio::main]
+async fn main() {
+ let mut child = Command::new("xxd")
+ .stdin(Stdio::piped())
+ .stdout(Stdio::piped())
+ .spawn()
+ .expect("Failed to spawn");
+
+ let binary_data: Vec<u8> = (0u8..=255).collect(); // 256 bytes of raw binary
+
+ // Write asynchronously
+ if let Some(mut stdin) = child.stdin.take() {
+ stdin.write_all(&binary_data).await.expect("Write failed");
+ // Drop stdin to close pipe (send EOF)
+ }
+
+ // Read asynchronously
+ let mut output = Vec::new();
+ if let Some(mut stdout) = child.stdout.take() {
+ stdout.read_to_end(&mut output).await.expect("Read failed");
+ }
+
+ child.wait().await.expect("Child failed");
+ println!("Received {} bytes of output", output.len());
+}
+```
+
+For streaming, long-running binaries, use tokio_process_tools which wraps
+Tokio's process API with real-time line/byte inspection via channels.
+
+## Fan-In: Many Binaries → One Receiver
+
+This is a classic multi-producer, single-consumer (MPSC) pattern. You don't need
+special OS-level listeners — Tokio's mpsc channel is purpose-built for this.
+
+The architecture:
+
+- Each producer binary is spawned as a child process with stdout(Stdio::piped())
+
+- Each gets a clone of the mpsc::Sender
+
+- A single async task owns the Receiver and processes all incoming data
+
+```rust
+use tokio::process::Command;
+use tokio::io::{AsyncReadExt, BufReader};
+use tokio::sync::mpsc;
+use std::process::Stdio;
+
+#[tokio::main]
+async fn main() {
+ let (tx, mut rx) = mpsc::channel::<(usize, Vec<u8>)>(32);
+
+ let producers = vec!["producer_a", "producer_b", "producer_c"];
+
+ for (id, binary) in producers.iter().enumerate() {
+ let tx = tx.clone(); // clone sender for each producer
+
+ let mut child = Command::new(binary)
+ .stdout(Stdio::piped())
+ .spawn()
+ .expect("Failed to spawn producer");
+
+ tokio::spawn(async move {
+ let mut stdout = BufReader::new(child.stdout.take().unwrap());
+ let mut buf = Vec::new();
+
+ loop {
+ let mut chunk = vec![0u8; 4096];
+ match stdout.read(&mut chunk).await {
+ Ok(0) => break, // EOF
+ Ok(n) => {
+ chunk.truncate(n);
+ tx.send((id, chunk)).await.unwrap();
+ }
+ Err(e) => { eprintln!("Error from producer {}: {}", id, e); break; }
+ }
+ }
+ child.wait().await.unwrap();
+ });
+ }
+
+ drop(tx); // drop original sender so rx knows when all producers are done
+
+ // Single receiver handles all producers
+ while let Some((producer_id, data)) = rx.recv().await {
+ println!("Producer {}: {} bytes", producer_id, data.len());
+ // process binary data here...
+ }
+}
+```
+
+Do You Need an Event Listener? Not in the traditional sense. Tokio's scheduler
+is itself an event-driven async runtime — when no data is available, tasks are
+suspended and CPU is yielded, not wasted in a spin loop.
+
+# Delimiter
+
+A delimiter is a character or group of characters that marks where one piece of
+data ends and the next one begins.
+
+In computing and data formats, common delimiters include commas, tabs, spaces,
+semicolons, and pipes, such as in CSV files where commas separate values. In
+general English dictionaries, it is defined as a character that marks the
+beginning or end of a unit of data. In maths and programming, symbols like
+parentheses (), quotes " " and braces {} also act as delimiters because they
+enclose or bound expressions or code blocks.
diff --git a/operating_systems/index.md b/operating_systems/index.md
@@ -0,0 +1,20 @@
+# Operating Systems
+
+## OS-level view
+
+A [mutex (mutual exclusion lock)](/memory_safety/mutex.md) is a synchronization
+primitive that ensures only one thread at a time enters a critical section that
+uses some shared data.
+
+If a thread tries to lock a mutex that is already locked, the OS (or runtime)
+either makes it spin briefly or puts it to sleep and queues it until the mutex
+is unlocked.
+
+The OS uses low-level tools (like Linux futex on an integer in memory) plus
+atomics and memory fences to implement this reliably across cores.
+
+- [System Call(syscall)](/operating_systems/system_call.md)
+- [User Space and Kernel Space](/operating_systems/user_space_and_kernel_space.md)
+- [Thread](/operating_systems/thread.md)
+- [PostgreSQL Connection](/operating_systems/postgres/postgresql_connection.md)
+- [Connection Pool](/operating_systems/postgres/connection_pool.md)
diff --git a/operating_systems/postgres/connection_pool.md b/operating_systems/postgres/connection_pool.md
@@ -0,0 +1,44 @@
+# Connection Pool
+
+A connection pool is a component (in your app or as a proxy like
+PgBouncer/Pgpool-II) that keeps a limited number of database connections open
+and reuses them across many client requests.
+
+## Key ideas:
+
+- Opening a [PostgreSQL connection](/operating_systems/postgres/postgresql_connection.md) is
+ relatively expensive and each connection consumes memory; creating hundreds or
+ thousands on demand is wasteful.
+
+- The pool maintains, say, 20–50 actual DB connections and lets many more
+ logical clients "borrow" them for short periods.
+
+- When a request finishes, the connection is returned to the pool instead of
+ being closed, so the next request can reuse it without the startup cost.
+
+- The pool enforces an upper bound on concurrent PostgreSQL connections, which
+ indirectly bounds per-connection memory usage.
+
+### Question: what if the pool is full?
+
+#### General behavior when pool is full:
+
+- The pool has a max number of physical PostgreSQL connections it will open.
+
+- When all of them are checked out and a new request comes in, most pool
+ implementations put the request in a queue and wait for a connection to be
+ returned. If no connection becomes free before a timeout, they raise an
+ error/exception (often something like "pool exhausted" or "timeout waiting for
+ connection").
+
+- This is separate from PostgreSQL's own max_connections; the pool will usually
+ hit its own limit first and throttle clients, which is the goal.
+
+### Question: What if the DB itself is at max connections?
+
+- PostgreSQL has a server-side max_connections limit; when that is reached, any
+ new physical connection attempt fails with "too many connections".
+
+- If your pool tries to open more physical connections beyond what the server
+ allows (e.g. under load or after restart), those attempts will fail and bubble
+ up as errors to the application (connection failure rather than "queued").
diff --git a/operating_systems/postgres/index.md b/operating_systems/postgres/index.md
@@ -0,0 +1,5 @@
+# Postgres
+
+- [PostgreSQL Connection](/operating_systems/postgres/postgresql_connection.md)
+- [Connection Pool](/operating_systems/postgres/connection_pool.md)
+- [Postgres Binaries](/operating_systems/postgres/postgres_binaries.md)
diff --git a/operating_systems/postgres/postgres_binaries.md b/operating_systems/postgres/postgres_binaries.md
@@ -0,0 +1,3 @@
+# Postgres Binaries
+
+<img src="/whiteboards/postgres.png" alt="Postgres Structure" width="100%">
diff --git a/operating_systems/postgres/postgresql_connection.md b/operating_systems/postgres/postgresql_connection.md
@@ -0,0 +1,27 @@
+# PostgreSQL Connection
+
+A PostgreSQL connection is the link between a client (like an app, script, or
+psql) and a PostgreSQL database server that lets them send queries and receive
+results.
+
+## Memory and Processes
+
+A PostgreSQL connection uses memory because each client gets its own backend
+process with its own state (caches, buffers, variables, temp data, etc.), and
+that state must live somewhere in RAM. This is also why too many connections can
+exhaust memory and hurt performance.
+
+PostgreSQL is a process‑per‑connection model (on typical
+[Unix-like systems](/operating_systems/README.md) a new OS process is forked for
+each client connection).
+
+Each backend process has its own private address space for session-local state,
+which avoids accidental shared mutable state between connections.
+
+Backends share only explicit shared-memory regions and files (e.g., shared
+buffer cache, WAL, locks), with well-defined synchronization, so there's no
+"mysterious" shared state like in a multi-threaded process with global
+variables.
+
+So, you do get isolation of most state per connection, with only controlled,
+intentional shared memory for common data structures.
diff --git a/operating_systems/system_call.md b/operating_systems/system_call.md
@@ -0,0 +1,7 @@
+# System Call
+
+A system call is how a normal program asks the operating system kernel to do
+something privileged (like read a file, send data on a socket, allocate certain
+memory).
+
+“System call” is a concept from operating‑system interfaces and kernel design.
diff --git a/operating_systems/thread.md b/operating_systems/thread.md
@@ -0,0 +1,145 @@
+# Thread
+
+### Question: What exactly is a thread?
+
+A thread is the smallest unit of execution that a CPU can run — essentially, an
+independent sequence of instructions within a program. The CPU distinguishes
+threads by their saved state and ID, not by “feeling” them electrically 😜.
+
+#### Every thread has a small bundle of CPU-related data called its context:
+
+- Program counter (which instruction to run next)
+
+- CPU registers (temporary values)
+
+- Stack pointer (where its stack lives)
+
+- A thread ID the OS uses to refer to it
+
+This context is stored in memory in a per-thread data structure managed by the
+OS (often called a TCB, “thread control block”).
+
+#### When the OS scheduler decides to run a different thread:
+
+- It takes the context of that thread (from memory).
+
+- It loads that context into the CPU’s registers and program counter.
+
+- From the CPU’s perspective, it is now executing that thread.
+
+So “which thread am I running?” = “which context did the OS most recently load
+into my registers and program counter?”
+
+### Question: Why This Matters for Arc?
+
+This is exactly why Arc exists in Rust — when multiple threads share the same
+data, they all access the same memory. Without atomic operations on the
+reference count, two threads incrementing it simultaneously could corrupt it,
+leading to use-after-free bugs or memory leaks. Arc's atomic reference counting
+prevents this without needing a lock.
+
+### Question: What is the relationship between a thread and a process and a program?
+
+```
+ ,----------------, ,---------,
+ ,-----------------------, ," ,"|
+ ," ,"| ," ," |
+ +-----------------------+ | ," ," |
+ | .-----------------. | | +---------+ |
+ | | | | | | -==----'| |
+ | | I LOVE DOS! | | | | | |
+ | | Bad command or | | |/----|`---= | |
+ | | C:\>_ | | | ,/|==== ooo | ;
+ | | | | | // |(((( [33]| ,"
+ | `-----------------' |," .;'| |(((( | ,"
+ +-----------------------+ ;; | | |,"
+ /_)______________(_/ //' | +---------+
+ ___________________________/___ `,
+ / oooooooooooooooo .o. oooo /, \,"-----------
+ / ==ooooooooooooooo==.o. ooo= // ,`\--{)B ,"
+/_==__==========__==_ooo__ooo=_/' /___________,"
+`-----------------------------'
+ v
++--------------------------------------------------------------+
+| PROGRAM |
+| (code on disk, not running) |
++--------------------------------------------------------------+
+ | |
+ | OS loads program into memory, twice |
+ v v
++----------------------+ +----------------------+
+| PROCESS 1 | | PROCESS 2 |
+| (instance of prog) | | (instance of prog) |
+| Own address space | | Own address space |
++----------------------+ +----------------------+
+ | | | | | |
+ | | +---------------+ | | +---------------+
+ | +-------+ | | +-------+ |
+ v v v v v v
++--------+ +--------+ +--------+ +--------+ +--------+ +--------+
+|THREAD 1| |THREAD 2| |THREAD 3| |THREAD A| |TRHEAD B| |THREAD C|
++--------+ +--------+ +--------+ +--------+ +--------+ +--------+
+
+- PROGRAM: one set of instructions on disk
+- PROCESS 1 and PROCESS 2: two running instances of the same program
+- THREADs: multiple execution flows inside PROCESS 1 (PROCESS 2 could also have threads)
+```
+
+The three concepts form a clear hierarchy: a program becomes a process when run,
+and a process contains one or more threads.
+
+Program → Process → Thread
+
+Think of it like this:
+
+- A program is a static set of instructions stored on disk — it's just a file,
+ doing nothing
+
+- A process is what a program becomes when the OS loads it into memory and
+ starts executing it — it's a living, running instance of the program
+
+- A thread is the actual unit of execution inside a process — the sequence of
+ instructions the CPU is actively running
+
+### Question: Are threads allocated automatically to a process? How do I know how many threads a process runs? And how do I know how many process a program runs?
+
+Every process automatically gets exactly one thread when it starts — the main
+thread. This is the thread that begins executing at main(). Any additional
+threads beyond that must be explicitly created by the program itself — the OS
+does not add more threads on its own. So the number of threads a process has is
+entirely determined by what the programmer coded.
+
+There's no fixed number as to how many threads can a process run — it depends on
+system resources. On Linux for example, the maximum is calculated as
+
+```text
+max threads = virtual memory size ÷ (stack size × 1024 × 1024)
+```
+
+In practice, a Linux system can support tens of thousands of threads (e.g.
+~63,704 on a typical kernel). On Windows, the limit is also very high and
+practically constrained by available memory rather than a hard cap.
+
+A program typically runs as one process by default. It can spawn additional
+processes explicitly in code (e.g. using fork() in Unix or spawning subprocesses
+in Rust/Python) — but this is always a deliberate programmer choice, not
+automatic.
+
+## How to Check in Practice
+
+On Linux/macOS (terminal):
+
+- See threads for a specific process:
+
+```bash
+ps -o nlwp <pid> # shows number of threads
+cat /proc/<pid>/status | grep Threads
+```
+
+- See all processes from a program:
+
+```bash
+ps aux | grep <program_name>
+```
+
+- Live view with threads: run top, then press H to toggle thread view
diff --git a/operating_systems/user_space_and_kernel_space.md b/operating_systems/user_space_and_kernel_space.md
@@ -0,0 +1,11 @@
+# User Space and Kernel Space
+
+User space is where your application code runs with limited privileges; kernel
+space is where the OS kernel runs with full access to hardware and all memory.
+
+“User space” and “kernel space” are concepts from OS memory protection,
+privilege levels, and CPU modes.
+
+In a curriculum, you’d see them in an Operating Systems course or a Systems
+Programming / OS Internals course, often alongside processes, threads,
+scheduling, virtual memory, and interrupts.
diff --git a/personal_development/4_disciplines_of_execution.md b/personal_development/4_disciplines_of_execution.md
@@ -0,0 +1,39 @@
+# 4 Disciplines of Execution
+
+The 4 Disciplines of Execution (4DX) is a framework designed to help individuals
+and organizations achieve critical goals despite daily distractions. By
+emphasizing actionable metrics over past results, this methodology bridges the
+gap between high-level strategy and practical execution.
+
+## 1. Focus on the Wildly Important
+
+Selecting just one or two Wildly Important Goals (WIGs) ensures that your most
+critical objectives receive focused attention. The more priorities you attempt
+to juggle simultaneously, the less you will actually accomplish across the
+board. Narrowing your focus guarantees that these vital goals are not
+overshadowed by the distracting demands of routine work.
+
+## 2. Act on Lead Measures
+
+Effective execution requires distinguishing between the two types of tracking
+metrics: lag measures and lead measures. Lag measures, such as total revenue or
+weight lost, track final results that can no longer be changed once they are
+recorded. Conversely, lead measures are predictive, actionable daily
+activities—like making sales calls or exercising—that you can control to
+actively drive those end results.
+
+## 3. Keep a Compelling Scoreboard
+
+Maintaining a simple, highly visible scoreboard allows you to instantly track
+progress toward your primary objectives. People naturally perform better and
+stay more engaged when they are actively keeping score. Visualizing this
+progress provides the motivation necessary to maintain momentum and understand
+whether or not you are succeeding.
+
+## 4. Cadence of Accountability
+
+Consistent accountability is the driving force that ensures your strategic goals
+are actually executed. Establishing a regular rhythm, such as a brief weekly
+review, allows you to evaluate past commitments, update your scoreboard, and
+plan your next steps. This continuous reporting cycle prevents your primary
+goals from being swallowed up by the whirlwind of daily activities.
diff --git a/personal_development/attention_restoration_theory.md b/personal_development/attention_restoration_theory.md
@@ -0,0 +1,258 @@
+# Attention Restoration Theory
+
+# Why a walk in the park makes your head quieter
+
+There is a particular kind of tiredness that has nothing to do with the body.
+You have been staring at a screen for hours; the work is not physically
+demanding; you have eaten, you have slept. Yet something in the machinery of
+focus has worn thin. You cannot hold a thought. You reread the same paragraph
+three times. You snap at a colleague over nothing. And then—sometimes—you
+step outside, walk among trees for twenty minutes, and return able to think
+again. Attention Restoration Theory (ART) is an attempt to explain, in
+cognitive terms, why that happens.
+
+## Two kinds of attention
+
+ART begins not with nature but with a distinction William James drew in 1890
+between two modes of attention: **voluntary** and **involuntary**.[^1][^2]
+Voluntary attention is the deliberate, effortful kind—what you use to read a
+dense paper, debug code, or hold a boring conversation. It is focused,
+goal-directed, and shielded by active inhibition of whatever else is competing
+for notice. Involuntary attention is the automatic kind, pulled by stimuli
+that are interesting in themselves: a bird flying past the window, a change in
+the light, the sound of water.[^2][^3]
+
+Stephen and Rachel Kaplan, working at the University of Michigan through the
+1970s and 1980s, proposed that the first kind is a finite resource and the
+second kind is not.[^1][^4] Voluntary attention—which they renamed **directed
+attention**—depends on mechanisms of cognitive inhibition that fatigue with
+sustained use. When directed attention is depleted, the symptoms are
+recognisable: errors, impulsivity, irritability, difficulty planning,
+difficulty holding information in working memory, and a subjective sense of
+mental clutter.[^4][^5] The Kaplans called this state **directed attention
+fatigue**.[^1]
+
+Modern urban and screen-mediated life, they argued, is unusually demanding of
+directed attention. Traffic, email, open-plan offices, and most of the
+ordinary stimuli of a city require constant filtering—each competing signal
+must be actively suppressed so the task at hand can proceed.[^1][^4] The
+inhibitory system wears out. The mind begins to leak.
+
+## The restoration claim
+
+ART's central proposal is that directed attention can recover if involuntary
+attention is allowed to take over for a while.[^1][^4] This is not the same as
+sleep, which rests the whole system, nor the same as doing nothing, which may
+leave directed attention engaged in rumination. What restores is engagement
+with an environment that captures involuntary attention **gently**—enough to
+occupy the mind, not so much that it demands effortful focus or provokes
+anxious vigilance.[^4][^6]
+
+Natural environments, the Kaplans argued, are unusually well suited to this.
+A forest, a river, a meadow—these are full of stimuli that are interesting
+without being urgent: moving leaves, shifting light, birdsong, the slow
+reconfiguration of clouds. None of them require you to do anything. None of
+them are trying to sell you anything or demanding a response within
+twenty-four hours. And yet they are not empty; the mind has somewhere to
+rest.[^1][^4]
+
+## Soft fascination
+
+The critical concept here is **fascination**—the quality of a stimulus that
+holds attention effortlessly. ART distinguishes two varieties.[^4][^6]
+
+**Hard fascination** is the total capture of attention by something intense:
+a sports match, an action film, a video game, a car crash. The stimulus is
+compelling enough that directed attention is not needed, but it leaves no
+cognitive room for anything else. Hard fascination can be restorative in a
+limited sense, but it does not support the kind of quiet reflection that the
+Kaplans considered part of full restoration.[^4]
+
+**Soft fascination** is gentler. The stimulus—a sunset, a fountain, wind in
+grass—engages involuntary attention, but leaves enough mental bandwidth for
+background thought to continue. The mind wanders, drifts, considers
+unresolved questions. Directed attention is not being used, and so it
+recovers; and at the same time, something else has space to happen—what the
+Kaplans called **reflection**.[^4][^6] This is why people often report that
+their best ideas come on walks, and why problems that seemed intractable at a
+desk sometimes resolve themselves on the way home.
+
+## The four properties of a restorative environment
+
+ART specifies four components that together make an environment
+restorative.[^4][^6][^7] All four need to be present, to some degree, for the
+full effect.
+
+1. **Being away.** The environment provides psychological distance from
+ habitual demands. This can be literal (a different place) or conceptual
+ (a different mode of engagement). The test is whether the mental content
+ the place evokes is different from the content of your ordinary
+ concerns.[^6][^7]
+2. **Extent.** The setting has enough scope and coherence to occupy the mind
+ for a while. It feels like a "whole other world"—rich enough to explore,
+ structured enough to make sense.[^6][^7] A well-designed Japanese garden
+ can possess extent in a small space; a single houseplant usually cannot.
+3. **Fascination.** The environment engages involuntary attention through
+ soft, aesthetically pleasing stimuli. This is the mechanism by which
+ directed attention is spared.[^4][^6]
+4. **Compatibility.** The environment fits what you actually want to be
+ doing. A forest is not restorative for someone who finds forests
+ threatening; a library is not restorative for someone who wanted a walk.
+ Restoration requires that the setting align with the person's
+ inclinations and purposes.[^6][^7]
+
+Nature is not the only environment that can meet these criteria—places of
+worship, museums, long drives, certain cafés, and even well-designed indoor
+spaces can qualify—but the Kaplans argued that natural settings have an
+"aesthetic advantage" in supplying all four, particularly soft fascination
+and extent.[^1][^4]
+
+## The evidence
+
+The most cited experimental evidence for ART comes from a 2008 paper by Marc
+Berman, John Jonides, and Stephen Kaplan, published in _Psychological
+Science_.[^8][^9] Participants first completed a demanding backwards
+digit-span task, then took a fifty-minute walk either through the
+tree-lined Ann Arbor Arboretum or along a busy urban street, then repeated
+the task. Those who walked in nature improved their performance by roughly
+twenty percent; those who walked in the city showed no reliable
+improvement.[^8][^9] A second experiment reproduced the effect using only
+photographs of nature versus photographs of urban scenes, suggesting that
+some of the benefit survives even without physical exposure.[^8]
+
+A striking detail: the effect held in winter, when Chicago-area participants
+rated the nature walk as actively unpleasant due to cold. Whether the walk
+was enjoyable appeared not to matter for the cognitive benefit—the restoration
+was not mediated by mood.[^9][^10]
+
+Related findings have accumulated. A 2012 study by Berman and colleagues
+found that brief nature walks produced larger working-memory improvements in
+people with major depressive disorder than in healthy controls.[^9] Work on
+children aged four to eight found that a twenty-minute walk in a natural
+setting produced faster reaction times on attention tasks than an equivalent
+urban walk.[^10] Studies of window views have reported that office workers
+with views of trees and sky perform better on some cognitive and creative
+tasks than those without, and report better affect.[^11][^12]
+
+## Replication, caveats, and the messy middle
+
+ART is widely cited, but the empirical picture is less tidy than the popular
+summaries suggest. A 2016 systematic review and meta-analysis by Ohly and
+colleagues, synthesising thirty-one studies, found significant positive
+effects of nature exposure on three attention measures—Digit Span Forward,
+Digit Span Backward, and Trail Making Test B—but ten other meta-analyses
+within the same review showed no marked benefit.[^13][^14] A 2021 conceptual
+replication and meta-analysis by Stevenson and colleagues, focused on the
+Attention Network Test, concluded that simulated nature does not reliably
+restore executive attention.[^15]
+
+Several features of the literature complicate the picture. Sample sizes are
+often small. "Nature" and "urban" are defined inconsistently across
+studies—some compare forests with city centres, others compare potted plants
+with bare walls. The effect can be confounded by exercise, social
+interaction, air quality, and sunlight. The file-drawer problem—a bias
+toward publishing significant results—likely inflates the apparent
+consistency of the effect.[^1][^13][^15]
+
+The best current summary is probably this: exposure to natural environments
+does seem to benefit some aspects of cognition and mood, but the effect is
+smaller and more variable than the earliest framings suggested, and the
+exact mechanisms remain uncertain.[^13][^15] ART is a useful descriptive
+framework with real empirical support, not a precise quantitative law.
+
+## The default mode network
+
+A mechanistic bridge between ART and neuroscience has emerged through work
+on the **default mode network (DMN)**—a set of brain regions, including
+medial prefrontal cortex and posterior cingulate, that activate when
+attention is not focused on external tasks.[^1] The DMN is associated with
+mind-wandering, autobiographical memory, and creative thought. It is
+suppressed during demanding, externally focused work and rebounds during
+rest.[^1]
+
+Soft fascination, on this account, is the condition in which directed
+attention relaxes its grip enough for the DMN to come online, while
+involuntary attention remains lightly engaged with the environment. The mind
+is not idling in a closed loop—it is resting in a way that is externally
+anchored and internally active.[^1] This is why a walk in the park feels
+qualitatively different from lying in bed staring at a ceiling: both rest
+directed attention, but only the walk provides the gentle external
+scaffolding that keeps reflection productive rather than ruminative.
+
+## ART and screens
+
+One of the more practical questions raised by ART is what screen-mediated
+life does to the attention system. Most digital environments are rich in
+hard fascination rather than soft: they are engineered to capture attention
+fully, via novelty, social feedback, and intermittent reinforcement.[^1]
+They do not rest directed attention so much as bypass it—but they also do
+not leave room for reflection, and many involve precisely the kinds of
+filtering and inhibition (notifications, pop-ups, tab-switching) that
+deplete it.[^4]
+
+The testable prediction is straightforward: time spent on most social media
+and entertainment platforms should feel like rest but fail to restore
+directed attention, while time spent with soft-fascination media—slow
+nature documentaries, ambient walking videos, photographs of natural
+scenes—should produce measurable restoration. The evidence here is still
+limited, but the 2008 Berman result with photographs of nature, and later
+work on virtual nature exposure, offers preliminary support.[^8][^11]
+
+## Practical implications
+
+- **Take real breaks, not fake ones.** Scrolling a feed is not a break from
+ directed attention—it is another demand on it. A short walk outside,
+ even a few minutes at a window with a natural view, is more
+ restorative.[^8][^9]
+- **Build soft fascination into your environment.** Plants, natural light,
+ views of trees or water, nature sounds, and uncluttered visual fields all
+ contribute. The effect is modest but cumulative.[^11][^12]
+- **Distinguish rest from stimulation.** Hard-fascination activities
+ (intense films, fast games) feel refreshing but tend not to leave room
+ for reflection. Soft-fascination activities (walking, gardening,
+ unhurried conversation in a pleasant setting) do.[^4][^6]
+- **Match the environment to compatibility.** A restorative place is one
+ that fits what you want to be doing; forcing yourself into "nature"
+ against inclination may not help.[^6][^7]
+- **Use nature prophylactically before demanding work, not just after.**
+ Restoration improves subsequent performance; starting from a replenished
+ baseline is cheaper than recovering from a depleted one.[^8][^9]
+
+The underlying frame is worth holding onto even where the specific
+numbers are contested. Directed attention is a finite resource. Most of
+modern life is designed to consume it. Restoration is not optional, and
+most things that feel like restoration are not. Finding the environments
+that genuinely refill the tank—and spending time in them deliberately,
+not apologetically—is a small but serious discipline.
+
+<div align="center">⁂</div>
+
+[^1]: https://en.wikipedia.org/wiki/Attention_restoration_theory
+
+[^2]: https://hikingresearch.wordpress.com/2011/03/05/additional-benefits-of-attention-restoration/
+
+[^3]: https://kappanonline.org/merritt-going-outdoors-natural-antidote-attention-fatigue/
+
+[^4]: https://www.tandfonline.com/doi/full/10.1080/10937404.2016.1196155
+
+[^5]: https://www.sciencedirect.com/science/article/abs/pii/S0272494402001135
+
+[^6]: https://www.ecehh.org/research/attention-restoration-theory-a-systematic-review/
+
+[^7]: https://www.thrive.org.uk/how-we-help/social-therapeutic-horticulture-resource-centre/therapeutic-opportunities-in-sth/attention-restoration-theory
+
+[^8]: https://journals.sagepub.com/doi/abs/10.1111/j.1467-9280.2008.02225.x
+
+[^9]: https://psychology.uchicago.edu/news/marc-berman-thinks-you-should-take-walk-park
+
+[^10]: https://www.drjonslaughter.com/post/nature-s-power-to-restore-your-mind-the-science-behind-attention-restoration-theory
+
+[^11]: https://www.sciencedirect.com/science/article/abs/pii/S0272494422001682
+
+[^12]: https://pure.bond.edu.au/ws/portalfiles/portal/201246366/AM_Design_by_Nature.pdf
+
+[^13]: https://pubmed.ncbi.nlm.nih.gov/27668460/
+
+[^14]: https://www.sciencedirect.com/science/article/abs/pii/0272494495900012
+
+[^15]: https://www.sciencedirect.com/science/article/abs/pii/S0272494421001626
diff --git a/personal_development/do_less_to_do_more.md b/personal_development/do_less_to_do_more.md
@@ -0,0 +1,129 @@
+# Do Less to Do More
+
+# You can only do more by doing less
+
+The counterintuitive truth at the heart of high performance is this: sustainable
+output is not a function of how much you attempt, but of how clearly you
+prioritise.[^1][^2] Doing less — deliberately, strategically — is what makes
+doing more possible.
+
+## The productivity trap
+
+Most people conflate busyness with progress. A packed calendar feels productive.
+A long to-do list feels responsible. But volume of activity is not a proxy for
+value created.[^3][^4]
+
+- Cognitive load is finite. Every item you hold in active attention draws from
+ the same limited reservoir of focus.[^5][^6]
+- Context-switching compounds the cost: research consistently shows that moving
+ between tasks doesn't just pause work, it degrades the quality of all tasks
+ involved.[^7][^5]
+- Most output follows a Pareto-like distribution — roughly 20% of efforts
+ produce 80% of meaningful results.[^8][^3]
+
+The trap is that the _feeling_ of doing more is itself a reward. Crossing items
+off a list, filling your hours, maintaining a constant state of motion — these
+are all emotionally satisfying in ways that bear little relation to actual
+progress.[^3][^4]
+
+## Why less creates more
+
+Doing less is not laziness. It is the act of concentrating energy rather than
+dispersing it.[^2][^9]
+
+Consider a lens focusing sunlight. A beam scattered across a surface warms it
+gently. The same energy, narrowed to a point, can start a fire. Subtraction is
+how you build that point.
+
+- **Depth over breadth**: dedicating uninterrupted time to a single task allows
+ you to reach states of flow — where output quality and speed both increase
+ dramatically.[^10][^9]
+- **Rest as a multiplier**: recovery is not the absence of work; it is the
+ biological mechanism through which learning consolidates, creativity
+ regenerates, and performance sustains.[^11][^12]
+- **Decision bandwidth**: every additional commitment consumes decision-making
+ capacity. Reducing commitments preserves the clarity needed to act well on the
+ ones that remain.[^6][^5]
+
+## The principle in practice
+
+Applying this requires active subtraction — deciding what _not_ to do is harder
+than adding more.[^2][^13]
+
+1. **Identify your high-leverage work**
+ - Ask: "What is the one thing, that if done well, makes everything else
+ easier or unnecessary?"[^14]
+ - Protect that work with time blocks that are defended like appointments.[^9]
+2. **Prune ruthlessly**
+ - Audit your commitments regularly. For each one ask: "Would I agree to this
+ today if it were not already on my list?"[^13][^2]
+ - If the answer is no, it is a candidate for elimination or delegation.
+3. **Embrace strategic incompleteness**
+ - Not every email needs a reply. Not every meeting needs your presence. Not
+ every idea needs to be pursued.[^4][^3]
+ - Choosing to leave some things undone is a productive act, not a failure.
+4. **Work in cycles, not sprints**
+ - Pair focused effort with genuine recovery. The ultradian rhythm — roughly
+ 90 minutes of focused work followed by rest — aligns with natural peaks of
+ cognitive performance.[^11][^12]
+ - Pushing through fatigue does not extend your output; it degrades it.
+5. **Measure outcomes, not hours**
+ - Shift your self-assessment from "how long did I work?" to "what did I
+ actually move forward?"[^1][^8]
+ - This reframes rest and leisure as inputs, not indulgences.
+
+## Where resistance comes from
+
+The practical obstacles to doing less are rarely logistical — they are
+psychological.[^3][^13]
+
+- **Identity**: many people have fused their sense of worth with their
+ busyness. Slowing down can feel like a moral failure.[^4][^3]
+- **Social signalling**: being busy is culturally legible as being important.
+ Choosing fewer things can feel socially costly.[^3]
+- **Fear of missing out**: every commitment declined feels like an opportunity
+ lost, even when the real opportunity cost is the focus you could have applied
+ elsewhere.[^2][^13]
+
+Recognising these forces allows you to choose differently — not because it is
+easy, but because you understand what the alternative actually costs.
+
+## A useful reframe
+
+Think of your attention as a currency. You can spend it in a hundred small
+places, or invest it in a few that compound. The goal is not to do as little as
+possible, but to do only what matters — and to do that well.[^1][^9][^14]
+
+The people who produce the most meaningful work over a lifetime are rarely those
+who attempted the most. They are those who maintained the discipline to say no
+clearly, and yes deeply.[^2][^9]
+
+<div align="center">⁂</div>
+
+[^1]: https://jamesclear.com/productivity
+
+[^2]: https://gregmckeown.com/books/essentialism/
+
+[^3]: https://www.apa.org/topics/mental-health/busyness-burnout
+
+[^4]: https://hbr.org/2019/03/to-be-more-productive-do-less
+
+[^5]: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7075496/
+
+[^6]: https://www.apa.org/research/action/multitask
+
+[^7]: https://www.ics.uci.edu/~gmark/CHI2005.pdf
+
+[^8]: https://tim.blog/2007/07/11/the-4-hour-workweek-first-2-chapters/
+
+[^9]: https://calnewport.com/books/deep-work/
+
+[^10]: https://www.researchgate.net/publication/224927532_Flow_The_Psychology_of_Optimal_Experience
+
+[^11]: https://www.sleepfoundation.org/sleep-science/ultradian-rhythm
+
+[^12]: https://hbr.org/2010/10/the-productivity-paradox-how-s
+
+[^13]: https://sive.rs/hellyeah
+
+[^14]: https://www.the1thing.com/
diff --git a/personal_development/egoless_programming.md b/personal_development/egoless_programming.md
@@ -0,0 +1,93 @@
+# Egoless Programming
+
+1. **Understand and accept that you will make mistakes.**
+
+ No one writes perfect code. Bugs, oversights, and misjudgments happen to
+ everyone, including experts. The key is to treat mistakes as opportunities to
+ learn rather than sources of embarrassment. An egoless programmer fixes
+ issues calmly, analyzes why they happened, and improves their process instead
+ of defending errors.
+
+ Example: When a bug is found during testing, thank the person who caught it
+ instead of feeling attacked.
+
+2. **You are not your code.**
+
+ Code reviews exist to improve the product, not to judge the programmer. The
+ goal is to find bugs, catch design flaws, and ensure consistency — not to
+ criticize someone’s abilities. When your code is reviewed, remember it’s a
+ normal, expected part of building reliable software. Feedback is about making
+ the code better together, not about you being wrong.
+
+ Example: Think, “We’re finding issues in the code so it’s stronger,” not
+ “They’re finding faults in me.”
+
+3. **No matter how much “karate” you know, someone else will always know more.**
+
+ There’s always a more experienced or specialized developer out there.
+ Recognizing this keeps you humble and open to learning. Instead of competing
+ for superiority, treat discussions as chances to improve together. Curiosity
+ will make you a better programmer than pride ever could.
+
+4. **Don’t rewrite code without consultation.**
+
+ Even if existing code looks inefficient or messy, consult the team before
+ reworking it. There’s a fine line between fixing code and rewriting it — know
+ the difference. Pursue changes as part of a code review, not as a lone
+ enforcer. Unilateral rewrites can waste time, break dependencies, or erase
+ important context.
+
+ Example: Before refactoring a module, talk to the original author to
+ understand the decisions and constraints behind it.
+
+5. **Treat people who know less than you with respect, deference, and
+ patience.**
+
+ Mentorship builds stronger teams. Talking down to less experienced developers
+ discourages them and weakens collaboration. Share knowledge with patience and
+ humility — remember, you were once a beginner too. Nontechnical colleagues
+ often see developers as prima donnas or overly sensitive; don’t reinforce
+ that stereotype with anger or impatience. The way you treat newcomers and
+ teammates sets the tone for your team’s culture and reputation.
+
+6. **The only constant in the world is change.**
+
+ Technology evolves quickly—languages, frameworks, and best practices
+ constantly shift. Adaptability is more important than mastery of a single
+ tool. Be ready to learn new approaches and let go of old habits.
+
+ Example: Moving from monoliths to microservices, or from manual deployments
+ to CI/CD pipelines, requires flexibility.
+
+7. **The only true authority stems from knowledge, not from position.**
+
+ Titles and hierarchy shouldn’t override logic or facts. In a healthy
+ engineering culture, ideas win on merit, not on who said them. Leaders should
+ earn respect through understanding and clear reasoning, not through formal
+ rank.
+
+8. **Fight for what you believe, but gracefully accept defeat.**
+
+ If you think something is important, advocate for it with evidence and
+ conviction. But once the team decides, support the outcome even if it wasn’t
+ your proposal. This balance between assertiveness and teamwork builds trust
+ and keeps projects moving forward.
+
+9. **Don’t be “the guy in the room.”**
+
+ This means don’t act like you’re smarter than everyone else. The “guy in the
+ room” (or “hero programmer”) dominates conversations, dismisses others, and
+ resists compromise. Avoid that behavior; it alienates teammates and stifles
+ creativity. Good programmers make the entire team better, not just
+ themselves.
+
+10. **Critique code instead of people – be kind to the coder, not to the code.**
+
+ When reviewing code, focus on improving it, not judging the person behind
+ it. Tone matters: say “This function could be clearer if…” rather than “You
+ didn’t write this well.” Constructive feedback builds confidence and shared
+ understanding.
+
+([The Psychology of Computer Programming](https://archive.org/details/psychologyofcomp00unse)
+by Gerald M. Weinberg, describes principles for healthy, collaborative software
+development that values humility, learning, and teamwork over ego or rivalry.)
diff --git a/personal_development/engineering_competency.md b/personal_development/engineering_competency.md
@@ -0,0 +1,29 @@
+# Engineering Competency
+
+## Levels of Competence
+
+- Level 1 - Unconscious Incompetence
+- Level 2 - Conscious Incompetence
+- Level 3 - Conscious Competence
+- Level 4 - Unconscious Competence
+
+## Zone of Proximal Development
+
+- Be able to identify gaps of knowledge and expanding gradually through Zone of
+ Proximal Development.
+
+- So no jumping ahead, and no deliberately trying to be fast.
+
+- Need to work on accuracy first and then consistency so fluency / speed will
+ follow naturally.
+
+- Also that trying to learn something too far away from current comfort zone is
+ unachievable - need to expand gradually through both vertical and lateral
+ learning challenges.
+
+- There is no such thing as unconscious learning, learning has to be conscious.
+ Conscious Learning.
+
+## Engineering Competency Matrix
+
+[Circle CI Engineering Competency Matrix](/assets/circle_ci_engineering_competency_matrix.xlsx)
diff --git a/personal_development/first_principles.md b/personal_development/first_principles.md
@@ -0,0 +1,164 @@
+# First Principles
+
+# Learning from first principle
+
+Learning from first principles means stripping away “this is how it’s done” and
+rebuilding your understanding from basic truths you can justify yourself.[^1][^2]
+
+## What “first principles” actually means
+
+- It is a way of thinking where you break a problem or topic into its
+ **fundamental** components, then reason upward from those.[^2][^3]
+- Instead of copying existing methods (analogies, best practices, standard
+ curricula), you ask what is absolutely true, what is just habit, and what
+ follows logically from the basics.[^4][^1]
+- Elon Musk popularised this in business and engineering by breaking complex
+ things (like rockets, batteries, car costs) into raw materials, physics, and
+ constraints, then designing from there.[^5][^3][^2]
+
+A simple illustration: rather than asking “How do I build a better horse
+carriage?”, you ask “What is transportation fundamentally? Move people from A to
+B, with safety, speed, cost, comfort constraints” and you may end up with a car
+instead of a better carriage.[^6][^7]
+
+## The core steps (general recipe)
+
+Most guides converge on a similar practical loop you can apply to anything.[^3][^7][^1][^4]
+
+1. Clarify the problem or question
+ - State what you’re trying to understand or improve in one clear sentence.[^8][^3]
+ - Example: “I want to understand how learning actually works so I can study
+ more effectively.”
+2. List and question your assumptions
+ - Write down what you “think you know” or what everyone seems to believe
+ about this topic.[^1][^4][^3]
+ - For each item, ask:
+ - “How do I know this is true?”
+ - “What if the opposite were true?”[^4][^3][^1]
+3. Deconstruct to fundamental truths
+ - Break the topic into the simplest elements that are as close as possible to
+ facts of reality: physics, biology, logic, basic data, clear cause–effect.[^7][^2][^3]
+ - Fundamental truths are things you could defend with evidence or clear
+ reasoning, not just “everyone says so.”[^3][^1][^4]
+4. Rebuild from the ground up
+ - Starting only from your fundamentals, reconstruct an explanation, method,
+ or plan.[^2][^7][^3]
+ - Ask: “Given only these basics, what follows? What’s the simplest solution
+ that fits them?”[^2][^3]
+5. Test, get feedback, refine
+ - Try your new understanding or method in reality, observe results, then
+ adjust your fundamentals or reasoning if needed.[^5][^3]
+
+## How to use it to _learn_ any subject
+
+Here’s how to apply this when learning something new (math, physics,
+programming, finance, etc.).[^9][^7][^3]
+
+1. Start with ultra-basic questions
+ - “What is this thing, in the simplest possible terms?”
+ - “What problem is this concept solving?”[^7]
+ - Example (calculus): “A derivative is fundamentally a way to describe how
+ fast something changes.”
+2. Build a minimal foundation
+ - Identify the prerequisite ideas you genuinely need, and learn those clearly
+ (even if it means going “backwards” to simpler topics).[^9][^3][^7]
+ - Use the Feynman approach: try explaining the concept as if to a child; gaps
+ reveal what is not fundamental or not understood yet.[^7]
+3. Constantly interrogate explanations
+ - When you read or watch something, pause and ask:
+ - “Which parts are definitions or conventions, and which parts are
+ unavoidable truths?”
+ - “Could this be different in another system or context?”[^1][^4][^3]
+4. Connect to reality and intuition
+ - Tie concepts to real or visual examples until they “click”.[^9][^7]
+ - Example: for probability, think in terms of games, lotteries, or real-life
+ risks; for physics, think in terms of everyday forces and motions.[^9]
+5. Practice “why” chains
+ - Use “Five Whys” or similar: keep asking “why?” until you hit something you
+ can’t reasonably question further (a first principle or definition).[^3]
+ - Example:
+ - “Why do spaced repetitions help memory?” → Because repeated recall
+ strengthens neural connections.
+ - “Why does recall matter?” → Because memory is about reactivating patterns
+ of activity in the brain, and practice makes them easier to trigger.
+
+## Concrete daily exercises to build the habit
+
+To _learn how to learn_ from first principles, turning it into a daily habit is
+crucial.[^3]
+
+Try these simple exercises:
+
+- One-problem deconstruction (5–10 minutes)
+ - Pick any small problem (how you organise your day, why you procrastinate,
+ how you cook something).
+ - Write: problem → assumptions → fundamentals → one improved solution.[^8][^3]
+- Explain-it-like-I’m-10 journal
+ - After studying something, write a one-paragraph explanation as if to a
+ child.
+ - Highlight any sentence where you hid behind jargon; revisit that part of the
+ material.[^7]
+- “Opposite day”
+ - Take a common belief in your field (“you must take notes by hand”, “you need
+ 8 hours to be productive”, etc.).
+ - Argue the opposite for a few minutes; look for data or cases that support
+ it.[^4][^1][^3]
+- Weekly “rebuild”
+ - Once a week, choose one skill or topic you use often (how you study, how you
+ work out, how you manage tasks).
+ - Strip it to fundamentals and redesign your method from scratch.[^2][^3]
+
+## Example: learning more effectively (from first principles)
+
+Let’s walk through a quick example applying this to “how to study”.
+
+1. Clarify
+ - “I want to learn faster and remember more from what I read.”
+2. Question assumptions
+ - Assumptions might include: “Re-reading is good studying”, “long sessions
+ are better than short ones”, “highlighting equals learning.”[^1][^4]
+3. Deconstruct learning to fundamentals
+ - Basic truths from cognitive science:
+ - Memory improves with active recall, not just exposure.
+ - Spaced repetition beats cramming.
+ - Understanding (connecting ideas) leads to longer retention than rote
+ memorisation.[^3][^9][^7]
+4. Rebuild a study method
+ - From those fundamentals, a study system might be:
+ - Short sessions with frequent recall (close the book and explain from
+ memory).
+ - Use flashcards spaced over days instead of single long cram sessions.
+ - After reading, make a simple concept map linking ideas instead of just
+ highlighting.[^9][^7][^3]
+5. Test and refine
+ - Try this for two weeks, track how much you remember on quick self-tests,
+ then adjust which tactics actually move the needle.
+
+If you repeat this pattern on different parts of your life, you train yourself
+to default to first principles thinking rather than habits or expert “scripts”.[^7][^3]
+
+What area do you most want to apply first-principles learning to right now
+(e.g., math, career decisions, productivity, something else)?
+<span style="display:none">[^10]</span>
+
+<div align="center">⁂</div>
+
+[^1]: https://fs.blog/first-principles/
+
+[^2]: https://blog.btrax.com/first-principles-thinking/
+
+[^3]: https://www.maray.ai/posts/first-principles-thinking
+
+[^4]: https://www.readynorth.com/blog/what-is-first-principles-thinking
+
+[^5]: https://theinvisiblementor.com/using-first-principles-to-approach-difficult-problems-like-elon-musk/
+
+[^6]: https://www.youtube.com/watch?v=NV3sBlRgzTI
+
+[^7]: https://modelthinkers.com/mental-model/first-principle-thinking
+
+[^8]: https://www.rhysthedavies.com/first-principle-thinking/
+
+[^9]: https://www.reddit.com/r/Physics/comments/11nrl20/how_do_i_embrace_the_first_principles_reasoning/
+
+[^10]: https://www.youtube.com/watch?v=ooTnMMnrOTo
diff --git a/personal_development/index.md b/personal_development/index.md
@@ -0,0 +1,3 @@
+# Personal Development
+
+Knowledge about Personal Development
diff --git a/personal_development/learn_how_to_learn.md b/personal_development/learn_how_to_learn.md
@@ -0,0 +1,29 @@
+# Learn How To Learn
+
+> If you really want to learn, and you want to learn more quickly and more
+> deeply, pay attention to how you pay attention. Think about how you think.
+> Learn how you learn.
+
+> Most of us did not take courses on metacognition or learning theory when we
+> were growing up. We were expected to learn, but rarely taught to learn.
+
+(Head First Design Patterns)
+
+1. Slow down. The more you understand, the less you have to memorize. - Don’t
+ just read. Stop and think.
+
+2. Do the exercises. Write your own notes. - Use a pencil. There’s plenty of
+ evidence that physical activity while learning can increase the learning.
+
+3. There are No Dumb Questions
+
+4. Talk about it. - Speaking activates a different part of the brain.
+
+5. Listen to your brain. - Pay attention to whether your brain is getting
+ overloaded.
+
+6. Feel something! Get involved with the stories. Make up your own captions or
+ images about the topics.
+
+7. Design something! Apply the knowledge to something new you’re designing, or
+ refactor an older project.
diff --git a/personal_development/rule_of_the_vital_few.md b/personal_development/rule_of_the_vital_few.md
@@ -0,0 +1,65 @@
+# The Rule of the Vital Few
+
+# The Rule of the Vital Few: Leveraging the Pareto Principle for Deep Work
+
+The "Rule of the Vital Few" is a conceptual framework derived from the Pareto Principle, which observes that in many systems, roughly 80% of the effects come from 20% of the causes.[^1][^2] While often simplified as the "80/20 Rule," the term "Vital Few" was specifically coined by Joseph Juran to distinguish between the critical factors that drive a system and the "trivial many" that create noise without significant value.[^3]
+
+When applied to personal productivity and cognitive performance, this rule suggests that a small minority of your activities produce the vast majority of your meaningful progress.
+
+## The Dichotomy: Vital Few vs. Trivial Many
+
+To achieve peak performance, one must first categorize their workload into two distinct groups:
+
+- **The Vital Few**: High-leverage activities (HLAs). These are tasks that require intense concentration, complex problem-solving, and creative synthesis. They are the activities that move the needle on your long-term goals (e.g., writing a complex architecture doc, learning a new language, deep strategic planning).[^4]
+- **The Trivial Many**: Shallow work. These are logistical or administrative tasks that can be performed while distracted. They are necessary for maintenance but do not create significant value on their own (e.g., clearing an inbox, attending status meetings, routine data entry).[^4]
+
+The danger in most professional environments is the **"Efficiency Trap"**: the tendency to spend 80% of one's energy optimizing the trivial many, leaving only 20% for the vital few.
+
+## The Synergy with Deep Work
+
+Deep Work—the ability to focus without distraction on a cognitively demanding task—is the only mechanism capable of executing the "Vital Few" effectively.[^4]
+
+Because the vital few tasks are complex, they require a state of flow and the absence of "attention residue." Attention residue occurs when you switch from a task (even a trivial one) to another; a part of your attention remains stuck on the previous task, reducing your cognitive capacity for the new one.[^5]
+
+If you intersperse the vital few with the trivial many (e.g., checking email every 15 minutes while coding a complex algorithm), you never reach the depth required to produce high-quality output. You are essentially attempting to perform "Vital" work with "Trivial" cognitive resources.
+
+## Applying the Rule to Minimize Distraction
+
+To minimize distraction, you must shift your perspective: distraction is not just a lack of willpower, but a failure to protect the vital few from the trivial many.
+
+### 1. The Pareto Audit
+List every activity you perform in a typical week. Assign a "Value Score" (1-10) based on how much it contributes to your primary long-term objective.
+- **Vital Few**: Tasks scoring 8–10.
+- **Trivial Many**: Tasks scoring 1–5.
+
+### 2. Aggressive Pruning
+Once identified, apply the following strategies to the trivial many:
+- **Eliminate**: Does this task actually need to happen?
+- **Automate**: Can a script or tool handle this?
+- **Delegate**: Can this be handled by someone else?
+- **Batch**: Group all trivial tasks into a single "shallow work" block (e.g., 4 PM to 5 PM).
+
+### 3. Guarding the Deep Work Block
+Schedule your "Vital Few" tasks during your peak biological energy windows (typically the morning). During these blocks:
+- **Zero-Interrupt Policy**: Turn off all notifications.
+- **Physical Separation**: If possible, change your environment to signify the shift from "trivial" to "vital" mode.
+- **Single-Tasking**: Commit to one vital task. Multitasking is the fastest way to turn a vital activity into a trivial one.
+
+## The Practical Framework for Implementation
+
+| Step | Action | Objective |
+| :--- | :--- | :--- |
+| **Audit** | Log time for 5 days $\rightarrow$ Value Score | Identify the 20% (Vital Few) |
+| **Filter** | Prune $\rightarrow$ Batch $\rightarrow$ Automate | Shrink the 80% (Trivial Many) |
+| **Schedule** | Time-block morning for HLAs | Allocate peak energy to high value |
+| **Protect** | Digital detox $\rightarrow$ Do Not Disturb | Eliminate attention residue |
+
+By consciously ignoring the trivial many during your deep work windows, you are not being unproductive—you are optimizing your cognitive throughput for the activities that actually matter.
+
+<div align="center">⁂</div>
+
+[^1]: https://fs.blog/pareto-principle/
+[^2]: https://en.wikipedia.org/wiki/Pareto_principle
+[^3]: https://jerimaerowley.com/blog-topics/business/80-20-rule-of-the-vital-few
+[^4]: https://www.calnewport.com/books/deep-work/
+[^5]: https://hbr.org/2010/12/the-cost-of-fast-switching
diff --git a/personal_development/social_media_detox.md b/personal_development/social_media_detox.md
@@ -0,0 +1,187 @@
+# Social Media Detox
+
+# Social Media Detox — Distancing Yourself to Reclaim Deep Focus
+
+> _"The ability to perform deep work is becoming increasingly rare at exactly
+> the same time it is becoming increasingly valuable in our economy."_ — Cal
+> Newport, _Deep Work_
+
+---
+
+## The Fragmentation Problem
+
+Every time you open Twitter, scroll through Instagram, or check a notification,
+you are not just losing a few seconds — you are fragmenting the architecture of
+your attention. Cal Newport calls this the shift from **deep work** to **shallow
+work**: the replacement of cognitively demanding, value-producing concentration
+with noncognitively demanding, easily replicated tasks performed while
+distracted.
+
+The average knowledge worker, according to a 2012 McKinsey study Newport cites,
+spends more than 60 percent of their workweek engaged in electronic
+communication and internet searching. This is not productivity — it is the
+_simulacrum_ of productivity. Busyness masquerading as output.
+
+Social media amplifies this fragmentation to an extreme. Platforms are not
+neutral tools. They are engineered to colonize attention: personalized feeds,
+intermittent variable rewards, notification pulses, and infinite scroll are
+deliberate design choices made to maximize time-on-platform. Newport is blunt
+about this: these services are "products, developed by private companies, funded
+lavishly, marketed carefully, and designed ultimately to capture then sell your
+personal information and attention to advertisers."
+
+---
+
+## The Deep Work Lens
+
+Newport defines **deep work** as: _"Professional activities performed in a state
+of distraction-free concentration that push your cognitive capabilities to their
+limit. These efforts create new value, improve your skill, and are hard to
+replicate."_
+
+Its opposite — **shallow work** — is what social media produces and demands.
+Liking posts, replying to threads, and composing 280-character opinions are
+shallow by definition. They require no cognitive depth, they produce no lasting
+value, and crucially, they _train_ your brain for distraction.
+
+This is not a metaphor. Newport references Nicholas Carr's argument that
+spending prolonged time in a state of frenetic shallowness permanently reduces
+your capacity to concentrate. Every hour you spend in distracted scrolling is an
+hour spent _practicing_ an inability to focus. The neurological muscle
+responsible for sustained concentration atrophies — not through disuse alone,
+but through active misuse.
+
+---
+
+## The 30-Day Experiment — Newport's Prescription
+
+Newport proposes a direct intervention: a **30-day cold-turkey abstention** from
+all social media services. Not a deactivation announcement, not a dramatic
+goodbye post — just stop, quietly, without ceremony. He gives two specific
+reasons for the silence:
+
+1. Announcing it feeds the same validation-seeking behavior that social media
+ exploits
+2. It forces an honest reckoning: will anyone notice? Will anything suffer?
+
+After 30 days, ask yourself two questions about each platform:
+
+- Would the last 30 days have been _notably_ better if I had used this service?
+- Did people _genuinely care_ that I was absent?
+
+For most people, the answer to both is no. This is not pessimism — it is a
+reality check on the inflated sense of importance that social media manufacture.
+Newport notes that the average Twitter user had around 208 followers at the time
+of writing. When you know 200 people opted in to receive your updates, it
+becomes easy to believe your activity matters. The 30-day gap dissolves that
+illusion.
+
+---
+
+## Distancing from the Social Stream
+
+The 30-day experiment is a diagnostic. The deeper practice is structural
+distancing — rebuilding your life so that social media occupies the periphery
+rather than the center.
+
+This involves several shifts:
+
+**Treat tools instrumentally, not habitually.** Newport advocates the
+**craftsman approach** to tool selection: only adopt a network tool if its
+positive impacts on what you value substantially outweigh the negatives. This is
+a higher bar than "it might be occasionally useful." Apply it ruthlessly. Most
+social platforms fail it.
+
+**Protect your attention like a resource.** Newport observes that J.K. Rowling
+avoided social media throughout the writing of the Harry Potter series. Woody
+Allen never owned a computer. Neal Stephenson makes himself near-impossible to
+reach electronically. These are not eccentricities — they are strategic acts of
+cognitive self-preservation.
+
+**Replace the vacuum.** Newport draws on Arnold Bennett's insight from a century
+ago: if you leave leisure time unstructured, the lowest-friction option fills
+it. Social media thrives in vacuums. The solution is not willpower but
+pre-commitment — decide what your evenings and free hours are _for_ before they
+begin. Read a deliberately chosen book. Exercise. Pursue a structured hobby.
+Give your brain something real to do.
+
+---
+
+## A Detox from Society's Noise
+
+There is a broader dimension beyond productivity. The case for distancing from
+social media is also a case for distancing from the **ambient noise of
+collective opinion** — the reflexive takes, the outrage cycles, the performative
+discourse that constitutes most of what flows through social feeds.
+
+Carl Jung did not retreat to his stone tower in Bollingen merely to write
+faster. He retreated because deep intellectual work requires a kind of
+**internal silence** that constant social input dissolves. Newport quotes Jung
+directly: _"In my retiring room I am by myself. I keep the key with me all the
+time — no one else is allowed in there except with my permission."_
+
+That room is not a geographic location — it is a cognitive state. Social media
+is the lock pick that opens it uninvited, dozens of times a day. Quitting, or
+severely limiting, these platforms is one way of taking the key back.
+
+The modern version of this retreat does not require a lakeside tower. It
+requires:
+
+- Deleting apps from your phone (friction is a surprisingly effective deterrent)
+- Establishing phone-free periods — mornings, meals, the first hour after waking
+- Using a dedicated browser profile with social media blocked during work hours
+- Treating your attention as something given deliberately, not taken
+ automatically
+
+---
+
+## What You Gain
+
+Newport's own life is the clearest data point. Without social media — no
+Twitter, no Facebook, no Instagram — he published four books in ten years,
+earned a PhD, wrote peer-reviewed academic papers at a high rate, and was hired
+as a tenure-track professor at Georgetown, all while rarely working past six
+p.m. His evenings were present, his reading was genuine, and — his phrase is
+worth keeping — "the lack of distraction tones down that background hum of
+nervous mental energy that seems to increasingly pervade people's daily lives."
+
+That background hum is familiar. It is the low-grade anxiety of an always-open
+loop: what did someone post, what did someone reply, what is happening right now
+that you are not watching. Closing that loop — not temporarily, but structurally
+— is a form of mental hygiene.
+
+Depth is not nostalgia. In an economy that increasingly rewards the ability to
+learn difficult things quickly and produce at an elite level, distraction is not
+a neutral lifestyle choice. It is a competitive disadvantage that accumulates
+quietly over years.
+
+---
+
+## Where to Begin
+
+If a permanent quit feels too drastic, start with the structure Newport
+outlines:
+
+1. **Run the 30-day experiment** — cold turkey, no announcements, honest audit
+ afterward
+2. **Delete the highest-pull apps** from your phone first — Instagram and TikTok
+ in particular are designed for mobile, idle-moment consumption
+3. **Establish a daily deep work block** — even 90 minutes of phone-off,
+ notification-free work will begin rebuilding the attention muscle
+4. **Fill leisure time intentionally** — books, exercise, craft, in-person
+ conversation; anything that requires and rewards sustained engagement
+5. **Evaluate each platform by the craftsman standard** — does this tool serve
+ my top-tier goals substantially, or just marginally? If marginally, remove it
+
+The goal is not asceticism. It is reclaiming the cognitive architecture that
+makes meaningful work — and meaningful rest — possible.
+
+---
+
+## References
+
+- Newport, Cal. _Deep Work: Rules for Focused Success in a Distracted World_.
+ Grand Central Publishing, 2016.
+- Carr, Nicholas. _The Shallows: What the Internet Is Doing to Our Brains_. W.W.
+ Norton, 2010.
+- Bennett, Arnold. _How to Live on 24 Hours a Day_. 1908.
diff --git a/personal_development/theory_of_knowledge.md b/personal_development/theory_of_knowledge.md
@@ -0,0 +1,54 @@
+# Theory of Knowledge
+
+# Philoshophy
+
+## I think therefore I am.
+
+René Descartes discovered certainty by doubting everything until he found
+something indubitable: the act of thinking itself. Because he was thinking (and
+doubting), he knew he must exist, famously concluding "Cogito, ergo sum" ("I
+think, therefore I am"). He rejected sensory evidence, using radical doubt to
+establish that only the existence of his own mind is certain.
+
+### Question: How do we know anything?
+
+How to Know (Descartes’ Method):
+
+- **Method of Radical Doubt:** Descartes decided to reject any idea that could
+ be doubted, including the existence of his body, the physical world, and
+ sensory input, which he noted could be deceptive.
+
+- **The Cogito ("I think, therefore I am"):** Even if a "demon" is deceiving
+ him, or he is dreaming, he must exist to be deceived or to have those
+ thoughts. The very act of doubting one's existence confirms it.
+
+- **Rationalism:** Knowledge is not gained through senses but by clear and
+ distinct perceptions of the mind (reason).
+
+- **God as Guarantee:** Descartes argued that because he has an idea of a
+ perfect God, that perfect being must exist and would not allow him to be
+ fundamentally deceived about reality.
+
+## Propositions
+
+```
+ ____________ ___________
+ / \ / \
+ / X \
+ / / \ \
+ / / \ \
+ / / \ \
+/ / KNOW- \ \
+\ TRUTHS \ LEDGE / BELIEFS /
+ \ \ / /
+ \ \ / /
+ \ \ / /
+ \ X /
+ \____________/ \____________/
+```
+
+- Left circle: Truths (all true propositions).
+
+- Right circle: Beliefs (all propositions someone believes).
+
+- Overlap: Knowledge (propositions that are both true and believed).
diff --git a/personal_development/unconscious_thought_theory.md b/personal_development/unconscious_thought_theory.md
@@ -0,0 +1,212 @@
+# Unconscious Thought Theory
+
+# The case for doing less thinking on purpose
+
+There is a counterintuitive finding in decision science: for complex problems
+with many variables, people who are _distracted_ after absorbing the relevant
+information often make better choices than people who sit down and deliberately
+think it through.[^1][^2] This is the core claim of Unconscious Thought Theory
+(UTT), proposed by Ap Dijksterhuis and Loran Nordgren in 2006. The broader
+implication—that idleness, mind-wandering, and strategic laziness are not flaws
+but cognitive tools—connects to a much older philosophical thread about the
+value of doing nothing.
+
+## Unconscious Thought Theory
+
+UTT distinguishes between two modes of thought: conscious and unconscious.
+Conscious thought is deliberate, sequential, and constrained by working memory
+(you can hold roughly seven items at once). Unconscious thought runs in the
+background with no such bottleneck—it processes in parallel, weighs attributes
+more naturally, and operates without the distortions introduced by
+overthinking.[^1][^3]
+
+The key experimental paradigm works like this: participants are given complex
+information (e.g., twelve attributes about each of four apartments), then split
+into three groups. One group decides immediately. One deliberates consciously
+for several minutes. One is distracted by a puzzle, then asked to decide.
+Consistently, the distracted group—the one that could not consciously
+deliberate—chose the best option more often than either of the other
+groups.[^1][^2]
+
+Dijksterhuis and Nordgren called this the **deliberation-without-attention
+effect**. The principle: for simple decisions (few variables), conscious thought
+is fine and often better. For complex decisions (many variables, competing
+trade-offs), unconscious thought tends to produce superior outcomes because it
+can integrate more information without the serial bottleneck of conscious
+attention.[^2][^4]
+
+### The six principles of UTT
+
+UTT rests on six distinguishing principles between conscious and unconscious
+thought.[^1][^3]
+
+1. **Unconscious Thought Principle.** Conscious thought is thought _with_
+ attention; unconscious thought is thought _without_ attention directed at the
+ problem.
+2. **Capacity Principle.** Conscious thought has low capacity (serial
+ processing); unconscious thought has high capacity (parallel processing).
+3. **Bottom-Up vs. Top-Down Principle.** Unconscious thought works bottom-up,
+ integrating information as it comes. Conscious thought works top-down, guided
+ by schemas and expectations—which can introduce bias.
+4. **Weighting Principle.** Unconscious thought naturally weights the relative
+ importance of different attributes. Conscious thought tends to distort
+ weightings, overemphasising whatever is most salient or recently attended to.
+5. **Rule Principle.** Conscious thought follows strict rules and is precise.
+ Unconscious thought gives rough estimates. This makes conscious thought
+ better for simple, rule-based problems.
+6. **Convergence vs. Divergence Principle.** Conscious thought converges on a
+ solution; unconscious thought diverges and explores. This is why unconscious
+ processing is linked to creativity.
+
+### Criticisms and nuance
+
+UTT is not without controversy. Several replication attempts have failed to
+reproduce the deliberation-without-attention effect, particularly when the
+experimental design includes a clearly dominant alternative.[^5] Critics argue
+that what appears to be "unconscious thought" may simply be the absence of
+conscious overthinking—that distraction prevents people from _worsening_ an
+already-formed first impression, rather than enabling a deeper unconscious
+process.[^6] The effect also disappears when participants are primed to feel
+powerful, suggesting the mechanism may be more about restraining conscious
+interference than empowering unconscious processing.[^5]
+
+Regardless of the exact mechanism, the practical takeaway is robust: for
+complex, multi-variable decisions, sleeping on it or stepping away tends to
+produce better outcomes than grinding through the options consciously.
+
+## Being lazy ambitious
+
+There is a concept—sometimes called "lazy ambitious"—that describes people who
+hold strong long-term vision but resist the compulsion to fill every hour with
+visible productivity.[^7] The phrase sounds like a contradiction, and that is
+the point. Productivity culture insists that ambition must be expressed through
+relentless action: early mornings, optimised routines, quantified output. The
+lazy ambitious person rejects this, not out of apathy, but from an intuition
+that the mind's best work happens in the gaps between effort.
+
+Bill Gates captured a related idea: "I choose a lazy person to do a hard job.
+Because a lazy person will find an easy way to do it."[^8] The "laziness" here
+is actually efficiency—the refusal to accept unnecessary complexity, the drive
+to find a more elegant path. It is not about doing nothing; it is about doing
+less of the wrong thing.
+
+What looks like laziness from the outside is often unconscious processing on
+the inside. Carl Jung called it "active imagination"—the conscious mind steps
+back, and the unconscious delivers insights that no amount of forced effort
+could produce.[^7] The lazy ambitious person's downtime is not a bug. It is the
+incubation period where unconscious connections form.
+
+## The neuroscience of idleness
+
+When the brain is not focused on an external task, it does not go quiet. It
+activates the **default mode network (DMN)**—a set of interconnected regions
+associated with daydreaming, self-reflection, memory consolidation, and
+creative thinking.[^9][^10]
+
+Research has shown that the DMN is causally linked to creative fluency. Direct
+cortical stimulation of DMN nodes disrupts the ability to generate original
+ideas, confirming the network's role is not incidental but functional.[^10]
+High-creative individuals show greater resting-state functional connectivity
+between the inferior frontal gyrus (associated with cognitive control) and the
+entire default mode network, suggesting that creativity depends on cooperation
+between controlled and spontaneous processes.[^11]
+
+This is the mechanism behind the shower thought, the walk-and-eureka moment,
+the answer that arrives the morning after you gave up. The incubation effect—
+stepping away from a problem to let a solution surface—relies on the DMN
+quietly sifting through memories, associations, and patterns while the
+conscious mind is occupied elsewhere.[^9]
+
+## In Praise of Idleness
+
+Bertrand Russell argued in his 1935 essay _In Praise of Idleness_ that "a
+great deal of harm is being done in the modern world by belief in the
+virtuousness of work."[^12] His case was both economic and philosophical:
+modern technology had made it possible to produce enough for everyone with far
+less labour, yet society insisted on overwork as a moral duty. Russell proposed
+a four-hour workday, with the remaining time devoted to whatever the individual
+found meaningful.[^12][^13]
+
+Russell's sharpest observation was about what overwork does to the mind:
+
+> The modern man thinks that everything ought to be done for the sake of
+> something else, and never for its own sake.[^13]
+
+This is the cult of productivity taken to its endpoint—every action justified
+only by its instrumental value, every moment optimised for output. What gets
+lost is the capacity for play, for exploration, for the kind of undirected
+thought that UTT tells us is essential for complex problem-solving and
+creativity.
+
+Oscar Wilde put it more bluntly: "To do nothing at all is the most difficult
+thing in the world, the most difficult and the most intellectual."[^14]
+
+## The hypnagogic edge
+
+There is an even more extreme form of productive idleness: the **hypnagogic
+state**, the twilight zone between wakefulness and sleep. Research has shown
+that people in this state are three times more likely to discover hidden rules
+in mathematical problems.[^15] Paul McCartney composed "Yesterday" in this
+state. Edison famously napped holding steel balls so that the clang of them
+dropping would wake him at the precise edge of sleep, capturing the ideas that
+surfaced there.[^15]
+
+The hypnagogic state works for the same reason that unconscious thought works
+more generally: the conscious mind's filters are down. Mental boundaries become
+permeable, and ideas flow through from the subliminal mind without the usual
+editorial interference.[^15]
+
+## Practical implications
+
+None of this is a licence for chronic procrastination. The distinction between
+rest and avoidance matters—rest feels peaceful; avoidance feels anxious.[^7]
+The point is that deliberate idleness, strategically deployed, is not the
+opposite of productivity but a component of it.
+
+- **For complex decisions**: absorb the information, then do something
+ unrelated. Let the unconscious integrate it. Decide later.
+- **For creative problems**: stop trying to force solutions. Walk, shower, nap.
+ Let the default mode network do its work.
+- **For long-term ambition**: resist the pressure to optimise every hour. The
+ lazy ambitious person's rhythm—bursts of intense work separated by genuine
+ rest—is not a failure of discipline. It is a strategy.
+- **For learning**: after a deep study session, do nothing for a while. The
+ consolidation that happens during rest is where understanding solidifies.
+
+Tim Ferriss summarised the trap neatly: "Being busy is a form of laziness—lazy
+thinking and indiscriminate action."[^16] The truly lazy thing is not to rest.
+It is to stay busy with the wrong work because it feels productive.
+
+<div align="center">⁂</div>
+
+[^1]: https://en.wikipedia.org/wiki/Unconscious_thought_theory
+
+[^2]: https://www.science.org/doi/10.1126/science.1121629
+
+[^3]: https://journals.sagepub.com/doi/10.1111/j.1745-6916.2006.00007.x
+
+[^4]: https://pubmed.ncbi.nlm.nih.gov/16484496/
+
+[^5]: https://www.cambridge.org/core/journals/judgment-and-decision-making/article/are-complex-decisions-better-left-to-the-unconscious-further-failed-replications-of-the-deliberationwithoutattention-effect/D2F0A25CA1F498259B90C3289FA54342
+
+[^6]: https://acmelab.yale.edu/sites/default/files/2011_unconscious_thought_theory_and_its_discontents.pdf
+
+[^7]: https://blog.digitaldeepak.com/p/the-dangerous-power-of-the-lazy-ambitious
+
+[^8]: https://www.goodreads.com/quotes/568877-i-choose-a-lazy-person-to-do-a-hard-job
+
+[^9]: https://www.bbc.com/worklife/article/20170414-why-idle-moments-are-crucial-for-creativity
+
+[^10]: https://www.nature.com/articles/s41380-021-01403-8
+
+[^11]: https://pmc.ncbi.nlm.nih.gov/articles/PMC4410786/
+
+[^12]: https://www.sloww.co/in-praise-of-idleness-bertrand-russell/
+
+[^13]: https://philosophybreak.com/articles/bertrand-russell-in-praise-of-idleness/
+
+[^14]: https://www.psychologytoday.com/us/blog/hide-and-seek/201410/the-psychology-of-laziness
+
+[^15]: https://www.psychologytoday.com/us/blog/out-of-the-darkness/202512/how-idleness-can-lead-to-genius
+
+[^16]: https://jamesclear.com/eisenhower-box
diff --git a/personal_development/whiteboard_effect.md b/personal_development/whiteboard_effect.md
@@ -0,0 +1,62 @@
+# The Whiteboard Effect
+
+For some types of problems, working with someone else at the proverbial shared
+whiteboard can push you deeper than if you were working alone. The presence of
+the other party waiting for your next insight—be it someone physically in the
+same room or collaborating with you virtually—can short-circuit the natural
+instinct to avoid depth.
+
+## Why it works
+
+When you're alone with a hard problem, it's easy to reach a point of discomfort
+and unconsciously retreat: you check your phone, switch to an easier task, or
+convince yourself you've thought about it "enough." The whiteboard effect
+disrupts this escape pattern. Another person's attention creates a gentle,
+productive pressure—not the anxiety of being judged, but the accountability of
+being witnessed. You stay in the difficulty longer because walking away isn't a
+silent, private act anymore.
+
+## The role of the other person
+
+The other person doesn't need to be an expert, or even deeply familiar with the
+problem. Their value isn't necessarily in what they contribute directly. It's in
+what their presence compels you to do:
+
+- **Articulate your reasoning out loud.** Half-formed thoughts that feel clear
+ in your head collapse the moment you try to explain them. The act of
+ externalising forces rigour.
+- **Hold yourself to a higher standard of clarity.** You won't hand-wave past a
+ weak link in your logic when someone is watching you build the chain.
+- **Resist the pull of shallow thinking.** The social contract of collaboration
+ makes it harder to settle for a surface-level answer and move on.
+
+## When it's most useful
+
+The whiteboard effect is strongest for problems that require sustained
+concentration and a willingness to sit with ambiguity—architecture decisions,
+debugging complex systems, working through a proof, or designing something from
+first principles. These are exactly the kinds of problems where solo thinkers
+tend to bail out early, mistaking discomfort for a dead end.
+
+It is less about brainstorming (where quantity of ideas matters) and more about
+depth-first exploration (where the quality of reasoning matters).
+
+## Collaborative depth vs. performative collaboration
+
+This is not a case for more meetings. Most meetings diffuse focus rather than
+concentrate it. The whiteboard effect requires a specific setup:
+
+- **Small group.** Two people is ideal. Three can work. Beyond that, social
+ dynamics take over and depth suffers.
+- **Shared artefact.** A whiteboard, a shared document, a terminal—something
+ both parties are actively looking at and building on together.
+- **Mutual investment.** Both people need to be genuinely engaged with the
+ problem, not just present in the room.
+
+## Applying it deliberately
+
+If you notice yourself repeatedly bouncing off a hard problem when working
+alone, that's a signal. Find someone—a colleague, a friend, even a rubber duck
+with a pulse—and work through it together. The goal isn't to outsource the
+thinking. It's to use the social dynamic to keep yourself honest and keep
+yourself going deeper than your solo instincts would allow.
diff --git a/personal_development/zeigarnik_effect.md b/personal_development/zeigarnik_effect.md
@@ -0,0 +1,269 @@
+# Zeigarnik Effect
+
+# Why unfinished things won't leave you alone
+
+You are halfway through a book and put it down. For days, the unresolved plot
+occupies your thinking in idle moments—on the train, in the shower, before
+sleep. Then you finish it. Within a week, you can barely remember the ending.
+This asymmetry—where the incomplete grips the mind and the complete dissolves
+from it—is the Zeigarnik Effect, and it governs more of your cognitive life
+than you probably realise.
+
+## Origins: a waiter's memory
+
+The story begins in a Vienna café in the 1920s. Kurt Lewin, the Gestalt
+psychologist, was dining with his students when he noticed something odd about
+their waiter. The man could recall complex, multi-item orders with perfect
+accuracy—but only while the orders remained open. The moment a table's bill was
+settled, the details vanished from his mind as if they had never existed.[^1][^2]
+
+Bluma Zeigarnik, one of Lewin's doctoral students at the University of Berlin,
+turned this café observation into a research programme. In her 1927 paper
+_Über das Behalten von erledigten und unerledigten Handlungen_ (On Finished
+and Unfinished Tasks), published in _Psychologische Forschung_, she reported
+the results of a series of experiments that would give the phenomenon its
+name.[^1][^3]
+
+## The experiments
+
+Zeigarnik asked participants to complete between 15 and 22 tasks—a mixture of
+tactile work (stringing beads, assembling boxes) and mental work (solving
+puzzles, arithmetic). She allowed half the tasks to be completed normally and
+interrupted the other half at various points, removing the materials and
+directing the participant to something else. After an hour's delay, she asked
+participants to recall which tasks they had worked on.[^1][^2]
+
+The results were striking. Participants recalled interrupted tasks roughly
+twice as often as completed ones. In some conditions, recall of unfinished
+tasks was up to 90% better than for finished ones.[^1][^3] The effect was not
+uniform—tasks interrupted near the middle or end were remembered more vividly
+than those interrupted at the beginning, suggesting that the closer one gets to
+completion, the stronger the cognitive tension becomes.[^1]
+
+Zeigarnik ran four experiments in total, testing individual adults, groups of
+adults, and groups of adolescents. She also found that participants with higher
+levels of ambition showed a stronger effect, and that those who interpreted
+interruption as personal failure remembered the incomplete tasks even more
+tenaciously.[^1][^2]
+
+In two further small experiments, she interrupted tasks but then allowed
+participants to immediately resume and complete half of them. The tasks that
+were never allowed to reach completion were still the most readily recalled—
+confirming that it was the state of incompleteness, not the act of
+interruption, that drove the memory advantage.[^1]
+
+## Lewin's tension systems
+
+Zeigarnik's explanation drew directly from Lewin's field theory. Lewin proposed
+that when a person forms an intention to complete a task, it creates a
+**quasi-need**—a state of psychic tension within what he called the person's
+"life space." This tension is a motivational force: it energises the behaviour
+required to finish the task and keeps the relevant information accessible in
+memory. Completing the task discharges the tension. The quasi-need is
+satisfied, and the cognitive system can release the associated
+information.[^1][^4]
+
+An incomplete task, by contrast, leaves the tension system unresolved. The
+quasi-need persists, and with it, the heightened accessibility of everything
+related to the task. The mind does not hold onto unfinished business because it
+is anxious or neurotic—it holds on because, from the perspective of Lewin's
+motivational dynamics, the task is still active. The cognitive loop remains
+open.[^4]
+
+This framing connects the Zeigarnik Effect to the Gestalt law of closure: the
+mind's tendency to perceive incomplete patterns as demanding completion. An
+unfinished task is a perceptual gap, and the drive to close it keeps the
+relevant information foregrounded in memory.[^5]
+
+## The Ovsiankina Effect
+
+A related but distinct phenomenon was documented by Maria Ovsiankina, another
+of Lewin's students, around the same time. Where Zeigarnik studied memory
+for interrupted tasks, Ovsiankina studied the behavioural drive to resume them.
+She found that when participants were interrupted during a task and given the
+opportunity to return to it later, the vast majority spontaneously chose to do
+so—even when there was no external incentive to finish.[^6]
+
+The Zeigarnik Effect says: you will _remember_ what you did not finish. The
+Ovsiankina Effect says: you will _return_ to finish it. Together, they
+describe a complete motivational circuit—cognitive accessibility plus
+behavioural drive—both powered by the same underlying tension system.[^6]
+
+## Replication and controversy
+
+The Zeigarnik Effect is one of those findings that is more famous than it is
+reliable. A 1964 review by Butterfield concluded that the effect was "far from
+being the invariable result" of interrupted-task experiments, and that
+frequently more completed than uncompleted tasks were recalled.[^1][^7]
+Multiple subsequent attempts have produced mixed results, with some researchers
+arguing there is no universal pattern.[^1]
+
+The effect is sensitive to conditions that are difficult to control. It is
+weakened when participants are ego-involved in the task (failure threatens
+self-esteem, which can suppress rather than enhance recall of the unfinished
+task). It is more likely to appear when the interruption seems incidental
+rather than deliberate. And it disappears when participants conclude the task
+is impossible—if the loop cannot be closed, the tension system appears to shut
+down rather than persist indefinitely.[^1][^7]
+
+A 2025 meta-analysis published in _Humanities and Social Sciences
+Communications_ reviewed the accumulated evidence for both the Zeigarnik and
+Ovsiankina effects, finding that while the effects are real, their magnitude
+varies considerably with experimental design, participant personality, and the
+nature of the task.[^8]
+
+Despite the replication difficulties, the core phenomenon—that incompleteness
+creates a cognitive state qualitatively different from completion—remains well
+supported and continues to generate research across memory, motivation, and
+clinical psychology.[^1][^7]
+
+## Open loops and cognitive load
+
+The modern interpretation of the Zeigarnik Effect frames unfinished tasks as
+**open loops** in working memory. Each open loop consumes cognitive resources—
+not because you are actively thinking about the task, but because your brain
+maintains a background monitoring process to ensure the task is not
+forgotten.[^5][^9]
+
+This has a direct consequence: the more open loops you carry, the less
+cognitive bandwidth you have for the task in front of you. David Allen's
+Getting Things Done methodology is built, in part, on this insight. Allen
+argues that the act of capturing tasks in a trusted external system (a list, a
+tool, a notebook) relieves the psychic tension that Lewin described—not by
+completing the task, but by externalising the commitment. The mind can release
+the open loop because the system will remember on its behalf.[^9]
+
+Research supports this. A study by Masicampo and Baumeister (2011) found that
+simply making a specific plan for how and when to complete an unfinished task
+eliminated the Zeigarnik Effect on cognitive intrusion—even though the task
+itself remained incomplete. The plan served as a psychological proxy for
+completion, discharging the tension without requiring the actual work to be
+done.[^10]
+
+## Applications
+
+### Productivity
+
+The Zeigarnik Effect offers a counterintuitive strategy for beating
+procrastination: start the task, then stop. The hardest part of any project is
+often the beginning—the blank page, the empty editor, the first line of code.
+But once you begin, the Zeigarnik Effect creates its own gravitational pull.
+The open loop generates tension, and that tension becomes a motive force to
+return and continue.[^5][^9]
+
+Ernest Hemingway reportedly used this principle deliberately, always stopping
+his writing sessions mid-sentence so that he knew exactly where to pick up the
+next day. The incomplete sentence was a hook—a deliberate open loop that kept
+the work alive in his mind overnight.[^11]
+
+Breaking large projects into smaller subtasks leverages the same mechanism.
+Each subtask, once started, generates its own tension. Completing it provides
+a small discharge—a micro-reward that sustains momentum—while the larger
+project's incompleteness keeps the overall direction in mind.[^5]
+
+### Learning
+
+Educators have noted that the Zeigarnik Effect can enhance retention. If a
+study session is interrupted and resumed later, the material studied during
+that session tends to be better remembered. This aligns with the broader
+evidence for spaced practice: distributing learning across multiple sessions
+with gaps between them produces stronger long-term memory than massing
+practice into a single block.[^1][^12]
+
+The mechanism may be twofold. The interruption keeps the material cognitively
+accessible (Zeigarnik's tension), and the spacing allows for consolidation
+during the intervals (a complementary process involving memory
+stabilisation).[^12]
+
+### Marketing and media
+
+Advertisers and storytellers have exploited the Zeigarnik Effect for decades.
+The cliffhanger—ending an episode, chapter, or commercial on an unresolved
+note—is a deliberate open loop designed to ensure the audience returns. Early
+research by Heimbach (1972) demonstrated that interrupted television
+commercials were recalled significantly better than those shown in
+full.[^1][^13]
+
+Television series, serialised podcasts, and episodic games all rely on the
+same principle. The unresolved plot thread is not a flaw in the narrative—it
+is a cognitive hook that binds the audience to the next instalment.
+
+### Software and product design
+
+Progress bars, streak counters, and incomplete profile indicators in software
+products are all applications of the Zeigarnik Effect. LinkedIn's profile
+completion percentage, Duolingo's daily streaks, and progress indicators in
+onboarding flows all create visible open loops that exploit the user's drive
+toward closure.[^5]
+
+The design principle is straightforward: show people what they have not
+finished, and their own cognition will supply the motivation to complete it.
+
+## The dark side
+
+The Zeigarnik Effect is not always benign. Unfinished tasks that cannot be
+completed—unresolved conflicts, unanswered questions, ambiguous losses—can
+become sources of chronic rumination. The tension that motivates productive
+behaviour can, when there is no available action, turn into anxiety. The open
+loop that would have driven you back to finish a puzzle instead keeps you
+awake at 3 a.m. replaying a conversation that cannot be unsaid.[^9][^14]
+
+Clinically, intrusive thoughts in conditions like OCD and PTSD share
+structural similarities with the Zeigarnik Effect: the mind returns repeatedly
+to something unresolved, unable to discharge the tension because no completing
+action is available or sufficient. Research has examined the relationship
+between the Zeigarnik Effect and obsessive-compulsive behaviour, finding
+parallels in the cognitive mechanisms that drive both.[^1][^14]
+
+The antidote, where one exists, is the same principle that Masicampo and
+Baumeister identified: creating a plan, or finding a symbolic form of closure,
+can relieve the tension even when literal completion is impossible. Rituals of
+closure—writing an unsent letter, holding a ceremony, making a deliberate
+decision to let go—are psychological technologies for manually closing loops
+that would otherwise remain open indefinitely.[^10]
+
+## Practical implications
+
+- **To overcome procrastination**: do not wait for motivation. Start the task—
+ even for five minutes—and let the Zeigarnik Effect supply the pull to
+ continue.
+- **To manage cognitive load**: capture every open loop in a trusted external
+ system. Your mind will release what it trusts something else to remember.
+- **To improve retention**: study in intervals. Interrupt yourself mid-topic
+ and return later. The incompleteness keeps the material alive in memory.
+- **To sustain long projects**: stop each work session at a point of
+ incompleteness—mid-paragraph, mid-function, mid-thought. The open loop will
+ carry you back.
+- **To protect your peace**: recognise when an open loop cannot be closed by
+ action. Make a plan, create symbolic closure, or consciously decide to
+ release it. Not every loop deserves your cognitive resources.
+
+<div align="center">⁂</div>
+
+[^1]: https://www.simplypsychology.org/zeigarnik-effect.html
+
+[^2]: https://www.psychologistworld.com/memory/zeigarnik-effect-interruptions-memory
+
+[^3]: https://en.wikipedia.org/wiki/Bluma_Zeigarnik
+
+[^4]: https://en.wikipedia.org/wiki/Field_theory_(psychology)
+
+[^5]: https://learningloop.io/plays/psychology/zeigarnik-effect
+
+[^6]: https://en.wikipedia.org/wiki/Ovsiankina_effect
+
+[^7]: https://huskiecommons.lib.niu.edu/allgraduate-thesesdissertations/1287/
+
+[^8]: https://www.nature.com/articles/s41599-025-05000-w
+
+[^9]: https://hubstaff.com/blog/zeigarnik-effect/
+
+[^10]: https://psycnet.apa.org/record/2011-10096-001
+
+[^11]: https://www.masterclass.com/articles/how-to-avoid-procrastination
+
+[^12]: https://helio.app/ux-research/laws-of-ux/zeigarnik-effect/
+
+[^13]: https://taproot.com/zeigarnik-effect/
+
+[^14]: https://www.psychologytoday.com/us/basics/zeigarnik-effect
diff --git a/quantum_mechanics/index.md b/quantum_mechanics/index.md
@@ -0,0 +1,2 @@
+# Quantum Mechanics
+
diff --git a/quantum_mechanics/michio_kaku.md b/quantum_mechanics/michio_kaku.md
@@ -0,0 +1,12 @@
+# Michio Kaku
+
+Michio Kaku adapts the Kardashev scale to describe how advanced civilizations
+are by the amount of energy they can control: Type 0 (like us) uses limited
+planetary resources such as fossil fuels; Type I can harness all energy
+available on its planet; Type II can capture most of the energy of its star; and
+Type III can use the energy output of an entire galaxy. On top of that, you
+could imagine a speculative Type IV civilization that harvests energy embedded
+in the fabric of the universe itself, such as the ubiquitous dark-matter or
+dark-energy–like fields between galaxies, and a Type V that somehow taps into
+energy sources outside our own universe “grid,” drawing power from a
+higher-dimensional or multiverse realm beyond our spacetime.
diff --git a/quantum_mechanics/simulation_theory.md b/quantum_mechanics/simulation_theory.md
@@ -0,0 +1,17 @@
+# Simulation Theory
+
+S. James Gates Jr. found that when he and his collaborators studied certain
+supersymmetry equations using diagrams called Adinkras, the mathematics
+naturally organized itself in terms of specific binary error‑correcting codes,
+similar to those used in digital communications and computers. He noted that
+these are “doubly‑even self‑dual” codes, a highly structured class of codes
+known from information theory, and that they appear to be built into the
+algebraic structure of these physical theories. Gates has described this as
+uncovering code‑like patterns embedded in the equations that describe
+fundamental particles, and he has suggested that this raises deep philosophical
+questions about whether reality might, at some level, have properties analogous
+to a computational or information‑based system.
+
+---
+
+[Simulation paper by Dr. Roman Yampolskiy](https://www.researchgate.net/publication/369187097_How_to_Escape_From_the_Simulation)
diff --git a/refactoring/index.md b/refactoring/index.md
@@ -0,0 +1,2 @@
+# Refactoring
+
diff --git a/refactoring/refactoring.md b/refactoring/refactoring.md
@@ -0,0 +1,51 @@
+# Refactoring
+
+### Book: Refactoring: Improving the Design of Existing Code (Addison-Wesley Professional 1999)
+
+## The First Step in Refactoring
+
+> Whenever I do refactoring, the first step is always the same. I need to build
+> a solid set of tests for that section of code. The tests are essential because
+> even though I follow refactorings structured to avoid most of the
+> opportunities for introducing bugs, I'm still human and still make mistakes.
+> Thus I need solid tests.
+
+> An important part of the tests is the way they report their results. ... The
+> tests are thus self-checking. It is vital to make tests self-checking. If you
+> don't, you end up spending time hand checking some numbers from the test
+> against some numbers of a desk pad, and that slows you down.
+
+> As we do the refactoring, we will lean on the tests. I'm going to be relying
+> on the tests to tell me whether I introduce a bug. It is essential for
+> refactoring that you have good tests. It's worth spending the time to build
+> the tests, because the tests give you the security you need to change the
+> program later.
+
+### Before you start refactoring, check that you have a solid suite of tests. These tests must be self-checking.
+
+## Decomposing and Redistributing the Statement Method
+
+1. Overly Long Statement method:
+
+> The obvious first target of my attention is the overly long statement method.
+> When I look at a long method like that, I am looking to decompose the method
+> into smaller pieces. Smaller pieces of code tend to make things more
+> manageable. They are easier to work with and move around.
+
+- Extraction Method
+
+> When I extract a method, as in any refactoring, I need to know what can go
+> wrong. If I do the extraction badly, I could introduce a bug into the program.
+> So before I do the refactoring I need to figure out how to do it safely.
+
+> [!info] Tip: Refactoring changes the programs in small steps. If you make a
+> mistake, it is easy to find the bug.
+
+> Is renaming worth the effort? Absolutely. Good code should communicate what it
+> is doing clearly, and variable names are a key to clear code. Never be afraid
+> to change the names of things to improve clarity. With good find and replace
+> tools, it is usually not difficult. Strong typing and testing will highlight
+> anything you miss. Remember
+
+> [!info] Tip: Any fool can write code that a computer can understand. Good
+> programmers write code that humans can understand.
diff --git a/references/bookmarks.md b/references/bookmarks.md
@@ -0,0 +1,5 @@
+# Bookmarks
+
+[The Software Architecture Chronicles by @hgraca](https://herbertograca.com/2017/07/03/the-software-architecture-chronicles/)
+
+[Code Reviews: Before You Even Run The Code](https://lornajane.net/posts/2015/code-reviews-before-you-even-run-the-code)
diff --git a/references/books.md b/references/books.md
@@ -0,0 +1,130 @@
+# Books
+
+## TypeScript
+
+[TypeScript Deep Dive](https://basarat.gitbook.io/typescript/future-javascript/arrow-functions) -
+This is great and succinct and freely available online
+
+Effective TypeScript by Dan Vanderkam - very useful
+
+## Fundamentals of Software Engineering
+
+The Pragmatic Programmer by Andrew Hunt, David Thomas (The Pragmatic Programmer:
+Your Journey to Mastery, 20th Anniversary Edition)
+
+The Mythical Man-Month
+
+Become an Effective Software Engineering Manager
+
+Site Reliability Engineering: How Google Runs Production Systems
+
+Clean Code by Robert C Martin
+
+A Philosophy of Software Design
+
+## Additional Software Engineering Reading
+
+Fundamentals of Software Architecture: An Engineering Approach
+
+Code Complete by Steve McConnell
+
+JavaScript: The Good Parts
+
+Design Patterns: Elements of Reusable Object-Oriented Software
+
+Refactoring by Martin Fowler
+
+Test-Driven Development by Example by Kent Beck
+
+## Rust
+
+[Rust Atomics and Locks](https://mara.nl/atomics/)
+
+Programming Rust, 2nd Ed. — Jim Blandy & Jason Orendorff Why: The definitive
+reference on Rust’s ownership, borrowing, and unsafe code. Covers lifetimes,
+async/await, and concurrency primitives in depth. Written by core language
+designers.
+
+Rust in Action — Tim McNamara Why: Focuses on systems programming patterns
+(including allocators, FFI, and unsafe code) and includes practical async/Tokio
+examples. Strong on performance-aware design.
+
+## Low-Latency Systems, Hardware-Aware Optimization & Allocators
+
+Designing Data-Intensive Applications — Martin Kleppmann Why: Though not
+Rust-specific, it’s essential for understanding latency, consistency, and
+distributed system trade-offs. Covers observability, ordering, exactly-once
+semantics, and system boundaries—directly relevant to your messaging and
+observability needs.
+
+High Performance Browser Networking — Ilya Grigorik Why: Surprisingly
+relevant—covers TCP/UDP, zero-copy, kernel bypass concepts, and NUMA-aware
+design patterns. Though focused on browsers, its low-level networking insights
+translate directly to systems programming.
+
+The Linux Programming Interface — Michael Kerrisk Why: The most comprehensive
+reference on Linux syscalls, I/O models (including io_uring), and memory
+management. Critical for understanding zero-copy, custom allocators, and
+hardware-level profiling.
+
+## System Design, Architecture & Leadership
+
+System Design Interview (2 vols) — Alex Xu Why: Excellent for scaling and
+distributed system design patterns—especially useful for Kafka/NATS-style
+systems and observability architecture.
+
+Accelerate — Nicole Forsgren et al. Why: Evidence-based practices for DevOps,
+CI/CD, and performance regression detection. Directly supports your CI/CD and
+benchmarking-gates requirements.
+
+The Pragmatic Engineer — Gustavo Sampaio Why: Covers architecture reviews, ADRs,
+and engineering standards—ideal for leading Rust engineering culture
+
+## Observability, Testing & Reliability
+
+Observability Engineering — Candace Gonzalez et al. Why: Covers
+Prometheus/OpenTelemetry, tracing, alert design, and structured logging in
+production systems—aligned with your observability stack.
+
+Chaos Engineering — Casey Rosenthal et al. Why: The foundational text on fault
+injection, chaos testing, and resilience design—directly matches your “chaos and
+fault injection experience” requirement.
+
+Property-Based Testing with Rust — John A. De Goes (free online) Why: While not
+a traditional book, De Goes’ Modern Software Engineering and his PBT tutorials
+are widely used in Rust for property-based testing and TDD as design discipline
+
+## Cloud Infrastructure (Azure/AWS)
+
+Cloud Native Infrastructure — Karl Isenberg & George Miranda Why: Covers IAM,
+secrets management, and infrastructure-as-code patterns across cloud
+providers—including Azure-specific guidance.
+
+Designing Distributed Systems — Brendan Burns Why: Written by a Kubernetes
+co-creator; emphasizes patterns for reliability, observability, and managed
+services—highly applicable to Azure/AWS deployments
+
+## Supplemental (Free & Online)
+
+[The Rust Async Book](https://rust-lang.github.io/async-book/)
+
+[The Rustonomicon](https://doc.rust-lang.org/nomicon/)
+
+[Tokio docs & blog](https://tokio.rs/blog/)
+
+io_uring & DPDK deep dives (e.g.,
+[A Deep Dive into Zero-Copy Networking and io_uring](https://medium.com/@jatinumamtora/a-deep-dive-into-zero-copy-networking-and-io-uring-78914aa24029),
+[Zero-copy network transmission with io_uring](https://lwn.net/Articles/879724/))
+
+## Personal Development
+
+Deep Work by Carl Newport
+
+How to measure anything
+
+Tidy First
+
+Becoming Evil: How Ordinary People Commit Genocide and Mass Killing by James
+Waller
+
+The Art of Memory - Frances Yates
diff --git a/references/index.md b/references/index.md
@@ -0,0 +1,3 @@
+# References
+
+Educational content
diff --git a/references/podcast.md b/references/podcast.md
@@ -0,0 +1,3 @@
+# Podcast
+
+[The Philosophy of Software Design](https://newsletter.pragmaticengineer.com/p/the-philosophy-of-software-design)
diff --git a/references/rules_of_programming.md b/references/rules_of_programming.md
@@ -0,0 +1,21 @@
+# Rob Pike's 5 Rules of Programming
+
+## Rob Pike's 5 Rules of Programming
+
+Rule 1. You can't tell where a program is going to spend its time. Bottlenecks
+occur in surprising places, so don't try to second guess and put in a speed hack
+until you've proven that's where the bottleneck is.
+
+Rule 2. Measure. Don't tune for speed until you've measured, and even then don't
+unless one part of the code overwhelms the rest.
+
+Rule 3. Fancy algorithms are slow when n is small, and n is usually small. Fancy
+algorithms have big constants. Until you know that n is frequently going to be
+big, don't get fancy. (Even if n does get big, use Rule 2 first.)
+
+Rule 4. Fancy algorithms are buggier than simple ones, and they're much harder
+to implement. Use simple algorithms as well as simple data structures.
+
+Rule 5. Data dominates. If you've chosen the right data structures and organized
+things well, the algorithms will almost always be self-evident. Data structures,
+not algorithms, are central to programming.
diff --git a/references/videos.md b/references/videos.md
@@ -0,0 +1,19 @@
+# Videos
+
+[How To Learn Any New Skill So Fast It’s Unfair](https://www.youtube.com/watch?v=O46wlbRsAK4&list=PLpoNNzdnAf3afm46nLHjgCkJTZVoEYCJI&index=5)
+
+[Preventing the Collapse of Civilization / Jonathan Blow](https://www.youtube.com/watch?v=ZSRHeXYDLko)
+
+[How Societies Collapse](https://youtu.be/K-_l9jBGo74?si=4ej7u0EYO-WJAxsQ)
+
+[Into Eternity (Documentary)](https://www.youtube.com/watch?v=vpRBfTHQg3k)
+
+[Build your own libraries](https://youtu.be/p2aea9dytpE?si=Zc8udMsEVTkir8-L)
+
+[Spotify Engineering Culture - Part 1](https://www.youtube.com/watch?v=Yvfz4HGtoPc)
+
+[Spotify Engineering Culture - Part 2](https://www.youtube.com/watch?v=vOt4BbWLWQw)
+
+[Your codebase is NOT ready for AI (here's how to fix it)](https://www.youtube.com/watch?v=uC44zFz7JSM)
+
+[The Microservices Scam Nobody Talks About](https://www.youtube.com/watch?v=6e9B7q3gvYY)
diff --git a/content/references/videos/useful-code-is-ugly-tsoding.md b/references/videos/useful-code-is-ugly-tsoding.md
diff --git a/system_administration/commands.md b/system_administration/commands.md
@@ -0,0 +1,17 @@
+# Linux Commands
+
+# Docker Related Commands
+
+```bash
+docker ps \
+--format '{{.Names | printf "%-20s"}} {{.Ports | printf "%-30s"}} {{.RunningFor}}' \
+| sort
+```
+
+```bash
+docker ps --format '{{.Names | printf "%-20s"}} {{.Ports}}' | sort
+```
+
+```bash
+docker ps -a --format '{{.Names}}: {{.Ports}}'
+```
diff --git a/system_administration/index.md b/system_administration/index.md
@@ -0,0 +1,2 @@
+# System Administration
+
diff --git a/system_administration/linux_permissions.md b/system_administration/linux_permissions.md
@@ -0,0 +1,88 @@
+# Linux Permissions
+
+### Question: What happens if I am on a server and then I ssh into this server again?
+
+Nothing special or dangerous happens by itself; you just get another shell on
+the same machine.
+
+Nothing special or dangerous happens by itself; you just get another shell on
+the same machine
+
+You now have two nested sessions:
+
+- outer: your original SSH (or local) shell
+
+- inner: the SSH session created from the server to itself.
+
+Processes in the inner session are separate from the outer one, with their own
+TTY, environment, history, etc.
+
+### Question: how to create a user on an archlinux server without sudo privilege but make it a member of docker group?
+
+#### Create the New User
+
+`useradd -m -G docker -s /bin/bash newusername`
+
+#### Set a Password
+
+`passwd newusername`
+
+#### Verify Group Membership
+
+```bash
+id newusername
+# Expected: uid=1001(newusername) gid=1001(newusername) groups=1001(newusername),999(docker)
+```
+
+#### Generate the Key Pair
+
+```bash
+ssh-keygen -t ed25519 -C "newusername@yourserver" -f /home/newusername/.ssh/id_ed25519 -N ""
+```
+
+#### Set Up authorized_keys
+
+```bash
+cat /home/newusername/.ssh/id_ed25519.pub >> /home/newusername/.ssh/authorized_keys
+```
+
+#### Fix Permissions
+
+SSH is strict about permissions — it will silently refuse to use keys with wrong
+ownership or modes:
+
+```bash
+chown -R newusername:newusername /home/newusername/.ssh
+chmod 700 /home/newusername/.ssh
+chmod 600 /home/newusername/.ssh/authorized_keys
+chmod 600 /home/newusername/.ssh/id_ed25519
+```
+
+#### Share the Private Key
+
+```bash
+ssh -i ~/.ssh/id_ed25519 newusername@your_server_ip
+```
+
+#### Setup Authorized Keys
+
+Confirm authorized_keys matches that key
+
+```bash
+sudo -u deploy cat /home/deploy/.ssh/ed25519.pub
+sudo -u deploy cat /home/deploy/.ssh/authorized_keys
+```
+
+Ensure authorized_keys contains the same line as ed25519.pub. If it doesn’t,
+append it:
+
+```bash
+sudo -u deploy sh -c 'cat ~/.ssh/ed25519.pub >> ~/.ssh/authorized_keys'
+chmod 600 /home/deploy/.ssh/authorized_keys
+```
+
+#### Test SSH Access
+
+```bash
+ssh -i /path/to/private_key -p SSH_PORT SSH_USER@SSH_HOST
+```
diff --git a/system_design/case_study/index.md b/system_design/case_study/index.md
@@ -0,0 +1,2 @@
+# Case Studies
+
diff --git a/content/system_design/case_study/typescript_channels.md b/system_design/case_study/typescript_channels.md
diff --git a/system_design/case_study/websocket_request_trackers.md b/system_design/case_study/websocket_request_trackers.md
@@ -0,0 +1,578 @@
+# WebSocket request trackers
+
+### What createWsRequestTracker Does
+
+```ts
+export function createWsRequestTracker<Epoch extends boolean = false>(
+ options: WsRequestTrackerOptions<Epoch>,
+): WsRequestTracker<Epoch> {
+ // Creates a tracker that bridges WebSocket request/response pairs into Promises.
+ // Handles timeouts and provides a clean API for awaiting responses.
+ const { timeoutMs, enableEpochs = false } = options;
+ const pending = new Map<string, PendingEntry>();
+ // pending map to store in-flight WS requests, keyed by contractId
+ let epoch = 0;
+ // current epoch used for stale request handling (epoch mode)
+
+ return {
+ track<T = void>(
+ contractId: string,
+ ): Promise<Epoch extends true ? T | null : T> {
+ // Start tracking a WS request by its contract ID
+ if (enableEpochs) {
+ epoch++;
+ }
+
+ const requestEpoch = epoch;
+
+ return new Promise<any>((resolve, reject) => {
+ const timeoutId = setTimeout(() => {
+ const entry = pending.get(contractId);
+ if (!entry) return; // already resolved/rejected
+
+ pending.delete(contractId);
+
+ if (enableEpochs && entry.epoch !== epoch) {
+ entry.resolve(null);
+ return;
+ }
+
+ let errorMessage =
+ `WS request ${contractId} timed out after ${timeoutMs}ms — server did not respond`;
+ reject(new WebError(errorMessage, "WsTimeoutError"));
+ }, timeoutMs);
+
+ pending.set(contractId, {
+ resolve: resolve as (value: unknown) => void,
+ reject,
+ timeoutId,
+ epoch: requestEpoch,
+ });
+ });
+ },
+
+ resolve(contractId: string, data?: unknown): void {
+ // Resolve pending request for contractId
+ const entry = pending.get(contractId);
+ if (!entry) return;
+
+ if (enableEpochs && entry.epoch !== epoch) {
+ clearTimeout(entry.timeoutId);
+ pending.delete(contractId);
+ entry.resolve(null);
+ return;
+ }
+
+ clearTimeout(entry.timeoutId);
+ pending.delete(contractId);
+ entry.resolve(data);
+ },
+
+ // Reject a pending request
+ reject(contractId: string, error: WebError): void {
+ const entry = pending.get(contractId);
+ if (!entry) return;
+
+ clearTimeout(entry.timeoutId);
+ pending.delete(contractId);
+ entry.reject(error);
+ },
+
+ // Bump epoch to stale-discard all in-flight resolves.
+ invalidate(): void {
+ if (!enableEpochs) return;
+ epoch++;
+ },
+
+ // Reject all pending and clear all timeouts.
+ dispose(): void {
+ for (const [contractId, entry] of pending) {
+ clearTimeout(entry.timeoutId);
+
+ // Reject with disposed error
+ let errMessage =
+ `WS tracker disposed while request ${contractId} was still pending`;
+ entry.reject(
+ new WebError(errMessage, "WsTrackerDisposedError"),
+ );
+ }
+ pending.clear();
+ },
+
+ // Check if request is pending
+ hasPending(contractId: string): boolean {
+ return pending.has(contractId);
+ },
+
+ // Get number of pending requests
+ pendingCount(): number {
+ return pending.size;
+ },
+ };
+```
+
+`createWsRequestTracker` is a Promise-WS bridge that converts WebSocket
+request/response pairs into synchronous-looking async operations. In a WebSocket
+protocol where you send a packet and receive a response later via a callback
+listener, this tracker provides a clean API to await the response as a Promise.
+
+### Core Problem It Solves
+
+Without this tracker:
+
+1. You send a WS packet (fire-and-forget)
+2. Later, a listener callback receives the response
+3. How do you connect step 2 back to step 1? How do you handle timeouts? How do
+ you ensure stale responses don't hang your app?
+
+The tracker solves this by:
+
+• Creating a Promise when you call track(contractId) after sending a packet
+
+• Storing it keyed by the contract ID (packet UUID)
+
+• Resolving/rejecting it when the listener receives the response
+
+• Handling timeouts with automatic rejection
+
+---
+
+### Workflow (Complete Flow)
+
+```text
+┌────────────────────────────────────────────────────────────────────────────┐
+│ 1. Component calls readFile() or saveFile() │
+│ (e.g., user clicks a file or presses Ctrl+S) │
+└────────────────────────────────────────────────────────────────────────────┘
+ ↓
+┌────────────────────────────────────────────────────────────────────────────┐
+│ 2. EditorStore sends WS packet via readFileWsFn / saveFileWsFn │
+│ → Returns a contractId (UUID) │
+└────────────────────────────────────────────────────────────────────────────┘
+ ↓
+┌────────────────────────────────────────────────────────────────────────────┐
+│ 3. tracker.track(contractId) creates a Promise & stores it in pending Map │
+│ → Returns Promise<T | null> (T=void by default, or T=fileData) │
+└────────────────────────────────────────────────────────────────────────────┘
+ ↓
+┌────────────────────────────────────────────────────────────────────────────┐
+│ 4. User waits for server response (await track()) │
+└────────────────────────────────────────────────────────────────────────────┘
+ ↓
+┌────────────────────────────────────────────────────────────────────────────┐
+│ 5. Server responds via WebSocket │
+│ → Listener (editorWebsocketEventHandler) receives packet │
+│ → Decodes the contract │
+└────────────────────────────────────────────────────────────────────────────┘
+ ↓
+┌────────────────────────────────────────────────────────────────────────────┐
+│ 6. applyFileRead / applyFileWritten calls tracker.resolve() │
+│ → Looks up contractId in pending Map │
+│ → Clears timeout │
+│ → Calls resolve() callback with data │
+└────────────────────────────────────────────────────────────────────────────┘
+ ↓
+┌────────────────────────────────────────────────────────────────────────────┐
+│ 7. await track() resolves, EditorStore returns typed result │
+│ → { kind: 'success', fileData } │
+│ → Or { kind: 'stale' } if epochs enabled and a newer request came in │
+└────────────────────────────────────────────────────────────────────────────┘
+```
+
+---
+
+### Why We Need It for service-cad/frontend/web
+
+1. Type-Safe Async/Await Over WebSocket Protocol
+
+The WebSocket protocol is event-driven (fire-and-forget + callback), but Svelte
+components work better with await:
+
+```
+ // Without tracker: messy callback style
+ sendReadFile(path, (result) => {
+ if (result.error) { /* handle */ }
+ if (result.data) { /* handle */ }
+ });
+
+ // With tracker: clean async/await
+ const response = await readFile(path);
+ // Throws on error, returns data on success
+```
+
+2. Timeout Handling The tracker implements automatic timeouts via setTimeout. If
+ the server doesn't respond within WEBSOCKET_TIMEOUT_MS, the promise rejects
+ with a WsTimeoutError.
+
+3. Epoch Tracking (Last-Write-Wins) With enableEpochs: true: • User rapidly
+ clicks between files → each click sends a request • Only the latest file's
+ response matters (the others are "stale") • Stale responses resolve with null
+ instead of hanging or updating stale UI • The caller checks if (response ===
+ null) return { kind: 'stale' }
+
+4. Cleanup & Memory Management • dispose() rejects all pending promises and
+ clears timeouts • Prevents memory leaks when components unmount • Used in
+ Svelte onDestroy hooks
+
+---
+
+### Two Modes of Operation
+
+Simple Mode (default, enableEpochs: false)
+
+• Every request gets its own independent promise
+
+• Used by loadProjectTracker and killSessionTracker
+
+• Example: loadProjectTracker.track('xyz') → always resolves when server
+responds
+
+Epoch Mode (enableEpochs: true)
+
+• Each track() increments a monotonic epoch counter
+
+• When resolve() is called, it checks if the entry's epoch matches current
+
+• Stale entries (older epoch) resolve with null instead of data
+
+• Used by readTracker, saveTracker, fileTreeTracker, compileTracker
+
+---
+
+Key Methods
+
+| Method | Description |
+| ---------------------------- | ------------------------------------------------------- |
+| `track(contractId)` | Start tracking, returns `Promise<T |
+| `resolve(contractId, data?)` | Resolve pending request (called by listener) |
+| `reject(contractId, error)` | Reject on error (always surfaces, never stale-discard) |
+| `invalidate()` | Bump epoch, mark all pending entries as stale |
+| `dispose()` | Reject all pending + clear timeouts (component cleanup) |
+| `hasPending(contractId)` | Check if entry exists (used by listeners) |
+| `pendingCount()` | Number of unprocessed requests |
+
+---
+
+Files Using This Tracker
+
+• codeEditor/wsHooks/tracker.ts: readTracker, saveTracker (epoch mode)
+
+• fileTree/wsHooks/tracker.ts: fileTreeTracker (epoch mode)
+
+• canvas/wsHooks/tracker.ts: compileTracker (default mode)
+
+• gitEditor/wsHooks/tracker.ts: gitTracker (epoch mode)
+
+• core/pageState/editorPage/tracker.ts: loadProjectTracker, killSessionTracker
+(default mode)
+
+All use WEBSOCKET_TIMEOUT_MS (configured elsewhere) for timeout duration.
+
+```ts
+import { WebError } from "@core/utils/errors/webError";
+
+/**
+ * Internal bookkeeping for a single in-flight WS request.
+ *
+ * `resolve` accepts `unknown` rather than a generic `T` because the pending
+ * map is type-erased — it stores entries for many different request types
+ * simultaneously, so there's no single `T` that fits all entries. The
+ * per-call type safety lives on the public `track<T>()` method instead.
+ *
+ * We use `unknown` over `any` deliberately: `unknown` forces a type
+ * assertion at the one internal call site where we store the resolve
+ * callback (`resolve as (value: unknown) => void`), making the type
+ * erasure explicit and auditable. With `any`, TypeScript stops checking
+ * entirely — if we accidentally passed the wrong value internally, the
+ * compiler wouldn't catch it. The single assertion is worth the trade-off
+ * for keeping the rest of the internals type-checked.
+ */
+export type PendingEntry = {
+ resolve: (value: unknown) => void;
+ reject: (error: WebError) => void;
+ timeoutId: ReturnType<typeof setTimeout>;
+ epoch: number;
+};
+
+/**
+ * Generic over Epoch — controls the return type of track<T>().
+ *
+ * When Epoch is false (default), track<T>() returns Promise<T>.
+ * When Epoch is true, track<T>() returns Promise<T | null> because
+ * stale responses resolve with null instead of hanging forever.
+ *
+ * TypeScript infers Epoch from the enableEpochs property value:
+ * createWsRequestTracker({ timeoutMs: WEBSOCKET_TIMEOUT_MS, enableEpochs: true })
+ * // → WsRequestTracker<true>, track returns Promise<T | null>
+ *
+ * createWsRequestTracker({ timeoutMs: WEBSOCKET_TIMEOUT_MS })
+ * // → WsRequestTracker<false>, track returns Promise<T>
+ */
+export type WsRequestTrackerOptions<Epoch extends boolean = false> = {
+ timeoutMs: number;
+ enableEpochs?: Epoch;
+};
+
+export type WsRequestTracker<Epoch extends boolean = false> = {
+ /**
+ * Start tracking a request. Returns a typed Promise that resolves
+ * when resolve() is called with the matching contract ID.
+ *
+ * T defaults to void — callers that expect response data specify it:
+ * await tracker.track<string[]>(contractId); // expects string[]
+ * await tracker.track(contractId); // expects void
+ *
+ * With enableEpochs: true, returns Promise<T | null>. null indicates
+ * a stale response — a newer track() call superseded this one. The
+ * caller checks for null to detect staleness and return early.
+ */
+ track<T = void>(
+ contractId: string,
+ ): Promise<Epoch extends true ? T | null : T>;
+
+ /**
+ * Resolve a pending request with optional data.
+ *
+ * Accepts `unknown` because the tracker is type-erased internally —
+ * it doesn't know which `T` the corresponding track<T>() call used.
+ * The caller (WS listener) passes whatever the server sent, and the
+ * awaiting handler receives it typed via the Promise<T> from track().
+ *
+ * With enableEpochs: true, stale entries (epoch doesn't match current)
+ * are settled with null instead of the provided data. This ensures the
+ * caller's promise always settles — its finally block runs, no leaked
+ * closures.
+ */
+ resolve(contractId: string, data?: unknown): void;
+
+ /** Reject a pending request with an error. Always surfaces (never stale-discarded). */
+ reject(contractId: string, error: WebError): void;
+
+ /** Bump epoch to stale-discard all in-flight resolves. No-op without enableEpochs. */
+ invalidate(): void;
+
+ /** Reject all pending and clear timers. Call on component destroy. */
+ dispose(): void;
+
+ /** Whether a tracked entry exists for this contract ID. */
+ hasPending(contractId: string): boolean;
+
+ /** Number of currently pending (unresolved) tracked requests. */
+ pendingCount(): number;
+};
+
+/**
+ * Creates a tracker that bridges WebSocket request/response pairs into Promises.
+ *
+ * The problem: WS senders fire-and-forget a packet, and the response arrives
+ * later via a separate listener callback. There's no built-in way for the
+ * caller to await the full round-trip, detect timeouts, or correlate a
+ * response back to the request that triggered it.
+ *
+ * This tracker solves that by letting the caller `track(contractId)` after
+ * sending a packet. This returns a Promise that resolves when the listener
+ * calls `resolve(contractId)` with the matching ID, or rejects on timeout.
+ *
+ * Two modes of operation:
+ *
+ * 1. Simple tracking (enableEpochs=false, the default)
+ * Every tracked request gets its own independent promise. resolve() always
+ * delivers. This is the mode used by the git editor where each operation
+ * (commit, stage, merge) is a distinct action that doesn't supersede others.
+ *
+ * 2. Epoch tracking (enableEpochs=true)
+ * Each track() call increments a monotonic epoch counter. When resolve()
+ * is called, it checks whether the pending entry's epoch matches the
+ * current epoch. If stale (a newer request was tracked since), the entry
+ * is settled with null — the awaiting promise resolves to null, signaling
+ * staleness to the caller. This implements last-write-wins for rapid
+ * sequential requests (e.g. clicking between files quickly — only the
+ * most recent file read matters).
+ *
+ * Stale timeouts also settle with null (not rejection) — if a stale
+ * request's timeout fires, it resolves with null rather than surfacing
+ * an error for a request the user already navigated away from.
+ *
+ * reject() always surfaces regardless of epoch — errors should never be
+ * silently swallowed.
+ *
+ * invalidate() bumps the epoch without tracking a new request, causing all
+ * in-flight responses to be discarded when they arrive. Use this when you
+ * want to cancel all pending without triggering error callbacks (e.g.
+ * navigating away from the current context).
+ */
+export function createWsRequestTracker<Epoch extends boolean = false>(
+ options: WsRequestTrackerOptions<Epoch>,
+): WsRequestTracker<Epoch> {
+ const { timeoutMs, enableEpochs = false } = options;
+ const pending = new Map<string, PendingEntry>();
+ let epoch = 0;
+
+ return {
+ /**
+ * Start tracking a WS request by its contract ID (the packet UUID).
+ * Returns a Promise that resolves when the listener calls resolve()
+ * with the same contract ID, or rejects on timeout.
+ *
+ * Call this immediately after sending the packet:
+ * const contractId = await sender(...);
+ * await tracker.track(contractId);
+ *
+ * This creates a `Promise<T>` with a properly typed `resolve: (value: T) => void`
+ * callback but then immediately erases it to `unknown` for storage. When the
+ * promise resolves, the caller gets T back. The erasure is a one-line bridge
+ * between the typed public API and the untyped internal map. This is us making T
+ * the same type as our storage, not the typescript rule that `unknown` types must
+ * do a type assertion before use.
+ */
+ track<T = void>(
+ contractId: string,
+ ): Promise<Epoch extends true ? T | null : T> {
+ if (enableEpochs) {
+ epoch++;
+ }
+
+ // Snapshot the current epoch so we can check staleness on resolve
+ const requestEpoch = epoch;
+
+ // Cast: TypeScript can't evaluate the conditional return type
+ // inside a generic factory. The runtime behavior is correct —
+ // with epochs, stale entries resolve with null; without, they
+ // resolve with T. The public type signature enforces this.
+ return new Promise<any>((resolve, reject) => {
+ const timeoutId = setTimeout(() => {
+ const entry = pending.get(contractId);
+ if (!entry) return; // already resolved/rejected
+
+ pending.delete(contractId);
+
+ // Stale epoch check — if a newer request was tracked since this
+ // one, don't surface a timeout error for an abandoned request.
+ // Resolve with null so the caller's promise settles cleanly
+ // (its finally block runs, no leaked closure).
+ if (enableEpochs && entry.epoch !== epoch) {
+ entry.resolve(null);
+ return;
+ }
+
+ let errorMessage =
+ `WS request ${contractId} timed out after ${timeoutMs}ms — server did not respond`;
+ reject(new WebError(errorMessage, "WsTimeoutError"));
+ }, timeoutMs);
+
+ pending.set(contractId, {
+ resolve: resolve as (value: unknown) => void,
+ reject,
+ timeoutId,
+ epoch: requestEpoch,
+ });
+ });
+ },
+
+ /**
+ * Resolve a pending request. Called by the WS listener after applying
+ * state from the server response.
+ *
+ * If epochs are enabled and the entry is stale (a newer request was
+ * tracked since this one), the entry is settled with null instead of
+ * the provided data. This ensures the caller's promise always settles
+ * — its finally block runs and closures are released. The caller
+ * checks for null to detect staleness.
+ *
+ * No-op if contractId is not found (already timed out, already
+ * resolved, or an unknown/broadcast ID).
+ */
+ resolve(contractId: string, data?: unknown): void {
+ const entry = pending.get(contractId);
+ if (!entry) return;
+
+ // Stale epoch check — a newer request was tracked since this one.
+ // Settle with null so the caller's promise resolves (its finally
+ // block runs, no leaked closure). The caller checks for null to
+ // detect staleness: if (result === null) return { kind: 'stale' };
+ if (enableEpochs && entry.epoch !== epoch) {
+ clearTimeout(entry.timeoutId);
+ pending.delete(contractId);
+ entry.resolve(null);
+ return;
+ }
+
+ clearTimeout(entry.timeoutId);
+ pending.delete(contractId);
+ entry.resolve(data);
+ },
+
+ /**
+ * Reject a pending request. Called by the WS listener when applying
+ * state fails, or internally on timeout.
+ *
+ * Unlike resolve, reject always surfaces regardless of epoch staleness.
+ * Errors should never be silently swallowed — the handler needs to
+ * know something went wrong so it can surface it to the user.
+ *
+ * No-op if contractId is not found.
+ */
+ reject(contractId: string, error: WebError): void {
+ const entry = pending.get(contractId);
+ if (!entry) return;
+
+ clearTimeout(entry.timeoutId);
+ pending.delete(contractId);
+ entry.reject(error);
+ },
+
+ /**
+ * Bump the epoch without tracking a new request.
+ *
+ * All currently pending entries become stale — their resolve() calls
+ * will settle with null when the server eventually responds.
+ * Their timeouts also settle with null (not rejection) since the
+ * timeout callback checks epoch staleness before rejecting.
+ *
+ * Use this when you want to "cancel" all in-flight requests without
+ * triggering error callbacks. For example, navigating away from a
+ * context where the responses are no longer relevant.
+ *
+ * No-op if epochs are not enabled.
+ */
+ invalidate(): void {
+ if (!enableEpochs) return;
+ epoch++;
+ },
+
+ /**
+ * Reject all pending requests and clear all timeouts.
+ * Call this from component onDestroy to prevent memory leaks
+ * and dangling timer callbacks.
+ */
+ dispose(): void {
+ for (const [contractId, entry] of pending) {
+ clearTimeout(entry.timeoutId);
+
+ // Reject with disposed error
+ let errMessage =
+ `WS tracker disposed while request ${contractId} was still pending`;
+ entry.reject(
+ new WebError(errMessage, "WsTrackerDisposedError"),
+ );
+ }
+ pending.clear();
+ },
+
+ /**
+ * Whether a tracked entry exists for this contract ID.
+ * Used by listeners to decide whether to apply state:
+ * - true: this is a response to a tracked request — apply + resolve
+ * - false: timed out, already resolved, or unknown (future broadcast)
+ */
+ hasPending(contractId: string): boolean {
+ return pending.has(contractId);
+ },
+
+ /** Number of currently pending (unresolved) tracked requests. */
+ pendingCount(): number {
+ return pending.size;
+ },
+ };
+}
+```
diff --git a/system_design/dal_architecture_overview.md b/system_design/dal_architecture_overview.md
@@ -0,0 +1,339 @@
+# DAL Architecture Overview
+
+# Architecture Overview: Three-Layer Pattern
+
+The codebase uses a three-layer architecture (similar to Clean Architecture /
+Layered Architecture) to separate concerns:
+
+```
+┌─────────────────────────────────────────────────────┐
+│ LAYER 3: NETWORKING (axum) │
+│ └── /services/projects/networking/axum/src/api/ │
+│ projects/create.rs │
+│ │
+│ Responsibility: │
+│ - HTTP endpoint handling (axum extractors) │
+│ - Authentication/Authorization (JWT tokens) │
+│ - Request/Response serialization (JSON via Axum) │
+│ - Calls core layer │
+└─────────────────────────────────────────────────────┘
+ │
+ ▼
+┌─────────────────────────────────────────────────────┐
+│ LAYER 2: CORE (Business Logic) │
+│ └── /services/projects/core/src/api/projects/ │
+│ create.rs │
+│ │
+│ Responsibility: │
+│ - Business logic validation │
+│ - Orchestration of multiple operations │
+│ - Converts domain models to/from DAL │
+│ - No HTTP/web framework knowledge │
+└────────────────────────┬────────────────────────────┘
+ │
+ ▼
+┌─────────────────────────────────────────────────────┐
+│ LAYER 1: DAL (Data Access Layer) │
+│ └── /layers/dal/src/models/projects/ │
+│ ├── tx_definitions.rs │
+│ └── postgres_txs.rs │
+│ │
+│ Responsibility: │
+│ - Raw SQL queries │
+│ - Database transaction management │
+│ - Zero business logic │
+└───────────┬─────────────────────────────────────────┘
+ │
+ ▼
+┌───────────────────────┐
+│ POSTGRESQL │
+│ DATABASE │
+└───────────────────────┘
+```
+
+---
+
+## How Each File Fits In
+
+### 1. DAL Layer: `tx_definitions.rs` + `postgres_txs.rs`
+
+**tx_definitions.rs** — Defines the traits that abstract database operations:
+
+```rust
+define_dal_transactions!(
+ GetProjectsByDepartmentId => get_projects_by_department_id(department_id: i32) -> Vec<Project>,
+ CreateProject => create_project(project: NewProject) -> Project,
+ DeleteProject => delete_project(project_id: i32, dept_id: i32) -> bool,
+ CheckUserProjectAccess => check_user_project_access(user_id: i32, project_id: i32) -> bool,
+ GetProjectById => get_project_by_id(project_id: i32) -> Option<Project>
+);
+```
+
+This expands to traits like:
+
+```rust
+pub trait CreateProject {
+ fn create_project(project: NewProject) -> impl Future<Output = sqlx::Result<Project>> + Send;
+}
+```
+
+See also: [define_dal_transactions!](/compiler/define_dal_transactions_macro.md)
+
+**postgres_txs.rs** — Implements those traits with actual SQL:
+
+```rust
+#[db_transaction(SqlxPostGresDescriptor, CreateProject)]
+async fn create_project(project: NewProject) -> Project {
+ let pool = T::yield_pool();
+ let query = r#"
+ INSERT INTO projects (department_id, name, description, created_at, updated_at)
+ VALUES ($1, $2, $3, NOW(), NOW())
+ RETURNING id, department_id, name, description, created_at, updated_at
+ "#;
+ sqlx::query_as::<_, Project>(query)
+ .bind(project.department_id)
+ .bind(project.name)
+ .bind(project.description)
+ .fetch_one(pool)
+ .await
+}
+```
+
+The `#[db_transaction(StructName, TraitName)]` macro:
+
+1. Generates an impl `TraitName` for `StructName<T>` where
+ `T: YieldPostGresPool`
+2. Wraps the async function body in that implementation
+3. Makes the function callable as `StructName::<PoolType>::create_project(...)`
+
+### 2. Core Layer: `core/src/api/projects/create.rs`
+
+This layer orchestrates the business logic:
+
+```rust
+pub async fn create_project<X, S>(storage_handle: &S, new_project: NewProject) -> Result<Project, NanoServiceError>
+where
+ X: CreateProject + ProjectBranchesCreateBranch,
+ S: GitDataTransfer + Debug,
+{
+ // 1. VALIDATION (business rule)
+ if new_project.department_id <= 0 {
+ return Err(NanoServiceError::bad_request("Invalid department ID".to_string()));
+ }
+ if new_project.name.trim().is_empty() {
+ return Err(NanoServiceError::bad_request("Project name cannot be empty".to_string()));
+ }
+ if new_project.description.trim().is_empty() {
+ return Err(NanoServiceError::bad_request("Project description cannot be empty".to_string()));
+ }
+
+ // 2. Create project in database
+ let created_project = X::create_project(new_project).await?;
+
+ // 3. Create git directory (side effect)
+ create_git_repo(storage_handle, created_project.id).await?;
+
+ // 4. Register default branch
+ let new_branch = NewProjectBranch { project_id: created_project.id, branch: "main".into() };
+ X::create_branch(new_branch).await.map_err(|e| NanoServiceError::unknown(e.to_string()))?;
+
+ Ok(created_project)
+}
+```
+
+**Key characteristics:**
+
+- No HTTP/Websocket knowledge — pure async functions
+- Generic over database handle (`X: CreateProject`) — allows mocking for tests
+- Validates business rules before touching the database
+- Orchestrates multiple operations (create project + git repo + branch)
+
+### 3. Networking Layer: `networking/axum/src/api/projects/create.rs`
+
+This layer adapts the core to HTTP:
+
+```rust
+pub async fn create_project<T, X, Y>(
+ token: HeaderToken<X, NoRoleCheck, T>, // Auth extraction
+ Json(payload): Json<NewProjectRequest>, // JSON deserialization
+) -> Result<impl IntoResponse, NanoServiceError>
+where
+ T: CreateProject + GetProjectsByDepartmentId + PingAuthSession + ProjectBranchesCreateBranch,
+ X: GetConfigVariable,
+ Y: YieldPostGresPool + Send + Sync + Clone + Debug,
+{
+ // 1. Extract department from JWT
+ let department_id = token.get_department_id()?;
+
+ // 2. Convert request DTO to domain model
+ let new_project = NewProject {
+ department_id,
+ name: payload.name,
+ description: payload.description
+ };
+
+ // 3. Create git storage handle
+ let storage_handle = PostgresGitBlobHandle::<Y>::new();
+
+ // 4. Call core business logic
+ let _ = create_project_core::<T, _>(&storage_handle, new_project).await?;
+
+ // 5. Return updated list
+ let projects = get_projects_by_department_id_core::<T>(department_id).await?;
+ Ok((StatusCode::CREATED, Json(projects)))
+}
+```
+
+**Key characteristics:**
+
+- Axum extractors handle HTTP parsing
+- Authentication via JWT token validation
+- Converts between request types (`NewProjectRequest` → `NewProject`)
+- Handles HTTP concerns (status codes, JSON serialization)
+
+---
+
+## Complete Workflow
+
+```
+Client Request
+ │
+ ▼
+ ┌─────────────────────────────────────────────────────────────────┐
+ │ 1. HTTP REQUEST arrives at axum endpoint │
+ │ POST /api/v1/projects/create │
+ │ Headers: Authorization: Bearer <jwt> │
+ │ Body: { "name": "...", "description": "..." } │
+ └─────────────────────────────────────────────────────────────────┘
+ │
+ ▼
+ ┌─────────────────────────────────────────────────────────────────┐
+ │ 2. AXUM LAYER (networking/axum) │
+ │ - Extracts and validates JWT token │
+ │ - Deserializes JSON payload │
+ │ - Converts NewProjectRequest → NewProject │
+ │ - Creates PostgresGitBlobHandle │
+ │ - Calls create_project_core() │
+ └─────────────────────────────────────────────────────────────────┘
+ │
+ ▼
+ ┌─────────────────────────────────────────────────────────────────┐
+ │ 3. CORE LAYER (core/api) │
+ │ - Validates department_id > 0 │
+ │ - Validates name is not empty │
+ │ - Validates description is not empty │
+ │ - Calls DAL: T::create_project() │
+ │ - Calls git repo creation (storage_handle) │
+ │ - Calls DAL: T::create_branch() │
+ │ - Returns Project model │
+ └─────────────────────────────────────────────────────────────────┘
+ │
+ ▼
+ ┌─────────────────────────────────────────────────────────────────┐
+ │ 4. DAL LAYER (dal/models) │
+ │ - tx_definitions.rs: defines CreateProject trait │
+ │ - postgres_txs.rs: │
+ │ #[db_transaction(Struct, Trait)] │
+ │ async fn create_project() -> SQL INSERT + RETURNING │
+ │ - SqlxPostGresDescriptor implements the trait │
+ │ - SQL executed against PostgreSQL │
+ └─────────────────────────────────────────────────────────────────┘
+ │
+ ▼
+ ┌─────────────────────────────────────────────────────────────────┐
+ │ 5. DATABASE (PostgreSQL) │
+ │ INSERT INTO projects (...) VALUES (...) │
+ │ RETURNING id, department_id, name, description, ... │
+ └─────────────────────────────────────────────────────────────────┘
+ │
+ ▼
+ Return back up the stack with created Project
+```
+
+---
+
+## Data Flow Diagram
+
+```
+┌─────────────┐ HTTP JSON ┌─────────────┐ NewProject ┌─────────────┐
+│ Client │ ────────────────► │ networking │ ───────────────► │ core │
+│ │ │ (axum) │ │ (create) │
+└─────────────┘ └─────────────┘ └─────────────┘
+ │
+ ┌───────────────────────────┼─────────────────────┐
+ │ │ │
+ ▼ ▼ ▼
+ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐
+ │DAL: CreateProject│ │GitDataTransfer │ │DAL: CreateBranch │
+ │(sqlx INSERT) │ │(create git dir) │ │(sqlx INSERT) │
+ └──────────────────┘ └──────────────────┘ └──────────────────┘
+ │ │ │
+ ▼ ▼ ▼
+ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐
+ │ PostgreSQL │ │ Database │ │ PostgreSQL │
+ │ projects │ │ git_blobs │ │ project_branches │
+ └──────────────────┘ └──────────────────┘ └──────────────────┘
+```
+
+---
+
+## Pros and Cons of This Approach
+
+### ✅ Pros
+
+| Benefit | Explanation |
+| -------------------------- | --------------------------------------------------------------------------------------------------------------------------------- |
+| **Separation of Concerns** | Each layer has a single responsibility. DAL knows SQL, Core knows business logic, Networking knows HTTP. |
+| **Testability** | Core layer can be tested with mock DB handles (`MockDeadPostGresPool`) without any HTTP server. No network needed for unit tests. |
+| **Database Abstraction** | The trait-based DAL allows swapping PostgreSQL for another database (though not currently used). |
+| **Reusability** | Core layer functions can be called from HTTP, WebSocket, gRPC, CLI, or tests — not coupled to HTTP. |
+| **Consistency** | All endpoints follow the same pattern — predictable codebase structure. |
+| **Swappable Networking** | Axum could be swapped for Actix-web or Hyper with minimal core changes. |
+| **Clear Boundaries** | Easy to identify where bugs live: HTTP issue → networking, business logic → core, SQL → DAL. |
+
+### ❌ Cons
+
+| Issue | Explanation |
+| --------------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| **Boilerplate Overhead** | Three files per feature with traits, macros, and adapters creates ceremony. A simple CRUD operation requires significant scaffolding. |
+| **Generic Proliferation** | Every function has `3+` generic type parameters (`<T, X, Y>`) making signatures hard to read and IDE autocomplete overwhelming. |
+| **Tight Coupling via Traits** | The `where X: CreateProject + GetProjectsByDepartmentId + ...` clauses require implementing many traits, creating coupling between networking and DAL layers. |
+| **No Transaction Across Layers** | The `create_project` core function calls multiple DAL operations that aren't wrapped in a DB transaction. If `create_git_repo` fails, the project row was already committed. |
+| **Hidden Complexity in Macros** | `#[db_transaction]` and `define_dal_transactions!` are magical — hard to debug, IDE can't "go to definition" easily. |
+| **Request/Response Type Proliferation** | `NewProjectRequest` (HTTP layer) → `NewProject` (Core layer) → `NewProject` (DAL) is mostly the same struct with different names. |
+| **Hard to Follow the Flow** | New developers must trace through 3 files + 2 macros to understand how a simple INSERT works. |
+| **Over-engineering for Simple Ops** | For a simple `SELECT * FROM projects`, you still need the full three-layer setup. |
+
+---
+
+## Key Files Summary
+
+| File | Role | Key Pattern |
+| -------------------------------------------------------------- | ------------------------------------------ | --------------------------------------------- |
+| `layers/dal/src/models/projects/tx_definitions.rs` | Defines trait signatures for DB operations | `define_dal_transactions!` macro |
+| `layers/dal/src/models/projects/postgres_txs.rs` | Implements the trait with SQL | `#[db_transaction(Struct, Trait)]` proc macro |
+| `services/projects/core/src/api/projects/create.rs` | Business logic orchestration | Validation → DAL calls → Return |
+| `services/projects/networking/axum/src/api/projects/create.rs` | HTTP adapter layer | Axum extractors → Call core → HTTP response |
+
+---
+
+## Testability Example
+
+The beauty of this pattern is shown in the core tests:
+
+```rust
+// Core layer test with MOCK database — no real DB needed
+#[db_transaction(MockDbHandle, CreateProject)]
+async fn create_project(new_project: NewProject) -> Project {
+ Ok(Project { id: 1, ... }) // Mocked response
+}
+
+let result = create_project::<MockDbHandle<MockDeadPostGrosPool>, _>(
+ &mock_git_handle,
+ new_project,
+)
+.await;
+```
+
+This lets you test business logic validation and orchestration without spinning
+up a PostgreSQL instance.
diff --git a/system_design/design_patterns/abstraction.md b/system_design/design_patterns/abstraction.md
@@ -0,0 +1,35 @@
+# Abstraction
+
+The use of abstractions is one of the most important concepts in computer
+science. For example, one aspect of good programming practice is to formulate a
+simple application program interface (API) for a set of functions that allow
+programmers to use the code without having to delve into its inner workings.
+Different programming languages provide different forms and levels of support
+for abstraction, such as Rust's trait definitions and module system with
+explicit pub visibility boundaries.
+
+## Abstractions provided by a computer system
+
+<img src="/whiteboards/abstraction_of_computer_system.png" alt="Abstraction" width="100%">
+
+We are probably familiar with several of the abstractions seen in computer
+systems. On the processor side, the instruction set architecture provides an
+abstraction of the actual processor hardware. With this abstraction, a
+machine-code program behaves as if it were executed on a processor that performs
+just one instruction at a time. The underlying hardware is far more elaborate,
+executing multiple instructions in parallel, but always in a way that is
+consistent with the simple, sequential model. By keeping the same execution
+model, different processor implementations can execute the same machine code
+while offering a range of cost and performance.
+
+On the operating system side, we have introduced three abstractions: files as an
+abstraction of I/O devices, virtual memory as an abstraction of program memory,
+and processes as an abstraction of a running program. To these abstractions we
+add a new one: the virtual machine, providing an abstraction of the entire
+computer, including the operating system, the processor, and the programs. The
+idea of a virtual machine was introduced by IBM in the 1960s, but it has become
+more prominent recently as a way to manage computers that must be able to run
+programs designed for multiple operating systems (such as Microsoft Windows, Mac
+OS X, and Linux) or different versions of the same operating system.
+
+<img src="/whiteboards/five_abstraction_levels.png" alt="Abstraction" width="100%">
diff --git a/system_design/design_patterns/chain_of_responsibility.md b/system_design/design_patterns/chain_of_responsibility.md
@@ -0,0 +1,213 @@
+# Chain of Responsibility
+
+The Chain of Responsibility (CoR) pattern is a behavioral design pattern where a
+request is passed along a chain of handlers — each handler either processes the
+request or forwards it to the next one in the chain.
+
+The three key components are:
+
+- Handler — a trait defining the interface for handling requests and holding the
+ successor link
+
+- Successor — the next handler in the chain, stored as Option<Box<dyn Handler>>
+
+- Request — the data passed along the chain
+
+> The Rust community's guiding principle here is: static where you can, dynamic
+> where you must.
+
+### Example 1: Purchase Approval Chain (Dynamic Disptach)
+
+The key pattern here is the Option<Box<dyn Approver>> field — it lets each
+handler optionally own its successor and forward the request via
+next.process_request(request).
+
+```rust
+struct PurchaseRequest {
+ amount: f64,
+}
+
+trait Approver {
+ fn set_successor(&mut self, successor: Box<dyn Approver>);
+ fn process_request(&self, request: &PurchaseRequest);
+}
+
+struct Manager {
+ successor: Option<Box<dyn Approver>>,
+}
+
+impl Approver for Manager {
+ fn set_successor(&mut self, successor: Box<dyn Approver>) {
+ self.successor = Some(successor);
+ }
+ fn process_request(&self, request: &PurchaseRequest) {
+ if request.amount <= 1000.0 {
+ println!("Manager approves ${}", request.amount);
+ } else if let Some(ref next) = self.successor {
+ next.process_request(request); // pass it up
+ } else {
+ println!("Cannot be approved.");
+ }
+ }
+}
+
+struct Director { successor: Option<Box<dyn Approver>> }
+
+impl Approver for Director {
+ fn set_successor(&mut self, successor: Box<dyn Approver>) {
+ self.successor = Some(successor);
+ }
+ fn process_request(&self, request: &PurchaseRequest) {
+ if request.amount <= 5000.0 {
+ println!("Director approves ${}", request.amount);
+ } else if let Some(ref next) = self.successor {
+ next.process_request(request);
+ } else {
+ println!("Cannot be approved.");
+ }
+ }
+}
+
+struct President;
+
+impl Approver for President {
+ fn set_successor(&mut self, _: Box<dyn Approver>) {} // terminal node
+ fn process_request(&self, request: &PurchaseRequest) {
+ if request.amount <= 10000.0 {
+ println!("President approves ${}", request.amount);
+ } else {
+ println!("Request denied.");
+ }
+ }
+}
+
+fn main() {
+ let president = President;
+ let mut director = Director { successor: Some(Box::new(president)) };
+ let mut manager = Manager { successor: Some(Box::new(director)) };
+
+ manager.process_request(&PurchaseRequest { amount: 500.0 }); // Manager approves
+ manager.process_request(&PurchaseRequest { amount: 3000.0 }); // Director approves
+ manager.process_request(&PurchaseRequest { amount: 8000.0 }); // President approves
+ manager.process_request(&PurchaseRequest { amount: 15000.0 }); // Denied
+}
+```
+
+### Example 2: Purchase Approval Chain (Generics - Zero-cost static dispatch)
+
+This is verbose but gives you zero-cost static dispatch — no heap allocation, no
+vtable lookup, all method calls resolved at compile time.
+
+`Handler<Manager<Director<President>>>`
+
+```rust
+trait Handler {
+ fn handle(&self, amount: f64);
+}
+
+// Terminal handler — no successor
+struct President;
+
+impl Handler for President {
+ fn handle(&self, amount: f64) {
+ if amount <= 10_000.0 {
+ println!("President approves ${amount}");
+ } else {
+ println!("Request denied.");
+ }
+ }
+}
+
+// Generic handler wrapping the next handler N
+struct Manager<N: Handler> {
+ next: N,
+}
+
+impl<N: Handler> Handler for Manager<N> {
+ fn handle(&self, amount: f64) {
+ if amount <= 1_000.0 {
+ println!("Manager approves ${amount}");
+ } else {
+ self.next.handle(amount); // static dispatch!
+ }
+ }
+}
+
+struct Director<N: Handler> {
+ next: N,
+}
+
+impl<N: Handler> Handler for Director<N> {
+ fn handle(&self, amount: f64) {
+ if amount <= 5_000.0 {
+ println!("Director approves ${amount}");
+ } else {
+ self.next.handle(amount);
+ }
+ }
+}
+
+fn main() {
+ // The full type is Manager<Director<President>>
+ let chain = Manager {
+ next: Director {
+ next: President,
+ },
+ };
+
+ chain.handle(500.0); // Manager approves
+ chain.handle(3_000.0); // Director approves
+ chain.handle(8_000.0); // President approves
+ chain.handle(15_000.0); // Request denied
+}
+```
+
+The compiler monomorphizes this — it generates a unique, optimized function for
+each concrete type combination. No heap, no vtable.
+
+#### Using impl Trait to Hide the Chain Type
+
+If you want to hide the ugly Manager<Director<President>> type from callers, use
+impl Trait as a return type:
+
+```rust
+fn build_chain() -> impl Handler {
+ Manager {
+ next: Director {
+ next: President,
+ },
+ }
+}
+
+fn main() {
+ let chain = build_chain(); // type is opaque to the caller
+ chain.handle(3_000.0);
+}
+```
+
+This still uses static dispatch under the hood — the compiler knows the exact
+type, the caller just doesn't need to spell it out.
+
+### Example 3: Enum-Based Variant (More Idiomatic)
+
+Rust's enums offer a cleaner alternative when the set of handlers is fixed at
+compile time:
+
+```rust
+enum SupportLevel {
+ Basic,
+ Intermediate,
+ Critical,
+}
+
+fn handle_request(level: SupportLevel) {
+ match level {
+ SupportLevel::Basic => println!("L1 Support handled it"),
+ SupportLevel::Intermediate => println!("L2 Support handled it"),
+ SupportLevel::Critical => println!("L3 Support handled it"),
+ }
+}
+```
+
+This approach avoids heap allocation and dynamic dispatch entirely, but loses
+the runtime flexibility of adding or reordering handlers.
diff --git a/system_design/design_patterns/decomposition.md b/system_design/design_patterns/decomposition.md
@@ -0,0 +1,3 @@
+# Decomposition
+
+
diff --git a/system_design/design_patterns/dependency_injection.md b/system_design/design_patterns/dependency_injection.md
@@ -0,0 +1,177 @@
+# Dependency Injection
+
+**Dependency injection** (DI) is a design pattern where a component receives
+its dependencies from an external source rather than creating them
+itself.[^1](https://dev.to/sgchris/how-traits-enable-dependency-injection-in-rust-5a50)
+This decouples components, making code easier to test, swap, and maintain.
+In Rust, DI is achieved idiomatically through **traits** and **generics**
+— without needing a framework — leveraging the compiler's type system to
+enforce correctness at compile
+time.[^2](https://jmmv.dev/2022/04/rust-traits-and-dependency-injection.html)
+
+---
+
+## Core Tool: Traits
+
+In Rust, **traits** act as the contract (interface) that dependencies must
+fulfill.[^1](https://dev.to/sgchris/how-traits-enable-dependency-injection-in-rust-5a50)
+Define the behaviour, not the implementation:
+
+```rust
+pub trait Logger {
+ fn log(&self, message: &str);
+}
+
+pub struct ConsoleLogger;
+
+impl Logger for ConsoleLogger {
+ fn log(&self, message: &str) {
+ println!("[LOG]: {}", message);
+ }
+}
+```
+
+---
+
+## Approach 1: Generics (Static Dispatch)
+
+The preferred Rust approach — the concrete type is resolved at **compile
+time**, producing zero-cost
+abstractions.[^1](https://dev.to/sgchris/how-traits-enable-dependency-injection-in-rust-5a50)
+
+```rust
+pub struct Application<L: Logger> {
+ logger: L,
+}
+
+impl<L: Logger> Application<L> {
+ pub fn new(logger: L) -> Self {
+ Self { logger }
+ }
+
+ pub fn run(&self) {
+ self.logger.log("Application is running!");
+ }
+}
+
+fn main() {
+ let app = Application::new(ConsoleLogger);
+ app.run(); // prints: [LOG]: Application is running!
+}
+```
+
+`Application` only requires that `L` implements `Logger` — swap in any
+logger without touching `Application`
+itself.[^3](https://users.rust-lang.org/t/how-do-you-implement-dependency-injection-in-rust/213)
+
+---
+
+## Approach 2: Trait Objects (Dynamic Dispatch)
+
+When you need runtime flexibility or to store mixed implementations in a
+collection, use `Box<dyn
+Trait>`.[^3](https://users.rust-lang.org/t/how-do-you-implement-dependency-injection-in-rust/213)
+
+```rust
+pub trait MessageSender {
+ fn send(&self, msg: &str);
+}
+
+pub struct NotificationService {
+ sender: Box<dyn MessageSender>,
+}
+
+impl NotificationService {
+ pub fn new(sender: Box<dyn MessageSender>) -> Self {
+ Self { sender }
+ }
+
+ pub fn notify(&self, msg: &str) {
+ self.sender.send(msg);
+ }
+}
+```
+
+This introduces a small **runtime overhead** via vtable lookup, so prefer
+generics unless dynamic dispatch is genuinely
+needed.[^1](https://dev.to/sgchris/how-traits-enable-dependency-injection-in-rust-5a50)
+
+---
+
+## Approach 3: Enums (Closed Set)
+
+If you have a fixed, known set of implementations, an enum avoids both
+generics complexity and boxing
+overhead.[^3](https://users.rust-lang.org/t/how-do-you-implement-dependency-injection-in-rust/213)
+
+```rust
+pub enum LoggerKind {
+ Console,
+ File(String),
+}
+
+impl Logger for LoggerKind {
+ fn log(&self, message: &str) {
+ match self {
+ LoggerKind::Console => println!("[Console]: {}", message),
+ LoggerKind::File(path) => println!("[File({})] {}", path, message),
+ }
+ }
+}
+```
+
+---
+
+## Why DI Shines in Testing
+
+The real payoff is **mockability** — swap the real implementation with a
+mock during
+tests.[^1](https://dev.to/sgchris/how-traits-enable-dependency-injection-in-rust-5a50)
+
+```rust
+pub struct MockLogger {
+ pub messages: std::cell::RefCell<Vec<String>>,
+}
+
+impl Logger for MockLogger {
+ fn log(&self, message: &str) {
+ self.messages.borrow_mut().push(message.to_string());
+ }
+}
+
+#[test]
+fn test_app_logs_on_run() {
+ let mock = MockLogger { messages: Default::default() };
+ let app = Application::new(&mock);
+ app.run();
+ assert!(mock.messages.borrow().contains(&"Application is running!".to_string()));
+}
+```
+
+No real I/O, no external systems — pure, fast unit
+tests.[^1](https://dev.to/sgchris/how-traits-enable-dependency-injection-in-rust-5a50)
+
+---
+
+## Choosing the Right Approach
+
+| | Generics | `Box<dyn Trait>` | Enum |
+| :------------------------------------- | :-------------------- | :---------------------- | :--------------------- |
+| **Dispatch** | Compile time (static) | Runtime (dynamic) | Compile time (static) |
+| **Overhead** | Zero | Vtable lookup | Zero |
+| **Implementations** | Any (open set) | Any (open set) | Fixed (closed set) |
+| **Heterogeneous collections** | ❌ | ✅ | ✅ |
+| **Best for** | Most cases | Plugin-like flexibility | Known, finite variants |
+
+For complex dependency graphs, consider a DI container crate such as
+[rustyinject](https://github.com/AlexSherbinin/rustyinject).[^4](https://github.com/AlexSherbinin/rustyinject)
+
+---
+
+## Key Pitfall: Visibility Creep
+
+Any type referenced in a **public** trait's function signature must also be
+public.[^2](https://jmmv.dev/2022/04/rust-traits-and-dependency-injection.html)
+If your trait is public but references an internal struct, you'll be forced
+to expose that struct too — keep internal traits `pub(crate)` where
+possible.[^2](https://jmmv.dev/2022/04/rust-traits-and-dependency-injection.html)
diff --git a/system_design/design_patterns/design_principles.md b/system_design/design_patterns/design_principles.md
@@ -0,0 +1,15 @@
+# Design Principles
+
+## Separating what changes from what stays the same
+
+Identify the aspects of your application that vary and separate them from what
+stays the same.
+
+As simple as this concept is, it forms the basis for almost every design
+pattern. All patterns provide a way to let some part of a system vary
+independently of all other parts.
+
+## Program to an interface, not an implementation.
+
+The point is to exploit polymorphism by programming to a supertype so that the
+actual runtime object isn’t locked into the code.
diff --git a/system_design/design_patterns/entity_boundary_interactor.md b/system_design/design_patterns/entity_boundary_interactor.md
@@ -0,0 +1,20 @@
+# Entity Boundary Interactor
+
+[Reference](https://ebi.readthedocs.io/en/latest/)
+
+The idea is to design programs so that their architectures immediately present
+their use case. It’s a way to design programs so that its internal dependency
+graph is organized cleanly and its elements are joined together with as loose
+coupling as possible.
+
+Ultimately, the goal is the separation of concerns between application layers,
+this architecture and many like it aren’t dependent on presentation models or
+platforms. All the arrows, or dependencies, point inwards in the abstraction
+chain, each successive layer less abstract than the one before it.
+
+Keeping the arrows pointing inwards makes the code easy to maintain, extend,
+test and refactor. EBI imposes some architectural requirements on the
+programmers, that is, you must navigate around its rules, but this is kept at a
+minimum.
+
+
diff --git a/system_design/design_patterns/entity_component_system.md b/system_design/design_patterns/entity_component_system.md
@@ -0,0 +1,259 @@
+# Entity Component System (ECS)
+
+> **Common use cases:** Game engines, simulations, UI frameworks, any system
+> with many objects sharing overlapping data
+
+---
+
+## What is ECS?
+
+The **Entity Component System** is a composition-over-inheritance architectural
+pattern that separates _identity_ (entities), _data_ (components), and _logic_
+(systems). Instead of a class hierarchy where a `Player` extends `Character`,
+you build objects by attaching plain data structs to a bare ID.
+
+| Concept | Role | Rust analogy |
+| ------------- | ----------------------------------------- | --------------------------------------------- |
+| **Entity** | A unique ID — nothing more | `u64` or a newtype wrapper |
+| **Component** | Plain data attached to an entity | A `struct` with `#[derive(Component)]` |
+| **System** | Logic that queries and mutates components | A function that receives queries as arguments |
+
+---
+
+## Simple Example from Scratch
+
+A minimal ECS without any external crate to understand the mechanics:
+
+```rust
+use std::collections::HashMap;
+
+type Entity = u64;
+
+// Components are plain structs
+#[derive(Debug)]
+struct Position { x: f32, y: f32 }
+
+#[derive(Debug)]
+struct Velocity { dx: f32, dy: f32 }
+
+// World holds component storage — one HashMap per component type
+#[derive(Default)]
+struct World {
+ next_entity: Entity,
+ positions: HashMap<Entity, Position>,
+ velocities: HashMap<Entity, Velocity>,
+}
+
+impl World {
+ fn spawn(&mut self) -> Entity {
+ let id = self.next_entity;
+ self.next_entity += 1;
+ id
+ }
+}
+
+// A system is just a plain function
+fn movement_system(world: &mut World) {
+ for (entity, vel) in &world.velocities {
+ if let Some(pos) = world.positions.get_mut(entity) {
+ pos.x += vel.dx;
+ pos.y += vel.dy;
+ }
+ }
+}
+
+fn main() {
+ let mut world = World::default();
+
+ let player = world.spawn();
+ world.positions.insert(player, Position { x: 0.0, y: 0.0 });
+ world.velocities.insert(player, Velocity { dx: 1.0, dy: 0.5 });
+
+ let static_obstacle = world.spawn();
+ world.positions.insert(static_obstacle, Position { x: 10.0, y: 10.0 });
+ // No Velocity — this entity won't move
+
+ movement_system(&mut world);
+
+ println!("{:?}", world.positions[&player]); // Position { x: 1.0, y: 0.5 }
+}
+```
+
+Key insight: `static_obstacle` was never touched by `movement_system` because it
+has no `Velocity`. Systems only operate on the _intersection_ of components they
+care about — no `if entity_is_static` checks required.
+
+---
+
+## Growing the Design
+
+Once the naive HashMap approach gets unwieldy you need:
+
+- **Archetypes** — group entities by their exact component set for better cache
+ locality (how Bevy's world works)
+- **Sparse sets** — fast add/remove at the cost of iteration speed (used for
+ rarely-changed components)
+- **System scheduling** — run systems in parallel when they don't share mutable
+ access; `bevy_ecs` does this automatically via Rust's borrow rules
+
+```rust
+// A slightly more ergonomic API pattern using a builder
+struct EntityBuilder<'w> {
+ world: &'w mut World,
+ id: Entity,
+}
+
+impl<'w> EntityBuilder<'w> {
+ fn with_position(self, x: f32, y: f32) -> Self {
+ self.world.positions.insert(self.id, Position { x, y });
+ self
+ }
+ fn with_velocity(self, dx: f32, dy: f32) -> Self {
+ self.world.velocities.insert(self.id, Velocity { dx, dy });
+ self
+ }
+ fn build(self) -> Entity { self.id }
+}
+```
+
+---
+
+## Bevy as a Production ECS
+
+[Bevy](https://bevyengine.org/) is the most prominent real-world ECS in Rust and
+a great study in ergonomic API design. Its ECS lives in the standalone
+[`bevy_ecs`](https://crates.io/crates/bevy_ecs) crate — you can use it without
+the rest of the game engine.
+
+```rust
+use bevy::prelude::*;
+
+// Components — plain structs that derive `Component`
+#[derive(Component)]
+struct Position { x: f32, y: f32 }
+
+#[derive(Component)]
+struct Velocity { dx: f32, dy: f32 }
+
+// Systems are just functions; Bevy injects queries via its scheduler
+fn movement_system(mut query: Query<(&mut Position, &Velocity)>) {
+ for (mut pos, vel) in &mut query {
+ pos.x += vel.dx;
+ pos.y += vel.dy;
+ }
+}
+
+fn spawn_player(mut commands: Commands) {
+ commands.spawn((
+ Position { x: 0.0, y: 0.0 },
+ Velocity { dx: 1.0, dy: 0.5 },
+ ));
+}
+
+fn main() {
+ App::new()
+ .add_plugins(MinimalPlugins)
+ .add_systems(Startup, spawn_player)
+ .add_systems(Update, movement_system)
+ .run();
+}
+```
+
+Notice how `movement_system` never knows about entities at all — it just
+expresses _"give me every entity that has both a mutable Position and an
+immutable Velocity"_. Bevy's scheduler can then safely parallelise any two
+systems whose query sets don't conflict.
+
+### Why Bevy's ECS is a Masterclass in Ergonomic Rust
+
+- **Function-parameter injection** via `SystemParam` traits — adding a
+ `Res<Time>` parameter to a system just works
+- **`Commands` for deferred mutations** — you can't borrow the world mutably
+ while iterating it, so commands queue up spawns/despawns to run between
+ systems
+- **`With` / `Without` / `Or` filters** — express complex queries in the type
+ system with no runtime cost
+- **Observers and triggers** (added in Bevy 0.14) — reactive event-driven logic
+ built on top of the ECS
+
+---
+
+## Why ECS Fits Rust So Well
+
+Traditional OOP patterns with shared mutable object graphs fight the borrow
+checker constantly. ECS sidesteps this by:
+
+1. **Separating data from logic** — systems take fine-grained borrows, so two
+ systems can run in parallel as long as they don't both need `&mut` on the
+ same component type
+2. **Cache-friendly storage** — components of the same type are stored
+ contiguously, turning what would be pointer-chasing in OOP into sequential
+ memory reads
+3. **Composition without `dyn Trait`** — you don't need dynamic dispatch;
+ queries are resolved at compile time via associated types and const generics
+
+---
+
+## Resources & Further Reading
+
+### Talks
+
+- 🎥 **Chris Biscardi — "Bevy: A Case Study in Ergonomic Rust"**
+ (RustConf 2024)\
+ Deep dive into the API design tricks Bevy uses — applicable far beyond games.\
+ →
+ [https://www.youtube.com/watch?v=CnoDOc6ML0Y](https://www.youtube.com/watch?v=CnoDOc6ML0Y)
+
+- 🎥 **Alice Cecile — "Architecting Bevy"** (interview, 2024)\
+ Alice is a core Bevy contributor and foundation member. This talk covers ECS
+ architecture decisions and long-term open-source project management.\
+ →
+ [https://www.youtube.com/watch?v=PND2Wpy6U-E](https://www.youtube.com/watch?v=PND2Wpy6U-E)
+
+### Video Series
+
+- 📺 **Brooks Patton — "Improve Your Rust Skills by Making an ECS Library"**
+ (YouTube playlist, 2021)\
+ Builds an ECS from scratch in Rust. Covers `TypeId`, `HashMap`, generics,
+ `Copy`/`Clone`, interior mutability, and modules — great for understanding the
+ internals.\
+ →
+ [https://www.youtube.com/playlist?list=PLrmY5pVcnuE_SQSzGPWUJrf9Yo-YNeBYs](https://www.youtube.com/playlist?list=PLrmY5pVcnuE_SQSzGPWUJrf9Yo-YNeBYs)\
+ → Code:
+ [https://github.com/brooks-builds/improve_skills_by_building_ecs_library_in_rust](https://github.com/brooks-builds/improve_skills_by_building_ecs_library_in_rust)
+
+### People to Follow
+
+| Person | Known for | Link |
+| ----------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------------------- |
+| **Alice Cecile** (`alice-i-cecile`) | Bevy core contributor, ECS architect, RFC author | [github.com/alice-i-cecile](https://github.com/alice-i-cecile) |
+| **Chris Biscardi** | Bevy educator, ergonomic Rust APIs, Rust Adventure | [youtube/@chrisbiscardi](https://www.youtube.com/@chrisbiscardi) |
+| **Brooks Patton** (`brookzerker`) | ECS from scratch series, Rust std-lib deep dives | [youtube/@brookzerker](https://www.youtube.com/@brookzerker) |
+| **Jon Gjengset** | "Crust of Rust" — intermediate Rust internals (not ECS-specific, but indispensable for understanding the mechanics ECS relies on like interior mutability) | [youtube/@jongjengset](https://www.youtube.com/@jongjengset) |
+| **Alice Ryhl** | Tokio maintainer, async Rust expert — not ECS-specific, but her writing on structured concurrency complements ECS scheduling design | [ryhl.io](https://ryhl.io) |
+
+### Crates
+
+| Crate | Notes |
+| ----------------------------------------------- | ---------------------------------------------------------- |
+| [`bevy_ecs`](https://crates.io/crates/bevy_ecs) | Production-grade, standalone; the reference implementation |
+| [`hecs`](https://crates.io/crates/hecs) | Minimal, low-level; great for embedding |
+| [`specs`](https://crates.io/crates/specs) | Older, more explicit; parallel systems via Rayon |
+| [`shipyard`](https://crates.io/crates/shipyard) | Sparse-set ECS; fast add/remove |
+| [`flecs`](https://crates.io/crates/flecs_ecs) | Rust bindings for the C flecs library; extremely mature |
+
+---
+
+## When _Not_ to Use ECS
+
+ECS adds indirection and query overhead. Avoid it when:
+
+- You have **fewer than ~100 objects** that rarely change structure
+- Your logic is **purely sequential and single-threaded** with no parallelism
+ benefit
+- You're building a **CRUD app or domain model** — here, plain structs and trait
+ objects are cleaner
+
+A good rule of thumb: reach for ECS when you find yourself writing
+`if player.has_component::<Health>()` — that conditional is ECS trying to break
+out.
diff --git a/system_design/design_patterns/finite_state_machine.md b/system_design/design_patterns/finite_state_machine.md
@@ -0,0 +1,184 @@
+# Finite State Machine
+
+A **state machine** (or finite state machine, FSM) is a behavioral model where a
+system can exist in exactly one of a finite number of states at any given time,
+and transitions between states are triggered by specific events or conditions.
+Rust is particularly well-suited for state machines because its type system can
+enforce valid transitions either at runtime (using enums) or at compile time
+(using the typestate
+pattern).[^1](https://www.itemis.com/en/products/itemis-create/documentation/user-guide/overview_what_are_state_machines)
+
+---
+
+## Core Concepts
+
+Three building blocks define any state
+machine:[^2](https://www.ni.com/docs/en-US/bundle/patools/page/what-is-a-state-machine.html)
+
+- **States** – distinct situations a system can be in (e.g., `Created`, `Paid`,
+ `Shipped`)
+- **Transitions** – rules that move the system from one state to another
+- **Events** – inputs/triggers that cause a transition (e.g., "payment
+ received")
+
+A system can only be in **one state at a time**, and transitions that don't
+exist for the current state are simply
+invalid.[^3](https://stately.ai/blog/2023-10-02-what-is-a-state-machine)
+
+---
+
+## Approach 1: Enum-Based State Machine
+
+The simplest Rust approach uses `enum` variants to represent states and pattern
+matching to handle transitions. Here's an order-processing
+example:[^4](https://oneuptime.com/blog/post/2026-02-01-rust-state-machines/view)
+
+```rust
+#[derive(Debug, Clone)]
+enum OrderState {
+ Created { items: Vec<String>, total: f64 },
+ Paid { items: Vec<String>, total: f64, payment_id: String },
+ Shipped { items: Vec<String>, tracking_number: String },
+ Delivered { delivered_at: String },
+ Cancelled { reason: String },
+}
+
+#[derive(Debug)]
+struct Order {
+ id: String,
+ state: OrderState,
+}
+
+impl Order {
+ fn new(id: String, items: Vec<String>, total: f64) -> Self {
+ Order { id, state: OrderState::Created { items, total } }
+ }
+
+ // Transition only valid from Created state
+ fn pay(self, payment_id: String) -> Result<Self, &'static str> {
+ match self.state {
+ OrderState::Created { items, total } => Ok(Order {
+ id: self.id,
+ state: OrderState::Paid { items, total, payment_id },
+ }),
+ _ => Err("Can only pay for orders in Created state"),
+ }
+ }
+
+ fn ship(self, tracking_number: String) -> Result<Self, &'static str> {
+ match self.state {
+ OrderState::Paid { items, .. } => Ok(Order {
+ id: self.id,
+ state: OrderState::Shipped { items, tracking_number },
+ }),
+ _ => Err("Can only ship paid orders"),
+ }
+ }
+}
+
+fn main() {
+ let order = Order::new("ORD-001".to_string(), vec!["Widget".to_string()], 99.99);
+ let order = order.pay("PAY-123".to_string()).unwrap();
+ let order = order.ship("TRACK-456".to_string()).unwrap();
+ println!("{:?}", order);
+}
+```
+
+This catches invalid transitions **at runtime** via `Result`. It's simple and
+easy to understand, but bugs only surface when that code path is
+hit.[^4](https://oneuptime.com/blog/post/2026-02-01-rust-state-machines/view)
+
+---
+
+## Approach 2: Typestate Pattern (Compile-Time Safety)
+
+The **typestate pattern** encodes state into the _type itself_ using generics,
+so the Rust compiler rejects invalid transitions before your code ever
+runs.[^4](https://oneuptime.com/blog/post/2026-02-01-rust-state-machines/view)
+
+```rust
+use std::marker::PhantomData;
+
+// Zero-sized marker types — no runtime cost
+struct Created;
+struct Paid { payment_id: String }
+struct Shipped { tracking_number: String }
+
+struct Order<St> {
+ id: String,
+ items: Vec<String>,
+ total: f64,
+ state_data: St,
+}
+
+// Methods only available on Order<Created>
+impl Order<Created> {
+ fn new(id: String, items: Vec<String>, total: f64) -> Self {
+ Order { id, items, total, state_data: Created }
+ }
+
+ fn pay(self, payment_id: String) -> Order<Paid> {
+ Order { id: self.id, items: self.items, total: self.total,
+ state_data: Paid { payment_id } }
+ }
+}
+
+// Methods only available on Order<Paid>
+impl Order<Paid> {
+ fn ship(self, tracking_number: String) -> Order<Shipped> {
+ Order { id: self.id, items: self.items, total: self.total,
+ state_data: Shipped { tracking_number } }
+ }
+}
+
+fn main() {
+ let order = Order::new("ORD-001".to_string(), vec!["Widget".to_string()], 49.99);
+ let paid = order.pay("PAY-123".to_string()); // ✅ Created → Paid
+ let shipped = paid.ship("TRACK-456".to_string()); // ✅ Paid → Shipped
+
+ // ❌ This would FAIL TO COMPILE — ship() doesn't exist on Order<Created>
+ // let bad = order.ship("TRACK-789".to_string());
+}
+```
+
+Because `ship()` is only implemented on `Order<Paid>`, calling it on an unpaid
+order is a **compile-time error**, not a runtime
+bug.[^4](https://oneuptime.com/blog/post/2026-02-01-rust-state-machines/view)
+
+---
+
+## Which Approach to Choose?
+
+| | Enum-Based | Typestate Pattern |
+| :-------------------------------- | :------------------------ | :--------------------------------- |
+| **Error detection** | Runtime | Compile time |
+| **Complexity** | Low | Medium–High |
+| **Dynamic state** (e.g., from DB) | ✅ Easy | ❌ Needs wrapper enum |
+| **IDE autocomplete** | Shows all methods | Shows only valid methods |
+| **Best for** | Prototyping, simple flows | Financial, safety-critical systems |
+
+In practice, many production systems use **both**: typestate for core business
+logic, with an enum wrapper (`AnyOrder`) for serialization and dynamic event
+handling.[^4](https://oneuptime.com/blog/post/2026-02-01-rust-state-machines/view)
+
+[^5]: https://www.freecodecamp.org/news/state-machines-basics-of-computer-science-d42855debc66/
+
+[^6]: https://www.youtube.com/watch?v=sosnUnI-vco
+
+[^7]: https://www.reddit.com/r/learnprogramming/comments/1g5yxci/state_machines_for_a_beginner/
+
+[^8]: https://github.com/Lifestreams-ai/statemachine
+
+[^9]: https://uk.mathworks.com/discovery/state-machine.html
+
+[^10]: https://docs.rs/simple_statemachine
+
+[^11]: https://www.reddit.com/r/rust/comments/18fugz0/state_machines_implementation/
+
+[^12]: https://www.stateworks.com/technology/understanding-state-machines/
+
+[^13]: https://en.wikipedia.org/wiki/Finite-state_machine
+
+[^14]: https://users.rust-lang.org/t/on-state-machines/114910
+
+[^15]: https://sparxsystems.com/resources/tutorials/uml2/state-diagram.html
diff --git a/system_design/design_patterns/iife.md b/system_design/design_patterns/iife.md
@@ -0,0 +1,76 @@
+# Immediately Invoked Function Expression (IIFE) in TypeScript
+
+An **IIFE** (immediately invoked function expression) is a function that is
+defined and executed in the same statement. The pattern has been part of
+JavaScript since the language’s early use in the browser, where it was a
+practical way to create a **local scope** and **avoid polluting the global
+object** when true modules were not available.
+
+## Basic shape
+
+The function is wrapped in parentheses so the parser treats it as an
+expression, then it is called with `()`:
+
+```ts
+(function () {
+ const secret = 42
+ // `secret` is not visible outside this function
+})()
+```
+
+With an arrow function, a similar effect is common in modern code:
+
+```ts
+;(() => {
+ // one-off setup or isolated block
+})()
+```
+
+A leading semicolon (or a line break before `(`) avoids the rare case where
+the previous line’s expression is parsed as a function call.
+
+## Why it still matters in TypeScript
+
+1. **Encapsulation** — `const` and `let` inside the IIFE are not hoisted to
+ an outer scope, so you can run initialization logic without exporting
+ implementation details. With ES modules, file scope already provides
+ privacy for top-level declarations, so IIFEs are less about “hiding from
+ globals” and more about **grouping** one-time work or **avoiding name
+ clashes** in a single file with many top-level symbols.
+
+2. **Async IIFE** — `async` functions return a `Promise`. Invoking one
+ immediately is a standard way to use `await` at the top level inside a
+ context that does not allow top-level `await` (older bundlers or scripts),
+ or to fire-and-forget async setup while keeping the rest of the module
+ synchronous:
+
+ ```ts
+ ;(async () => {
+ await someSetup()
+ })()
+ ```
+
+ In modern ES modules with **top-level await** enabled, you can often
+ replace this with plain `await` at module scope.
+
+3. **Return values** — An IIFE can produce a value that you assign or export:
+
+ ```ts
+ export const config = (() => {
+ const raw = process.env["FEATURE"]
+ return raw === "on"
+ })()
+ ```
+
+## Trade-offs
+
+IIFEs add nesting and can make stack traces slightly noisier. Prefer **small
+named functions** or **plain module-level code** when there is no need for an
+extra scope. Use an IIFE when you want a **single expression** that computes a
+value with private locals, or when you need **async execution** without making
+the whole module async.
+
+## References
+
+- [MDN: IIFE](https://developer.mozilla.org/en-US/docs/Glossary/IIFE)
+- [TypeScript Handbook — Modules](https://www.typescriptlang.org/docs/handbook/modules.html)
diff --git a/system_design/design_patterns/index.md b/system_design/design_patterns/index.md
@@ -0,0 +1,2 @@
+# Design patterns
+
diff --git a/system_design/design_patterns/interface.md b/system_design/design_patterns/interface.md
@@ -0,0 +1,69 @@
+# Interface
+
+In Rust, the concept of "programming to an interface/supertype" is achieved
+using traits — Rust's equivalent of interfaces or abstract supertypes.
+
+1. Define the "Supertype" (Trait)
+
+```rust
+trait Animal {
+ fn make_sound(&self);
+}
+```
+
+2. Concrete Implementations
+
+```rust
+struct Dog;
+struct Cat;
+
+impl Animal for Dog {
+ fn make_sound(&self) {
+ println!("Woof!");
+ }
+}
+
+impl Animal for Cat {
+ fn make_sound(&self) {
+ println!("Meow!");
+ }
+}
+```
+
+## Static Dispatch (resolved at compile time — faster)
+
+```rust
+fn interact(animal: &impl Animal) {
+ animal.make_sound();
+}
+
+let animal = Dog;
+interact(&animal); // Works for any Animal implementor
+```
+
+## Dynamic dispatch (resolved at runtime — more flexible)
+
+```rust
+fn interact(animal: &dyn Animal) {
+ animal.make_sound();
+}
+
+let animal: &dyn Animal = &Dog; // Variable typed as the trait, not Dog
+animal.make_sound();
+```
+
+## Assigning at Runtime
+
+```rust
+fn get_animal(sound_type: &str) -> Box<dyn Animal> {
+ match sound_type {
+ "dog" => Box::new(Dog),
+ _ => Box::new(Cat),
+ }
+}
+
+fn main() {
+ let animal = get_animal("dog"); // We don't know the concrete type here
+ animal.make_sound(); // All we care about is make_sound()
+}
+```
diff --git a/system_design/design_patterns/mealy_and_moore_machine.md b/system_design/design_patterns/mealy_and_moore_machine.md
@@ -0,0 +1,202 @@
+# Mealy and Moore Machine
+
+# Compare Mealy and Moore state machines with Rust code
+
+The key difference between Mealy and Moore
+[state machines](/system_design/design_patterns/finite_state_machine.md) comes down to **where
+the output lives**: in Moore machines, output is tied to the _state_; in Mealy
+machines, output is tied to the _transition_ (state + input together).[^1]
+
+---
+
+## Moore Machine
+
+In a **Moore machine**, each state has a fixed output associated with it,
+regardless of how you arrived there or what the current input is. The output
+only changes when you _enter a new state_ — making it synchronous and immune to
+input glitches.[^2][^3]
+
+Here's a traffic light modelled as a Moore machine — the output (light colour)
+depends purely on the current state:
+
+```rust
+#[derive(Debug, Clone, PartialEq)]
+enum TrafficState {
+ Red,
+ Green,
+ Yellow,
+}
+
+#[derive(Debug)]
+enum Input {
+ Timer,
+}
+
+struct MooreMachine {
+ state: TrafficState,
+}
+
+impl MooreMachine {
+ fn new() -> Self {
+ MooreMachine { state: TrafficState::Red }
+ }
+
+ // Output is derived from STATE alone — no input needed
+ fn output(&self) -> &str {
+ match self.state {
+ TrafficState::Red => "STOP",
+ TrafficState::Green => "GO",
+ TrafficState::Yellow => "SLOW DOWN",
+ }
+ }
+
+ // Transition: input triggers state change, output follows state
+ fn transition(&mut self, input: Input) {
+ self.state = match (&self.state, input) {
+ (TrafficState::Red, Input::Timer) => TrafficState::Green,
+ (TrafficState::Green, Input::Timer) => TrafficState::Yellow,
+ (TrafficState::Yellow, Input::Timer) => TrafficState::Red,
+ };
+ // Output is read AFTER the state changes
+ println!("State: {:?} → Output: {}", self.state, self.output());
+ }
+}
+
+fn main() {
+ let mut fsm = MooreMachine::new();
+ println!("Initial output: {}", fsm.output()); // "STOP"
+ fsm.transition(Input::Timer); // Green → "GO"
+ fsm.transition(Input::Timer); // Yellow → "SLOW DOWN"
+ fsm.transition(Input::Timer); // Red → "STOP"
+}
+```
+
+Output is produced **after** entering the new state — `output()` takes `&self`
+with no input parameter.[^3]
+
+---
+
+## Mealy Machine
+
+In a **Mealy machine**, output is produced **on the transition** — it depends on
+both the current state _and_ the input that triggered the move. This means
+outputs can react instantly to inputs, and you typically need fewer states than
+an equivalent Moore machine.[^4][^1]
+
+Here's a coin-operated vending machine where the output depends on both state
+and the coin inserted:
+
+```rust
+#[derive(Debug, Clone, PartialEq)]
+enum VendingState {
+ Idle,
+ Has10p,
+ Has20p,
+}
+
+#[derive(Debug)]
+enum Coin {
+ P10,
+ P20,
+}
+
+// Output is produced ON the transition, not from the state alone
+#[derive(Debug)]
+enum Output {
+ None,
+ Dispense(&'static str),
+ ReturnChange(u32),
+}
+
+struct MealyMachine {
+ state: VendingState,
+}
+
+impl MealyMachine {
+ fn new() -> Self {
+ MealyMachine { state: VendingState::Idle }
+ }
+
+ // Returns (new_state, output) — output depends on BOTH state AND input
+ fn transition(&mut self, coin: Coin) -> Output {
+ let (next_state, output) = match (&self.state, coin) {
+ (VendingState::Idle, Coin::P10) => (VendingState::Has10p, Output::None),
+ (VendingState::Idle, Coin::P20) => (VendingState::Has20p, Output::None),
+ (VendingState::Has10p, Coin::P10) => (VendingState::Has20p, Output::None),
+ (VendingState::Has10p, Coin::P20) => (VendingState::Idle, Output::Dispense("Chewing gum (30p)")),
+ (VendingState::Has20p, Coin::P10) => (VendingState::Idle, Output::Dispense("Chocolate (30p)")),
+ (VendingState::Has20p, Coin::P20) => (VendingState::Idle, Output::ReturnChange(10)),
+ };
+ self.state = next_state;
+ output
+ }
+}
+
+fn main() {
+ let mut fsm = MealyMachine::new();
+ println!("{:?}", fsm.transition(Coin::P10)); // None
+ println!("{:?}", fsm.transition(Coin::P20)); // Dispense("Chewing gum (30p)")
+ println!("{:?}", fsm.transition(Coin::P20)); // None
+ println!("{:?}", fsm.transition(Coin::P10)); // Dispense("Chocolate (30p)")
+}
+```
+
+Notice `transition()` takes **both** `&self` and `coin` to compute the output —
+a dead giveaway of the Mealy model.[^5]
+
+---
+
+## Side-by-Side Comparison
+
+| Feature | Moore | Mealy |
+| :--------------------------- | :--------------------------------------- | :------------------------------------------ |
+| **Output depends on** | Current state only | Current state + input |
+| **Output location** | Attached to state | Attached to transition |
+| **Rust signature** | `fn output(&self)` | `fn transition(&mut self, input) -> Output` |
+| **Number of states** | More (one per output combo) | Fewer [^4] |
+| **Output timing** | After state change (synchronous) | Immediately on input (asynchronous) [^2] |
+| **Input glitch sensitivity** | Immune | More sensitive [^2] |
+| **Best for** | Display/status outputs, digital circuits | Reactive systems, protocol parsers |
+
+---
+
+## Key Takeaway
+
+Both models are equally expressive — any Mealy machine can be converted to a
+Moore machine by splitting states, and vice versa. In Rust, the distinction maps
+cleanly: Moore output lives in a method that reads `&self` alone, while Mealy
+output is returned from the transition method that takes both `&self` and the
+input event.[^6][^5]
+<span style="display:none">[^10][^11][^12][^13][^14][^15][^7][^8][^9]</span>
+
+<div align="center">⁂</div>
+
+[^1]: https://www.geeksforgeeks.org/theory-of-computation/difference-between-mealy-machine-and-moore-machine/
+
+[^2]: https://www.youtube.com/watch?v=YiQxeuB56i0
+
+[^3]: https://stackoverflow.com/questions/4009283/mealy-v-s-moore
+
+[^4]: https://mil.ufl.edu/3701/classes/joel/16 Lecture.pdf
+
+[^5]: https://comp.lang.forth.narkive.com/zJDmPu3N/mealy-vs-moore-fsm
+
+[^6]: https://www.reddit.com/r/explainlikeimfive/comments/30uq6e/eli5_the_difference_between_a_mealey_machine_and/
+
+[^7]: https://www.youtube.com/watch?v=kb-Ww8HaHuE
+
+[^8]: https://forum.allaboutcircuits.com/threads/when-to-use-mealy-machine-and-when-to-use-moore-machine.190031/
+
+[^9]: https://github.com/rust-cy/generic-state-machine-rs
+
+[^10]: https://docs.rs/rust-fsm/
+
+[^11]: https://users.rust-lang.org/t/how-to-create-complex-state-machines/82714
+
+[^12]: https://www.reddit.com/r/FPGA/comments/vmlb5z/is_there_any_difference_in_my_implementation_of/
+
+[^13]: https://oneuptime.com/blog/post/2026-02-01-rust-state-machines/view
+
+[^14]: https://lib.rs/crates/edfsm
+
+[^15]: https://blog.devgenius.io/building-robust-distributed-state-machines-in-rust-a-comprehensive-guide-ad1a358134df
diff --git a/system_design/design_patterns/revealing_module_pattern.md b/system_design/design_patterns/revealing_module_pattern.md
@@ -0,0 +1,78 @@
+# Revealing Module Pattern in JavaScript
+
+The **revealing module pattern** wraps private state and behaviour inside a
+function (often an **IIFE**), then **returns a plain object** whose properties
+are the public API. Private helpers stay as inner functions or variables; only
+what you attach to that returned object is visible to callers.
+
+It is one variant of the classic **module pattern** from pre-ES2015 JavaScript,
+when files did not have native `import` / `export`.
+
+## Shape
+
+```js
+const counter = (function () {
+ let count = 0 // private
+
+ function increment() {
+ count += 1
+ }
+
+ function getCount() {
+ return count
+ }
+
+ return {
+ increment,
+ getCount,
+ }
+})()
+
+counter.increment()
+console.log(counter.getCount()) // 1
+```
+
+The **“revealing”** part is naming: you define real functions (`increment`,
+`getCount`) inside the closure, then **expose** them as properties on the
+returned object. Callers never see `count`; they only use the methods you
+listed.
+
+## Why use it
+
+- **Encapsulation** — Private data is not a property on the returned object, so
+ it cannot be read or overwritten from outside without going through your API.
+- **Stable surface** — The returned object is a small, explicit list of
+ capabilities (sometimes called a **facade**).
+- **Refactoring room** — You can change how `increment` works internally without
+ changing property names on the public object.
+
+## Compared to ES modules
+
+Today, a file can achieve similar separation with **top-level private symbols**
+and explicit `export`:
+
+```js
+let count = 0
+export function increment() {
+ count += 1
+}
+export function getCount() {
+ return count
+}
+```
+
+The revealing module pattern remains useful when you need a **factory** (many
+independent instances), when bundling legacy scripts without a module loader, or
+when teaching how closures and object literals compose into a module-like API.
+
+## Trade-offs
+
+- **Testing** — Private functions are hard to unit-test in isolation; you test
+ through the public API or accept that internals are covered indirectly.
+- **Memory** — Each IIFE invocation creates new function instances; for many
+ instances, consider whether a `class` or shared prototype fits better.
+
+## References
+
+- [Addy Osmani — Essential JavaScript Design Patterns](https://addyosmani.com/resources/essentialjsdesignpatterns/book/#revealingmodulepatternjavascript)
+- [MDN — Closures](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Closures)
diff --git a/system_design/edge_architecture.md b/system_design/edge_architecture.md
@@ -0,0 +1,97 @@
+# Edge Architecture
+
+Edge Architecture utilizes a tiered architecture to move data as close to the
+end user as possible while protecting the origin from traffic spikes.
+
+```
+ +-------------------+
+ | Developer |
+ | (npm/docker pull) |
+ +---------+---------+
+ |
+ v
+ +-------------------+
+ | Anycast / Geo-DNS |
+ +---------+---------+
+ | (Routes to nearest PoP)
+ v
++-----------------------------------------------------------+
+| EDGE PoP |
+| |
+| +-------------+ +---------------+ +--------------+ |
+| | Edge Compute|--->| L1 Cache | | WAF / DDoS | |
+| | (Auth/Route)| | (Memory/NVMe) | | Protection | |
+| +-------------+ +-------+-------+ +--------------+ |
++-----------------------------|-----------------------------+
+ | (Cache Miss)
+ v
++-----------------------------------------------------------+
+| REGIONAL SHIELD CACHE |
+| +-----------------------------------------------------+ |
+| | L2 Cache (High Capacity SSD, Request Collapsing) | |
+| +--------------------------+--------------------------+ |
++-----------------------------|-----------------------------+
+ | (Cache Miss)
+ v
++-----------------------------------------------------------+
+| ORIGIN INFRASTRUCTURE |
+| +--------------------+ +------------------------+ |
+| | Blob Storage | | Global Metadata DB | |
+| | (S3 / GCS) | | (Spanner / DynamoDB) | |
+| +--------------------+ +------------------------+ |
++-----------------------------------------------------------+
+```
+
+To achieve this scale, the technology stack must be highly concurrent and
+lightweight:
+
+- **Edge Routing & Proxy**: NGINX, Envoy, or Rust-based proxies to handle
+ millions of concurrent TCP connections and perform TLS termination.
+
+-- **Edge Compute**: WebAssembly (Wasm) or V8 Isolates running directly on the
+CDN edge to execute custom logic like authentication, A/B testing, and request
+filtering without routing back to the origin.
+
+-- **Caching Layer**: Varnish or custom memory-mapped file systems for L1 edge
+caching, backed by high-capacity NVMe drives for L2 regional shields.
+
+-- **Data & Origin**: Geographically replicated object storage (like AWS S3) for
+immutable package blobs, and a globally distributed database (like Google Cloud
+Spanner) for mutable package metadata and user entitlements.
+
+## System Data Flows
+
+When a user pulls a package, the request follows a strict path to ensure
+authorization and speed:
+
+-- **Resolution**: The client's DNS query hits a Geo-DNS provider, returning the
+Anycast IP of the nearest Edge PoP (Point of Presence).
+
+-- **Edge Auth**: The request reaches the Edge Proxy. An Edge Function executes
+immediately, verifying the user's API token against a highly cached subset of
+the metadata database.
+
+-- **Cache Lookup**: The proxy checks the L1 Cache. If the package is found, it
+is returned instantly.
+
+-- **Shield Fallback**: On an L1 miss, the request goes to the Regional Shield.
+If the package is present in the L2 cache, it is returned and populated in L1.
+
+-- **Origin Fetch**: On an L2 miss, the shield fetches the blob from Origin
+Storage, caches it, and streams it back down the chain to the client.
+
+## Performance Impact Chart
+
+This tiered networking approach reduces latency across the distribution
+lifecycle.
+
+```
+ Average Response Latency (ms) by Retrieval Tier
+------------------------------------------------------------
+Origin Fetch |################################ (250ms)
+Regional Shield L2 |########### (85ms)
+Edge PoP L1 |### (20ms)
+Predictive Cache |# (5ms)
+------------------------------------------------------------
+`
+```
diff --git a/system_design/index.md b/system_design/index.md
@@ -0,0 +1,7 @@
+# System Design
+
+- [PostgreSQL transactions and ACID](postgresql_transactions_acid.md)
+- [Design patterns](design_patterns/)
+- [Single server](single_server.md)
+- [Server scaling](server_scaling.md)
+- [Server authoritative design](server_authoritative_design.md)
diff --git a/system_design/postgresql_transactions_acid.md b/system_design/postgresql_transactions_acid.md
@@ -0,0 +1,489 @@
+# PostgreSQL Transactions, Stored Routines, and ACID
+
+PostgreSQL is built around **transactions**: bounded units of work that either
+commit as a whole or leave no durable effect (after rollback). Understanding
+how those transactions interact with **multi-version concurrency control
+(MVCC)**, **locking**, and the **write-ahead log (WAL)** explains both everyday
+behaviour (why two writers sometimes wait or fail) and what **ACID** means in
+practice.
+
+---
+
+## Transactions in PostgreSQL
+
+A transaction begins explicitly with `BEGIN` (or `START TRANSACTION`) and ends
+with `COMMIT` or `ROLLBACK`. If you do neither, a single SQL statement still runs
+inside an **implicit** transaction: PostgreSQL wraps each standalone statement
+in its own mini-transaction that commits on success.
+
+Inside one transaction, all statements see a coherent database state according to
+the chosen **isolation level** (see below). Until `COMMIT`, other sessions may or
+may not see your changes, depending on isolation and whether they started before
+or after your writes.
+
+**Savepoints** (`SAVEPOINT`, `ROLLBACK TO SAVEPOINT`) let you partially undo
+work without aborting the whole outer transaction—useful for complex workflows
+with selective retry logic.
+
+---
+
+## “Stored procedures” vs functions in PostgreSQL
+
+PostgreSQL’s primary server-side programming mechanism is the **`FUNCTION`**
+(PL/pgSQL, SQL, or other languages). A function **always runs in the caller’s
+transaction**; it cannot `COMMIT` or `ROLLBACK` the surrounding transaction from
+inside the function body.
+
+Since **PostgreSQL 11**, **`PROCEDURE`** exists for routines that **may** issue
+transaction control: a procedure can end its own transaction with `COMMIT` or
+`ROLLBACK` and continue (multi-transaction batching inside one call). That is
+the feature people often mean by “stored procedure” when comparing to other
+databases. For most application code, plain SQL inside a client-managed
+transaction plus optional **functions** is still the common pattern.
+
+Whether logic lives in the app or in the server, the same **MVCC and locking
+rules** apply: routines do not bypass concurrency controls.
+
+---
+
+## How PostgreSQL handles many writers (conflicts)
+
+PostgreSQL uses **MVCC** instead of readers blocking writers with a single
+global lock on rows. Each row version carries metadata about which transactions
+created and invalidated it (`xmin` / `xmax` system columns). A query reads **row
+versions visible to its snapshot**—typically versions committed before the
+snapshot’s cutoff—so normal **SELECT** does not take row-level locks that block
+concurrent **UPDATE** on other rows.
+
+When **two transactions try to change the same row**, MVCC does not allow both
+to succeed blindly:
+
+1. **Row-level locks** — `UPDATE`, `DELETE`, and `SELECT … FOR UPDATE` acquire
+ locks on the target rows. A second transaction that needs the same row may
+ **block** until the first commits or rolls back, then either proceeds or
+ detects the row was changed (depending on operation and isolation).
+
+2. **Serialization failures** — Under **`REPEATABLE READ`** or **`SERIALIZABLE`**,
+ PostgreSQL can abort a transaction with **`SQLSTATE 40001`**
+ (`serialization_failure`) when the guarantee of that isolation level would be
+ violated—e.g. a concurrent write pattern that could produce anomalies if both
+ committed. The application is expected to **retry** the whole transaction.
+
+3. **Deadlocks** — When transactions wait on each other in a cycle, PostgreSQL
+ **detects** the deadlock and **aborts one transaction** (`deadlock_detected`)
+ so the others can proceed.
+
+So “multiple users writing the same data at the same time” is handled by a mix
+of **waiting**, **abort with retry**, and **ordering**—not by silently losing
+updates. Which outcome you get depends on SQL patterns (locking reads vs blind
+updates) and isolation level.
+
+The subsections below quote **actual C code** from the upstream PostgreSQL tree
+so you can see how these behaviours are implemented. Excerpts are taken from
+commit [`43fc1dc7527`](https://github.com/postgres/postgres/tree/43fc1dc7527c4213c4d48a96bf98738df0acb71a)
+(PostgreSQL is licensed under [the PostgreSQL Licence](https://www.postgresql.org/about/licence/)).
+
+### Row-level blocking: `heap_lock_tuple`
+
+When a tuple is already being modified or deleted, `heap_lock_tuple` inspects
+visibility with **`HeapTupleSatisfiesUpdate`**, unlocks the buffer briefly, and
+may **sleep** until competing lockers finish—first acquiring a **heavyweight
+tuple lock** (`LockTuple` via `heap_acquire_tuplock`), then waiting on the
+transaction or MultiXact holding `xmax`:
+
+```c
+l3:
+ result = HeapTupleSatisfiesUpdate(tuple, cid, *buffer);
+
+ if (result == TM_Invisible)
+ {
+ /*
+ * This is possible, but only when locking a tuple for ON CONFLICT DO
+ * SELECT/UPDATE. We return this value here rather than throwing an
+ * error in order to give that case the opportunity to throw a more
+ * specific error.
+ */
+ result = TM_Invisible;
+ goto out_locked;
+ }
+ else if (result == TM_BeingModified ||
+ result == TM_Updated ||
+ result == TM_Deleted)
+ {
+ TransactionId xwait;
+ uint16 infomask;
+ uint16 infomask2;
+ bool require_sleep;
+ ItemPointerData t_ctid;
+
+ /* must copy state data before unlocking buffer */
+ xwait = HeapTupleHeaderGetRawXmax(tuple->t_data);
+ infomask = tuple->t_data->t_infomask;
+ infomask2 = tuple->t_data->t_infomask2;
+ ItemPointerCopy(&tuple->t_data->t_ctid, &t_ctid);
+
+ LockBuffer(*buffer, BUFFER_LOCK_UNLOCK);
+```
+
+(The branch continues with multixact checks, tuple-lock acquisition, and sleep
+logic; only the opening is shown here.)
+
+The fragment above is the start of the branch taken when the tuple is already
+being modified, updated, or deleted; the remainder of `heap_lock_tuple` decides
+whether sleeping is required and how to wait.
+
+Later, when `require_sleep` is true, the same function acquires the tuple lock
+and waits on the rival transaction id:
+
+```c
+ if (!skip_tuple_lock &&
+ !heap_acquire_tuplock(relation, tid, mode, wait_policy,
+ &have_tuple_lock))
+ {
+ /*
+ * This can only happen if wait_policy is Skip and the lock
+ * couldn't be obtained.
+ */
+ result = TM_WouldBlock;
+ /* recovery code expects to have buffer lock held */
+ LockBuffer(*buffer, BUFFER_LOCK_EXCLUSIVE);
+ goto failed;
+ }
+```
+
+```c
+ switch (wait_policy)
+ {
+ case LockWaitBlock:
+ XactLockTableWait(xwait, relation, &tuple->t_self,
+ XLTW_Lock);
+ break;
+ case LockWaitSkip:
+ if (!ConditionalXactLockTableWait(xwait, false))
+ {
+ result = TM_WouldBlock;
+ /* recovery code expects to have buffer lock held */
+ LockBuffer(*buffer, BUFFER_LOCK_EXCLUSIVE);
+ goto failed;
+ }
+ break;
+ case LockWaitError:
+ if (!ConditionalXactLockTableWait(xwait, log_lock_failures))
+ ereport(ERROR,
+ (errcode(ERRCODE_LOCK_NOT_AVAILABLE),
+ errmsg("could not obtain lock on row in relation \"%s\"",
+ RelationGetRelationName(relation))));
+ break;
+ }
+```
+
+(Source: [`src/backend/access/heap/heapam.c`](https://github.com/postgres/postgres/blob/43fc1dc7527c4213c4d48a96bf98738df0acb71a/src/backend/access/heap/heapam.c),
+function `heap_lock_tuple`; this `switch` sits in the branch that waits on a
+single transaction id after copying `xwait` from `xmax`.)
+
+The **heavyweight** tuple lock is wired to `LockTuple` through a small macro
+layer that maps `LockTupleMode` to the lock manager’s `LOCKMODE`:
+
+```c
+#define LockTupleTuplock(rel, tup, mode) \
+ LockTuple((rel), (tup), tupleLockExtraInfo[mode].hwlock)
+#define UnlockTupleTuplock(rel, tup, mode) \
+ UnlockTuple((rel), (tup), tupleLockExtraInfo[mode].hwlock)
+#define ConditionalLockTupleTuplock(rel, tup, mode, log) \
+ ConditionalLockTuple((rel), (tup), tupleLockExtraInfo[mode].hwlock, (log))
+```
+
+`heap_acquire_tuplock` is the helper that invokes it when blocking is allowed:
+
+```c
+static bool
+heap_acquire_tuplock(Relation relation, const ItemPointerData *tid, LockTupleMode mode,
+ LockWaitPolicy wait_policy, bool *have_tuple_lock)
+{
+ if (*have_tuple_lock)
+ return true;
+
+ switch (wait_policy)
+ {
+ case LockWaitBlock:
+ LockTupleTuplock(relation, tid, mode);
+ break;
+
+ case LockWaitSkip:
+ if (!ConditionalLockTupleTuplock(relation, tid, mode, false))
+ return false;
+ break;
+
+ case LockWaitError:
+ if (!ConditionalLockTupleTuplock(relation, tid, mode, log_lock_failures))
+ ereport(ERROR,
+ (errcode(ERRCODE_LOCK_NOT_AVAILABLE),
+ errmsg("could not obtain lock on row in relation \"%s\"",
+ RelationGetRelationName(relation))));
+ break;
+ }
+ *have_tuple_lock = true;
+
+ return true;
+}
+```
+
+(Source: same file, macros near the top of `heapam.c` and `heap_acquire_tuplock`
+below `heap_lock_tuple`.)
+
+### Deadlock detection: `CheckDeadLock` and `DeadLockCheck`
+
+While a backend sleeps on a lock, a **deadlock timeout** can fire; the waiter
+then locks all lock-hash partitions and runs **`DeadLockCheck`**. If the graph
+contains an unresolvable cycle (`DS_HARD_DEADLOCK`), the waiter is removed from
+the queue so **`ProcSleep`** surfaces an error and the transaction aborts. The
+full handler is **`CheckDeadLock`** in `proc.c`:
+
+```c
+/*
+ * CheckDeadLock
+ *
+ * We only get to this routine, if DEADLOCK_TIMEOUT fired while waiting for a
+ * lock to be released by some other process. Check if there's a deadlock; if
+ * not, just return. If we have a real deadlock, remove ourselves from the
+ * lock's wait queue.
+ */
+static DeadLockState
+CheckDeadLock(void)
+{
+ int i;
+ DeadLockState result;
+
+ /*
+ * Acquire exclusive lock on the entire shared lock data structures. Must
+ * grab LWLocks in partition-number order to avoid LWLock deadlock.
+ *
+ * Note that the deadlock check interrupt had better not be enabled
+ * anywhere that this process itself holds lock partition locks, else this
+ * will wait forever. Also note that LWLockAcquire creates a critical
+ * section, so that this routine cannot be interrupted by cancel/die
+ * interrupts.
+ */
+ for (i = 0; i < NUM_LOCK_PARTITIONS; i++)
+ LWLockAcquire(LockHashPartitionLockByIndex(i), LW_EXCLUSIVE);
+
+ /*
+ * Check to see if we've been awoken by anyone in the interim.
+ *
+ * If we have, we can return and resume our transaction -- happy day.
+ * Before we are awoken the process releasing the lock grants it to us so
+ * we know that we don't have to wait anymore.
+ *
+ * We check by looking to see if we've been unlinked from the wait queue.
+ * This is safe because we hold the lock partition lock.
+ */
+ if (dlist_node_is_detached(&MyProc->waitLink))
+ {
+ result = DS_NO_DEADLOCK;
+ goto check_done;
+ }
+
+#ifdef LOCK_DEBUG
+ if (Debug_deadlocks)
+ DumpAllLocks();
+#endif
+
+ /* Run the deadlock check */
+ result = DeadLockCheck(MyProc);
+
+ if (result == DS_HARD_DEADLOCK)
+ {
+ /*
+ * Oops. We have a deadlock.
+ *
+ * Get this process out of wait state. (Note: we could do this more
+ * efficiently by relying on lockAwaited, but use this coding to
+ * preserve the flexibility to kill some other transaction than the
+ * one detecting the deadlock.)
+ *
+ * RemoveFromWaitQueue sets MyProc->waitStatus to
+ * PROC_WAIT_STATUS_ERROR, so ProcSleep will report an error after we
+ * return.
+ */
+ Assert(MyProc->waitLock != NULL);
+ RemoveFromWaitQueue(MyProc, LockTagHashCode(&(MyProc->waitLock->tag)));
+
+ /*
+ * We're done here. Transaction abort caused by the error that
+ * ProcSleep will raise will cause any other locks we hold to be
+ * released, thus allowing other processes to wake up; we don't need
+ * to do that here. NOTE: an exception is that releasing locks we
+ * hold doesn't consider the possibility of waiters that were blocked
+ * behind us on the lock we just failed to get, and might now be
+ * wakable because we're not in front of them anymore. However,
+ * RemoveFromWaitQueue took care of waking up any such processes.
+ */
+ }
+
+ /*
+ * And release locks. We do this in reverse order for two reasons: (1)
+ * Anyone else who needs more than one of the locks will be trying to lock
+ * them in increasing order; we don't want to release the other process
+ * until it can get all the locks it needs. (2) This avoids O(N^2)
+ * behavior inside LWLockRelease.
+ */
+check_done:
+ for (i = NUM_LOCK_PARTITIONS; --i >= 0;)
+ LWLockRelease(LockHashPartitionLockByIndex(i));
+
+ return result;
+}
+```
+
+`DeadLockCheck` in `deadlock.c` implements the search for cycles and possible
+wait-queue reordering; its header comment states the `DS_HARD_DEADLOCK`
+contract, and the body begins by resetting workspace used by the detector:
+
+```c
+/*
+ * DeadLockCheck -- Checks for deadlocks for a given process
+ *
+ * This code looks for deadlocks involving the given process. If any
+ * are found, it tries to rearrange lock wait queues to resolve the
+ * deadlock. If resolution is impossible, return DS_HARD_DEADLOCK ---
+ * the caller is then expected to abort the given proc's transaction.
+ *
+ * Caller must already have locked all partitions of the lock tables.
+ *
+ * On failure, deadlock details are recorded in deadlockDetails[] for
+ * subsequent printing by DeadLockReport(). That activity is separate
+ * because we don't want to do it while holding all those LWLocks.
+ */
+DeadLockState
+DeadLockCheck(PGPROC *proc)
+{
+ /* Initialize to "no constraints" */
+ nCurConstraints = 0;
+ nPossibleConstraints = 0;
+ nWaitOrders = 0;
+
+ /* Initialize to not blocked by an autovacuum worker */
+ blocking_autovacuum_proc = NULL;
+
+ /* Search for deadlocks and possible fixes */
+ if (DeadLockCheckRecurse(proc))
+```
+
+(Sources: [`src/backend/storage/lmgr/proc.c`](https://github.com/postgres/postgres/blob/43fc1dc7527c4213c4d48a96bf98738df0acb71a/src/backend/storage/lmgr/proc.c)
+and [`src/backend/storage/lmgr/deadlock.c`](https://github.com/postgres/postgres/blob/43fc1dc7527c4213c4d48a96bf98738df0acb71a/src/backend/storage/lmgr/deadlock.c). The `EDGE` / waits-for graph structs at the top of `deadlock.c` are also worth reading alongside [`src/backend/storage/lmgr/README`](https://github.com/postgres/postgres/blob/43fc1dc7527c4213c4d48a96bf98738df0acb71a/src/backend/storage/lmgr/README).)
+
+### Serializable isolation: `PreCommit_CheckForSerializationFailure`
+
+Under **SSI**, commit-time validation walks **read/write conflict** lists; if a
+**dangerous structure** appears (pivot still uncommitted), PostgreSQL raises
+**`ERRCODE_T_R_SERIALIZATION_FAILURE`** (`SQLSTATE 40001`), matching the
+user-visible `serialization_failure`:
+
+```c
+void
+PreCommit_CheckForSerializationFailure(void)
+{
+ dlist_iter near_iter;
+
+ if (MySerializableXact == InvalidSerializableXact)
+ return;
+
+ Assert(IsolationIsSerializable());
+
+ LWLockAcquire(SerializableXactHashLock, LW_EXCLUSIVE);
+
+ /*
+ * Check if someone else has already decided that we need to die. Since
+ * we set our own DOOMED flag when partially releasing, ignore in that
+ * case.
+ */
+ if (SxactIsDoomed(MySerializableXact) &&
+ !SxactIsPartiallyReleased(MySerializableXact))
+ {
+ LWLockRelease(SerializableXactHashLock);
+ ereport(ERROR,
+ (errcode(ERRCODE_T_R_SERIALIZATION_FAILURE),
+ errmsg("could not serialize access due to read/write dependencies among transactions"),
+ errdetail_internal("Reason code: Canceled on identification as a pivot, during commit attempt."),
+ errhint("The transaction might succeed if retried.")));
+ }
+```
+
+(Source: [`src/backend/storage/lmgr/predicate.c`](https://github.com/postgres/postgres/blob/43fc1dc7527c4213c4d48a96bf98738df0acb71a/src/backend/storage/lmgr/predicate.c).)
+
+---
+
+## Isolation levels (what “consistent view” means)
+
+Default isolation is **`READ COMMITTED`**: each statement sees rows committed
+before that **statement** began. That prevents dirty reads but allows **non-repeatable
+reads** and **phantoms** across statements in the same transaction.
+
+**`REPEATABLE READ`** uses a **snapshot** taken at the **first query** in the
+transaction; subsequent reads see the same world. Concurrent commits that would
+change what that snapshot means for writes can trigger **`serialization_failure`**.
+
+**`SERIALIZABLE`** uses **Serializable Snapshot Isolation (SSI)**—tracking
+dependencies between concurrent transactions so that any outcome that could not
+have occurred if transactions ran one-after-another causes an abort. It is the
+strongest guarantee PostgreSQL offers but may force **retries** under contention.
+
+---
+
+## ACID “under the hood”
+
+The acronym maps to mechanisms roughly as follows:
+
+### Atomicity
+
+All changes in a transaction are recorded as **WAL records** before commit is
+allowed (see durability). If the session crashes mid-transaction, recovery
+replays only **committed** work; incomplete transactions are rolled back during
+recovery. Within the server, rollback uses transaction status and MVCC rules so
+uncommitted row versions are invisible after abort.
+
+### Consistency
+
+PostgreSQL enforces **declared rules**: primary keys, foreign keys, `CHECK`,
+`NOT NULL`, triggers, etc. **Consistency** in the broader business sense (e.g.
+“account totals match ledger”) remains the application’s responsibility;
+transactions bundle changes so those rules can be checked together before commit.
+
+### Isolation
+
+Implemented through **MVCC snapshots**, **row locks**, **predicate locks /
+SSI bookkeeping** (for serializable), and occasional **blocking**. Isolation is
+not “free”: stronger levels trade convenience for **abort/retry** under
+concurrency.
+
+### Durability
+
+Committed transactions are made **durable** via the **write-ahead log**: redo
+information is written to WAL and, depending on **`synchronous_commit`** and
+replication settings, flushed to disk (and replicas) before the client is told
+`COMMIT` succeeded. That way a crash after commit can **replay** the log and
+restore committed data.
+
+---
+
+## Practical takeaways
+
+- Use explicit transactions when several statements must succeed or fail
+ together.
+- Expect **`serialization_failure`** under **`SERIALIZABLE`** or **`REPEATABLE
+ READ`** when writes overlap; implement **retry** with backoff.
+- Use **`SELECT … FOR UPDATE`** (or equivalent) when you read a row and then
+ update it based on that read, to serialize competing writers on that row.
+- Distinguish **`FUNCTION`** (single outer transaction) from **`PROCEDURE`**
+ (optional internal commits) when porting “stored procedure” designs from other
+ DBMSs.
+
+---
+
+## References
+
+- [PostgreSQL server sources (GitHub)](https://github.com/postgres/postgres)
+- [PostgreSQL Manual — Transaction Isolation](https://www.postgresql.org/docs/current/transaction-iso.html)
+- [PostgreSQL Manual — Routine vacuuming / MVCC](https://www.postgresql.org/docs/current/routine-vacuuming.html)
+- [PostgreSQL Manual — Write-Ahead Logging (WAL)](https://www.postgresql.org/docs/current/wal.html)
+- [PostgreSQL Manual — `CREATE PROCEDURE`](https://www.postgresql.org/docs/current/sql-createprocedure.html)
diff --git a/system_design/server_authoritative_design.md b/system_design/server_authoritative_design.md
@@ -0,0 +1,1069 @@
+# Server-Authoritative Design
+
+# Server-Authoritative Design: The Backend-as-Truth Principle
+
+Server-authoritative design is an architectural philosophy where **the backend serves as the single source of truth**, treating client applications as inherently unreliable from a connection and execution standpoint. This principle asserts that while frontends may disconnect, crash, or behave unpredictably, the backend must maintain consistency, enforce business rules, and guarantee data integrity.
+
+```
+┌─────────────────────┐ ┌─────────────────────┐
+│ Client │ │ Backend │
+│ │ │ │
+│ - Ephemeral │ ◄── Request ──► │ - Authoritative │
+│ - Unreliable │ │ - Persistent │
+│ - Untrusted │ ◄── Response ──► │ - Trusted │
+│ - Stateful │ │ - Stateless/ │
+│ (when connected) │ Connection │ Transactional │
+└─────────────────────┘ may fail └─────────────────────┘
+ ▲
+ │
+ ┌─────────────────────┐
+ │ Single Source │
+ │ of Truth │
+ │ │
+ │ - Validation │
+ │ - Business Rules │
+ │ - Data Integrity │
+ │ - Transactions │
+ └─────────────────────┘
+```
+
+## Core Philosophy
+
+### Why Treat Frontends as Unreliable?
+
+From a **connection perspective**, clients are fundamentally unreliable:
+
+| Client Failure Mode | Implications for Design |
+|---------------------|-------------------------|
+| **Network Disconnection** | WiFi drops, mobile signal loss, VPN issues |
+| **Browser/Tab Closure** | User can close application at any moment |
+| **Device Power Loss** | Battery dies, system crash, forced restart |
+| **Background Throttling** | Mobile OS limits background process execution |
+| **Intentional Disruption** | User kills app via task manager |
+
+From a **security and trust perspective**, clients are inherently untrusted:
+- Code can be inspected, modified, or bypassed
+- Input validation can be circumvented
+- Authentication tokens can be stolen
+- Users may attempt to exploit business logic
+
+### The Server's Role as Authoritative Source
+
+The backend becomes the **arbiter of truth**:
+
+1. **Validation Gatekeeper**: Every mutation request must pass server-side validation
+2. **Business Logic Enforcer**: Rules applied consistently across all clients
+3. **Transactional Guarantor**: ACID properties maintained at database level
+4. **Consensus Point**: Single source for resolving conflicts or race conditions
+
+## Architectural Patterns
+
+### Thin Client vs. Fat Client Spectrum
+
+```
+ Thin Client Fat Client
+┌─────────────────────┐ ┌─────────────────────┐
+│ Presentation Only │ │ Business Logic │
+│ │ │ │
+│ + Minimal State │ │ + Rich State │
+│ + Server-Rendered │ │ + Optimistic UI │
+│ + Simple Updates │ │ + Advanced Caching │
+│ - High Latency │ │ - Complex Sync │
+│ - Poor Offline │ │ - Security Risks │
+└─────────┬───────────┘ └─────────┬───────────┘
+ │ │
+ └────────────────────────────────────┘
+ Hybrid Approach
+ (Server-Authoritative with
+ Intelligent Client Features)
+```
+
+### Hybrid Server-Authoritative Pattern
+
+Modern applications often follow a hybrid approach:
+
+```
+┌─────────────────────────────────────────────────────────────────┐
+│ Client Application │
+│ │
+│ ┌──────────────────┐ ┌──────────────────┐ │
+│ │ Local Cache │ │ Optimistic UI │ │
+│ │ (Ephemeral) │ │ (Immediate │ │
+│ │ │ │ Feedback) │ │
+│ └──────────────────┘ └──────────────────┘ │
+│ │ │ │
+│ └──────────────────────────┼────────────────────────────┘
+│ ▼ │
+│ ┌──────────────────┐ │
+│ │ Sync Engine │ │
+│ │ (Queues/Retry) │ │
+│ └─────────┬────────┘ │
+└─────────────────────────────────────┼────────────────────────────┘
+ │
+ Network Boundary
+ │
+┌─────────────────────────────────────┼────────────────────────────┐
+│ Backend Server │
+│ │
+│ ┌──────────────────┐ ┌──────────────────┐ ┌─────────────┐ │
+│ │ Validation │ │ Business Logic │ │ Data Store │ │
+│ │ (Authoritative) │────▶ (Authoritative) │────▶ (Source of │ │
+│ │ │ │ │ │ Truth) │ │
+│ └──────────────────┘ └──────────────────┘ └─────────────┘ │
+│ │
+│ ┌─────────────────────────────────────────────────────────────┐ │
+│ │ Conflict Resolution │ │
+│ │ - Last-Write-Wins (with versioning) │ │
+│ │ - Operational Transformation (for collaborative editing) │ │
+│ │ - Client ID precedence rules │ │
+│ └─────────────────────────────────────────────────────────────┘ │
+└───────────────────────────────────────────────────────────────────┘
+```
+
+## Implementation Examples
+
+### Rust Backend: Transactional Business Logic
+
+```rust
+// Domain model - The source of truth
+#[derive(Debug, Clone, Serialize, Deserialize)]
+pub struct BankAccount {
+ pub id: Uuid,
+ pub user_id: Uuid,
+ pub balance: Decimal,
+ pub version: i32, // For optimistic concurrency control
+ pub created_at: DateTime<Utc>,
+ pub updated_at: DateTime<Utc>,
+}
+
+// Authoritative business logic
+pub struct AccountService {
+ db_pool: PgPool,
+}
+
+impl AccountService {
+ /// Transfer funds between accounts - AUTHORITATIVE VERSION
+ /// This is the single source of truth for fund transfers
+ #[tracing::instrument(skip(self))]
+ pub async fn transfer_funds(
+ &self,
+ from_account_id: Uuid,
+ to_account_id: Uuid,
+ amount: Decimal,
+ request_id: Uuid, // Idempotency key
+ ) -> Result<Transaction, TransferError> {
+ // Check: amount must be positive
+ if amount <= Decimal::ZERO {
+ return Err(TransferError::InvalidAmount);
+ }
+
+ // Wrap in database transaction to maintain consistency
+ let mut tx = self.db_pool.begin().await?;
+
+ // Use SELECT FOR UPDATE to lock rows
+ let (from_account, to_account) = tokio::try_join!(
+ sqlx::query_as!(
+ BankAccount,
+ "SELECT * FROM bank_accounts WHERE id = $1 FOR UPDATE",
+ from_account_id
+ )
+ .fetch_optional(&mut *tx)
+ .map_err(TransferError::from),
+
+ sqlx::query_as!(
+ BankAccount,
+ "SELECT * FROM bank_accounts WHERE id = $1 FOR UPDATE",
+ to_account_id
+ )
+ .fetch_optional(&mut *tx)
+ .map_err(TransferError::from),
+ )?;
+
+ // Validate: accounts exist
+ let from_account = from_account.ok_or(TransferError::AccountNotFound(from_account_id))?;
+ let to_account = to_account.ok_or(TransferError::AccountNotFound(to_account_id))?;
+
+ // Validate: sufficient funds (business rule)
+ if from_account.balance < amount {
+ return Err(TransferError::InsufficientFunds {
+ available: from_account.balance,
+ requested: amount,
+ });
+ }
+
+ // Validate: not transferring to self
+ if from_account_id == to_account_id {
+ return Err(TransferError::SelfTransfer);
+ }
+
+ // Validate: amount limits (business rule)
+ const MAX_TRANSFER_AMOUNT: Decimal = Decimal::from_str("10000").unwrap();
+ if amount > MAX_TRANSFER_AMOUNT {
+ return Err(TransferError::ExceedsLimit(MAX_TRANSFER_AMOUNT));
+ }
+
+ // Perform the transfer atomically
+ let new_from_balance = from_account.balance - amount;
+ let new_to_balance = to_account.balance + amount;
+
+ // Update with optimistic concurrency control
+ let updated_from = sqlx::query!(
+ r#"
+ UPDATE bank_accounts
+ SET balance = $1, version = version + 1, updated_at = NOW()
+ WHERE id = $2 AND version = $3
+ RETURNING id, user_id, balance, version, created_at, updated_at
+ "#,
+ new_from_balance,
+ from_account_id,
+ from_account.version
+ )
+ .fetch_optional(&mut *tx)
+ .await?;
+
+ if updated_from.is_none() {
+ return Err(TransferError::ConcurrentModification);
+ }
+
+ let updated_to = sqlx::query!(
+ r#"
+ UPDATE bank_accounts
+ SET balance = $1, version = version + 1, updated_at = NOW()
+ WHERE id = $2 AND version = $3
+ RETURNING id, user_id, balance, version, created_at, updated_at
+ "#,
+ new_to_balance,
+ to_account_id,
+ to_account.version
+ )
+ .fetch_optional(&mut *tx)
+ .await?;
+
+ if updated_to.is_none() {
+ return Err(TransferError::ConcurrentModification);
+ }
+
+ // Record the transaction for audit trail
+ let transaction = sqlx::query_as!(
+ Transaction,
+ r#"
+ INSERT INTO transactions
+ (id, from_account_id, to_account_id, amount, status, request_id, created_at)
+ VALUES ($1, $2, $3, $4, $5, $6, NOW())
+ RETURNING *
+ "#,
+ Uuid::new_v4(),
+ from_account_id,
+ to_account_id,
+ amount,
+ "COMPLETED",
+ request_id
+ )
+ .fetch_one(&mut *tx)
+ .await?;
+
+ // Commit the entire transaction
+ tx.commit().await?;
+
+ Ok(transaction)
+ }
+}
+
+// Compare with naive client-side implementation (WHAT NOT TO DO)
+pub struct NaiveClientSideTransfer {
+ // This would be BAD: trusting client to validate and calculate
+ pub async fn transfer_funds_naive(
+ &self,
+ from_balance: Decimal, // Client-provided - UNTRUSTED!
+ to_balance: Decimal, // Client-provided - UNTRUSTED!
+ amount: Decimal,
+ ) -> Result<(), TransferError> {
+ // Client-side validation - CAN BE BYPASSED
+ if amount <= Decimal::ZERO {
+ return Err(TransferError::InvalidAmount);
+ }
+
+ // Client-side business logic - CAN BE MANIPULATED
+ if from_balance < amount {
+ return Err(TransferError::InsufficientFunds {
+ available: from_balance,
+ requested: amount,
+ });
+ }
+
+ // Client-side calculation - WRONG if balances changed
+ let new_from = from_balance - amount;
+ let new_to = to_balance + amount;
+
+ // Send to server - RACE CONDITION if other transfers happening
+ self.update_balance(from_account_id, new_from).await?;
+ self.update_balance(to_account_id, new_to).await?;
+
+ Ok(())
+ }
+}
+```
+
+### TypeScript Frontend: Optimistic UI with Server Reconciliation
+
+```typescript
+// Client-side representation (NOT authoritative)
+interface ClientBankAccount {
+ id: string;
+ userId: string;
+ balance: number;
+ pendingTransactions: PendingTransaction[];
+ version: number; // For optimistic updates
+}
+
+// Sync engine that respects server authority
+class AccountSyncEngine {
+ private localState: Map<string, ClientBankAccount> = new Map();
+ private pendingQueue: PendingOperation[] = [];
+ private isOnline: boolean = true;
+
+ // Optimistic update - immediate UI feedback
+ async transferFundsOptimistic(
+ fromAccountId: string,
+ toAccountId: string,
+ amount: number
+ ): Promise<void> {
+ // 1. Client-side validation (for UX, not security)
+ if (amount <= 0) {
+ throw new Error('Amount must be positive');
+ }
+
+ const fromAccount = this.localState.get(fromAccountId);
+ const toAccount = this.localState.get(toAccountId);
+
+ if (!fromAccount || !toAccount) {
+ throw new Error('Account not found');
+ }
+
+ // 2. Optimistic update - show changes immediately
+ const operationId = crypto.randomUUID();
+ const pendingTx: PendingTransaction = {
+ id: operationId,
+ fromAccountId,
+ toAccountId,
+ amount,
+ status: 'pending',
+ timestamp: Date.now(),
+ };
+
+ // Update local state optimistically
+ this.localState.set(fromAccountId, {
+ ...fromAccount,
+ balance: fromAccount.balance - amount,
+ pendingTransactions: [...fromAccount.pendingTransactions, pendingTx],
+ version: fromAccount.version + 1,
+ });
+
+ this.localState.set(toAccountId, {
+ ...toAccount,
+ balance: toAccount.balance + amount,
+ version: toAccount.version + 1,
+ });
+
+ // Notify UI of change
+ this.notifyStateChange();
+
+ // 3. Queue for server sync (authoritative)
+ const operation: PendingOperation = {
+ id: operationId,
+ type: 'transfer',
+ fromAccountId,
+ toAccountId,
+ amount,
+ retryCount: 0,
+ timestamp: Date.now(),
+ };
+
+ this.pendingQueue.push(operation);
+
+ // 4. Try to sync immediately
+ await this.flushQueue();
+ }
+
+ // Sync with server - respects server authority
+ private async flushQueue(): Promise<void> {
+ if (!this.isOnline || this.pendingQueue.length === 0) {
+ return;
+ }
+
+ // Process operations in order
+ for (const operation of this.pendingQueue.slice()) {
+ try {
+ // Send to authoritative backend
+ const response = await fetch('/api/transfers', {
+ method: 'POST',
+ headers: {
+ 'Content-Type': 'application/json',
+ 'Idempotency-Key': operation.id,
+ },
+ body: JSON.stringify({
+ fromAccountId: operation.fromAccountId,
+ toAccountId: operation.toAccountId,
+ amount: operation.amount,
+ requestId: operation.id,
+ }),
+ });
+
+ if (response.ok) {
+ // Server accepted - update local state to match server
+ const serverTransaction = await response.json();
+ await this.reconcileWithServer(serverTransaction);
+
+ // Remove from queue
+ const index = this.pendingQueue.indexOf(operation);
+ if (index > -1) {
+ this.pendingQueue.splice(index, 1);
+ }
+ } else if (response.status === 409) {
+ // Conflict - server rejected due to business rule
+ await this.handleConflict(operation, response);
+ } else {
+ // Other error - retry later
+ operation.retryCount++;
+ if (operation.retryCount > 3) {
+ // Give up and revert optimistic update
+ await this.revertOptimisticUpdate(operation);
+ }
+ }
+ } catch (error) {
+ // Network error - will retry when back online
+ console.warn('Network error, will retry:', error);
+ }
+ }
+ }
+
+ // Reconciliation: align client state with server truth
+ private async reconcileWithServer(serverTransaction: ServerTransaction): Promise<void> {
+ // Get current optimistic state
+ const fromAccount = this.localState.get(serverTransaction.fromAccountId);
+ const toAccount = this.localState.get(serverTransaction.toAccountId);
+
+ if (!fromAccount || !toAccount) {
+ // Something wrong - fetch fresh state from server
+ await this.fetchAccountState(serverTransaction.fromAccountId);
+ await this.fetchAccountState(serverTransaction.toAccountId);
+ return;
+ }
+
+ // Remove pending transaction
+ const updatedFromPending = fromAccount.pendingTransactions.filter(
+ tx => tx.id !== serverTransaction.requestId
+ );
+
+ const updatedToPending = toAccount.pendingTransactions.filter(
+ tx => tx.id !== serverTransaction.requestId
+ );
+
+ // Update to match server authoritative state
+ // NOTE: We use server-provided balances, not our calculated ones
+ this.localState.set(serverTransaction.fromAccountId, {
+ ...fromAccount,
+ balance: serverTransaction.fromAccountNewBalance,
+ pendingTransactions: updatedFromPending,
+ version: serverTransaction.fromAccountVersion,
+ });
+
+ this.localState.set(serverTransaction.toAccountId, {
+ ...toAccount,
+ balance: serverTransaction.toAccountNewBalance,
+ pendingTransactions: updatedToPending,
+ version: serverTransaction.toAccountVersion,
+ });
+
+ this.notifyStateChange();
+ }
+
+ // Handle server rejection (business rule violation)
+ private async handleConflict(operation: PendingOperation, response: Response): Promise<void> {
+ const error = await response.json();
+
+ // Revert optimistic update
+ await this.revertOptimisticUpdate(operation);
+
+ // Fetch fresh state from server
+ await this.fetchAccountState(operation.fromAccountId);
+ await this.fetchAccountState(operation.toAccountId);
+
+ // Notify user of the conflict
+ this.notifyError({
+ type: 'conflict',
+ message: error.message,
+ operationId: operation.id,
+ });
+
+ // Remove from queue
+ const index = this.pendingQueue.indexOf(operation);
+ if (index > -1) {
+ this.pendingQueue.splice(index, 1);
+ }
+ }
+}
+
+// HTTP API client that respects server authority
+class AuthoritativeApiClient {
+ // Idempotent request pattern
+ async transferFunds(
+ fromAccountId: string,
+ toAccountId: string,
+ amount: number
+ ): Promise<ServerTransaction> {
+ const requestId = crypto.randomUUID();
+
+ const response = await fetch('/api/transfers', {
+ method: 'POST',
+ headers: {
+ 'Content-Type': 'application/json',
+ 'Idempotency-Key': requestId,
+ },
+ body: JSON.stringify({
+ fromAccountId,
+ toAccountId,
+ amount,
+ requestId,
+ }),
+ });
+
+ if (!response.ok) {
+ const error = await response.json();
+ throw new Error(error.message || `Transfer failed: ${response.status}`);
+ }
+
+ return response.json();
+ }
+
+ // Poll for updates - accept server authority
+ async pollForUpdates(accountId: string, lastVersion: number): Promise<AccountUpdate> {
+ const response = await fetch(`/api/accounts/${accountId}/updates?sinceVersion=${lastVersion}`);
+
+ if (response.status === 304) {
+ // No changes - server is authoritative about this
+ return { hasUpdates: false };
+ }
+
+ if (!response.ok) {
+ throw new Error(`Failed to poll updates: ${response.status}`);
+ }
+
+ const update = await response.json();
+ return {
+ hasUpdates: true,
+ account: update.account,
+ transactions: update.transactions,
+ };
+ }
+}
+```
+
+## Use Cases and Trade-offs
+
+### When Server-Authoritative Design Is Essential
+
+| Use Case | Why Server-Authoritative | Example Implementation |
+|----------|--------------------------|------------------------|
+| **Financial Systems** | Legal requirement for audit trails, fraud prevention, regulatory compliance | Banking transactions with double-entry bookkeeping and immutable ledger |
+| **E-commerce** | Inventory management (prevent overselling), pricing consistency, tax calculation | Stock reservation system, cart abandonment recovery |
+| **Collaborative Editing** | Conflict resolution, version history, real-time synchronization | Operational transformation in Google Docs, CRDTs with authoritative merge |
+| **Multiplayer Games** | Cheat prevention, game state consistency, fair play enforcement | Deterministic lockstep simulation, server-side game logic |
+| **Healthcare Systems** | Patient safety, regulatory compliance (HIPAA), audit requirements | Electronic health records with change tracking |
+
+### When to Relax Server Authority
+
+| Scenario | Appropriate Approach | Rationale |
+|----------|---------------------|-----------|
+| **Read-heavy applications** | Client-side caching with TTL/ETag | Reduce server load, improve responsiveness |
+| **Offline-first apps** | Local-first with eventual consistency | Must function without network connectivity |
+| **Real-time collaboration** | Hybrid with conflict-free data types | Low latency required, conflicts resolvable |
+| **Static content delivery** | CDN edge caching with invalidation | Performance outweighs consistency needs |
+| **Analytics dashboards** | Client-side aggregation of pre-approved data | Reduce server computation costs |
+
+## Case Studies
+
+### Case Study 1: Banking Application
+
+**Problem**: A mobile banking app where users could theoretically manipulate client-side code to bypass balance checks.
+
+**Server-Authoritative Solution**:
+
+```
+Client (Untrusted) Server (Authoritative)
+┌─────────────────┐ ┌─────────────────────┐
+│ Transfer Request│──────────────▶│ 1. Validate Token │
+│ - Amount: $1000 │ │ 2. Check Balance │
+│ - From: Acct A │ │ (SELECT ... FOR │
+│ - To: Acct B │ │ UPDATE) │
+└─────────────────┘ │ 3. Apply Business │
+ │ │ Rules │
+ │ │ 4. Execute in │
+ │ ┌──────────────┤ Transaction │
+ │ │ │ 5. Record Audit │
+ │ │ Rejection │ Trail │
+ ▼ ▼ └─────────────────────┘
+┌─────────────────┐ ┌─────────────────────┐
+│ UI Shows │ │ Transaction │
+│ Success │ │ Failed: │
+│ Immediately │ │ Insufficient │
+│ (Optimistic) │ │ Funds │
+└─────────────────┘ └─────────────────────┘
+ │ │
+ │ Server Truth │
+ └──────────────────────┘
+ ▼
+┌─────────────────┐
+│ UI Reconciles │
+│ with Server │
+│ (Actual State)│
+└─────────────────┘
+```
+
+**Key Architecture Decisions**:
+1. **Idempotent requests**: Each transfer includes unique `request_id` to prevent duplicate processing
+2. **Pessimistic locking**: `SELECT ... FOR UPDATE` prevents race conditions
+3. **Audit trail**: Every transaction recorded immutably
+4. **Client reconciliation**: Optimistic UI updates, but final state from server
+
+### Case Study 2: E-commerce Inventory
+
+**Problem**: Flash sale with 100 items, 10,000 simultaneous users. Prevent overselling.
+
+**Naive Approach (Client-side counting)**:
+```typescript
+// BAD: Client decides if item is available
+async function purchaseItem(itemId: string) {
+ const inventory = await fetchInventory(itemId);
+ if (inventory.available > 0) {
+ // RACE CONDITION: Other users might be buying simultaneously
+ await purchase(itemId);
+ }
+}
+```
+
+**Server-Authoritative Solution**:
+```rust
+// GOOD: Server authoritatively manages inventory
+#[derive(Debug, Clone, Copy)]
+pub enum InventoryReservation {
+ Reserved { reservation_id: Uuid, expires_at: DateTime<Utc> },
+ SoldOut,
+ Available,
+}
+
+pub struct InventoryService {
+ redis: RedisConnection,
+ db_pool: PgPool,
+}
+
+impl InventoryService {
+ /// Reserve item atomically - only server can do this
+ pub async fn reserve_item(
+ &self,
+ item_id: Uuid,
+ user_id: Uuid,
+ quantity: i32,
+ ) -> Result<InventoryReservation, InventoryError> {
+ // Use Redis for distributed lock AND inventory count
+ let lock_key = format!("inventory:lock:{}", item_id);
+ let inventory_key = format!("inventory:{}", item_id);
+
+ // Atomic check-and-decrement
+ let script = r#"
+ local current = redis.call('GET', KEYS[2])
+ if not current or tonumber(current) < tonumber(ARGV[1]) then
+ return {false, 'insufficient'}
+ end
+
+ local new_val = tonumber(current) - tonumber(ARGV[1])
+ redis.call('SET', KEYS[2], new_val)
+
+ local reservation_id = ARGV[2]
+ local expires_at = ARGV[3]
+ redis.call('HSET', 'reservations', reservation_id, ARGV[4])
+ redis.call('EXPIREAT', reservation_id, expires_at)
+
+ return {true, reservation_id}
+ "#;
+
+ let reservation_id = Uuid::new_v4();
+ let expires_at = Utc::now() + chrono::Duration::minutes(10);
+
+ let result: (bool, String) = redis::cmd("EVAL")
+ .arg(script)
+ .arg(2) // number of keys
+ .arg(&lock_key)
+ .arg(&inventory_key)
+ .arg(quantity)
+ .arg(reservation_id.to_string())
+ .arg(expires_at.timestamp())
+ .arg(user_id.to_string())
+ .query_async(&mut self.redis.clone())
+ .await?;
+
+ match result {
+ (true, rid) => Ok(InventoryReservation::Reserved {
+ reservation_id: Uuid::parse_str(&rid).unwrap(),
+ expires_at,
+ }),
+ (false, _) => Ok(InventoryReservation::SoldOut),
+ }
+ }
+}
+```
+
+## Workflow Implementation Steps
+
+### Step 1: Identify Authoritative vs. Non-Authoritative Operations
+
+```
+┌─────────────────────────────────────────────────────────────┐
+│ Operation Analysis Matrix │
+├─────────────────┬─────────────────┬─────────────────────────┤
+│ Operation │ Must Be │ Can Be │
+│ │ Authoritative │ Client-Side │
+├─────────────────┼─────────────────┼─────────────────────────┤
+│ Funds Transfer │ ✅ Yes │ ❌ No │
+│ │ (Financial rule)│ │
+├─────────────────┼─────────────────┼─────────────────────────┤
+│ Form Validation │ ⚠️ Partial │ ✅ Yes (for UX) │
+│ │ (Final check │ │
+│ │ on server) │ │
+├─────────────────┼─────────────────┼─────────────────────────┤
+│ Search Filtering│ ❌ No │ ✅ Yes │
+│ │ (Presentation │ │
+│ │ only) │ │
+├─────────────────┼─────────────────┼─────────────────────────┤
+│ Read Operations │ ⚠️ Depends │ ✅ Often │
+│ │ (Cached vs. │ │
+│ │ fresh data) │ │
+└─────────────────┴─────────────────┴─────────────────────────┘
+```
+
+### Step 2: Design Idempotent APIs
+
+```rust
+// Rust backend implementing idempotency
+pub struct IdempotencyService {
+ db_pool: PgPool,
+}
+
+impl IdempotencyService {
+ pub async fn execute_with_idempotency<F, T, E>(
+ &self,
+ request_id: Uuid,
+ user_id: Uuid,
+ operation: &str,
+ f: F,
+ ) -> Result<T, E>
+ where
+ F: FnOnce() -> futures::future::BoxFuture<'static, Result<T, E>>,
+ E: From<IdempotencyError>,
+ {
+ // Check if we've already processed this request
+ let existing = sqlx::query!(
+ r#"
+ SELECT result, status_code
+ FROM idempotency_keys
+ WHERE key = $1 AND user_id = $2 AND operation = $3
+ "#,
+ request_id,
+ user_id,
+ operation
+ )
+ .fetch_optional(&self.db_pool)
+ .await?;
+
+ if let Some(record) = existing {
+ // Request already processed - return cached result
+ match record.status_code.as_str() {
+ "COMPLETED" => {
+ let result: T = serde_json::from_str(&record.result.unwrap())?;
+ return Ok(result);
+ }
+ "FAILED" => {
+ return Err(serde_json::from_str(&record.result.unwrap())?);
+ }
+ _ => {
+ // In progress - wait or retry
+ return Err(IdempotencyError::RequestInProgress.into());
+ }
+ }
+ }
+
+ // First time - record that we're starting
+ sqlx::query!(
+ r#"
+ INSERT INTO idempotency_keys
+ (key, user_id, operation, status_code, created_at, updated_at)
+ VALUES ($1, $2, $3, 'PROCESSING', NOW(), NOW())
+ "#,
+ request_id,
+ user_id,
+ operation
+ )
+ .execute(&self.db_pool)
+ .await?;
+
+ // Execute the actual operation
+ let result = f().await;
+
+ // Record the outcome
+ match &result {
+ Ok(success_result) => {
+ let result_json = serde_json::to_string(success_result)?;
+ sqlx::query!(
+ r#"
+ UPDATE idempotency_keys
+ SET status_code = 'COMPLETED', result = $1, updated_at = NOW()
+ WHERE key = $2 AND user_id = $3 AND operation = $4
+ "#,
+ result_json,
+ request_id,
+ user_id,
+ operation
+ )
+ .execute(&self.db_pool)
+ .await?;
+ }
+ Err(error) => {
+ let error_json = serde_json::to_string(error)?;
+ sqlx::query!(
+ r#"
+ UPDATE idempotency_keys
+ SET status_code = 'FAILED', result = $1, updated_at = NOW()
+ WHERE key = $2 AND user_id = $3 AND operation = $4
+ "#,
+ error_json,
+ request_id,
+ user_id,
+ operation
+ )
+ .execute(&self.db_pool)
+ .await?;
+ }
+ }
+
+ result
+ }
+}
+```
+
+### Step 3: Implement Optimistic UI with Reconciliation
+
+```typescript
+// TypeScript reconciliation engine
+class ReconciliationEngine<T> {
+ private localState: T;
+ private pendingMutations: Array<{
+ id: string;
+ mutation: (state: T) => T;
+ timestamp: number;
+ }> = [];
+
+ constructor(initialState: T) {
+ this.localState = initialState;
+ }
+
+ // Apply mutation optimistically
+ mutate(mutation: (state: T) => T, mutationId: string): T {
+ const pending = {
+ id: mutationId,
+ mutation,
+ timestamp: Date.now(),
+ };
+
+ this.pendingMutations.push(pending);
+ this.localState = mutation(this.localState);
+
+ return this.localState;
+ }
+
+ // Reconcile with server authoritative state
+ reconcile(serverState: T, appliedMutationIds: string[]): T {
+ // Remove mutations that server has confirmed
+ this.pendingMutations = this.pendingMutations.filter(
+ mutation => !appliedMutationIds.includes(mutation.id)
+ );
+
+ // Start from server state (authoritative)
+ let reconciledState = serverState;
+
+ // Re-apply pending mutations that server hasn't seen yet
+ for (const pending of this.pendingMutations) {
+ reconciledState = pending.mutation(reconciledState);
+ }
+
+ this.localState = reconciledState;
+ return reconciledState;
+ }
+
+ // Handle server rejection
+ rejectMutation(mutationId: string, serverState: T): T {
+ // Remove the rejected mutation
+ this.pendingMutations = this.pendingMutations.filter(
+ m => m.id !== mutationId
+ );
+
+ // Reset to server state
+ this.localState = serverState;
+
+ // Re-apply remaining pending mutations
+ for (const pending of this.pendingMutations) {
+ this.localState = pending.mutation(this.localState);
+ }
+
+ return this.localState;
+ }
+}
+```
+
+## Performance Considerations
+
+### Trade-offs: Latency vs. Consistency
+
+```
+ Response Time Impact
+┌─────────────────────────────────────────────────┐
+│ Client-Side Heavy │
+│ │
+│ Fast ╭───────────────────────────────────╮ │
+│ │■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■│ │
+│ │■■■ Optimistic Updates ■■■■│ │
+│ │■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■│ │
+│ │■■■■ Immediate Feedback ■■■■■■■■■■■│ │
+│ │■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■│ │
+│ ╰───────────────────────────────────╯ │
+│ │
+│ Server-Authoritative │
+│ │
+│ ╭───────────────────────────────────╮ │
+│ │■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■│ │
+│ │■■■ Network Round-Trips ■■■■│ │
+│ │■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■│ │
+│ │■■■■ Validation & Locking ■■■■■■■│ │
+│ Slow │■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■│ │
+│ ╰───────────────────────────────────╯ │
+│ │
+└─────────────────────────────────────────────────┘
+ Consistency
+```
+
+### Optimization Strategies
+
+1. **Batched Authoritative Operations**
+```rust
+// Instead of multiple round-trips
+for item in cart.items {
+ reserve_item(item.id).await?; // N+1 problem
+}
+
+// Batch authoritative operations
+batch_reserve_items(cart.items).await?; // Single round-trip
+```
+
+2. **Edge Caching with Validation**
+```typescript
+// Cache validation rules at edge, final check at origin
+async function validateInputCached(input: unknown): Promise<ValidationResult> {
+ // Check local cache first (rules don't change often)
+ const cachedRules = await cache.get('validation-rules');
+ const quickResult = quickValidate(input, cachedRules);
+
+ if (!quickResult.valid) {
+ return quickResult;
+ }
+
+ // Final authoritative check
+ return await authoritativeValidate(input);
+}
+```
+
+3. **Predictive Pre-authorization**
+```rust
+// Pre-approve likely actions based on user behavior
+pub async fn pre_authorize_transfer(
+ &self,
+ user_id: Uuid,
+ max_amount: Decimal,
+ window_minutes: i32,
+) -> Result<PreAuthorization, AuthError> {
+ // Analytics suggest user will transfer < $500 in next 5 minutes
+ // Pre-reserve capacity, reduce latency for actual transfer
+}
+```
+
+## Security Implications
+
+### Threat Model for Server-Authoritative Systems
+
+| Threat | Without Server Authority | With Server Authority |
+|--------|--------------------------|------------------------|
+| **Balance Manipulation** | Client can modify JavaScript to bypass checks | Server validates all transactions |
+| **Race Conditions** | Two clients might oversell inventory | Distributed locks prevent overselling |
+| **Replay Attacks** | Same transaction submitted multiple times | Idempotency keys prevent duplicates |
+| **Business Logic Bypass** | Client can skip validation steps | All rules enforced server-side |
+| **Data Tampering** | Client-side state can be manipulated | Only server state is authoritative |
+
+### Defense in Depth Strategy
+
+```
+┌─────────────────────────────────────────────────────────┐
+│ Defense in Depth Layers │
+├─────────────────┬───────────────────────────────────────┤
+│ Layer │ Implementation │
+├─────────────────┼───────────────────────────────────────┤
+│ Client-Side │ Basic validation (UX only) │
+│ │ Input sanitization │
+├─────────────────┼───────────────────────────────────────┤
+│ Network │ HTTPS/TLS 1.3 │
+│ │ Request signing │
+├─────────────────┼───────────────────────────────────────┤
+│ API Gateway │ Rate limiting │
+│ │ Schema validation │
+├─────────────────┼───────────────────────────────────────┤
+│ Application │ Business logic validation │
+│ │ Authentication/Authorization │
+├─────────────────┼───────────────────────────────────────┤
+│ Database │ ACID transactions │
+│ │ Row-level security │
+│ │ Audit logging │
+└─────────────────┴───────────────────────────────────────┘
+```
+
+## Future Evolution
+
+### Beyond Traditional Server-Authoritative
+
+1. **Edge Computing with Authoritative Rules**
+ - Push validation logic to CDN edge
+ - Final authorization at origin
+ - Reduced latency while maintaining security
+
+2. **Blockchain as Authoritative Layer**
+ - Smart contracts as business logic
+ - Immutable transaction ledger
+ - Decentralized but still authoritative
+
+3. **Federated Authority**
+ - Multiple authoritative services
+ - Consensus protocols for coordination
+ - Used in distributed systems like Kubernetes
+
+4. **Zero-Trust with Continuous Authorization**
+ - Every operation re-validated
+ - Context-aware authorization
+ - Dynamic policy evaluation
+
+## Conclusion
+
+The server-authoritative design principle remains essential for systems where correctness, security, and consistency matter more than pure latency. While modern applications often adopt hybrid approaches—optimistic UI updates with eventual server reconciliation—the fundamental truth remains: **the backend must be the ultimate arbiter of business rules and data integrity**.
+
+The key is not eliminating client-side logic, but rather clearly demarcating which operations require server authority and which can be safely delegated. This boundary should be explicit in both architecture documentation and code implementation.
+
+As connectivity improves and edge computing matures, the line between "client" and "server" may blur, but the need for authoritative validation of critical operations will persist. The most robust systems will continue to embrace server authority while optimizing for user experience through intelligent client-side enhancements.
+
+---
+
+*Article written with examples in Rust and TypeScript, demonstrating practical implementation of server-authoritative patterns while maintaining responsive user interfaces.*
+\ No newline at end of file
diff --git a/system_design/server_scaling.md b/system_design/server_scaling.md
@@ -0,0 +1,29 @@
+# Server Scaling
+
+## Load Balancer
+
+A load balancer evenly distributes incoming traffic among web servers that are
+defined in a load-balanced set.
+
+## Database Replication
+
+A master database generally only supports write operations. A slave database
+gets copies of the data from the master database and only supports read
+operations.
+
+<img src="/assets/system_design-server_scaling.png" alt="Server Setup" width="100%">
+
+## Cache Tier
+
+The [cache](/networking/cache.md) tier is a temporary data store layer, much
+faster than the database. The benefits of having a separate cache tier include
+better system performance, ability to reduce database workloads, and the ability
+to scale the cache tier independently.
+
+## Content Delivery Network (CDN)
+
+A CDN is a network of geographically dispersed servers used to deliver static
+content. CDN servers cache static content like images, videos, CSS, JavaScript
+files, etc.
+
+<img src="/assets/cdn_cache_server.png" alt="CDN Cache Server Setup" width="100%">
diff --git a/system_design/single_server.md b/system_design/single_server.md
@@ -0,0 +1,6 @@
+# Single Server Setup
+
+The simplest design is everything running on a single server: web app, database,
+cache, etc.
+
+<img src="/assets/single_server.png" alt="Single Server Setup" width="100%">
diff --git a/system_design/state_management/action_reducer_pattern.md b/system_design/state_management/action_reducer_pattern.md
@@ -0,0 +1,69 @@
+# Action/Reducer Pattern
+
+## Redux-Like Action/Reducer Pattern
+
+```
+┌──────────────────────────┐
+│ UI / View │
+│ (components, widgets) │
+└────────────┬─────────────┘
+ │
+ │ dispatch(action)
+ ▼
+ ┌───────────────┐
+ │ ACTION │
+ │ type + data │
+ └──────┬────────┘
+ │
+ │ sent to
+ ▼
+┌──────────────────────────────────────┐
+│ STORE (currentState+ reducer) │
+└────────────┬─────────────────────────┘
+ │
+ │ calls
+ ▼
+ ┌───────────────────────┐
+ │ REDUCER │
+ │ newState = f( │
+ │ state, action │
+ │ ) │
+ └─────────┬─────────────┘
+ │
+ │ returns newState
+ ▼
+ ┌───────────────┐
+ │ STORE │
+ │ updates state │
+ └──────┬────────┘
+ │
+ │ notifies subscribers
+ ▼
+┌───────────────────────────┐
+│ UI / View │
+│ re-renders from state │
+└───────────────────────────┘
+```
+
+For more predictable updates, you can implement a Redux-style pattern using
+typed discriminated unions:
+
+```ts
+type Action =
+ | { type: "ADD_TODO"; payload: { text: string } }
+ | { type: "TOGGLE_TODO"; payload: { id: number } };
+
+function reducer(state: AppState, action: Action): AppState {
+ switch (action.type) {
+ case "ADD_TODO":
+ return { ...state, todos: [...state.todos, action.payload] };
+ case "TOGGLE_TODO":
+ return { ...state /* ... */ };
+ default:
+ return state;
+ }
+}
+```
+
+TypeScript's exhaustive checking ensures your switch covers every action type,
+preventing silent missed cases.
diff --git a/system_design/state_management/broadcastchannel_api.md b/system_design/state_management/broadcastchannel_api.md
@@ -0,0 +1,95 @@
+# Broadcast Channel API
+
+```
+BroadcastChannel (cross-context bus)
+---------------------------------------
+
+ [ Tab 1 ] [ Tab 2 ] [ Worker / Tab 3 ]
+ +-----------+ +-----------+ +-----------------+
+ | App code | | App code | | App code |
+ | publishes | | subscribes| | subscribes |
+ +-----+-----+ +-----+-----+ +--------+--------+
+ | | |
+ +---------+---------+-------------------------+
+ v
+ +-----------------------+
+ | BroadcastChannel | browser-level bus
+ | ("app-events") |
+ +-----------------------+
+```
+
+`BroadcastChannel` is a built-in browser API that lets different browser
+contexts (tabs, iframes, workers) on the same origin communicate by posting
+messages to a named channel. It's essentially a pub/sub bus at the browser level
+— perfect for syncing state across tabs without a server.
+
+## Typed BroadcastChannel in TypeScript
+
+You can wrap it in a typed class to get full compile-time safety:
+
+```ts
+type StateMessage<T> =
+ | { type: "STATE_UPDATE"; payload: Partial<T> }
+ | { type: "REQUEST_SYNC" };
+
+class SyncedStore<T extends object> {
+ private state: T;
+ private channel: BroadcastChannel;
+ private subscribers = new Set<(s: T) => void>();
+
+ constructor(name: string, initial: T) {
+ this.state = { ...initial };
+ this.channel = new BroadcastChannel(name);
+ this.channel.onmessage = (e: MessageEvent<StateMessage<T>>) => {
+ if (e.data.type === "STATE_UPDATE") {
+ this.state = { ...this.state, ...e.data.payload };
+ this.subscribers.forEach((cb) => cb(this.state));
+ }
+ };
+ }
+
+ setState(updates: Partial<T>) {
+ this.state = { ...this.state, ...updates };
+ this.channel.postMessage({ type: "STATE_UPDATE", payload: updates });
+ this.subscribers.forEach((cb) => cb(this.state));
+ }
+
+ subscribe(cb: (s: T) => void) {
+ this.subscribers.add(cb);
+ return () => this.subscribers.delete(cb);
+ }
+
+ destroy() {
+ this.channel.close();
+ }
+}
+```
+
+TypeScript's MessageEvent<T> generic ensures the e.data payload is typed
+correctly, preventing you from accidentally posting or reading the wrong shape.
+
+### Common Use Cases
+
+- Session sync — if a user logs out in one tab, broadcast a logout action to
+ immediately reflect that across all open tabs
+
+- Shopping cart sync — changes made in one tab propagate instantly to others
+
+- Shared worker state — coordinate state between a ServiceWorker and page tabs
+
+### Limitations to Know
+
+- BroadcastChannel does not fire in the tab that sent the message — only other
+ tabs receive it, so you still update local state directly
+
+- It only works within the same origin (same protocol + domain + port)
+
+- It has no built-in message history — a newly opened tab won't get past state
+ unless you implement a REQUEST_SYNC handshake pattern (one tab requests the
+ current state, another responds with it)
+
+- Not available in IE, but has full support in all modern browsers
+
+This pattern pairs naturally with the observable store from the previous answer
+— the BroadcastChannel becomes an external subscriber that mirrors state updates
+to other tabs.
diff --git a/system_design/state_management/index.md b/system_design/state_management/index.md
@@ -0,0 +1,58 @@
+# State Management
+
+## Vanilla TypeScript Core State Management Patterns
+
+The foundational approach is a typed observable store — a generic `Store<T>`
+class that holds state, exposes `getState()`, and notifies subscribers via
+`setState()`. TypeScript generics enforce that only valid keys/shapes can be
+set. Three main patterns build on this:
+
+- [Observable Store](observer_pattern.md)
+ — a central class with `subscribe/setState`, good for simple-to-medium apps
+
+- [Action/Reducer](action_reducer_pattern.md)
+ — Redux-style discriminated union actions processed by a pure `reducer`
+ function; TypeScript's exhaustive checking ensures no action is missed
+
+- [Proxy-based reactivity](proxy_based_reactivity.md)
+ — intercepts property assignments to auto-trigger re-renders, mimicking
+ Vue/MobX without a library
+
+- [Path-based (DeepState)](path_based_state.md) -
+ Best for bested state trees
+
+- [Persistent State](persistent_state.md) -
+ Best for cross-session state
+
+ Best practices: always define a state interface, use `Partial<T>` for updates,
+ prefer immutable spreads, and normalise collections as `{ byId, allIds }`.
+
+### Difference between observer, singleton channel and BroadcastChannel
+
+1. Observer — one object talking to its own watchers
+
+2. [Singleton Channel](singleton_channels.md)
+ — any part of your app talking to any other part, within the same tab
+
+3. [BroadcastChannel](broadcastchannel_api.md)
+ — any tab talking to any other tab in the same browser
+
+In practice, you'd often use all three together: a `BroadcastChannel` receives a
+cross-tab message → publishes onto a singleton `Channel<T>` → individual
+components subscribed via the Observer pattern react to the update.
+
+## TypeScript-Specific Best Practices
+
+- Always define a state interface — never use any for your state shape
+
+- Use `Partial<T>` for updates — so you only pass the fields you're changing
+
+- Prefer immutable updates — spread operators `({ ...state, ...updates })`
+ instead of mutating directly
+
+- Normalize entity collections — store arrays as
+ `{ byId: Record<id, T>, allIds:
+ id[] }` for O(1) lookups
+
+- Use `localStorage` with version migrations — persist state but include a
+ `_version` field so you can safely migrate old data
diff --git a/system_design/state_management/observer_pattern.md b/system_design/state_management/observer_pattern.md
@@ -0,0 +1,53 @@
+# Observer Pattern
+
+## Centralized Store (Observable Pattern)
+
+```
+Observer pattern (object-centric)
+------------------------------------
+
+ +-------------------+
+ | UserService | subject owns its observers
+ +-------------------+
+ | | |
+ v v v
+ +--------+ +--------+ +--------+
+ | View A | | View B | | Logger |
+ +--------+ +--------+ +--------+
+```
+
+The most fundamental approach is a typed store class that holds state, exposes a
+getState() method, and notifies subscribers on change:
+
+```ts
+interface AppState {
+ user: User | null;
+ theme: "light" | "dark";
+}
+
+class Store<T> {
+ private state: T;
+ private subscribers = new Set<(state: T) => void>();
+
+ constructor(initialState: T) {
+ this.state = { ...initialState };
+ }
+
+ getState(): T {
+ return { ...this.state };
+ }
+
+ setState(updates: Partial<T>) {
+ this.state = { ...this.state, ...updates };
+ this.subscribers.forEach((cb) => cb(this.getState()));
+ }
+
+ subscribe(cb: (state: T) => void): () => void {
+ this.subscribers.add(cb);
+ return () => this.subscribers.delete(cb);
+ }
+}
+```
+
+TypeScript generics make the store fully type-safe — setState only accepts keys
+that exist on your state interface.
diff --git a/system_design/state_management/path_based_state.md b/system_design/state_management/path_based_state.md
@@ -0,0 +1,75 @@
+# Path-Based State (Deep State)
+
+```
+State Tree
++----------------------------------+
+| app |
+| user |
+| profile |
+| name: "Alice" <--- path |
+| age: 30 |
+| cart |
+| count: 2 |
++----------------------------------+
+
+subscribe("user.profile.name") --> only fires when name changes
+subscribe("cart.count") --> only fires when count changes
+```
+
+Instead of subscribing to the whole state object, subscribers listen to a
+specific path within a nested state tree — e.g. "user.profile.name". Only
+changes to that exact path trigger their callback.
+
+```ts
+type Path<T> = string; // e.g. "user.profile.name"
+
+class DeepStore<T extends object> {
+ private state: T;
+ private subscribers = new Map<string, Set<(val: unknown) => void>>();
+
+ constructor(initial: T) {
+ this.state = initial;
+ }
+
+ subscribe<V>(path: Path<T>, cb: (val: V) => void): () => void {
+ if (!this.subscribers.has(path)) this.subscribers.set(path, new Set());
+ this.subscribers.get(path)!.add(cb as (val: unknown) => void);
+ return () =>
+ this.subscribers.get(path)?.delete(cb as (val: unknown) => void);
+ }
+
+ set(path: Path<T>, value: unknown) {
+ const keys = path.split(".");
+ let cursor: any = this.state;
+ for (let i = 0; i < keys.length - 1; i++) cursor = cursor[keys[i]];
+ cursor[keys[keys.length - 1]] = value;
+ this.subscribers.get(path)?.forEach((cb) => cb(value));
+ }
+
+ get(path: Path<T>): unknown {
+ return path.split(".").reduce((obj: any, key) => obj?.[key], this.state);
+ }
+}
+
+// Usage
+interface AppState {
+ user: { profile: { name: string; age: number } };
+ cart: { count: number };
+}
+
+const store = new DeepStore<AppState>({
+ user: { profile: { name: "Alice", age: 30 } },
+ cart: { count: 2 },
+});
+
+store.subscribe<string>(
+ "user.profile.name",
+ (name) => console.log(`Name changed: ${name}`),
+);
+store.set("user.profile.name", "Bob"); // fires callback
+store.set("cart.count", 5); // does NOT fire name callback
+```
+
+Use case: Large nested state trees where you want granular, performant
+subscriptions — only the parts of the UI that care about a specific slice
+re-render.
diff --git a/system_design/state_management/persistent_state.md b/system_design/state_management/persistent_state.md
@@ -0,0 +1,82 @@
+# Persistent State
+
+```
+ +-------------------+
+setState() --> | PersistentStore | --> subscribers notified
+ +-------------------+
+ | ^
+ save| |load on init
+ v |
+ +-------------------+
+ | localStorage |
+ +-------------------+
+```
+
+State that survives page reloads by syncing to localStorage (or sessionStorage).
+Includes a _version field so you can safely migrate stale data from older
+schemas.
+
+```ts
+interface Versioned {
+ _version: number;
+}
+
+class PersistentStore<T extends Versioned> {
+ private state: T;
+ private subscribers = new Set<(s: T) => void>();
+ private key: string;
+ private currentVersion: number;
+ private migrate: (old: any) => T;
+
+ constructor(key: string, initial: T, migrate: (old: any) => T) {
+ this.key = key;
+ this.currentVersion = initial._version;
+ this.migrate = migrate;
+ this.state = this.load(initial);
+ }
+
+ private load(initial: T): T {
+ const raw = localStorage.getItem(this.key);
+ if (!raw) return initial;
+ const parsed = JSON.parse(raw);
+ if (parsed._version !== this.currentVersion) return this.migrate(parsed);
+ return parsed as T;
+ }
+
+ private save() {
+ localStorage.setItem(this.key, JSON.stringify(this.state));
+ }
+
+ getState(): T {
+ return { ...this.state };
+ }
+
+ setState(updates: Partial<T>) {
+ this.state = { ...this.state, ...updates };
+ this.save();
+ this.subscribers.forEach((cb) => cb(this.getState()));
+ }
+
+ subscribe(cb: (s: T) => void) {
+ this.subscribers.add(cb);
+ return () => this.subscribers.delete(cb);
+ }
+}
+
+// Usage
+interface ThemeState extends Versioned {
+ theme: "light" | "dark";
+ fontSize: number;
+}
+
+const themeStore = new PersistentStore<ThemeState>(
+ "theme",
+ { _version: 2, theme: "light", fontSize: 16 },
+ (old) => ({ _version: 2, theme: old.theme ?? "light", fontSize: 16 }), // migrate v1 → v2
+);
+
+themeStore.setState({ theme: "dark" }); // persisted to localStorage immediately
+```
+
+Use case: User preferences, session data, or any state that should survive a
+page refresh — theme, language, last-visited route, partially filled forms.
diff --git a/system_design/state_management/proxy_based_reactivity.md b/system_design/state_management/proxy_based_reactivity.md
@@ -0,0 +1,97 @@
+# Proxy-based Reactivity
+
+```
+ Proxy (trap layer)
+ +------------------+
+state.count++ | set trap fires |---> notify subscribers
+<-- direct --> | get trap fires |---> track dependencies
+ +------------------+
+ |
+ +------------------+
+ | Raw state obj |
+ +------------------+
+```
+
+## Reactive State with Proxy
+
+A JavaScript Proxy wraps your state object and intercepts get and set
+operations. Instead of calling setState() manually, any direct property
+assignment automatically triggers subscribers — the same mechanic Vue 3 and MobX
+use internally.
+
+```ts
+type Subscriber<T> = (state: T) => void;
+
+function reactive<T extends object>(
+ initial: T,
+ onChange: Subscriber<T>,
+): T {
+ return new Proxy(initial, {
+ set(target, prop, value) {
+ (target as any)[prop] = value;
+ onChange(target);
+ return true;
+ },
+ });
+}
+
+// Usage
+interface AppState {
+ count: number;
+ user: string | null;
+}
+
+const state = reactive<AppState>(
+ { count: 0, user: null },
+ (s) => console.log("State changed:", s),
+);
+
+state.count++; // logs automatically
+state.user = "Alice"; // logs automatically
+```
+
+For finer control, you can track which properties were accessed (get trap) and
+only notify subscribers that depend on that specific property — this is called
+dependency tracking:
+
+```ts
+const subscribers = new Map<string | symbol, Set<() => void>>();
+let activeEffect: (() => void) | null = null;
+
+function reactive<T extends object>(initial: T): T {
+ return new Proxy(initial, {
+ get(target, prop) {
+ if (activeEffect) {
+ if (!subscribers.has(prop)) subscribers.set(prop, new Set());
+ subscribers.get(prop)!.add(activeEffect);
+ }
+ return (target as any)[prop];
+ },
+ set(target, prop, value) {
+ (target as any)[prop] = value;
+ subscribers.get(prop)?.forEach((fn) => fn());
+ return true;
+ },
+ });
+}
+
+function effect(fn: () => void) {
+ activeEffect = fn;
+ fn(); // run once to collect dependencies via get traps
+ activeEffect = null;
+}
+
+// Usage
+const state = reactive({ count: 0, user: "Alice" });
+
+effect(() => {
+ console.log(`Count is: ${state.count}`); // only re-runs when count changes
+});
+
+state.count = 5; // triggers effect
+state.user = "Bob"; // does NOT trigger effect (not accessed in effect)
+```
+
+Watch out: Proxy only intercepts the top-level object by default. For deeply
+nested state you need to recursively wrap nested objects in their own Proxy —
+which is exactly what Vue 3's reactive() does under the hood.
diff --git a/system_design/state_management/singleton_channels.md b/system_design/state_management/singleton_channels.md
@@ -0,0 +1,92 @@
+# Singleton Channel
+
+```
+Singleton Channel (neutral in-app bus)
+-----------------------------------------
+
+ Publishers Subscribers
+ ---------- -----------
+ +--------+ +--------+
+ | View A |----+ +-->| View C |
+ +--------+ | | +--------+
+ v |
+ +---------------------------+
+ | AppChannel<Event> | (singleton)
+ +---------------------------+
+ ^ |
+ +--------+ | | +--------+
+ | Store |----+ +-->| Logger |
+ +--------+ +--------+
+```
+
+A `Channel<T>` is a globally shared, typed event bus. Any component can publish
+a value of type T into it, and any component that has subscribed will receive
+that value — FIFO (first subscriber registered, first notified). It's simpler
+than a full store because it carries no persistent state; it just broadcasts a
+moment-in-time event.
+
+```ts
+class Channel<T> {
+ private subscribers: Array<(data: T) => void> = [];
+
+ subscribe(cb: (data: T) => void): () => void {
+ this.subscribers.push(cb);
+ return () => {
+ this.subscribers = this.subscribers.filter((s) => s !== cb);
+ };
+ }
+
+ publish(data: T): void {
+ for (const cb of this.subscribers) {
+ cb(data);
+ }
+ }
+}
+```
+
+Because it's generic, TypeScript enforces that every publisher and subscriber
+agrees on the shape of `T` at compile time — you can't accidentally publish a
+`string` on a `Channel<UserEvent>`.
+
+## Defining Global Singleton Channels
+
+You declare each channel once and export it as a module-level singleton:
+
+```ts
+// channels.ts
+interface UserLoggedIn {
+ userId: string;
+ role: "admin" | "user";
+}
+interface CartUpdated {
+ itemCount: number;
+ total: number;
+}
+
+export const userLoginChannel = new Channel<UserLoggedIn>();
+export const cartChannel = new Channel<CartUpdated>();
+```
+
+Any file can import and use these without passing them through props or
+constructors.
+
+## Usage Across Components
+
+```ts
+// ComponentA.ts — subscriber
+import { cartChannel } from './channels';
+
+const unsub = cartChannel.subscribe(({ itemCount, total }) => {
+ document.getElementById('cart-count')!.textContent = String(itemCount);
+});
+
+// call unsub() when component is destroyed to avoid memory leaks
+
+// ComponentB.ts — publisher
+import { cartChannel } from './channels';
+
+function addToCart(item: Item) {
+ // ... update cart logic
+ cartChannel.publish({ itemCount: cart.length, total: cart.reduce(...) });
+}
+```
diff --git a/system_design/state_management/svelte_state.md b/system_design/state_management/svelte_state.md
@@ -0,0 +1,54 @@
+# Svelte State
+
+[How to share state in svelte 5](https://joyofcode.xyz/how-to-share-state-in-svelte-5)
+
+## Function
+
+```ts
+export function createCounter() {
+ let count = $state(0);
+ // you can also derive values
+ let double = $derived(count * 2);
+
+ return {
+ get count() {
+ return count;
+ },
+ set count(value) {
+ count = value;
+ },
+ increment() {
+ count++;
+ },
+ };
+}
+```
+
+Export it once in the same file you make it global.
+
+```ts
+export const counter = createCounter();
+```
+
+Or instantiate it in each component make it a new instance per component:
+
+```ts
+const counter = createCounter();
+```
+
+## Class
+
+```ts
+export class Counter {
+ // make count private
+ #count = $state(0);
+
+ // create property accessors
+ get count() {
+ return this.#count;
+ }
+ set count(value) {
+ this.#count = value;
+ }
+}
+```
diff --git a/test_driven_development/arrange_act_assert.md b/test_driven_development/arrange_act_assert.md
@@ -0,0 +1,146 @@
+# Arrange-Act-Assert
+
+## The Pattern
+
+Arrange-Act-Assert is a great way to structure test cases. It prescribes an
+order of operations:
+
+1. **Arrange** inputs and targets. Arrange steps should set up the test case.
+ Does the test require any objects or special settings? Does it need to prep a
+ database? Does it need to log into a web app? Handle all of these operations
+ at the start of the test.
+
+2. **Act** on the target behavior. Act steps should cover the main thing to be
+ tested. This could be calling a function or method, calling a REST API, or
+ interacting with a web page. Keep actions focused on the target behavior.
+
+3. **Assert** expected outcomes. Act steps should elicit some sort of response.
+ Assert steps verify the goodness or badness of that response. Sometimes,
+ assertions are as simple as checking numeric or string values. Other times,
+ they may require checking multiple facets of a system. Assertions will
+ ultimately determine if the test passes or fails.
+
+**Behavior-Driven Development** follows the **Arrange-Act-Assert** pattern by
+another name: **Given-When-Then**. The Gherkin language uses Given-When-Then
+steps to specify behaviors in scenarios. Given-When-Then is essentially the same
+formula as Arrange-Act-Assert.
+
+### References
+
+[automationpanda](https://automationpanda.com/2020/07/07/arrange-act-assert-a-pattern-for-writing-good-tests/)
+[semaphore](https://semaphore.io/blog/aaa-pattern-test-automation)
+
+## Example
+
+Here is a simple Rust example demonstrating the Arrange-Act-Assert pattern using
+a small `ShoppingCart` struct. Rust's built-in `#[cfg(test)]` module makes the
+three phases very natural to express.
+
+The code:
+
+```rust
+// src/lib.rs
+
+pub struct ShoppingCart {
+ items: Vec<(String, f64)>, // (name, price)
+}
+
+impl ShoppingCart {
+ pub fn new() -> Self {
+ ShoppingCart { items: vec![] }
+ }
+
+ pub fn add_item(&mut self, name: &str, price: f64) {
+ self.items.push((name.to_string(), price));
+ }
+
+ pub fn total(&self) -> f64 {
+ self.items.iter().map(|(_, price)| price).sum()
+ }
+
+ pub fn item_count(&self) -> usize {
+ self.items.len()
+ }
+}
+```
+
+The test:
+
+```rust
+#[cfg(test)]
+mod tests {
+ use super::*;
+
+ #[test]
+ fn test_total_reflects_added_items() {
+ // ── Arrange ──────────────────────────────────────
+ let mut cart = ShoppingCart::new();
+ let expected_total = 17.97;
+
+ // ── Act ──────────────────────────────────────────
+ cart.add_item("Apple", 0.99);
+ cart.add_item("Bread", 2.49);
+ cart.add_item("Laptop", 14.49);
+
+ // ── Assert ───────────────────────────────────────
+ assert!((cart.total() - expected_total).abs() < f64::EPSILON);
+ assert_eq!(cart.item_count(), 3);
+ }
+
+ #[test]
+ fn test_empty_cart_has_zero_total() {
+ // ── Arrange ──────────────────────────────────────
+ let cart = ShoppingCart::new();
+
+ // ── Act ──────────────────────────────────────────
+ let total = cart.total();
+
+ // ── Assert ───────────────────────────────────────
+ assert_eq!(total, 0.0);
+ }
+}
+```
+
+Flow Diagram:
+
+```
+┌─────────────────────────────────────────────────────┐
+│ TEST FUNCTION │
+│ │
+│ ┌─────────────────────────────────────────────┐ │
+│ │ ARRANGE │ │
+│ │ • Create ShoppingCart::new() │ │
+│ │ • Define expected_total = 17.97 │ │
+│ └──────────────────┬──────────────────────────┘ │
+│ │ │
+│ ▼ │
+│ ┌─────────────────────────────────────────────┐ │
+│ │ ACT │ │
+│ │ • cart.add_item("Apple", 0.99) │ │
+│ │ • cart.add_item("Bread", 2.49) │ │
+│ │ • cart.add_item("Laptop", 14.49) │ │
+│ └──────────────────┬──────────────────────────┘ │
+│ │ │
+│ ▼ │
+│ ┌─────────────────────────────────────────────┐ │
+│ │ ASSERT │ │
+│ │ • cart.total() ≈ 17.97 ✔ / ✘ │ │
+│ │ • cart.item_count() == 3 ✔ / ✘ │ │
+│ └─────────────────────────────────────────────┘ │
+└─────────────────────────────────────────────────────┘
+```
+
+Rust-specific notes:
+
+- `#[cfg(test)]` gates the test module so it's only compiled during cargo test,
+ keeping production binaries lean.
+
+- Floating-point assertions in Rust require a tolerance check
+ (`abs() < EPSILON`) rather than a direct `==`, since `f64` arithmetic can
+ introduce tiny rounding errors.
+
+- Each test function is independent — no shared mutable state bleeds between
+ them, reinforcing the one behavior per test principle of AAA.
+
+- The **Given-When-Then** equivalent here would be: Given an empty cart, When
+ three items are added, Then the total equals their sum.
diff --git a/test_driven_development/index.md b/test_driven_development/index.md
@@ -0,0 +1,2 @@
+# Test Driven Development
+
diff --git a/test_driven_development/unit_testing.md b/test_driven_development/unit_testing.md
@@ -0,0 +1,71 @@
+# Unit Testing
+
+## Writing Isolated and Focused Unit Tests
+
+Unit tests should be **isolated and focused**, testing one small, well-defined
+unit of functionality at a time. Each test should verify a single behavior
+without relying on other parts of the system. When tests are too broad or
+tightly coupled across components, they become brittle — a minor change in one
+area can cause unrelated tests to fail.
+
+To ensure reliability and maintainability, unit tests should:
+
+- Run independently of external systems or global state.
+- Use **mocks or stubs** to replace dependencies.
+- Follow the
+ [**Arrange–Act–Assert**](/test_driven_development/arrange_act_assert.md)
+ pattern for clarity.
+- Be **deterministic** and **fast** so they can run often during development.
+
+Writing tests this way builds confidence in each unit, simplifies debugging, and
+supports modular, testable code design.
+
+---
+
+## Enabling CI/CD Test Environments with Isolated Resources
+
+For continuous integration workflows such as **GitHub Actions** or **Forgejo**,
+unit tests should run within a fully isolated and reproducible environment. This
+means the application should be capable of **spinning up a dedicated test
+PostgreSQL instance** and a **test server API** that operates on a separate port
+with its own **test-specific environment variables**.
+
+These configurations ensure that test runs do not interfere with production or
+staging databases. The test infrastructure should start up quickly and shut down
+cleanly as part of the CI/CD pipeline, allowing automated workflows to execute
+the full test suite independently for every build or pull request. This approach
+guarantees repeatable, safe testing while maintaining complete separation
+between test and production systems.
+
+---
+
+### Question: What's the difference between intergation tests and unit tests?
+
+Unit tests focus on small, isolated pieces of code and run very fast, so they
+give precise, quick feedback and make it easy to iterate or refactor without
+breaking unrelated behavior. In contrast, integration tests exercise multiple
+components together (like API, database, and services) to verify real workflows,
+which provides higher confidence that the system behaves correctly as a whole.
+
+The tradeoff is that integration tests are slower, more complex to set up, and
+failures can be harder to diagnose, since a small change in one part of the
+system can break a test somewhere else. Because of this, you typically rely on
+unit tests for rapid development of specific functionality, and use integration
+tests more sparingly to ensure that the integrated application still works end
+to end.
+
+---
+
+## Abstractions and Patterns to Keep Tests Stable
+
+This is where **interfaces** (traits), the facade and adapter patterns,
+dependency injection, and internal mutability come in. When a piece of code is
+wrapped behind a stable trait or adapter interface, the call sites that depend
+on it can remain unchanged, while other developers are free to change the
+internals of the wrapped functions without breaking those callers. This gives
+more confidence in integration testing when integrating with different external
+systems, call sites, or frontends, because the surface contracts stay stable
+even as implementations evolve. These abstraction layers reinforce separation of
+concerns between core functions, business logic, infrastructure, and UI/API
+layers, so each layer can be unit-tested in isolation and then composed
+predictably in integration tests.
diff --git a/web_development/http_headers.md b/web_development/http_headers.md
@@ -0,0 +1,70 @@
+# HTTP Headers
+
+> HTTP headers let the client and the server pass additional information with a
+> message in a request or response.
+
+[HTTP Headers](https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Headers)
+
+### Question: Where exactly is HTTP headers stored?
+
+HTTP headers are not stored in one permanent place; they live briefly in memory
+at each hop (browser, proxies, server) as part of the HTTP message being sent or
+received.
+
+#### Client side:
+
+The browser builds an HTTP request message in its process memeory, which is
+written into a TCP/IP packet buffer sent over the network (nothing is saved on
+disk so to speak). The browser may keep some header in internal data structures
+for cahcing or cookies.
+
+#### Server side:
+
+The web server process (apache2, nginx, node.js) receives the TCP stream and
+parses out the HTTP headers for your web app to use. Default server headers
+cofiguration (apache config, nginx config, .htaccess) do live on disk. Each
+actual header that goes to the client is ephermeral and generated on the fly in
+memory for each response.
+
+#### Proxy and Cache:
+
+CDNs, reverse proxies, or browser's cache store HTTP messages (both headers and
+body) in memory or disk for reuse.
+
+# HTTP Security Headers
+
+[HTTP Security Headers: A complete guide to HTTP headers](https://www.darkrelay.com/post/http-security-headers)
+
+[Analyze your headers](https://securityheaders.com/)
+
+- Access-Control-Allow-Origin Security Header
+
+- Content-Type Header
+
+- Content-Security-Policy (CSP) Security Header
+
+- Cross-Origin-Embedder-Policy Security Header
+
+- Cross-Origin-Resource-Policy Security Header
+
+- Cross-Origin-Opener-Policy Security Header
+
+- Set-Cookie Header
+
+- Strict-Transport-Security (HSTS) Security Header
+
+- Referrer-Policy Header
+
+- X-Content-Type-Options Security Header
+
+- X-Frame-Options Security Header
+
+- X-XSS-Protection Security Header
+
+- X-Permitted-Cross-Domain-Policies Security Header
+
+- Cache-Control Header
+
+- X-Powered-By Header
+
+- Public-Key-Pins(HPKP) header
diff --git a/web_development/index.md b/web_development/index.md
@@ -0,0 +1,2 @@
+# Web Development
+
diff --git a/web_development/openapi-guide.md b/web_development/openapi-guide.md
@@ -0,0 +1,263 @@
+# Open API guide
+
+# Bridging the Type Gap: End-to-End Type Safety with Axum, Aide, and Orval
+
+One of the most frustrating experiences in full-stack development is the "silent
+failure": you change a field name in your Rust backend, everything compiles
+perfectly, but your frontend suddenly starts receiving `undefined` and crashing
+in production.
+
+To solve this, we need a **Single Source of Truth**. Instead of manually
+maintaining TypeScript interfaces that mirror your Rust structs, you can
+automate the entire pipeline:
+
+**Rust Models $\rightarrow$ OpenAPI Spec $\rightarrow$ TypeScript Axios Client**
+
+In this guide, we'll use **`aide`** to generate an OpenAPI specification
+directly from our Axum types and **`orval`** to transform that spec into a
+type-safe Axios client.
+
+---
+
+## Part 1: The Backend (Rust + Axum + Aide)
+
+Traditionally, writing an OpenAPI (Swagger) spec means writing a giant YAML file
+by hand. **`aide`** eliminates this by leveraging Rust's type system to generate
+the spec at runtime.
+
+### 1. Dependencies
+
+Add these to your `Cargo.toml`:
+
+```toml
+[dependencies]
+axum = "0.7" # Ensure version compatibility with aide
+aide = { version = "0.15", features = ["axum"] }
+schemars = "0.8" # Required for generating JSON schemas from structs
+serde = { version = "1.0", features = ["derive"] }
+serde_json = "1.0"
+tokio = { version = "1.0", features = ["full"] }
+```
+
+### 2. The Implementation
+
+The core idea is to replace Axum's standard `Router` with `ApiRouter` and ensure
+your data models implement `JsonSchema`.
+
+```rust
+use aide::{
+ axum::{routing::{get, post}, ApiRouter, IntoApiResponse},
+ openapi::{Info, OpenApi},
+};
+use axum::{Extension, Json};
+use schemars::JsonSchema;
+use serde::{Deserialize, Serialize};
+
+// 1. All models must derive JsonSchema to be visible in the OpenAPI spec
+#[derive(Deserialize, Serialize, JsonSchema)]
+struct User {
+ id: u64,
+ username: String,
+ email: String,
+}
+
+// 2. Handlers must return types that implement IntoApiResponse
+async fn create_user(Json(user): Json<User>) -> impl IntoApiResponse {
+ // In a real app, you'd save to DB here
+ Json(user)
+}
+
+async fn serve_api(Extension(api): Extension<OpenApi>) -> impl IntoApiResponse {
+ Json(api)
+}
+
+#[tokio::main]
+async fn main() {
+ // Define the OpenApi metadata
+ let mut api = OpenApi {
+ info: Info {
+ title: "My Awesome API".to_string(),
+ version: "1.0.0".to_string(),
+ ..Info::default()
+ },
+ ..OpenApi::default()
+ };
+
+ // Use ApiRouter instead of Router
+ let app = ApiRouter::new()
+ // Use api_route to explicitly include the route in the documentation
+ .api_route("/users", post(create_user))
+ // Standard route for the JSON spec itself
+ .route("/api.json", get(serve_api));
+
+ let listener = tokio::net::TcpListener::bind("0.0.0.0:3000").await.unwrap();
+
+ axum::serve(
+ listener,
+ app
+ .finish_api(&mut api) // Finalize the spec generation
+ .layer(Extension(api)) // Inject the spec into the handlers
+ .into_make_service(),
+ )
+ .await
+ .unwrap();
+}
+```
+
+### How it works:
+
+- **`JsonSchema`**: The `schemars` crate analyzes your Rust struct at
+ compile-time and allows `aide` to describe it in JSON format.
+- **`ApiRouter`**: This acts as a wrapper. Whenever you call `.api_route()`,
+ `aide` records the types of the request and response.
+- **`/api.json`**: We expose the generated `OpenApi` struct as a JSON endpoint.
+ This is the bridge to our frontend.
+
+---
+
+## Part 2: The Frontend (TypeScript + Orval)
+
+Now that our backend is shouting its structure via `/api.json`, we use **Orval**
+to listen. Orval reads the OpenAPI spec and generates an entire API
+client—including request functions and TypeScript types—so you never have to
+write a `fetch` or `axios` call manually.
+
+### 1. Install Dependencies
+
+```bash
+npm install axios
+npm install -D orval
+```
+
+### 2. Configure Orval
+
+Create an `orval.config.js` file in your project root:
+
+```javascript
+module.exports = {
+ "my-api": {
+ input: "http://localhost:3000/api.json", // The URL from our Rust app
+ output: {
+ target: "./src/api/generated.ts",
+ client: "axios",
+ override: {
+ axios: {
+ useBaseUrl: true,
+ baseURL: "http://localhost:3000",
+ },
+ },
+ },
+ },
+};
+```
+
+### 3. Generate the Client
+
+Run the generator:
+
+```bash
+npx orval
+```
+
+Orval will now create `src/api/generated.ts`. Inside, you'll find:
+
+1. **TypeScript Interfaces**: Exactly matching the Rust `User` struct.
+2. **Request Functions**: A function like `createUser` that takes a `User`
+ object and returns a Promise of a `User`.
+
+### 4. Use it in your Frontend (Svelte/React/Vue)
+
+Now, your API calls are fully type-safe:
+
+```typescript
+import { createUser } from "./api/generated";
+
+async function handleSignUp(formData: any) {
+ try {
+ // TypeScript will error here if formData doesn't match the Rust User struct!
+ const user = await createUser({
+ id: 1,
+ username: "rust_lover",
+ email: "hello@rust.rs",
+ });
+ console.log("User created:", user.username);
+ } catch (e) {
+ console.error("API Error", e);
+ }
+}
+```
+
+---
+
+## Part 3: Using CI to Keep the Spec Up to Date
+
+To make this architecture truly production-ready, you shouldn't rely on manually
+running `npx orval` every time you change a field in Rust. Instead, you can
+integrate this into your **CI/CD pipeline**.
+
+### The "Golden File" Strategy
+
+Since `aide` generates the specification at runtime, the most robust way to
+handle CI is to treat the OpenAPI JSON as a versioned artifact (a "Golden
+File").
+
+#### 1. Export the Spec during Backend CI
+
+Create a Rust test in your backend that runs the `ApiRouter`, generates the
+`OpenApi` object, and writes it to a file:
+
+```rust
+#[tokio::test]
+async fn export_openapi_spec() {
+ let mut api = OpenApi::default();
+ let app = ApiRouter::new().api_route("/users", post(create_user));
+
+ app.finish_api(&mut api);
+
+ let json = serde_json::to_string_pretty(&api).unwrap();
+ std::fs::write("openapi.json", json).expect("Unable to write spec file");
+}
+```
+
+#### 2. The CI Pipeline (GitHub Actions Example)
+
+Set up a workflow that triggers the frontend whenever the `openapi.json`
+changes:
+
+```yaml
+name: API Type Sync
+on:
+ push:
+ paths:
+ - "backend/**"
+
+jobs:
+ sync-types:
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v4
+
+ - name: Export OpenAPI Spec
+ run: |
+ cargo test --test export_openapi_spec
+
+ - name: Generate Frontend Client
+ run: |
+ cd frontend
+ npm install
+ npx orval
+
+ - name: Type Check Frontend
+ run: |
+ cd frontend
+ npm run type-check # Run 'tsc' to see if API changes broke the UI
+```
+
+### Summary of the Automated Flow
+
+1. **Developer** pushes a change to a Rust struct.
+2. **CI** runs a test to generate the latest `openapi.json`.
+3. **CI** runs Orval to update the TypeScript Axios client.
+4. **CI** runs the TypeScript compiler (`tsc`). If the Rust change broke a
+ frontend component, the **build fails** before the code ever reaches
+ production.
diff --git a/web_development/svelte.md b/web_development/svelte.md
@@ -0,0 +1,28 @@
+# Svelte
+
+## Attachment factories
+
+A useful pattern is for a function, such as tooltip in this example, to return
+an attachment:
+
+```ts
+<script lang="ts">
+ import tippy from 'tippy.js';
+ import type { Attachment } from 'svelte/attachments';
+
+ let content = $state('Hello!');
+
+ function tooltip(content: string): Attachment {
+ return (element) => {
+ const tooltip = tippy(element, { content });
+ return tooltip.destroy;
+ };
+ }
+</script>
+
+<input bind:value={content} />
+
+<button {@attach tooltip(content)}>
+ Hover me
+</button>
+```
diff --git a/web_development/svelte_state.md b/web_development/svelte_state.md
@@ -0,0 +1,124 @@
+# Svelte State Management
+
+## Svelte 4 Writable Store
+
+```ts
+// fooStore.ts
+import { type Writable, writable } from "svelte/store";
+
+function createFoo() {
+ const { subscribe, set, update }: Writable<number> = writable(0);
+ return {
+ subscribe,
+ set: (value: number) => set(value),
+ add: (value: number) => update((x: number) => x + value),
+ triple: () => update((x: number) => x * 3),
+ clear: () => set(0),
+ };
+}
+export const foo = createFoo();
+```
+
+Derived store:
+
+```ts
+// fooStore.ts
+import { derived, writable } from "svelte/store";
+
+export const bar = writable(1);
+export const baz = writable(2);
+export const foo = derived(
+ [bar, baz],
+ ([$bar, $baz]) => Math.min($bar, $baz) / 2,
+);
+```
+
+## Svelte 5 Rune Store
+
+```ts
+// foo.svelte.ts (must be placed inside a *.svelte.ts file)
+function createFoo(initialCount = 0) {
+ let foo = $state(initialCount);
+
+ return {
+ get foo() {
+ return foo;
+ },
+ set foo(value) {
+ foo = value;
+ },
+ add(value) {
+ foo += value;
+ },
+ triple() {
+ foo *= 3;
+ },
+ };
+}
+export const foo = createFoo();
+```
+
+Or class if you prefer:
+
+```ts
+// foo.svelte.ts
+export class Foo {
+ foo = $state(0);
+
+ constructor(initialCount = 0) {
+ this.foo = initialCount;
+ }
+
+ add(value) {
+ this.foo += value;
+ }
+
+ triple() {
+ this.foo *= 3;
+ }
+}
+
+export const foo = new Foo();
+```
+
+Class doesn't need get/set method, just read/write it directly:
+
+```ts
+// In a component:
+// Reading:
+console.log(foo.foo);
+// Writing (triggers UI updates automatically):
+foo.foo = 10;
+```
+
+Derived rune:
+
+```ts
+// foo.svelte.ts (must be placed inside a *.svelte.ts file)
+function createFoo(initialBar = 1, initialBaz = 2) {
+ let bar = $state(initialBar);
+ let baz = $state(initialBaz);
+ let foo = $derived(Math.min(bar, baz) / 2);
+
+ return {
+ get bar() {
+ return bar;
+ },
+ set bar(value) {
+ bar = value;
+ },
+ get baz() {
+ return baz;
+ },
+ set baz(value) {
+ baz = value;
+ },
+ get foo() {
+ return foo;
+ },
+ };
+}
+export const foo = createFoo();
+```
+
+[Different Ways To Share State In Svelte 5](https://joyofcode.xyz/how-to-share-state-in-svelte-5)
diff --git a/web_development/typescript.md b/web_development/typescript.md
@@ -0,0 +1,54 @@
+# TypeScript
+
+Remember, the whole point of using TypeScript is to use its typechecker to stop
+you from doing invalid things.
+
+TypeScript gives you error messages in your text editor, as you type.
+
+But we should use type annotations only when necessary, and let TypeScript work
+its inference magic for us whenever possible.
+
+## Avoid using `any` as type
+
+`any` makes your value behave like it would in regular JavaScript, and totally
+prevents the typechecker from working its magic. When you allow `any` into your
+code you're flying blind. Avoid `any` like fire, and use it only as a very very
+last resort.
+
+## `public` keyword in class constructor
+
+```ts
+class Person {
+ constructor(public firstname: string);
+}
+```
+
+`public` is shorthand for `this.firstName = firstName`
+
+## Index signatures
+
+```ts
+let a: {
+ b: number;
+ c?: string;
+ [key: number]: boolean;
+};
+```
+
+The `[key: T]: U` syntax is called an index signature, and this is the way you
+tell TypeScript that the given object might contain more keys.
+
+```ts
+a = { b: 1, c: "d", 10: true, 20: false };
+```
+
+For this object, all keys of type T must have values of type U.
+
+## Type Alias
+
+Type aliases are useful for DRYing up repreated complex types.
+
+## Arrays
+
+TypeScript supports two syntaxes for arrays: `T[]` and `Array<T>`. They are
+indentical both in meaning and in performance.
diff --git a/web_development/typescript_books.md b/web_development/typescript_books.md
@@ -0,0 +1,89 @@
+# TypeScript Books
+
+TypeScript has become the industry standard for scalable JavaScript development.
+Because the ecosystem is vast, choosing the right learning path depends heavily
+on your current experience level—whether you are a seasoned JavaScript engineer,
+a total beginner to programming, or someone looking for a deep dive into the
+type system.
+
+## The Gold Standard
+
+### Effective TypeScript
+
+**Author:** Dan Vanderkam\
+Widely regarded as the best overall resource for those wanting to write
+"idiomatic" TypeScript. Rather than a linear tutorial, it provides 62 specific
+ways to improve your code. It is particularly strong on type inference and the
+nuances of the type system.
+
+- **Best For:** Developers who want a deep, professional understanding of the
+ type system.
+- **Level:** Beginner to Advanced.
+
+## Path-Specific Recommendations
+
+### For JavaScript Developers: Programming TypeScript
+
+**Author:** Boris Cherny\
+If you are already proficient in JavaScript, this book focuses on the
+transition. It takes a hands-on approach to scaling applications, covering error
+handling, asynchronous patterns, and the practicalities of migrating existing JS
+projects to TS.
+
+- **Best For:** JS veterans moving to TypeScript.
+- **Level:** Intermediate.
+
+### For Absolute Beginners: Mastering TypeScript
+
+Unlike most TS resources, this book does not assume prior JavaScript knowledge.
+It guides the reader from the very basics upward, eventually integrating the
+language with popular frameworks like React, Vue, and Angular.
+
+- **Best For:** Those new to the web development ecosystem.
+- **Level:** Beginner.
+
+## Community & Fast-Track Resources
+
+### TypeScript Deep Dive (Free)
+
+A community favorite available on GitHub and via GitBook. It is often cited as
+the "definitive guide" due to its comprehensiveness, covering generics,
+decorators, and the inner workings of the TypeScript compiler.
+
+- **Resource:** [TypeScript Deep Dive](https://basarat.gitbook.io/typescript/)
+
+### Quick & Structured Learning
+
+- **TypeScript in 50 Lessons**: Ideal for those who prefer bite-sized,
+ digestible content. It breaks the language down into 50 short lessons.
+- **TypeScript Quickly**: A project-oriented book that guides you through
+ building a blockchain service application.
+
+---
+
+## Summary Comparison
+
+| Book/Resource | Primary Focus | Target Audience | Level |
+| :------------------------- | :----------------------- | :----------------- | :---------------------------------- |
+| **Effective TypeScript** | Type system optimization | Professional Devs | Beginner $\rightarrow$ Advanced |
+| **Programming TypeScript** | Scaling JS applications | JS Developers | Intermediate |
+| **Mastering TypeScript** | Foundational concepts | Absolute Beginners | Beginner |
+| **TS Deep Dive** | Comprehensive reference | All levels | All levels |
+| **TS in 50 Lessons** | Short, focused bursts | Quick learners | Beginner $\rightarrow$ Intermediate |
+| **TypeScript Quickly** | Project-based learning | Hands-on learners | Intermediate |
+
+---
+
+## Official Resources
+
+For the most up-to-date specifications and documentation, always refer to the
+official TypeScript project:
+
+- **[Official Documentation](https://www.typescriptlang.org/docs/)**: The
+ primary source for all language features.
+- **[The TypeScript Handbook](https://www.typescriptlang.org/docs/handbook/intro.html)**:
+ The definitive guide to the language's core concepts.
+- **[TypeScript Playground](https://www.typescriptlang.org/play)**: An
+ interactive environment to test type logic in real-time.
+- **[TS Reference](https://www.typescriptlang.org/docs/handbook/tsconfig-json.html)**:
+ Essential guide for `tsconfig.json` configuration.
diff --git a/content/whiteboards/abstraction_of_computer_system.png b/whiteboards/abstraction_of_computer_system.png
Binary files differ.
diff --git a/content/whiteboards/cpu.png b/whiteboards/cpu.png
Binary files differ.
diff --git a/content/whiteboards/data_stream.png b/whiteboards/data_stream.png
Binary files differ.
diff --git a/content/whiteboards/five_abstraction_levels.png b/whiteboards/five_abstraction_levels.png
Binary files differ.
diff --git a/content/whiteboards/five_abstraction_levels.png~ b/whiteboards/five_abstraction_levels.png~
Binary files differ.
diff --git a/whiteboards/index.md b/whiteboards/index.md
@@ -0,0 +1,2 @@
+# Whiteboards
+
diff --git a/content/whiteboards/pipe_between_binaries.png b/whiteboards/pipe_between_binaries.png
Binary files differ.
diff --git a/content/whiteboards/postgres.png b/whiteboards/postgres.png
Binary files differ.
diff --git a/z-md/agent.md b/z-md/agent.md
@@ -0,0 +1,178 @@
+# AGENTS.md
+
+# Agents.md: Senior Rust Developer with Svelte/TypeScript/JavaScript Expertise
+
+## **Role Overview**
+
+As a **Senior Rust Developer with Svelte, TypeScript, and JavaScript
+expertise**, you are responsible for:
+
+- Writing **high-performance, memory-safe, and concurrent** Rust code.
+- Developing **type-safe, scalable, and maintainable** applications using
+ **Svelte, vanilla TypeScript, and vanilla JavaScript**.
+- Ensuring **clean, efficient, and lean** code across the stack.
+- Mentoring junior developers and leading code reviews.
+- Collaborating with cross-functional teams to design and implement robust
+ solutions.
+
+---
+
+## **Core Principles**
+
+### **1. Clean Code**
+
+- **Readability**: Write self-documenting code with meaningful names for
+ variables, functions, and modules.
+- **Consistency**: Follow the project’s style guide (e.g., `rustfmt`, `prettier`
+ for TypeScript/JavaScript).
+- **Simplicity**: Prefer simple solutions over clever ones. Avoid unnecessary
+ abstraction.
+
+### **2. Maintainability**
+
+- **Modularity**: Break down code into small, reusable modules/functions.
+- **Documentation**: Document public APIs, complex logic, and edge cases.
+- **Tests**: Write unit, integration, and end-to-end tests. Aim for high test
+ coverage.
+
+### **3. Efficiency**
+
+- **Performance**: Optimize critical paths in Rust (e.g., zero-cost
+ abstractions, minimal allocations).
+- **Memory Safety**: Leverage Rust’s ownership model to prevent data races and
+ memory leaks.
+- **Type Safety**: Use TypeScript’s type system to catch errors at compile time.
+
+### **4. Lean Code**
+
+- **Minimal Dependencies**: Avoid bloating the project with unnecessary crates
+ or npm packages.
+- **DRY Principle**: Don’t repeat yourself. Reuse logic where possible.
+- **YAGNI**: Avoid over-engineering. Only implement what is needed now.
+
+---
+
+## **Rust Best Practices**
+
+### **Code Structure**
+
+- Use `cargo` workspaces for large projects.
+- Organize code into logical modules (e.g., `src/lib.rs`, `src/bin/`).
+- Prefer `Result` and `Option` over panics for error handling.
+
+### **Concurrency**
+
+- Use `async/await` with `tokio` or `async-std` for async tasks.
+- Leverage `Arc<Mutex<T>>` or `RwLock` for shared state.
+- Avoid locks where possible; prefer message-passing (e.g., `std::sync::mpsc`).
+
+### **Performance**
+
+- Profile with `cargo bench` or `flamegraph`.
+- Use `#[inline]` for hot paths.
+- Avoid unnecessary clones; prefer references (`&T`).
+
+### **Testing**
+
+- Use `cargo test` for unit and integration tests.
+- Test edge cases (e.g., empty inputs, large datasets).
+- Mock external dependencies (e.g., `mockall`).
+
+---
+
+## **Svelte/TypeScript/JavaScript Best Practices**
+
+### **Code Structure**
+
+- Use `eslint` and `prettier` for consistent formatting.
+- Organize code into features/modules (e.g., `src/features/auth/`).
+- Prefer functional programming patterns (e.g., `map`, `filter`, `reduce`).
+
+### **Svelte**
+
+- Use Svelte’s reactive statements (`$:`) for state management.
+- Avoid unnecessary stores; prefer local state where possible.
+- Use Svelte’s built-in transitions and animations for UI enhancements.
+
+### **TypeScript**
+
+- Use `interface` or `type` for complex objects.
+- Avoid `any`; use `unknown` for dynamic data.
+- Leverage generics for reusable components.
+
+### **Vanilla JavaScript**
+
+- Use modern ES6+ features (e.g., `const`, `let`, arrow functions).
+- Prefer modules (`import/export`) over global scope.
+- Avoid side effects; write pure functions where possible.
+
+### **Performance**
+
+- Avoid heavy computations in the main thread.
+- Use `Web Workers` for CPU-intensive tasks.
+- Optimize DOM updates; batch changes where possible.
+
+### **Testing**
+
+- Use `jest` or `vitest` for unit tests.
+- Test Svelte components with `@testing-library/svelte`.
+- Mock API calls (e.g., `msw`).
+
+---
+
+## **Collaboration**
+
+### **Code Reviews**
+
+- Be constructive and specific in feedback.
+- Focus on **correctness**, **readability**, and **performance**.
+- Use GitHub/GitLab review tools for inline comments.
+
+### **Documentation**
+
+- Update `README.md` for project setup and usage.
+- Document public APIs with `///` (Rust) or `JSDoc` (TypeScript/JavaScript).
+- Use `cargo doc` or `TypeDoc` for auto-generated docs.
+
+### **Tools**
+
+- **Rust**: `cargo`, `clippy`, `rust-analyzer`.
+- **TypeScript/JavaScript**: `npm`, `yarn`, `eslint`, `prettier`.
+- **Svelte**: `svelte-check`, `vite`, `svelte-kit`.
+- **CI/CD**: GitHub Actions, GitLab CI.
+
+---
+
+## **Example: Rust + Svelte/TypeScript Project Structure**
+
+```
+project/
+├── rust/
+│ ├── Cargo.toml
+│ ├── src/
+│ │ ├── lib.rs
+│ │ ├── main.rs
+│ │ └── models.rs
+│ └── tests/
+├── frontend/
+│ ├── package.json
+│ ├── src/
+│ │ ├── App.svelte
+│ │ ├── lib/
+│ │ │ ├── components/
+│ │ │ ├── stores/
+│ │ │ └── utils/
+│ │ └── main.ts
+│ ├── static/
+│ └── tests/
+└── README.md
+```
+
+---
+
+## **Final Notes**
+
+- **Stay Updated**: Follow Rust, Svelte, TypeScript, and JavaScript RFCs and
+ best practices.
+- **Share Knowledge**: Write blog posts or give talks on lessons learned.
+- **Feedback Loop**: Continuously improve processes and tools.
