notes

dynamic_libraries.md (10953B)

---

 # Dynamic Libraries
 
 A **dynamic library** is a compiled binary that is loaded into a program's
 address space at either load time or runtime, rather than being copied into the
 final executable by the linker. On Linux these files typically have a `.so`
 extension (shared object), on Windows `.dll` (dynamic-link library), and on
 macOS `.dylib` (dynamic library).
 
 Dynamic libraries enable **code sharing** between multiple processes: the
 operating system maps a single physical copy of the library into the virtual
 address space of every process that needs it. This reduces total memory
 footprint and allows a library to be updated independently of the executables
 that depend on it.
 
 ## Dynamic vs. Static Libraries
 
 | Feature | Static Library | Dynamic Library |
 | :------ | :------------- | :---------------- |
 | Linking phase | Compile / link time | Load time or runtime |
 | Binary size | Larger executable (library copied in) | Smaller executable (reference only) |
 | Memory sharing | Each process gets its own copy | OS shares one physical copy |
 | Updates | Recompile executable to update library | Replace library file, restart process |
 | Portability | Self-contained executable | Requires compatible library present at runtime |
 
 A static library (`.a` on Unix, `.lib` on Windows) is essentially an archive of
 object files. The linker extracts the needed object files and copies them into
 the final executable. Once linked, the static library is no longer needed to run
 the program.
 
 A dynamic library remains a separate file on disk. The executable contains a
 **reference** to the library — typically a recorded name and a symbol table of
 needed functions — and the operating system's dynamic loader resolves that
 reference when the process starts (load-time linking) or when the program
 explicitly requests it (runtime linking).
 
 ## How the OS Loads Dynamic Libraries
 
 When a program starts, the operating system's dynamic loader (e.g., `ld.so` on
 Linux, `dyld` on macOS, or the Windows loader) inspects the executable's
 **interpreter** and **dynamic section** to determine which shared libraries are
 required. It performs the following steps:
 
 1. **Dependency resolution** — read the list of needed libraries from the
    executable headers.
 2. **Library search** — locate each library on the search path (`LD_LIBRARY_PATH`,
    system cache `/etc/ld.so.cache`, `rpath`, `runpath`, or default system
    directories).
 3. **Loading and mapping** — `mmap` the library into the process's address space.
 4. **Symbol resolution** — walk the relocation tables and patch addresses so
    that function calls in the executable point to the correct offsets in the
    loaded library.
 5. **Initialization** — run constructor functions (e.g., `__attribute__((constructor))`
    in C) registered in the library.
 
 This process is known as **dynamic linking**. If a required library cannot be
 found, the loader aborts and the program fails to start.
 
 ## Runtime Loading with `dlopen` and Friends
 
 Programs can also load libraries explicitly after they have already started.
 This is **runtime dynamic linking** and is the mechanism behind plugin systems,
 extensible applications, and language interpreters that load native extensions.
 
 On POSIX systems the C standard library provides four key functions:
 
 - `dlopen(path, flags)` — load a shared object into the current address space
 - `dlsym(handle, symbol)` — retrieve the address of a named symbol (function or
   variable)
 - `dlclose(handle)` — decrement the reference count and possibly unload the library
 - `dlerror()` — return a human-readable string describing the last error
 
 On Windows the analogous APIs are `LoadLibraryA`, `GetProcAddress`, and
 `FreeLibrary`.
 
 ### Example: A Minimal Dynamic C Library
 
 Imagine a small C library that computes a checksum. Save this as `checksum.c`:
 
 ```c
 #include <stdint.h>
 
 uint32_t checksum(const uint8_t *data, size_t len) {
     uint32_t sum = 0;
     for (size_t i = 0; i < len; i++) {
         sum = (sum << 1) | (sum >> 31); // rotate left
         sum += data[i];
     }
     return sum;
 }
 ```
 
 Compile it into a shared object on Linux:
 
 ```bash
 gcc -shared -fPIC -o libchecksum.so checksum.c
 ```
 
 The `-fPIC` flag tells the compiler to emit **position-independent code** —
 machine code that can execute correctly regardless of where in memory it is
 mapped. This is mandatory for shared libraries because the OS may load them at
 different base addresses in different processes (or in the same process across
 restarts) for security reasons such as ASLR (Address Space Layout Randomization).
 
 ## Loading a Dynamic C Library from Rust
 
 Rust can interact with dynamic C libraries through two mechanisms:
 
 1. **Compile-time dynamic linking** — declare `#[link(name = "checksum")]` and
    let the Rust linker record a dependency on `libchecksum.so`. The OS loader
    resolves it automatically when the program starts.
 2. **Runtime dynamic loading** — use a crate such as `libloading` to `dlopen`
    the library manually and look up symbols on demand.
 
 Runtime loading is more flexible because the program can decide at execution time
 whether to load a library, handle failures gracefully, and even swap
 implementations without restarting.
 
 ### Using `libloading`
 
 Add `libloading` to your `Cargo.toml`:
 
 ```toml
 [dependencies]
 libloading = "0.8"
 ```
 
 Then load the library and call its function:
 
 ```rust
 use libloading::{Library, Symbol};
 use std::ffi::c_void;
 
 type ChecksumFn = unsafe extern "C" fn(*const u8, usize) -> u32;
 
 fn main() -> Result<(), Box<dyn std::error::Error>> {
     unsafe {
         let lib = Library::new("./libchecksum.so")?;
 
         let checksum: Symbol<ChecksumFn> = lib.get(b"checksum\0")?;
 
         let data = b"hello dynamic world";
         let result = checksum(data.as_ptr(), data.len());
 
         println!("Checksum: {}", result);
     }
 
     Ok(())
 }
 ```
 
 Key observations about this code:
 
 - The `unsafe` block is required because the compiler cannot verify the
   correctness of a C library's ABI, pointer usage, or thread safety.
 - The symbol name `b"checksum\0"` is a null-terminated byte string, matching the
   C ABI expectation.
 - `Library::new` calls `dlopen` (or `LoadLibrary` on Windows) under the hood.
 - `Symbol` is essentially a smart pointer to a function loaded from a
   [**vtable**](/compiler/vtable.md)-like structure inside the dynamic loader.
 
 ### Safety and ABI Mismatches
 
 When calling into a dynamic C library, Rust's usual memory-safety guarantees do
 not apply across the FFI boundary. The C library operates on raw pointers and
 expects a specific **application binary interface (ABI)**. If the Rust side
 misdeclares a function signature — for example, using `u64` instead of `usize`,
 or omitting `extern "C"` — the behavior is undefined and may result in a
 [segfault](/memory_safety/segfault.md) or silent data corruption.
 
 Best practices for dynamic C library interop:
 
 - Define a thin, audited FFI module that mirrors the C headers exactly.
 - Use `std::os::raw` or the `libc` crate for C types (`c_int`, `c_char`, etc.).
 - Keep `unsafe` blocks as small as possible; validate inputs before crossing the
   boundary.
 - Never pass Rust references (`&T`) directly to C code expecting mutable access
   unless you have proven alias safety manually.
 
 ## Relocation and the Global Offset Table
 
 Because a shared library can be loaded at any address, it cannot contain
 absolute addresses for its own functions or global data. Instead, the compiler
 emits **position-independent code** that refers to a **Global Offset Table
 (GOT)** and a **Procedure Linkage Table (PLT)**.
 
 - **GOT** — an array of pointers to global data. The code reads data indirectly
   through the GOT so that only the table entries need to be patched at load time,
   not every instruction that references the data.
 - **PLT** — a trampoline for external function calls. The first time a function
   is called through the PLT, the dynamic linker resolves the real address and
   patches the GOT entry; subsequent calls jump directly to the resolved target.
 
 This lazy resolution (often called **lazy binding**) improves startup time for
 large programs with many shared libraries because not every symbol needs to be
 resolved immediately.
 
 ## Rust as a Dynamic Library Producer
 
 Rust can also produce dynamic libraries for consumption by other languages. Two
 crate types are relevant:
 
 - **`cdylib`** — produces a C-compatible dynamic library (`.so`, `.dll`, `.dylib`).
   Use this when a C program (or Python via ctypes, or another language) needs to
   load your Rust code dynamically. The Rust compiler strips Rust-specific metadata
   and exports only functions marked `#[no_mangle]` and `pub extern "C"`.
 - **`dylib`** — produces a Rust-native dynamic library. This is primarily used
   for rustc plugins and is rarely appropriate for general FFI.
 
 Example `Cargo.toml` for a C-callable Rust library:
 
 ```toml
 [package]
 name = "rust_checksum"
 version = "0.1.0"
 edition = "2021"
 
 [lib]
 crate-type = ["cdylib"]
 ```
 
 And the corresponding Rust source:
 
 ```rust
 #[no_mangle]
 pub extern "C" fn rust_checksum(data: *const u8, len: usize) -> u32 {
     if data.is_null() {
         return 0;
     }
     let slice = unsafe { std::slice::from_raw_parts(data, len) };
     slice.iter().fold(0u32, |acc, &b| acc.wrapping_mul(31).wrapping_add(b as u32))
 }
 ```
 
 Other languages can now `dlopen` the resulting `.so` and call `rust_checksum`
 just as if it were a C function.
 
 ## When to Use Dynamic Libraries
 
 Prefer dynamic libraries when:
 
 - Multiple executables share the same code and you want to reduce disk and
   memory usage.
 - You need a plugin architecture where third-party code is loaded at runtime.
 - You want to update a library (e.g., a security patch) without rebuilding every
   dependent executable.
 - You are shipping a large framework and want consumers to link against a stable
   ABI without caring about your internal implementation details.
 
 Prefer static linking when:
 
 - You need a self-contained binary that runs without external dependencies
   (e.g., containers, embedded systems, or CLI tools distributed to unknown
   environments).
 - Maximum startup performance is critical and you want to avoid symbol
   resolution overhead.
 - You want whole-program optimization (LTO) to inline across library boundaries.
 
 ## Summary
 
 Dynamic libraries are separate binaries loaded by the OS loader either at
 process startup or on demand via `dlopen` / `LoadLibrary`. They reduce memory
 usage through sharing and enable runtime extensibility, but they introduce
 complexities around symbol resolution, ABI stability, and distribution.
 
 When loading a dynamic C library from Rust, you cross an **FFI boundary** where
 the compiler's usual safety checks no longer apply. Tools like `libloading`
 make runtime loading ergonomic, but correctness depends on matching the C ABI
 exactly and carefully auditing every `unsafe` call site.