Abhishek Tripathi

HTTP responses can be quite large and memory consumption can be a concern. In some cases, it is important to be able to handle large responses without loading the entire response into memory.

One such scenario is when you want to download a large file from a server. If you were to load the entire file into memory, it would require a large amount of memory and would be inefficient. Instead, you can use a streaming approach to download the file directly to disk.

This example will show you how to do just that using the reqwest and tokio crates (Rust). Here is the rough flow.

Deep Flattening in Rust: A Recursive Adventure​

Flattening nested data structures is a common problem in programming. However, flattening structures with an arbitrary depth—like nested Vecs within Vecs—can be tricky. Rust, with its strong type system and trait-based polymorphism, allows us to implement elegant solutions to such problems. In this post, we'll explore a recursive approach to deep flattening in Rust using traits, type inference, and iterators.

The Goal​

Given a deeply nested structure, such as:

let nested_vec = vec![
    vec![vec![1, 2, 3], vec![4, 5]],
    vec![vec![6], vec![7, 8, 9]],
];

Our goal is to flatten it into:

let flattened = vec![1, 2, 3, 4, 5, 6, 7, 8, 9];

A recent GitHub issue #6751 in the Caddy server repository revealed an interesting performance bottleneck when using multiple layers of reverse proxying. Here's what was discovered and how it was resolved.

The 1 Billion Row Challenge (1BRC) is a programming challenge focused on processing a large dataset of temperature measurements. If you're unfamiliar with it, you can learn more from these resources: 1 and 2.

This is a cheatsheet of optimisations done for 1brc challenges. It tries to summarise and put the optimisations in perspective.