Algorithms on Collections: Sort, Filter, Dedup, and the Art of Data Transformation

There's a moment in every programmer's career when they stop writing for loops for every data transformation and start chaining iterator methods instead. It feels wrong at first—like driving an automatic after years of manual transmission. "Where's the control?" you think. "Where's the loop variable? Where's the index? How do I know what's happening?"

And then you write your first .filter(|x| x > &threshold).map(|x| x * 2).collect::<Vec<_>>() chain, and it compiles, and it runs, and it produces the right answer, and you realize: the code reads like a description of what you want. Filter the values above the threshold. Double each one. Collect the results. No loop variable to manage. No index to get wrong. No off-by-one error lurking in a boundary condition. Just a pipeline of transformations, each one clear, each one composable, each one zero-cost.

Rust's collection algorithms—sort, filter, dedup, map, collect, and dozens more—transform how you write data processing code. They're not just syntactic sugar for loops (though they look like it). They're declarative descriptions of transformations that the compiler optimizes into code as fast as hand-written loops—and often faster, because the compiler has more information about the structure of the computation when it's expressed as an iterator chain rather than a manual loop.

At RantAI, our data pipelines are built on these operations. Sensor data arrives, gets filtered, normalized, sorted, deduplicated, and collected—all as iterator chains that read like specifications of what the pipeline does. This article, drawn from Chapter 4, Section 4.5 of our guide "The Rust Programming Language," covers the essential algorithms and shows you how to compose them into powerful data transformation pipelines.

Sorting: Order from Chaos#

fn main() {
    let mut numbers = vec![3, 1, 4, 1, 5, 9, 2, 6, 5, 3];

    numbers.sort();
    println!("Sorted: {:?}", numbers);  // [1, 1, 2, 3, 3, 4, 5, 5, 6, 9]

    // Sort by custom criteria
    let mut words = vec!["banana", "apple", "cherry", "date"];
    words.sort_by(|a, b| a.len().cmp(&b.len()));  // Sort by length
    println!("By length: {:?}", words);  // ["date", "apple", "banana", "cherry"]

    // Sort by key — cleaner syntax for single-field sorting
    let mut names = vec!["Charlie", "Alice", "Bob"];
    names.sort_by_key(|name| name.len());
    println!("By name length: {:?}", names);  // ["Bob", "Alice", "Charlie"]

    // Reverse sort
    let mut descending = vec![1, 5, 3, 8, 2];
    descending.sort_by(|a, b| b.cmp(a));  // Reverse comparison
    println!("Descending: {:?}", descending);  // [8, 5, 3, 2, 1]
}

sort() sorts in place—no new allocation. sort_by takes a comparator closure for custom ordering. sort_by_key extracts a sortable key from each element. All three are stable sorts (equal elements maintain their relative order) and run in O(n log n) time.

For floating-point data, use sort_by with partial_cmp since floats don't implement Ord (because NaN is weird):

let mut floats = vec![3.14, 1.41, 2.72, 0.577];
floats.sort_by(|a, b| a.partial_cmp(b).unwrap());

Filtering: Keeping What Matters#

fn main() {
    let numbers = vec![1, 2, 3, 4, 5, 6, 7, 8, 9, 10];

    let evens: Vec<i32> = numbers.iter()
        .filter(|&&x| x % 2 == 0)
        .copied()
        .collect();
    println!("Evens: {:?}", evens);  // [2, 4, 6, 8, 10]

    // In-place filtering with retain (modifies the original)
    let mut data = vec![10, -5, 20, -3, 15, -8];
    data.retain(|&x| x > 0);
    println!("Positive only: {:?}", data);  // [10, 20, 15]
}

filter creates a lazy iterator—elements are filtered on demand, not all at once. retain modifies the vector in place, removing elements that don't match the predicate. Use filter when you want a new collection; use retain when you want to modify the existing one.

Deduplication: Eliminating Redundancy#

fn main() {
    // dedup removes CONSECUTIVE duplicates (sort first for full dedup)
    let mut data = vec![1, 1, 2, 3, 3, 3, 2, 1, 1];
    data.dedup();
    println!("After dedup: {:?}", data);  // [1, 2, 3, 2, 1] — not fully unique!

    // Full deduplication: sort + dedup
    let mut full = vec![3, 1, 4, 1, 5, 9, 2, 6, 5, 3];
    full.sort();
    full.dedup();
    println!("Unique sorted: {:?}", full);  // [1, 2, 3, 4, 5, 6, 9]

    // Or use HashSet for unordered deduplication
    use std::collections::HashSet;
    let unique: HashSet<i32> = vec![3, 1, 4, 1, 5, 9, 2, 6, 5, 3].into_iter().collect();
    println!("Unique (unordered): {:?}", unique);
}

The important gotcha: dedup() only removes consecutive duplicates. If your data isn't sorted, you need sort() first, or use a HashSet for unordered deduplication. This catches people who assume dedup works like Python's set().

Chaining: The Power of Composition#

fn main() {
    let raw_data = vec![
        "  Alice  ", "bob", "  Charlie ", "alice", "BOB", "Diana", "bob",
    ];

    // Clean, normalize, deduplicate, sort — all in one chain
    let mut processed: Vec<String> = raw_data.iter()
        .map(|s| s.trim().to_lowercase())   // Clean whitespace, normalize case
        .collect::<HashSet<_>>()             // Deduplicate
        .into_iter()
        .collect();
    processed.sort();                         // Sort alphabetically

    println!("{:?}", processed);  // ["alice", "bob", "charlie", "diana"]
}

This chain reads like a recipe: trim and lowercase each name, deduplicate, sort. Each step is a single operation. The intent is crystal clear. And the compiler optimizes the entire chain into efficient machine code with minimal allocations.

Real-World Pipeline: Processing Measurement Data#

fn main() {
    let measurements = vec![23.1, -1.0, 24.5, 999.9, 22.8, -0.5, 25.0, 23.7, 1000.0];

    let valid_readings: Vec<f64> = measurements.iter()
        .copied()
        .filter(|&x| x > 0.0 && x < 100.0)  // Remove invalid readings
        .collect();

    let count = valid_readings.len();
    let sum: f64 = valid_readings.iter().sum();
    let avg = sum / count as f64;
    let min = valid_readings.iter().cloned().reduce(f64::min).unwrap();
    let max = valid_readings.iter().cloned().reduce(f64::max).unwrap();

    println!("Valid readings: {}", count);
    println!("Average: {:.1}", avg);
    println!("Range: {:.1} - {:.1}", min, max);
}

Filter invalid readings, compute statistics on the clean data. This is the kind of pipeline that would be a 20-line for loop with manual min/max tracking and edge case handling. With iterator methods, it's clear, correct, and resistant to off-by-one errors.

Broader Implications: Declarative Data Processing#

At RantAI, we treat iterator chains as the "specification language" for data transformations. When a data scientist says "filter outliers, normalize the values, and compute the moving average," we can almost translate that sentence directly into an iterator chain. The code is the specification. There's no gap between what we want and what the code says.

This approach also makes code reviews dramatically easier. Instead of tracing through loop variables, index manipulation, and mutable accumulators, reviewers read a sequence of named operations: filter, map, sort, dedup, collect. Each operation has a clear contract. The chain as a whole is verifiable by reading top-to-bottom.

Practical Applications & Strategic Takeaways#

For newcomers: Start replacing simple for loops with .iter().filter().map().collect(). The pattern becomes natural within a day, and your code becomes more readable immediately.

For data engineers: Chain operations to build data pipelines. sort + dedup for deduplication. filter + map + collect for transformation. fold for aggregation. These compose naturally into complex pipelines.

For performance engineers: Iterator chains are lazy—nothing is computed until collect() or another terminal operation. The compiler fuses the chain into a single pass over the data. This is often faster than multiple separate loops because it reduces memory traffic.

Our Commitment to Open Knowledge#

RantAI is committed to open education. Collection algorithms are covered in Chapter 4, Section 4.5 of our guide, "The Rust Programming Language," freely available online.

Explore these concepts further: https://trpl.rantai.dev

Support Our Mission & Get Your Handbook#

• Get the Handbook on Amazon KDP: https://www.amazon.com/dp/B0DHCMD3F2

• Get the Handbook on Google Play Books: https://play.google.com/store/books/details?id=INwfEQAAQBAJ

What's the longest iterator chain you've written? Or the moment when chaining replaced a multi-loop monstrosity? Share your data pipeline victories below!

#RustLang #Iterators #DataProcessing #FunctionalProgramming #RantAI #LearnRust #Sort #Filter #Algorithms #ZeroCost