Returning to Rust: a small beginning

After a long break from Rust, it can feel as though everything needs to be learned again. One smaller starting point is representing tokens for a tiny expression language.

Name the possibilities

A token might be a number, a variable name or an operator. Rust’s enum groups these possibilities into one type.

#[derive(Debug, PartialEq)]
enum Token {
    Number(f64),
    Identifier(String),
    Plus,
    Star,
}

Number carries a number and Identifier carries text. Plus and Star need no additional data. We do not need a separate string field to track the kind of token.

Consider every case with match

fn describe(token: &Token) -> String {
    match token {
        Token::Number(value) => format!("Number: {value}"),
        Token::Identifier(name) => format!("Name: {name}"),
        Token::Plus => "Addition".to_owned(),
        Token::Star => "Multiplication".to_owned(),
    }
}

The function receives &Token: it looks at the token without taking ownership. It produces a new, owned String. Adding a Minus variant later would make the compiler remind us to update this match.

Start with owned text

With Identifier(String), each name owns its text. This can make the first data model easier to understand before introducing lifetime relationships. Creating the name may require an allocation or a copy.

A later experiment could borrow &str slices from the source. Then the source’s lifetime limits how long the tokens may live. Both designs have reasons behind them; starting with the easier model can help us understand the problem.

A small exercise

1. Add Minus to the Token enum.
2. Read the compiler error for describe.
3. Add the missing match arm.
4. Print the result of describe(&Token::Number(14.0)).

The goal is to connect enum, match and borrowing in one example.

Vocabulary

Borrow: access a value without taking ownership. Variant: one possible form of an enum.

The function borrows a token and returns an owned string.

Further reading: The Rust Book — Enums and Pattern Matching.