Rethinking Systems Programming

View Github Repository Open presentation in a new window

thoughtram

See all presentation from thoughtram

Rethinking Systems Programming

0 3

rust-and-nickel

Slides for the Rust & Nickel talk

On Github thoughtram / rust-and-nickel

Rethinking Systems Programming

By Christoph Burgdorf

Christoph Burgdorf

@cburgdorf

DANGER ZONE

You're gonna learn from a Rust noob

3 Topics to talk about

  • Why Rust
  • Rust Features
  • Nickel.rs

Why Rust?

Certain areas need C/C++

  • Operating Systems
  • Browsers
  • Gaming

C/C++'s Offer

  • Direct Memory Access
  • Zero Cost Abstractions
  • No Garbage Collection
  • Compiled

What's wrong with C/C++

  • Manual memory management
  • Segfaults
  • Language Flaws
  • Data Races
Control / Performance Safety C C++ Go Java ML Haskell

"Rust is a systems programming language that runs blazingly fast, prevents almost all crashes*, and eliminates data races."

* In theory. Rust is a work-in-progress and may do anything it likes up to and including eating your laundry.

Control / Performance Safety C C++ Rust Go Java ML Haskell

Tell me more!

Rust governance

  • Created by Mozilla
  • Open Source posterchild
  • Lots of external contributors
  • RFC Process

The Zen of Rust

  • Modern C/C++ replacement
  • Memory Safety without Garbage Collection
  • No runtime needed
  • Rich Typesystem
  • Strong Functional Programming influence

Rust features

Ownership

Garbage Collection

class Circle {
  constructor(radius) {
    this.radius = radius;
  }

  calcArea () {
    return Math.PI * this.radius * this.radius;
  }
}

Garbage Collection

function calcCircles(numberOfCircles, radius) {
  var area = 0;
  for (var i = 0; i < numberOfCircles; i++) {
    let circle = new Circle(radius);
    area += circle.calcArea();
  }
  return area;
}

calcCircles(3, 5);
            Heap Memory
┌─────────────────────────────────┐
│                                 │
├─────────────────────────────────┤
│                                 │
├─────────────────────────────────┤
│                                 │
├─────────────────────────────────┤
│                                 │
└─────────────────────────────────┘

Garbage Collection

function calcCircles(numberOfCircles, radius) {
  var area = 0;
  for (var i = 0; i < numberOfCircles; i++) {
    let circle = new Circle(radius);
    area += circle.calcArea();
  }
  return area;
}

calcCircles(3, 5);
            Heap Memory
┌─────────────────────────────────┐
│      new Circle (radius)        │
├─────────────────────────────────┤
│                                 │
├─────────────────────────────────┤
│                                 │
├─────────────────────────────────┤
│                                 │
└─────────────────────────────────┘

Garbage Collection

function calcCircles(numberOfCircles, radius) {
  var area = 0;
  for (var i = 0; i < numberOfCircles; i++) {
    let circle = new Circle(radius);
    area += circle.calcArea();
  }
  return area;
}

calcCircles(3, 5);
            Heap Memory
┌─────────────────────────────────┐
│      new Circle (radius)        │
├─────────────────────────────────┤
│      new Circle (radius)        │
├─────────────────────────────────┤
│                                 │
├─────────────────────────────────┤
│                                 │
└─────────────────────────────────┘

Garbage Collection

function calcCircles(numberOfCircles, radius) {
  var area = 0;
  for (var i = 0; i < numberOfCircles; i++) {
    let circle = new Circle(radius);
    area += circle.calcArea();
  }
  return area;
}

calcCircles(3, 5);
            Heap Memory
┌─────────────────────────────────┐
│      new Circle (radius)        │
├─────────────────────────────────┤
│      new Circle (radius)        │
├─────────────────────────────────┤
│      new Circle (radius)        │
├─────────────────────────────────┤
│                                 │
└─────────────────────────────────┘

Garbage Collection

function calcCircles(numberOfCircles, radius) {
  var area = 0;
  for (var i = 0; i < numberOfCircles; i++) {
    let circle = new Circle(radius);
    area += circle.calcArea();
  }
  return area;
}

calcCircles(3, 5);
//some time later

            Heap Memory
┌─────────────────────────────────┐
│                                 │
├─────────────────────────────────┤
│                                 │
├─────────────────────────────────┤
│                                 │
├─────────────────────────────────┤
│                                 │
└─────────────────────────────────┘

Eventually GC cleans it up

Consider this JS Code

/*
var vehicle = {
  name: 'Schoolbus',
  passenger: []
};*/
var vehicle = getVehicle();
var bookingService = new BookingService(vehicle);
var repairService = new RepairService(vehicle);

Now both bookingService and repairService hold a reference to vehicle

Same thing in Rust

let vehicle = get_vehicle();
let booking_service = BookingService::new(vehicle);
let repair_service = RepairService::new(vehicle);

Doesn't compile!

error: use of moved value: `vehicle`
RepairService::new(vehicle);
                   ^~~~~~~
note: `vehicle` moved here because it has type `Vehicle`,
which is non-copyable BookingService::new(vehicle);

But why???

Ownership

  • Rust moves ownership by default
  • The owner has the right to destroy the thing it owns
  • The memory is freed as soon as the owned variable goes out of it's scope
  • Hence vehicle may already be destroyed at the point when it's passed to repair_service
  • Rust catches these errors at compile time

How am I supposed to write any code?!

You can lend things out

let vehicle = get_vehicle();
let booking_service = BookingService::new(&vehicle);
let third_service = ThirdService::new(&vehicle);

Borrowing

  • A reference is passed without transfering ownership
  • One can borrow immutable (&) or mutable (&mut) but not both at the same time
  • Borrowing is more polite than taking ownership
  • Shared ownership can be achieved through special pointers with runtime checks

Safe memory management without GC

Structs

struct Dog;

Structs are like lightweight classes

struct Dog {
    name: String,
    age: i32
}

They may have fields

let yeti = Dog {
    name: "Yeti".to_owned(),
    age: 15
}

They can be instanziated

let yeti = Dog {
    name: "Yeti".to_owned()
}
error: missing field: `age` [E0063]
let yeti = Dog {
    name: "Yeti".to_owned()
};

But not partially 

struct Dog {
    name: String,
    age: Option<i32>
}

Rust has the Option enum for that

let yeti = Dog {
    name: "Yeti".to_owned(),
    age: None
};

Which makes it explicit (No NullPointerExceptions!)

struct Dog {
    name: String
}

impl Dog {
  fn greet (&self) {
    println!("My name is {}", self.name);
  }
}

let banjo = Dog { name: "Banjo".to_owned() };
banjo.greet();

They can have methods

Traits

trait Talk {
    fn talk (&self);
}

Traits are like interfaces. They define a contract.

struct Human;
struct Dog;
struct Tree;

impl Talk for Human {
    fn talk (&self) { println!("blabla"); }
}

impl Talk for Dog {
    fn talk (&self)  { println!("bark bark"); }
}
fn talk(talkable: &Talk) {
    talkable.talk();
}
fn main() {
    let person = Human;
    let dog = Dog;
    let tree = Tree;
    talk(&person);
    talk(&dog);
    //compile time error since Tree doesn't impl Talk
    //talk(&tree);
}

trait Talk {
    fn talk (&self) {
        println!("strange noises");
    }
}

impl Talk for Cat {}

Traits can have default implementations

impl Talk for i32 {
    fn talk (&self) { println!("shifting bits around"); }
}

fn talk(talkable: &Talk) {
    talkable.talk();
}

fn main() {
    talk(&3);
}

Traits can be written for foreign types!

Isn't that wild west crazy?!

Trait limitations

  • Either the trait or the type you're writing the impl for must be inside your crate
  • Traits must be brought into scope with use

Generics

struct Person;
struct Car;
struct SmartPersonStorage;

impl SmartPersonStorage {
    fn add (&self, item: Person) { /*code*/ }
    fn get (&self, id: i32) -> Person { /*code*/ }
}

What if we need a SmartCarStorage, too?

struct Person;
struct Car;
struct SmartPersonStorage;
struct SmartCarStorage;

impl SmartPersonStorage {
    fn add (&self, item: Person) { /*code*/ }
    fn get (&self, id: i32) -> Person { /*code*/ }
}
impl SmartCarStorage {
    fn add (&self, item: Car) { /*code*/ }
    fn get (&self, id: i32) -> Car { /*code*/ }
}

Give up DRY

:( sad face

struct Person;
struct Car;
struct SmartStorage;

impl SmartStorage {
    fn add (&self, item: &Any) { /*code*/ }
    fn get (&self, id: i32) -> &Any { /*code*/ }
}

Give up type safety

:( sad face

struct Person;
struct Car;
struct SmartStorage;

impl SmartStorage<T> {
    fn add (&self, item: T) { /*code*/ }
    fn get (&self, id: i32) -> T { /*code*/ }
}

Give up nothing

:) happy face

struct Person;
struct Car;
struct SmartStorage;

impl SmartStorage<T: HasId> {
    fn add (&self, item: T) { /*code*/ }
    fn get (&self, id: i32) -> T { /*code*/ }
}

Can use trait bounds

 

Traits + Generics = <3

Let's refactor some earlier code! 

fn talk(talkable: &Talk) {
    talkable.talk();
}
fn main() {
    let person = Human;
    let dog = Dog;
    talk(&person);
    talk(&dog);
}

Can pass any reference to anything that implements the Talk trait 

struct Human;
struct Dog;

impl Talk for Human {
    fn talk (&self) { println!("blabla"); }
}

impl Talk for Dog {
    fn talk (&self)  { println!("bark bark"); }
}

Each struct has it's own talk implementation 

fn talk(talkable: &Talk) {
    talkable.talk();
}
fn main() {
    let person = Human;
    let dog = Dog;
    talk(&person);
    talk(&dog);
}

 

fn talk(talkable: &Talk) {
    talkable.talk();
}
fn main() {
    let person = Human;
    let dog = Dog;
    talk(&person);
    talk(&dog);
}

Which implementation of talk() to call here?

It has to be looked up in a vtable at run time

You're talking about dynamic dispatch, right?!

Let's refactor that

fn talk(talkable: &Talk) {
    talkable.talk();
}

fn main() {
    let person = Human;
    let dog = Dog;
    talk(&person);
    talk(&dog);
}

 

fn talk<T: Talk>(talkable: T) {
    talkable.talk();
}

fn main() {
    let person = Person;
    let dog = Dog;
    talk(person);
    talk(dog);
}

 

fn talk<T: Talk>(talkable: T) {
    talkable.talk();
}

fn main() {
    let person = Person;
    let dog = Dog;
    talk(person);
    talk(dog);
}

talk is now generic with a Talk trait bound 

fn talk_person(talkable: Person) {
    talkable.talk();
}

fn talk_dog(talkable: Dog) {
    talkable.talk();
}
fn main() {
    let person = Person;
    let dog = Dog;
    talk_person(person);
    talk_dog(dog);
}

The compiler generates specialized code

That's why they call it static dispatch!

Zero-cost abstractions

enums

(called the "sum type" of algebraic data types) 

enum Error {
    Critical,
    NonCritical
}

Error can either be Critical or NonCritical at any one time 

fn log_error(error:Error) {
    match error {
        Error::Critical => println!("Critical error happened"),
        Error::NonCritical => println!("Relax, it's probably fine ;)")
    }
}

fn main() {
    log_error(Error::Critical);
    log_error(Error::NonCritical);
}

enum HttpError {
    WithCode(i32),
    WithMessage(&'static str),
    WithCodeAndMessage(i32, &'static str)
}

Variants may contain data of any other (mixed) types 

fn log_http_error(error:HttpError) {
    match error {
        HttpError::WithCode(code) => println!("error with id {} occured", code),
        HttpError::WithMessage(msg) => println!("error with message {} occured", msg),
        HttpError::WithCodeAndMessage(code, msg) => println!("error with code {} and message {} occured", code, msg)
    }
}

fn main() {
    log_http_error(HttpError::WithCode(404));
    log_http_error(HttpError::WithMessage("Service unavailable"));
}

which we have access to via pattern matching 

enum Result<T, E> {
    Ok(T),
    Err(E),
}

Types can be generic 

match File::open("foo.txt") {
    Ok(file) => {/*do something with file */},
    Err(error) => {/* handle error (io::Error)*/}
}
match json::decode(json_str)) {
    Ok(model) => {/*do something with model*/},
    Err(error) => {/*handle error (json::DecoderError)*/}
}

In fact the std library uses generic enums a lot 

impl<T, E> Result<T, E> {
    fn is_err(&self) -> bool {
        !self.is_ok()
    }
}

Enums can implement methods 

impl <T: Clone, E: Clone> Clone for Result<T, E>  {
  fn clone(&self) -> Result<T, E> {
    match &*self {
      &Result::Ok(ref val) => Result::Ok(val.clone()),
      &Result::Err(ref val) => Result::Err(val.clone()),
    }
  }
}

Enums can implement traits

Macros

macro_rules! foo {
    (x => $e:expr) => (println!("mode X: {}", $e));
    (y => $e:expr) => (println!("mode Y: {}", $e));
}

fn main() {
    foo!(y => 3);
}

Macros allow us to abstract at a syntactic level

Nickel.rs

Nickel in a nutshell

  • Thin layer on top of pure http
  • Inspired by express.js
  • Extensible via custom middleware

hello world example

#[macro_use]
extern crate nickel;

use nickel::Nickel;
use nickel::router::http_router::HttpRouter;

fn main() {
    let mut server = Nickel::new();
    server.get("/hello", middleware!("hello world"));
    server.listen("127.0.0.1:6767");
}
#[macro_use]
extern crate nickel;

use nickel::Nickel;
use nickel::router::http_router::HttpRouter;

fn main() {
    let mut server = Nickel::new();
    server.get("/hello", middleware!("hello world"));
    server.listen("127.0.0.1:6767");
}

Tells the compiler to include macros from nickel crate 

#[macro_use]
extern crate nickel;

use nickel::Nickel;
use nickel::router::http_router::HttpRouter;

fn main() {
    let mut server = Nickel::new();
    server.get("/hello", middleware!("hello world"));
    server.listen("127.0.0.1:6767");
}

Tells the compiler to include the nickel crate 

#[macro_use]
extern crate nickel;

use nickel::Nickel;
use nickel::router::http_router::HttpRouter;

fn main() {
    let mut server = Nickel::new();
    server.get("/hello", middleware!("hello world"));
    server.listen("127.0.0.1:6767");
}

Imports nickel's facade 

#[macro_use]
extern crate nickel;

use nickel::Nickel;
use nickel::router::http_router::HttpRouter;

fn main() {
    let mut server = Nickel::new();
    server.get("/hello", middleware!("hello world"));
    server.listen("127.0.0.1:6767");
}

Brings HttpRouter trait into scope 

#[macro_use]
extern crate nickel;

use nickel::Nickel;
use nickel::router::http_router::HttpRouter;

fn main() {
    let mut server = Nickel::new();
    server.get("/hello", middleware!("hello world"));
    server.listen("127.0.0.1:6767");
}

The rust program entry function 

#[macro_use]
extern crate nickel;

use nickel::Nickel;
use nickel::router::http_router::HttpRouter;

fn main() {
    let mut server = Nickel::new();
    server.get("/hello", middleware!("hello world"));
    server.listen("127.0.0.1:6767");
}

Creates nickel's facade object 

#[macro_use]
extern crate nickel;

use nickel::Nickel;
use nickel::router::http_router::HttpRouter;

fn main() {
    let mut server = Nickel::new();
    server.get("/hello", middleware!("hello world"));
    server.listen("127.0.0.1:6767");
}

Mutability must be explicit 

#[macro_use]
extern crate nickel;

use nickel::Nickel;
use nickel::router::http_router::HttpRouter;

fn main() {
    let mut server = Nickel::new();
    server.get("/hello", middleware!("hello world"));
    server.listen("127.0.0.1:6767");
}

Respond with hello world on 127.0.0.1:6767/hello 

#[macro_use]
extern crate nickel;

use nickel::Nickel;
use nickel::router::http_router::HttpRouter;

fn main() {
    let mut server = Nickel::new();
    server.get("/hello", middleware!("hello world"));
    server.listen("127.0.0.1:6767");
}

Start listening

The middleware! macro is hot sauce!

middleware!("hello world")

handler without body 

middleware!({
    let first_name = "Pascal";
    let last_name = "Precht";
    format!("{} {}", first_name, last_name)
})

handler with body 

middleware! { |req|
    format!("Hello: {}", req.param("username"))
}

Accessing request params 

middleware! { |req, mut res|
    res.content_type(MediaType::Json);
    r#"{"foo":"bar"}"#
}

Setting the content type

The middleware! macro

  • Improves ergonomics where the language lacks
  • Improves stability on a syntactic level

HTTP verbs

#[macro_use]
extern crate nickel;

use nickel::Nickel;
use nickel::router::http_router::HttpRouter;

fn main() {
    let mut server = Nickel::new();
    server.get("/hello", middleware!("hello world"));
    server.listen("127.0.0.1:6767");
}
#[macro_use]
extern crate nickel;

use nickel::Nickel;
use nickel::router::http_router::HttpRouter;

fn main() {
    let mut server = Nickel::new();
    server.post("/hello", middleware!("hello world"));
    server.listen("127.0.0.1:6767");
}
#[macro_use]
extern crate nickel;

use nickel::Nickel;
use nickel::router::http_router::HttpRouter;

fn main() {
    let mut server = Nickel::new();
    server.put("/hello", middleware!("hello world"));
    server.listen("127.0.0.1:6767");
}
#[macro_use]
extern crate nickel;

use nickel::Nickel;
use nickel::router::http_router::HttpRouter;

fn main() {
    let mut server = Nickel::new();
    server.delete("/hello", middleware!("hello world"));
    server.listen("127.0.0.1:6767");
}
#[macro_use]
extern crate nickel;

use nickel::Nickel;
use nickel::router::http_router::HttpRouter;

fn main() {
    let mut server = Nickel::new();
    server.option("/hello", middleware!("hello world"));
    server.listen("127.0.0.1:6767");
}

Flexible Routing

#[macro_use]
extern crate nickel;

use nickel::Nickel;
use nickel::router::http_router::HttpRouter;

fn main() {
    let mut server = Nickel::new();
    server.get("/bar", middleware!("hello world"));
    server.listen("127.0.0.1:6767");
}

matches /bar 

#[macro_use]
extern crate nickel;

use nickel::Nickel;
use nickel::router::http_router::HttpRouter;

fn main() {
    let mut server = Nickel::new();
    server.get("/a/*/d", middleware!("hello world"));
    server.listen("127.0.0.1:6767");
}

matches /a/b/d

BUT NOT /a/b/c/d 

#[macro_use]
extern crate nickel;

use nickel::Nickel;
use nickel::router::http_router::HttpRouter;

fn main() {
    let mut server = Nickel::new();
    server.get("/a/**/d", middleware!("hello world"));
    server.listen("127.0.0.1:6767");
}

matches /a/b/d

AND ALSO /a/b/c/d

Middleware

#[macro_use]
extern crate nickel;

use nickel::Nickel;
use nickel::router::http_router::HttpRouter;

fn main() {
    let mut server = Nickel::new();
    server.utilize(StaticFilesHandler::new("examples/assets/"));
    server.get("/hello", middleware!("hello world"));
    server.listen("127.0.0.1:6767");
}

Serves files from example/assets

What else

  • Custom Middleware
  • JSON Support
  • Mounting
  • Error Handling
  • Encrypted Cookies
  • Session Support

Thank you

Rethinking Systems Programming By Christoph Burgdorf