Every Language Should Have Pipelining

ooooh you want to add ▷◁ to your language soooo bad

I read Pipelining might be my favorite language feature a little while back. It’s a good post!

It’s an argument for languages to have a pipelining operator: a way of chaining together the result of function calls, without needing to nest them within the syntax of the function call itself.

In other words, a pipelining operator allows for you to write this:

def get_ids (data : List Widget) : List Id :=
  collect(map(filter(iter(data), |w| w.alive), |w| w.id))

Instead like this:

def get_ids (data : List Widget) : List Id :=
  data
    |> iter() 
    |> filter(|w| w.alive) 
    |> map(|w| w.id) 
    |> collect()

Pretty good, right?

The points made by the article can be summarized briefly.

1. You already use pipelining. It’s the same thing as member access syntax / method call syntax. Why should such nice syntax be restricted to members and methods?
2. A pipelining operator allows for breaking long function composition calls apart. This benefits readability, git diffs, and everyone’s sanity – and encourages function composition. Function composition is a very good thing! It is important for purity and parallelism and modularity and everything in that vein: and this, in my mind, encourages better code.
3. The . operator popping up completions for members / methods is the single biggest value add of IDEs / LSPs. With a general pipeline operator, it is possible, on principle, to be able to offer this for any term and the functions that operate on it (in a typed language).

I strongly agree with all these points. Particularly the last. More languages should have pipelining.

The author then goes on to discuss the current state of pipelining in a couple of languages: SQL, design-pattern languages, Haskell, and Rust. Here, I think the article is missing some interesting languages and points in the PL design space, which I’d like to elaborate on.

uniform method call syntax

There is out there an idea of “uniform method call syntax”: overloading the . operator to function not just for members and methods, but for any function taking in a first argument. len("string")? "string".len()? Who cares? Let both work. It’s all the same, after all. Sometimes the prefix function call is clearer: sometimes the postfix pipelined call is desired.

This is, of course, exactly a pipelining operator, by a different name. There are a good number of languages that do this: Nim and D (little-used hobbyist languages) and Koka and Effekt (academic research languages). I think this specific formulation of pipelining is interesting to discuss / think about: and it’s been subject to criticism over the years, some of which is warranted.

Now – I came from Nim into Rust. I learned how to program in a language with this uniform function call syntax. And I think that Rust’s system of pipelining is bad, as a result. For a variety of reasons, Rust strongly discourages you from defining impls on types you don’t control – in fact, you straight up can’t, and have to use the newtype pattern to make a single-field wrapper struct around the type before being able to define impls… and by doing so, you lose out on all previous impls. As impl blocks are the only way to have access to method call syntax: this seriously sucks!! I find myself leaning for newtype patterns or even different types altogether purely to work around the syntactic verbosity not having a true pipelining operator brings.¹

That is all to say: I like pipelining. But Rust is not a stellar example of a language built for it.
Nim and Gleam and Lean all do a much better job in my opinion.

left and right pipelines

Speaking of Lean – let’s talk about Lean.

Lean is a combination theorem prover / general purpose programming language, in the ML tradition: function calls are by associativity, with parentheses only for disambiguation / tuples; the core language is purely functional with mutability / loops / etc expressed by monads and do notation; and it’s extremely strongly typed. (Dependently typed. That’s the “theorem prover” part.)

(Lean is notable to me personally by having consistently the best syntax out of any programming language I’ve ever used. Syntax is subjective, of course. But every time I think of some ideal syntax for a language – I sit back, think a little bit, and realize that’s what Lean already does.² )

Anyway, back to the point. Lean has pipelining, with the |> operator. But it also has a <| operator. What’s the <| operator do? And why does it look like the classic |> pipeline operator?

The Lean language reference describes these both as pipeline syntax: |> being “right pipe notation”, and <| being “left pipe notation”. In descriptive words, the expression of the left of |> is fed rightwards, and the expression on the right of <| is fed leftwards.

This makes a lot of sense in the context that Lean is an ML: functions are left-associative, so if we want to call another function taking arguments as not-the-first-argument to our first function, we’ll need parentheses. Or, we can use <|, and feed the expression on the right leftwards. But wait. This reminds me of something. Lean is an ML… this <| is changing associativity… haven’t we seen something like this before?

Yeah. Yup. It’s exactly Haskell’s $. So exactly, that $ is aliased to <| in Lean. There are in fact two salient notions of a pipeline operator. Though, the second is only really relevant for languages in the ML tradition of syntax. (The original article notes this in a footnote, but I wanted to highlight it – having both |> and <| makes really great sense conceptually, IMO. And there are languages out there that do this – by convention, everywhere, without historical baggage. Come learn Lean. 😈)

design considerations

That’s it. That’s all I really wanted to add on to the article.

I’ll close with some language design considerations, explicitly stated and restated.

1. Your language should have a pipelining operator.
2. Your pipelining operator must have good LSP support.
3. You should consider having both forward and backward pipes. If so, |> <| is good syntax.