Description
Piping is function application (f(x)) in reverse, so it takes the value to be applied and then the function that will operate with it. In some languages (like OCaml, F#, Elm) it’s implemented with the piping operator (|>). Hence, it can be used like this:
x |> f
where f is a unary function and x its parameter.
Before talking about why this operator is useful, let's take a look at the shortcomings of the common function application. Shortcomings of the common function application (f x) Function application comes with some disadvantages when combining multiple functions: It needs extra parentheses to pass the result from one function to the next one. It changes the normal flow of reading code (left to right) to start from the deepest nested function (right to left). Reading code demands more time than writing code. Hence, the importance of code readability. It keeps growing to the right as new functions are added to the computation, which can result in a long and difficult line to read, and code formatters don’t always help.
Let’s see an example:
myOuterFunction(myMiddleFunction(myDeepFunction(value)))
In the example above, it’s necessary to start reading from the right to know that the result of myDeepFunction(value) is going to be the argument of myMiddleFunction, and the result of myMiddleFunction will be the argument of myOuterFunction. Modifying the example above (e.g. adding another middle function) can make things harder to read because it needs to add extra parentheses and keep moving the line to the right. Code formatters could modify that automatically to make it grow from top to bottom but it can result in an arrow antipattern:
myOuterFunction(
myMiddleFunction(
myDeepFunction(value)
)
)
Using auxiliary variables is an alternative to flatten the code a little bit, but they introduce a level of indirection making the code more difficult to follow:
deepFunctionResult = myDeepFunction(value)
middleFunctionResult = myMiddleFunction(deepFunctionResult)
finalResult = myOuterFunction(middleFunctionResult)
Intermediate operations require auxiliary variables (which can be a pain in the neck to give a name and type), a pair of extra parentheses for every action, and it’s necessary to go from the variable to its definition to know where it comes from. This is annoying and that’s where Piping comes to the rescue. How will Piping help? Piping multiple functions will grow from top to bottom, without adding extra parentheses for intermediate actions or introducing a level of indirection in the code, so it doesn’t present the issues that we see in the common function application. Let’s see it in action in the following example:
Let’s get a summary of the users that are allowed to enter a club (over 18 and not banned):
[{ name: “John Paul”, age: 24, banned: true }
,{ name: “Craig”, age: 19, banned: false }
,{ name: “Kelsie”, age: 17, banned: false }
,{ name: “Bernard”, age: 27, banned: false }
]
|> filter ((user) => user.age >= 18 and not user.banned)
|> map ((user) => user.name)
|> intersperse “, ”
|> \names ->
if isEmpty names
then "None of them"
else concat names
“Craig, Bernard”
The above example is just pseudocode. In the next section, this example will be implemented in Elm, Haskell, Python and JavaScript using the features embedded in each language. Piping in some languages
Elm
Elm is one of my preferred languages and it provides the Pipe Operator (|>) and a reversed Pipe Operator (<|) in the core library, so it’s not necessary to import it explicitly. The example above will look like this:
allowedToEnter =
[{ name = "John Paul", age = 22, banned = True }
,{ name = "Craig", age = 19, banned = False }
,{ name = "Kelsie", age = 17, banned = False }
,{ name = "Bernard", age = 27, banned = False }
]
|> List.filter (\user -> user.age >= 18 && not user.banned)
|> List.map .name
|> List.intersperse ","
|> \names ->
if List.isEmpty names
then "None of them"
else String.concat names
In Elm, functions are curried, which means that a function with multiple arguments are actually functions that take only one argument and return another function. The combination of this feature and the pipe operator clarifies how the data flows from top to bottom. These features make the Elm implementation neat!
NOTE: The Elm documentation warns to avoid using the pipe operator beyond three or four steps, since beyond that point the data flow becomes clearer if it's broken down into smaller top functions.
Haskell
The reverse function application operator was added to Haskell around 2014 and it’s the (&) infix operator (perhaps you have seen this operator when working with lenses). So, let’s see how the Haskell version of the example would look:
import Data.Function ((&))
import Data.List (intersperse)
data User = User
{ name :: String
, age :: Int
, banned :: Bool
}
allowedToEnter =
[User { name = "John Paul", age = 22, banned = True }
,User { name = "Craig", age = 19, banned = False }
,User { name = "Kelsie", age = 17, banned = False }
,User { name = "Bernard", age = 27, banned = False }
]
& filter (\user -> age user >= 18 && not (banned user))
& map name
& intersperse ","
& \names ->
if null names
then "None of them"
else concat names
The Haskell implementation also benefits from curried functions and the operator is already defined in the language. However, it’s necessary to import the library whenever you’re going to use the operator and it doesn’t look like data flow at first glance compared with the pipe operator (|>).
NOTE: It’s interesting to know why the piping (|>) operator wasn’t chosen for Haskell so I suggest reading this GitHub Issue thread. Additionally, the flow library exposes the function application and reverse function application using (<|) and (|>) operators in case you’re interested in it.
Python
Python functions follow the standard function application style and it’s not possible to create new infix operators. Nevertheless, Python is a flexible language and its features will allow us to overwrite an existing infix operator to be the pipe.
Python allows operator overloading by overwriting some dunder methods (aka magic methods) and it can be defined based on the order of the operands in the infix operator (whether the instance is at the right or the left). This feature can be used to override the | operator taking the argument from the left to apply it to the value on the right. So, the class that will override this methods is the following:
NOTE: I chose the | operator since it’s similar to the Pipe operator (|>) and because it’s the one used in the Pipe infix toolkit.
from typing import Callable, Iterator
class Pipe:
def __init__(self, fn: Callable):
self.fn = fn
def __ror__(self, arg):
return self.fn(arg)
def __call__(self, *args, **kwargs) -> 'Pipe':
return Pipe(lambda x: self.fn(x, *args, **kwargs))
The __call__ method was also overwritten so that instances of Pipe can be callables that have more arguments, which will be passed to the initial wrapped function and return another Pipe as a result.
The instances of Pipe will wrap a function to be instantiated and Python decorators provide the syntax sugar that will make Pipe instantiation easier. For example:
@Pipe
def select(xs: Iterator, func: Callable):
return map(func, xs)
Instead of:
select = Pipe(lambda xs, func: map(func, xs))
Next, it’s necessary to define some Pipe instances for the exercise:
@Pipe
def select(xs: Iterator, func: Callable):
return map(func, xs)
@Pipe
def where(xs: Iterator, predicate: Callable):
return filter(predicate, xs)
@Pipe
def to_list(xs: Iterator):
return list(xs)
@Pipe
def intersperse(xs: list, sep: str):
return sep.join(xs)
Finally, it’s time to put all of these together for the example:
allowedToEnter = (
[
{ "name": "John Paul", "age": 22, "banned": True },
{ "name": "Craig", "age": 19, "banned": False },
{ "name": "Kelsie", "age": 17, "banned": False },
{ "name": "Bernard", "age": 27, "banned": False }
]
| where(lambda user: user['age'] >= 18 and not user['banned'])
| select(lambda user: user['name'])
| to_list
| intersperse(',')
| Pipe(lambda names: "None of them" if names == "" else names)
)
As you can see, the Python implementation gives all the benefits from the Pipe operator and looks fine just by defining a single class. Unfortunately, functions are not curried by default in Python so it’s up to us to curry them, Python lambdas are not very ergonomic and the type hints don’t fully work.
JavaScript
A Pipe operator for JavaScript was proposed some time ago but it isn’t available yet (as of this blog’s writing it’s on Stage 2), it doesn’t provide a way to create new infix operators, and it doesn’t support operator overloading. Hence, it needs a different strategy to implement this operator. Recently, I’ve been working with fp-ts (and I loved it!) and it provides a pipe function that allows piping. Although it’s different to the pipings that we’ve seen so far.
The implementation of pipe can be very simple taking advantage of variadic arguments. It will be a function that takes a value as the first parameter and then one or many unary functions as the following parameters that will be applied sequentially:
function pipe(x, ...fns) {
let resp = x
for(const fn of fns) {
resp = fn(resp)
}
return resp
}
Now, it’s possible to use pipe the fp-ts way. The example will look like this:
const allowedToEnter = pipe(
[{ name: "John Paul", age: 22, banned: true }
,{ name: "Craig", age: 19, banned: false }
,{ name: "Kelsie", age: 17, banned: false }
,{ name: "Bernard", age: 27, banned: false }
],
(x) => x.filter((user) => user.age >= 18 && !user.banned),
(x) => x.map((user) => user.name),
(x) => x.join(', '),
(x) => {
if(x === "") {
return 'None of them'
} else {
return x
}
}
)
In JavaScript pipe is just a function that takes advantage of variadic parameters to apply multiple functions and it doesn’t need to define class instances to pipe values. However, it requires functions to be curried, which is not the default case for most JavaScript core libraries.
Conclusion
Piping is very handy to define function application as a flow of data through a set of functions. It’s defined in some languages but it can be implemented in others, too – regardless of their paradigms. In this tutorial, the benefits of using the Pipe operator in Elm was described, how reverse function application works in Haskell, and how to implement the Pipe operator in Python and JavaScript with two different approaches (operator overloading and function with variadic parameters). As you can see, these languages provide a way to pipe a value through a set of functions which will help readability and maintainability in the long run.
