» Chevrons.jl

Your friendly >> chevron >> based syntax for piping data through multiple transformations.

A Julia package with all the good ideas from Chain.jl and Pipe.jl, but with nicer syntax and REPL integration.

Here is a simple example:

-repl julia> using Chevrons, DataFrames, TidierData

julia> Chevrons.enable_repl()  # magic to enable Chevrons syntax in the REPL
julia> df = DataFrame(name=["John", "Sally", "Roger"], age=[54, 34, 79], children=[0, 2, 4]) 3×3 DataFrame  Row │ name    age    children      │ String  Int64  Int64 ─────┼─────────────────────────    1 │ John       54         0    2 │ Sally      34         2    3 │ Roger      79         4
julia> df >> @filter(age > 40) >> @select(num_children=children, age) 2×2 DataFrame  Row │ num_children  age      │ Int64         Int64 ─────┼─────────────────────    1 │            0     54    2 │            4     79

Quick comparison with similar packages:

| Feature | Chevrons.jl | Chain.jl | Pipe.jl | | --- | --- | --- | --- | | Piping syntax | ✔️ (>>) | ✔️ (@chain) | ✔️ (\|>) | | Side effects | ✔️ (>>>) | ✔️ (@aside) | ❌ | | Pipe backwards | ✔️ (<<) | ❌ | ❌ | | Recursive syntax | ✔️ | ❌ | ❌ | | REPL integration | ✔️ | ❌ | ❌ | | Line numbers on errors | ❌ | ✔️ | ❌ |

Usage

Installation

Click ] to enter the Pkg REPL then do:

pkg> add Chevrons

Getting started

Chevrons exports a macro @chevrons which transforms expressions like x >> f(y, z) into f(x, y, z). These can be chained together, so that

@chevrons Int[] >> push!(5, 2, 4, 3, 1) >> sort!()

is equivalent to

sort!(push!(Int[], 5, 2, 4, 3, 1))

In fact we can see exactly what it is transformed to with @macroexpand. This is equivalent code but with intermediate results saved for clarity.

-repl julia> @macroexpand @chevrons Int[] >> push!(5, 2, 4, 3, 1) >> sort!() quote     var"##chevrons#241" = Int[]     var"##chevrons#242" = push!(var"##chevrons#241", 5, 2, 4, 3, 1)     sort!(var"##chevrons#242") end

REPL integration

If you are using the Julia REPL, you can activate Chevrons's REPL integration like

-repl julia> Chevrons.enable_repl()

This allows you to use this syntax from the Julia REPL without typing @chevrons every time. Use Chevrons.enable_repl(false) to disable it again. The rest of the examples here will be from the REPL.

Also see this tip for automatically enabling the REPL integration.

Basic piping syntax with >>

Expressions like x >> f(y, z) are transformed to insert x as an extra first argument in the function call, like:

-repl julia> [5,2,4,3,1] >> sort!() >> println() [1, 2, 3, 4, 5]

If you want the argument to appear elsewhere, you can indicate where with _:

-repl julia> [5,2,4,3,1] >> filter!(isodd, _) >> println() [5, 3, 1]

In fact, you can use any expression involving _:

-repl julia> [5,2,4,3,1] >> filter!(isodd, _ .+ 10) >> println() [15, 13, 11]

Side-effects with >>>

Sometimes you want to do something with an intermediate value in the pipeline, but then continue with the previous value. For this, you can use x >>> f() which is transformed to tmp = x; f(tmp); tmp. It is very similar to Chain.jl's @aside syntax.

One use for this is to log intermediate values for debugging:

-repl julia> [5,2,4,3,1] >> filter!(isodd, ) >>> println("x = ", ) >> sum() x = [5, 3, 1] 9

You can assign values, and even use them in later steps:

-repl julia> 10 >> ( * 2) >>> (x = ) >> (x^2 - _) 380

julia> x 20

It is also useful for functions which mutate the argument but do not return it:

-repl julia> [5,2,4,3,1] >> popat!(4) 3

julia> [5,2,4,3,1] >>> popat!(4) >> println() [5, 2, 4, 1]

Piping backwards with <<

You can use << to pipe backwards: f(y) << x is transformed to f(x, y).

This can be useful as a sort of "inline do-notation":

-repl julia> write("hello.txt", "ignore this line\nkeep this line!");

julia> (            "hello.txt"            >> open()            << (io -> io >>> readline() >> read(String))            >> uppercase()        ) "KEEP THIS LINE!"

You can instead just use regular do-notation:

julia> (            "hello.txt"            >> open() do io                io >>> readline() >> read(String)            end            >> uppercase()        ) "KEEP THIS LINE!"

Recursive usage

The @chevrons macro works recursively, meaning you can wrap an entire module (or script or function or any code block) and all >>/>>>/<< expressions will be converted.

For example here is the first example in this README converted to a script:

using Chevrons, DataFrames, TidierData
@chevrons begin     df = DataFrame(name=["John", "Sally", "Roger"], age=[54, 34, 79], children=[0, 2, 4])     df2 = df >> @filter(age > 40) >> @select(num_children=children, age)     df2 >> println("data:", _)     df2 >> size >> println("size:", _) end

Or the data manipulation step can be encapsulated as a function like so:

@chevrons munge(df) = df >> @filter(age > 40) >> @select(num_children=children, age)

Pro tips

Parentheses

If you surround your pipelines with parentheses then you can place each transformation on a separate line for clarity. This also allows you to easily comment out individual transformations.

@chevrons (
    df
    # >> @filter(age > 40)
    >> @select(nchildren=children, age),
)

Or you can use >>(x, y, z) syntax instead of x >> y >> z like so:

@chevrons >>(
    df,
    # @filter(age > 40),
    @select(nchildren=children, age),
)

Startup file

You can add the following lines to your startup.jl file (usually at ~/.julia/config/startup.jl) to enable Chevrons's REPL integration automatically:

if isinteractive()
    try
        using Chevrons
    catch
        @warn "Chevrons not available"
    end
    if @isdefined Chevrons
        Chevrons.enable_repl()
    end
end

Chevrons has no dependencies so is safe to add to your global environment - then it will always be available at the REPL.

Using bit-shift functions

If you want to use the actual >>/<</>>> bit-shifting functions in your code, you can do this by giving them a new name like so:

-repl julia> using Base: << as lshift

julia> lshift(1, 10) 1024

API

See the docstrings for more help:

• @chevrons ...: Transform and execute the given code.
• chevrons(expr): Transform the given expression.
• Chevrons.enable_repl(on=true): Enable/disable the REPL integration.