Introduction to Julia¶

Julia is a fast, dynamic, reproducible, composable, general and open source programming language that can be downloaded here. The full documentation of the current release (v1.7) is available here. This tutorial is designed to go over the basics of Julia, and to cover several topics on which Julia's speed is based.

use/add/remove/develop package
basic linear algebra
just-in-time (JIT) compiler
multiple dispatch
type-stability

Package management¶

In Julia, packages are loaded via using (like packages are loaded in python via import). We typically load all the packages we want to use at the start of the file as

using LinearAlgebra, Test, BenchmarkTools

To add/remove a package, we can follow Pkg.jl and run

using Pkg
Pkg.add("IterativeSolvers")

    Updating registry at `~/.julia/registries/General`
    Updating git-repo `https://github.com/JuliaRegistries/General.git`
    Updating registry at `~/.julia/registries/SLIMregistryJL`
    Updating git-repo `https://github.com/slimgroup/SLIMregistryJL.git`
   Resolving package versions...
    Updating `~/.julia/environments/v1.7/Project.toml`
  [42fd0dbc] ~ IterativeSolvers v0.9.2 `https://github.com/JuliaLinearAlgebra/IterativeSolvers.jl.git#master` ⇒ v0.9.2
    Updating `~/.julia/environments/v1.7/Manifest.toml`
  [42fd0dbc] ~ IterativeSolvers v0.9.2 `https://github.com/JuliaLinearAlgebra/IterativeSolvers.jl.git#master` ⇒ v0.9.2
Precompiling project...
  ✓ IterativeSolvers
  ✓ JOLI
  ✓ GenSPGL
  ✓ JUDI
  ✓ SetIntersectionProjection
  ✓ MECurvelets
  ✓ ImageGather
  ✓ InvertibleNetworks
  8 dependencies successfully precompiled in 16 seconds (356 already precompiled)

Pkg.rm("IterativeSolvers")

    Updating `~/.julia/environments/v1.7/Project.toml`
  [42fd0dbc] - IterativeSolvers v0.9.2
  No Changes to `~/.julia/environments/v1.7/Manifest.toml`

Note that we can also install a Julia package at a specific version, or from a GitHub repository

Pkg.add(name="IterativeSolvers", version="0.9.2")

   Resolving package versions...
    Updating `~/.julia/environments/v1.7/Project.toml`
  [42fd0dbc] + IterativeSolvers v0.9.2
  No Changes to `~/.julia/environments/v1.7/Manifest.toml`

Pkg.add(url="https://github.com/JuliaLinearAlgebra/IterativeSolvers.jl.git", rev="master")

    Updating git-repo `https://github.com/JuliaLinearAlgebra/IterativeSolvers.jl.git`
   Resolving package versions...
    Updating `~/.julia/environments/v1.7/Project.toml`
  [42fd0dbc] ~ IterativeSolvers v0.9.2 ⇒ v0.9.2 `https://github.com/JuliaLinearAlgebra/IterativeSolvers.jl.git#master`
    Updating `~/.julia/environments/v1.7/Manifest.toml`
  [42fd0dbc] ~ IterativeSolvers v0.9.2 ⇒ v0.9.2 `https://github.com/JuliaLinearAlgebra/IterativeSolvers.jl.git#master`
Precompiling project...
  ✓ IterativeSolvers
  ✓ JOLI
  ✓ GenSPGL
  ✓ JUDI
  ✓ SetIntersectionProjection
  ✓ ImageGather
  ✓ MECurvelets
  ✓ InvertibleNetworks
  8 dependencies successfully precompiled in 16 seconds (356 already precompiled)

We can also do these on julia's interactive command-line REPL (read-eval-print loop) via

] add IterativeSolvers
] rm IterativeSolvers
] add https://github.com/JuliaLinearAlgebra/IterativeSolvers.jl.git

etc.

Linear algebra¶

The syntax of creating multidimensional arrays is a bit similar to MATLAB. We use square brackets [] to set up a multidimensional array, and use semi-colon ; to switch rows.

A = [1 2; 3 4]

2×2 Matrix{Int64}:
 1  2
 3  4

Note that multidimensional arrays are stored in a column-major order, meaning that they fill up one column at a time. We can verify this by vectorizing A via vec or [:]

A[:]

4-element Vector{Int64}:
 1
 3
 2
 4

vec(A)

4-element Vector{Int64}:
 1
 3
 2
 4

It is also quite simple to create a vector using square bracket, and Julia assumes 1D vector as column vector.

x = [1, 2]

2-element Vector{Int64}:
 1
 2

There are built-in basic linear algebraic operations, such as

inv(A)

2×2 Matrix{Float64}:
 -2.0   1.0
  1.5  -0.5

A * x

2-element Vector{Int64}:
  5
 11

A \ x

2-element Vector{Float64}:
 0.0
 0.5

det(A)

-2.0

Just-in-time (JIT) compilation¶

Julia code is just-in-time compiled---i.e., every statement runs using compiled functions which are either compiled right before they are used, or cached compilations from before. This is different from e.g. C, which is compiled before executed. Due to the JIT compilation, the first run of a function/operation typically takes slightly longer time as it needs to compile first. An example is shown below

A = randn(Float32, 10^4, 10^4);
b = randn(Float32, 10^4);

# first run
@time A * b;

  0.080792 seconds (290.82 k allocations: 16.171 MiB, 81.98% compilation time)

# second run
@time A * b;

  0.017111 seconds (2 allocations: 39.109 KiB)

The first time of execution this "Float32 matrix and Float 32 vector product" takes longer time and more memory allocations as this is the first time to run such functionality in the current notebook. In order for robust estimate of the execution time and memory usage, we can use the @benchmark macro in BenchmarkTools.jl package.

@benchmark A * b

BenchmarkTools.Trial: 205 samples with 1 evaluation.
 Range (min … max):  16.436 ms …  27.088 ms  ┊ GC (min … max): 0.00% … 0.00%
 Time  (median):     24.335 ms               ┊ GC (median):    0.00%
 Time  (mean ± σ):   24.451 ms ± 909.266 μs  ┊ GC (mean ± σ):  0.00% ± 0.00%

                                                ▃▅█▆▁           
  ▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▂▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▂▁▂▅▇██████▇▅▆▅█▄▅▃ ▃
  16.4 ms         Histogram: frequency by time           26 ms <

 Memory estimate: 39.11 KiB, allocs estimate: 2.

Multiple dispatch¶

The core design to make Julia fast is type-stability through specialization via multiple-dispatch. A single function in Julia can take different types of input and compile the function in specialized ways. This "specialization" brings huge performance gains to Julia. Let's first go over a simple example, namely scalar multiplication, and use the macro @which to check the method being used

@which 1 * 2

In this example, we want to compute the multiplication of 2 integers. As we can see, the underlying method is running the multiplication of integers.

@which 1.0 * 2.0

If we multiply 2 floating point numbers, we can see that a different method is used, which is designed specifically for the type double (Float64 in Julia).

@which 1.0 * 2

From here, we can see that Julia can't find a specialized way to compute 1.0 * 2 according to the type of inputs. In the end, it chooses to use the very generic Number to describe the type of 1.0 and 2 and to perform the multiplication. Note that this is one of the cases that we want to avoid during large-scale computations. We usually want to make sure that the type of input and output is the same in our functions.

This example demonstrates that the same function (multiplication * in this case) is executed via different compiled code blocks that are specialized for different kinds of input. This feature brings huge performance gain and potentially makes Julia to run almost as fast as C/Fortran (which are strictly typed). With that being said, we should also take advantage of this type-stability when implementing functions---i.e., dispatch a function to multiple types of input.

Let's show a simple example: we can define a foo function that works for different kinds of input. Let's first build it to take String inputs.

foo(x::String, y::String) = println("My inputs x and y are both strings!")

foo (generic function with 1 method)

foo("hello", "hi!")

My inputs x and y are both strings!

Notice that now foo is a generic function with 1 method (a.k.a. taking 2 String and output a sentence). Now the foo function won't work for numbers.

foo(3, 4) # doesn't work here

MethodError: no method matching foo(::Int64, ::Int64)

Stacktrace:
 [1] top-level scope
   @ In[23]:1
 [2] eval
   @ ./boot.jl:373 [inlined]
 [3] include_string(mapexpr::typeof(REPL.softscope), mod::Module, code::String, filename::String)
   @ Base ./loading.jl:1196

To get foo to work on integer (Int) inputs, let's add ::Int onto our input arguments when we declare foo.

foo(x::Int, y::Int) = println("My inputs x and y are both integers!")

foo (generic function with 2 methods)

foo(3, 4) # works now

My inputs x and y are both integers!

Now foo(::Int, ::Int) works as well. Notice that it says foo (generic function with 2 methods). This means foo is a general function but contains 2 methods: the first one works for 2 strings and the second one works for 2 numbers. This means that the functionality of foo on 2 strings isn't overwritten when you define how to run foo on another type of input(s). Instead, we just added an additional method to the generic function foo.

foo("hello", "hi!")

My inputs x and y are both strings!

Then let's show an explanatory example to discuss how we should write code in Julia. Suppose we want to build a function to take the 2 norm of a scalar/vector/matrix (while we choose to ignore the existing norm functionality). A naive way to implement this would be

function my2norm(x)
    if isa(x, Number)
        return abs(x)
    elseif isa(x, Vector)
        return sqrt(sum(x.^2))
    elseif isa(x, Matrix)
        return return maximum(svdvals(x))
    else
        println("my function is not defined for this kind of input")
        return -1
    end
end

my2norm (generic function with 1 method)

In the above code block, we defined a function my2norm that uses nested if-else to find the type of x and perform the corresponding operation (absolute value for scalar, euclidean norm for vector, and largest singular value for matrix).

However, there is a smarter and cleaner way to write this function, which provides exactly the same functionality.

mysmart2norm(x::Number) = abs(x)
mysmart2norm(x::Vector) = sqrt(sum(x.^2))
mysmart2norm(x::Matrix) = maximum(svdvals(x))
mysmart2norm(x) = begin println("my function is not defined for this kind of input"); return -1; end

mysmart2norm (generic function with 4 methods)

x = randn(10^2, 10^2);
@test my2norm(x) == mysmart2norm(x)

Test Passed
  Expression: my2norm(x) == mysmart2norm(x)
   Evaluated: 19.768341325704956 == 19.768341325704956

This mysmart2norm will lead to cleaner and more maintainable code: when a new type comes to our sight (e.g. an abstract type defined by user), then we don't need to dig into the very long my2norm to add another set of ifelse. A simple mysmart2norm(x::NewType) will do the work. For example, check here to see how JUDI.jl defines norm for judiMultiSourceVector, an abstract type defined for seismic data.

Type stability¶

The multiple dispatch feature makes Julia easy to use like Python, and fast to run like C. In order for Julia compiler to produce highly efficient code and show ultimate performance, we should write type-stable code---i.e., given the source code and the concrete type of all arguments, the compiler is able to infer the concrete type of every variable and expression within the method.

Let's consider the following example. funcN takes a sequence of Number (... is the splatting operation) and it aims to compute the product of this entire array.

function funcN(a::Number...)
    c=copy(a[1])
    L=length(a)
    for l=2:L
        c=c*a[l]
    end
    return c
end

funcN (generic function with 1 method)

Now we want to compute the product of the sequence 1, 1.5, 2, 2.5, 3. We have 2 following options:

x1 = (1, 1.5, 2, 2.5, 3)
@benchmark funcN(x1...)

BenchmarkTools.Trial: 10000 samples with 413 evaluations.
 Range (min … max):  234.228 ns …  50.863 μs  ┊ GC (min … max): 0.00% … 99.35%
 Time  (median):     259.131 ns               ┊ GC (median):    0.00%
 Time  (mean ± σ):   272.421 ns ± 507.021 ns  ┊ GC (mean ± σ):  1.86% ±  0.99%

      ▄█ ▁ █  ▁                                                  
  ▃▃▂▃██▄█▄█▅▅█▄▅▅▃▃▄▃▃▅▆▄▃▂▃▃▂▂▃▂▂▂▂▂▂▂▂▂▂▂▂▃▂▂▂▂▂▂▂▂▂▂▂▂▁▁▂▂▂ ▃
  234 ns           Histogram: frequency by time          390 ns <

 Memory estimate: 96 bytes, allocs estimate: 6.

x2 = (1.0, 1.5, 2.0, 2.5, 3.0)
@benchmark funcN(x2...)

BenchmarkTools.Trial: 10000 samples with 803 evaluations.
 Range (min … max):  146.661 ns …  29.191 μs  ┊ GC (min … max): 0.00% … 99.30%
 Time  (median):     158.966 ns               ┊ GC (median):    0.00%
 Time  (mean ± σ):   171.676 ns ± 495.770 ns  ┊ GC (mean ± σ):  4.99% ±  1.72%

     ▆▁█ ▅ ▅                                                     
  ▃▅▄███▆█▅█▄▃▃▂▃▃▃▃▂▃▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▂▁▂▂▂▂▂ ▃
  147 ns           Histogram: frequency by time          259 ns <

 Memory estimate: 96 bytes, allocs estimate: 6.

Interesting! Why do they have different runtimes? Julia provides a macro @code_warntype to check whether the code is type-stable. Let's see.

@code_warntype funcN(x1...)

MethodInstance for funcN(::Int64, ::Float64, ::Int64, ::Float64, ::Int64)
  from funcN(a::Number...) in Main at In[30]:1
Arguments
  #self#::Core.Const(funcN)
  a::Tuple{Int64, Float64, Int64, Float64, Int64}
Locals
  @_3::Union{Nothing, Tuple{Int64, Int64}}
  L::Int64
  c::Union{Float64, Int64}
  l::Int64
Body::Union{Float64, Int64}
1 ─ %1  = Base.getindex(a, 1)::Int64
│         (c = Main.copy(%1))
│         (L = Main.length(a))
│   %4  = (2:L::Core.Const(5))::Core.Const(2:5)
│         (@_3 = Base.iterate(%4))
│   %6  = (@_3::Core.Const((2, 2)) === nothing)::Core.Const(false)
│   %7  = Base.not_int(%6)::Core.Const(true)
└──       goto #4 if not %7
2 ┄ %9  = @_3::Tuple{Int64, Int64}
│         (l = Core.getfield(%9, 1))
│   %11 = Core.getfield(%9, 2)::Int64
│   %12 = c::Union{Float64, Int64}
│   %13 = Base.getindex(a, l)::Union{Float64, Int64}
│         (c = %12 * %13)
│         (@_3 = Base.iterate(%4, %11))
│   %16 = (@_3 === nothing)::Bool
│   %17 = Base.not_int(%16)::Bool
└──       goto #4 if not %17
3 ─       goto #2
4 ┄       return c

@code_warntype funcN(x2...)

MethodInstance for funcN(::Float64, ::Float64, ::Float64, ::Float64, ::Float64)
  from funcN(a::Number...) in Main at In[30]:1
Arguments
  #self#::Core.Const(funcN)
  a::NTuple{5, Float64}
Locals
  @_3::Union{Nothing, Tuple{Int64, Int64}}
  L::Int64
  c::Float64
  l::Int64
Body::Float64
1 ─ %1  = Base.getindex(a, 1)::Float64
│         (c = Main.copy(%1))
│         (L = Main.length(a))
│   %4  = (2:L::Core.Const(5))::Core.Const(2:5)
│         (@_3 = Base.iterate(%4))
│   %6  = (@_3::Core.Const((2, 2)) === nothing)::Core.Const(false)
│   %7  = Base.not_int(%6)::Core.Const(true)
└──       goto #4 if not %7
2 ┄ %9  = @_3::Tuple{Int64, Int64}
│         (l = Core.getfield(%9, 1))
│   %11 = Core.getfield(%9, 2)::Int64
│   %12 = c::Float64
│   %13 = Base.getindex(a, l)::Float64
│         (c = %12 * %13)
│         (@_3 = Base.iterate(%4, %11))
│   %16 = (@_3 === nothing)::Bool
│   %17 = Base.not_int(%16)::Bool
└──       goto #4 if not %17
3 ─       goto #2
4 ┄       return c

A red line in the output indicates type-instability in the code. So what happened? In the function, we first read the variable c as the first entry of the sequence. In x1, the first entry is an Int64, so c will get the type of Int64 as well initially. However, c later multiplies with Float64 entries (namely 1.5 and 2.5). Therefore, it is forced to change its type of Float64.

When we are very sure of the input and output types of a function we expect (like in this case we know everything ends up to be Float64), we should keep in mind to write type-stable code so that Julia doesn't find a hard time to struggle on the type of the variables.

T22 - Julia for Geoscience

Notebooks

Linear algebra abstractions in JOLI.jl

Intro to Julia