---
title: "Using Julia inside targets"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Using Julia inside targets}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(collapse = TRUE, comment = "#>", eval = FALSE)
```

This vignette is the reference for running **Julia** scripts as `targets` steps with [JuliaCall](https://github.com/JuliaInterop/JuliaCall): every `tar_target_jl()` argument, how to choose the Julia install and project, and the common use cases. It mirrors `vignette("python")`: the *shape* of a Julia step is identical to a Python one, only the bridge and the environment arguments differ. For a gentler tour start with `vignette("get_started")`.

Code blocks are illustrative and **not executed** when the vignette builds.

## The two constructors

- `tar_target_jl()`: bare `name` (and unquoted `pattern`), for direct use in `_targets.R`.
- `tar_target_jl_raw()`: string `name`, for use inside targets factories.

Both return a single `targets` target and forward **every** `targets::tar_target_raw()` argument (`pattern`, `format`, `iteration`, `deployment`, `resources`, `cue`, ...).

These constructors depend on the [JuliaCall](https://github.com/JuliaInterop/JuliaCall) R package. As a result, any limitation of JuliaCall is inherited by the constructors. Reading JuliaCall's own documentation is recommended.

## Arguments

### Scripts, data, and output

These behave exactly as for Python (see `vignette("python")`):

| Argument | Meaning |
|------------------------------------|------------------------------------|
| `script` | Path to the Julia script to run (**required**). Runs in the `Main` module. |
| `pre_script` | Optional R script run **before** the Julia script. Assign a named list `to_jl` to push objects into `Main`. |
| `post_script` | Optional R script run **after** the Julia script. `jl_get()` and `jl_call()` are available; its last expression is the value (object mode) or file paths (file mode). |
| `inputs` | Named vector mapping in-step names to upstream targets, e.g. `c(df = "prepared")`. |
| `output` | `"object"` (default) or `"file"`. |
| `retrieve` | Julia variable name(s) to return when there is no post-script (object mode). |
| `files` | Paths to return when there is no post-script (file mode). |

As with Python, any of `script` / `pre_script` / `post_script` may be a literal path or a `tar_target_path("name")` reference to track the file.

### Choosing the Julia install / project

| Argument | Meaning |
|------------------------------------|------------------------------------|
| `julia_version` | A version string (e.g. `"1.11"`), resolved to a [juliaup](https://github.com/JuliaLang/juliaup)-managed install. Used when `julia_home` is not given. |
| `julia_home` | The directory containing the `julia` executable. Defaults to `getOption("tarpolyglot.julia_home")`; when unset and no `julia_version`, JuliaCall discovers Julia on `PATH`. |
| `julia_project` | A Julia project environment (folder with `Project.toml` / `Manifest.toml`) to `Pkg.activate()`. When `NULL`, Julia's default global environment (`@v#.#`) is used. |
| `julia_packages` | Character vector of packages to `using` before running the script. |

## The three-script model

```         
 upstream targets ─► pre_script (R) ─► script (.jl) ─► post_script (R) ─► target value
                     builds `to_jl`     computes         reads jl_get("name") /
                                        `result`         jl_call(fn, ...)
```

1.  **`script`**: the Julia file, run in `Main`.
2.  **`pre_script`**: the `inputs` are already bound by name. Assign a named list `to_jl`; each element is `julia_assign()`ed as a variable in `Main`.
3.  **`post_script`**: JuliaCall has no `py`-style proxy, so you read variables back through `jl_get("name")` (a shortcut for `JuliaCall::julia_eval("name")`) and call Julia functions with `jl_call` (an alias of `JuliaCall::julia_call()`).

## Object output

`jl/stats.jl`:

``` julia
# `x` was pushed from R by the pre-script.
seq = isa(x, AbstractVector) ? x : [x]
result = Dict("sum" => sum(seq), "n" => length(seq), "mean" => sum(seq) / length(seq))
```

`R/pre_push.R`:

``` r
to_jl <- list(x = x)   # `x` came from inputs = c(x = "prepared_x")
```

`R/post_result.R`:

``` r
res <- jl_get("result")
data.frame(sum = res$sum, n = res$n, mean = res$mean)   # last expression = value
```

`_targets.R`:

``` r
library(targets)
library(tarpolyglot)

list(
  tar_target(prepared_x, c(1, 2, 3, 4)),

  # (a) return a Julia variable directly with `retrieve` (no post-script)
  tar_target_jl(
    name = jl_direct,
    script = "jl/stats.jl",
    inputs = c(x = "prepared_x"),
    pre_script = "R/pre_push.R",
    retrieve = "result"
  ),

  # (b) reshape the result in a post-script
  tar_target_jl(
    name = jl_prepost,
    script = "jl/stats.jl",
    inputs = c(x = "prepared_x"),
    pre_script = "R/pre_push.R",
    post_script = "R/post_result.R"
  )
)
```

## File output

`jl/write.jl`: writes `x` to a file, keeps the path in `out_path` for the post-script to read back:

``` julia
out_path = tempname() * ".csv"
open(out_path, "w") do io
    println(io, join(x, ","))
end
```

`R/post_files.R`:

``` r
jl_get("out_path")
```

``` r
tar_target_jl(
  name = jl_file,
  script = "jl/write.jl",
  inputs = c(x = "prepared_x"),
  pre_script = "R/pre_push.R",
  post_script = "R/post_files.R",    # returns jl_get("out_path")
  output = "file"
)
```

## Dynamic branching (iris example)

`jl/fit.jl`: `df` (a per-species subset of `iris`, auto-converted to a Julia `DataFrame`) was pushed from R by the pre-script:

``` julia
result = sum(df[!, "Petal.Length"]) / nrow(df)
```

`R/pre_push_jl.R`:

``` r
to_jl <- list(df = df)   # `df` came from inputs = c(df = "iris_groups")
```

``` r
list(
  tar_target(iris_groups, split(iris, iris$Species), iteration = "list"),
  tar_target_jl(
    name = fit_by_group,
    script = "jl/fit.jl",
    inputs = c(df = "iris_groups"),
    pre_script = "R/pre_push_jl.R",
    retrieve = "result",
    pattern = map(iris_groups),      # one branch per species
    iteration = "list"
  )
)
```

Run the branches in parallel by setting a `crew` controller (see `vignette("get_started")`, the "Parallelism and isolation with crew" section, which also covers `polyglot_controller(log = tar_polyglot_log(...))` for per-step stdout/stderr log files, including a caveat specific to branching there, since every branch of a pattern shares one log file).

## Parallel computing (Distributed, not Threads)

Julia's native `Threads.@threads` multi-threading does **not** work through JuliaCall: `Threads.nthreads()` reports `1` from a `run_jl_step()` session regardless of the `JULIA_NUM_THREADS` environment variable, confirmed set both to a fixed number and to `"auto"` before `JuliaCall::julia_setup()` runs. This is a limitation of how JuliaCall embeds Julia, not something `tar_target_jl()` or `deployment` can work around.

`Distributed`, Julia's process-based parallelism, works correctly instead, confirmed both in the main session and inside a real `crew` worker:

```julia
# jl/parallel_sqrt.jl
using Distributed
addprocs(n_workers)
@everywhere sqrt_one(v) = sqrt(v)
result = pmap(sqrt_one, x)
```

```r
tar_target_jl(
  name = jl_parallel,
  script = "jl/parallel_sqrt.jl",
  inputs = c(x = "big_x", n_workers = "n_workers"),
  pre_script = tar_code({ to_jl <- list(x = x, n_workers = n_workers) }),
  retrieve = "result"
)
```

`addprocs()` spawns genuine separate `julia` processes, so the same oversubscription concern from `vignette("get_started")` applies: a `crew` worker has access to every core on the machine, not a restricted slice of it, so several workers each calling `addprocs()` at once can oversubscribe the machine. Put the parallel-heavy step on `deployment = "main"` if you have more than one, or cap `n_workers` per step, for the same reason described there.

## Choosing the Julia install and project

The use cases below are about the installation-selection arguments, not what each script computes.

### Use case 1: Julia on PATH (default)

Set nothing; JuliaCall discovers the `julia` on `PATH` and uses the global environment. `jl/probe.jl`:

``` julia
result = Sys.BINDIR
```

``` r
tar_target_jl(
  name = probe,
  script = "jl/probe.jl",
  retrieve = "result"
)
```

### Use case 2: a specific juliaup version

Reuses `jl/fit.jl` from "Dynamic branching" above as a stand-in script (without `inputs` here, so `df` is undefined, the point of this example is `julia_version`, not the computation):

``` r
tar_target_jl(
  name = fit,
  script = "jl/fit.jl",
  julia_version = "1.11",            # resolved via juliaup
  retrieve = "result"
)
```

### Use case 3: an explicit Julia home

If discovery fails (e.g. a fresh install not yet on `PATH`, or the Windows juliaup shim), point at the `bin` directory. You can set it once globally:

``` r
options(tarpolyglot.julia_home = "C:/Users/me/.julia/juliaup/.../bin")
```

or per step, again reusing `jl/fit.jl` as a stand-in:

``` r
tar_target_jl(
  name = fit,
  script = "jl/fit.jl",
  julia_home = "C:/Users/me/.julia/juliaup/.../bin",
  retrieve = "result"
)
```

### Use case 4: a pinned project (recommended)

For reproducibility, activate a Julia **project** with a committed `Manifest.toml`, and `using` the packages the script needs. `jl/solve.jl`:

``` julia
A = [2.0 0.0; 0.0 2.0]
result = tr(A)
```

``` r
tar_target_jl(
  name = solve,
  script = "jl/solve.jl",
  julia_project = "julia/MyEnv",     # folder with Project.toml + Manifest.toml
  julia_packages = c("LinearAlgebra", "Statistics"),
  retrieve = "result"
)
```

The requested project takes priority over an ambient `JULIA_PROJECT` environment variable (e.g. one inherited by `crew` workers): it is cleared for the duration of the Julia binding, so your explicit `julia_project` (or the global environment you get when none is given) wins.

> **One interpreter per session.** JuliaCall binds a single Julia per R session, so all Julia targets that run in the same session share one project and set of loaded packages. To use different projects in one pipeline, run those targets on separate `crew` workers (see `vignette("get_started")`).

## Tracking scripts as dependencies

Reuses `jl/fit.jl` from "Dynamic branching" above as a stand-in script (again without `inputs = c(df = ...)`, so `df` is undefined here, the point of this example is `tar_target_path()`, not the computation):

``` r
list(
  tar_target(fit_jl, "jl/fit.jl", format = "file"),

  tar_target_jl(
    name = fit,
    script = tar_target_path("fit_jl"),   # re-runs when jl/fit.jl changes
    inputs = c(x = "data"),
    retrieve = "result"
  )
)
```

## Conversion caveats

- Julia is **1-indexed** (unlike Python).
- Check the JuliaCall conversion rules for which R/Julia types round-trip; for anything that does not, use file mode.

See `?tar_target_jl` and `?run_jl_step` for the full argument reference.