---
title: "Getting Started with pkgaudit"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Getting Started with pkgaudit}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment  = "#>"
)
```

pkgaudit scans R packages for security-relevant files and code without executing
anything it scans. It reports what code does and when it runs, so code that runs
on install or load is distinguishable from code that runs only when called.

A finding is not an accusation. Nearly all of the patterns and matches flagged
by pkgaudit have legitimate uses in R packages. pkgaudit helps to identify what
deserves reviewer attention, not what is malicious.

```{r setup}
library(pkgaudit)
```

## Example package

`untrustedpkg` is a small source package shipped with pkgaudit for
demonstration. It is never built, checked, installed, or loaded; it exists only
to be scanned.

```{r example-package}
tarball <- system.file(
  "extdata", "untrustedpkg", "untrustedpkg_0.1.0.tar.gz",
  package = "pkgaudit"
)

exdir <- file.path(tempdir(), "untrustedpkg-example")
utils::untar(tarball, exdir = exdir)
pkg <- file.path(exdir, "untrustedpkg")
```

`untrustedpkg` contains the following files.

```{r example-list, echo = FALSE, comment = ""}
list.files(pkg, recursive = TRUE)
```

The script that generates this package is in
[data-raw/create_untrustedpkg.R](https://github.com/tylerjssmith/pkgaudit/blob/main/data-raw/create_untrustedpkg.R).

## Auditing a package

`audit_package()` scans an untarred source package. `print()` gives the scan
metadata and the number of findings by category.

```{r audit}
result <- audit_package(pkg)
print(result, path = FALSE)
```

`summary()` reports the number of findings by rule and by phase. It also
provides the MITRE ATT&CK techniques associated with each rule.

```{r summary}
summary(result, path = FALSE)
```

Phases overlap -- building a package with vignettes, for example, also installs
and loads it -- and one occurrence is counted under every phase it runs in. The
summary above reflects five findings, some counted under multiple phases. To see
only what is known to run automatically on `library()`, for example, pass an
argument to `phase`:

```{r summary-phase}
summary(result, phase = c("at_load", "at_attach"), path = FALSE)
```

Both `print()` and `summary()` accept `path = FALSE`, which omits the local
filesystem path. This can matter when sharing results, since the path may reveal
a username or directory layout.

## Examining results

A `pkgaudit` object is a named list of ordinary data frames, plus a named list
of scan metadata, so findings can be filtered, joined, and reported on directly.

```{r names}
names(result)
```

### File contexts

`$file_contexts` reports security-relevant files, whatever they contain: R runs
a `configure` script, which can execute shell commands, so the script should be
reviewed.

```{r file-contexts-columns}
result$file_contexts[, c("rule", "file_context")]
```

### Patterns

`$patterns` reports security-relevant R calls, located by `file_context`,
`line_number`, and `column_number`.

```{r patterns-columns1}
result$patterns[, c("rule", "file_context", "line_number", "column_number")]
```

Additionally, `code_context` indicates where code sits within a file. File and
code context are used to determine the phases. `guarded` is `TRUE` when a call
may be stopped from running in the expected phase, e.g., code in `\dontrun{}` in
an `\examples{}` block or a code chunk marked `eval=FALSE` in a vignette.
`indirect` is `TRUE` when a call is made through a function name, e.g.,
`do.call("system", ...)` instead of `system()`.

```{r patterns-columns2}
result$patterns[, c("code_context", "guarded", "indirect")]
```

`preview` provides a snippet of the code and its surroundings, which may allow
reviewers to determine the relevance of a finding without opening the file.

```{r patterns-columns3}
result$patterns[, c("preview")]
```

### Matches

`$matches` reports text matching regular expressions in shell and Make-like
files. Its columns mirror `$patterns`, but it does not have `code_context`,
`guarded`, or `indirect`.

```{r matches-columns1}
result$matches[, c("rule", "file_context", "line_number", "column_number")]
```

```{r matches-columns2}
result$matches[, c("preview")]
```

### Coverage

`$coverage` reports what pkgaudit made of each file it found: `parsed` for R,
which is matched against its XML parse tree; `matched` for shell and Make-like
files, which are matched as text; `exportable` for languages pkgaudit does not
read, such as C and Python; `unexamined` for files pkgaudit did not read, such
as serialized `.rda` and `.rds` objects; and `error` for files pkgaudit tried to
read and could not.

```{r coverage}
result$coverage[, c("file_context", "language", "status", "reason", "lines")]
```

### Errors

`$errors` reports files that pkgaudit tried to read and could not, with
information about what may have gone wrong. pkgaudit did not encounter errors
when scanning `untrustedpkg`.

### Metadata

`$metadata` is a named list of metadata.

```{r metadata}
result$metadata
```

### Subsetting by phase

Every findings frame carries one logical column per lifecycle phase:
`at_autoconf`, `at_build`, `at_check`, `at_install_src`, `at_install_bin`,
`at_load`, `at_attach`, `at_unload`, `at_detach`. A pattern inside an ordinary
function is `FALSE` for all of them, since it runs only if something calls it.
These can be used to subset the data frames and see what findings apply to each
phase. For example, to see what is known to run when `untrustedpkg` is loaded:

```{r by-phase}
result$patterns[result$patterns$at_load,
                c("rule", "file_context", "code_context")]
```

## Reviewing findings

pkgaudit is a guide to human review, not a substitute for human judgment. Below
we discuss how reviewers should evaluate the four patterns and one match for
`untrustedpkg` and inspect the relevant files. In this case, it appears that
`untrustedpkg` should *not* be used.

### Patterns

1. An `\Sexpr{}` macro in `man/fetch_data.Rd` calls `httr::POST()` when the help
page is rendered at build, check, and source installation. Reviewers should
consider whether a help page should make an HTTP POST request and what it may
send to an external host.

2. The `\examples{}` block in the same help file calls `download.file()` at
check. Reviewers should verify what is downloaded. Since this depends on an
external host, and what is downloaded may change over time, reviewers should
also consider how the download is used -- for example, could another function
execute code if the file ever contained it?

```{r example-rd, echo = FALSE, comment = ""}
cat(readLines(file.path(pkg, "man", "fetch_data.Rd")), sep = "\n")
```

3. `.onLoad()` in `R/zzz.R` calls `system()` and runs when users call
`library()`. It also runs at build, check, and source installation, each of
which loads the package. While `system()` calls are common in R packages, they
are less common in lifecycle hooks like `.onLoad()`, and reviewers should
inspect what commands would be run automatically on their systems.

```{r example-onload, echo = FALSE, comment = ""}
cat(readLines(file.path(pkg, "R", "zzz.R")), sep = "\n")
```

4. A regular function in `R/fetch.R` calls `download.file()`, but code inside a
regular function in `R/` is not known to run automatically, so this pattern is
reported under `none`. A reviewer may still want to inspect if and when the
function is called, and what would be downloaded.

```{r example-fetch, echo = FALSE, comment = ""}
cat(readLines(file.path(pkg, "R", "fetch.R")), sep = "\n")
```

### Matches

1. The `configure` script may invoke `curl` at build, check, and source
installation. Some R packages use `curl` to fetch dependencies that cannot be
vendored with the package. Reviewers should verify what is fetched and what
could happen if that changes.

```{r example-configure, echo = FALSE, comment = ""}
cat(readLines(file.path(pkg, "configure")), sep = "\n")
```

## Exporting findings

pkgaudit can integrate its scan with other tools. `emit_sarif()` renders its
results as SARIF 2.1.0, which editors and code-scanning platforms read directly.
It returns the document as a string, which users may write to disk.

```{r sarif}
sarif <- emit_sarif(result)
substr(sarif, 1, 200)
```

Written to a file and opened in an editor with a SARIF viewer, such as VS Code
with the
[SARIF Viewer extension](https://marketplace.visualstudio.com/items?itemName=MS-SarifVSCode.sarif-viewer),
each finding appears on the line it was found. `level` is `note` for every
result. When code executes is carried in `properties.phases`.

`export_unscanned()` exports code written in languages pkgaudit cannot read,
like C and Python, to a directory for a scanner like Semgrep that can. A whole
file is copied verbatim; a vignette chunk is written into a file of its own,
blank-padded so that its code sits at the same line numbers it occupies in the
source. A finding another tool reports at line 40 of `intro.python.py` would
therefore be at line 40 of `intro.Rmd`. `untrustedpkg` contains only R and
shell, so here `export_unscanned()` would create an empty directory.

## Auditing a tarball

`audit_tarball()` takes a `.tar.gz`, `.tgz`, or `.tar` source package, validates
it using `validate_tar()`, extracts it to a temporary directory, scans it, and
removes the directory. An untrusted archive is itself an attack surface, so
validation fails closed: the whole archive is refused rather than partially
extracted.

```{r tarball}
print(audit_tarball(tarball), path = FALSE)
```

The result is a `pkgaudit` object like any other, so everything above applies to
it unchanged.

## Rule database integrity

The rules live in a versioned SQLite database shipped with the package.
`load_rules()` reads it; `rules_version()` reports the version.

```{r rules}
rules_version()

rules <- load_rules()
vapply(rules, nrow, integer(1))
```

A modified database is one way to evade a scanner. `load_rules()` verifies the
database against its bundled SHA-256 sidecar on every call and refuses to load
a modified one. The hash of an installed copy can also be checked against the
value published in the
[README](https://github.com/tylerjssmith/pkgaudit#rule-database-integrity):

```{r hash}
digest::digest(
  system.file("db", "rules.db", package = "pkgaudit"),
  algo = "sha256",
  file = TRUE
)
```

The full rule set is documented in [Rule Coverage](rules.html). How pkgaudit
works internally is in [Internals](internals.html).