Getting Started with tezr

This vignette walks through the core features of tezr. Each section builds on the previous one, starting with simple keyword searches and progressing to multi-filter queries, detail retrieval, and cache management.

The vignette does not run live NTC queries during ordinary package builds. It uses representative output so readers can see the returned shapes without depending on the live portal. Set TEZR_LIVE_EXAMPLES=true before rendering if you want to execute the portal requests.

library(tezr)
library(dplyr)

tezr has three search functions: search_basic(), search_advanced(), and search_detailed(). The sections below cover each function in order.

Responsible Use

Use tezr for academic, reproducible research workflows. Cache or save large results locally, avoid repeated identical requests, and respect the NTC portal’s access controls and terms. Do not post private researcher data, access tokens, local cookies, or sensitive institutional details in issues or pull requests.

When publishing results, cite both tezr and the National Thesis Center or Council of Higher Education data source. Record the query terms, filters, retrieval date, and any completeness warnings returned by the package.

More Than Downloading

tezr does more than submit a web form and return rows. It resolves lookup labels, filters searches through structured portal fields, paginates capped result sets by adaptive year ranges, parses bilingual metadata, deduplicates expanded searches, annotates results with completeness attributes, caches repeated requests, and exposes statistics tables that help check retrieval completeness.

Retrieving Detailed Metadata

Search results contain core metadata (title, author, university, year, type, subject). If you need full details, such as abstracts, keywords, supervisor names, page counts, and PDF links, you can use detail() function.

Single Thesis

You can pass a single detail_id from search results to detail(). The function returns a one-row tibble.

# Search and get details for the first match
ankara_econ <- search_detailed(
  university = "Ankara Üniversitesi",
  division = "İktisat Ana Bilim Dalı",
  thesis_type = "phd",
  year_start = 2024,
  year_end = 2025
)

ankara_econ_details <- detail(ankara_econ$detail_id[2])

dplyr::glimpse(ankara_econ_details)

# English abstract
ankara_econ_details$abstract_translation
#> Rows: 1
#> Columns: 24
#> $ thesis_no            <chr> "9677551"
#> $ title_original       <chr> "Parasal aktarim mekanizmasi cercevesinde ozel se…
#> $ title_translation    <chr> "Domestic credit to private sector and monetary t…
#> $ author               <chr> "PERIHAN EZGI BALLI"
#> $ advisor              <chr> "PROF. DR. HASAN SAHIN"
#> $ co_advisor           <chr> NA
#> $ university           <chr> "Ankara Universitesi"
#> $ institute            <chr> "Sosyal Bilimler Enstitusu"
#> $ division             <chr> "Iktisat Ana Bilim Dali"
#> $ year                 <chr> "2020"
#> $ pages                <chr> "153"
#> $ thesis_type_tr       <chr> "Doktora"
#> $ thesis_type_en       <chr> "Doctorate"
#> $ language_tr          <chr> "Turkce"
#> $ language_en          <chr> "Turkish"
#> $ subject_tr           <chr> "Ekonomi; Enerji"
#> $ subject_en           <chr> "Economics; Energy"
#> $ abstract_original    <chr> "Enerji piyasasi duzenlemelerinin ana ekseninde e…
#> $ abstract_translation <chr> "This thesis consists of three essays on energy m…
#> $ keywords_tr          <chr> "Enerji piyasalari; Duzenleme; Elektrik"
#> $ keywords_en          <chr> "Energy markets; Regulation; Electricity"
#> $ access_status        <chr> "open"
#> $ pdf_url              <chr> "https://tez.yok.gov.tr/UlusalTezMerkezi/tezIndir…
#> $ detail_url           <chr> "https://tez.yok.gov.tr/UlusalTezMerkezi/tezDetay…
#> [1] "This thesis consists of three essays on energy market regulation, market design, and transformation."

Batch Retrieval

You can also pass a vector of detail_id values to fetch details for multiple theses. The function shows text progress updates by default and fetches uncached records in parallel (up to 5 active requests).

# Fetch details for all results
ankara_econ <- search_detailed(
  university = "Ankara Üniversitesi",
  division = "İktisat Ana Bilim Dalı",
  thesis_type = "phd",
  year_start = 2025,
  year_end = 2026
)

# Batch retrieval
ankara_econ_all_details <- detail(ankara_econ$detail_id)

Aggregate Statistics

These functions return summary statistics tables from the NTC.

# Thesis counts by year
year_stats <- stats_years()
tail(year_stats)
#> # A tibble: 5 × 4
#>    year yuksek_lisans doktora toplam
#>   <int>         <int>   <int>  <int>
#> 1  2021         28906    7128  36034
#> 2  2022         33102    7481  40583
#> 3  2023         35987    8026  44013
#> 4  2024         40124    8841  48965
#> 5  2025         22615    4416  27031
# Thesis counts by university
uni_stats <- stats_universities()
head(uni_stats)

# Thesis counts by subject
subject_stats <- stats_subjects()
head(subject_stats)

# Total counts by thesis type
type_stats <- stats_types()
type_stats

Cache Management

tezr caches search results, detail records, year-range queries, and lookup lists in memory. Caching speeds up repeated queries and reduces server load.

Viewing Cache Status

# Shows: enabled status, item counts, and TTL settings
cache_info()
#> $enabled
#> [1] TRUE
#> 
#> $search_count
#> [1] 0
#> 
#> $range_count
#> [1] 0
#> 
#> $detail_count
#> [1] 0
#> 
#> $search_ttl
#> [1] 3600
#> 
#> $detail_ttl
#> NULL

The output includes search_count, range_count, detail_count, search_ttl, and detail_ttl. Search cache defaults to 3600 seconds (1 hour). Detail cache defaults to NULL (session lifetime; entries stay until you clear them or restart R).

Clearing Cache

You can clear specific cache types or everything at once. The "lookups" option clears cached university/subject/division lists.

# Clear search results only
cache_clear("searches")

# Clear detail records only
cache_clear("details")

# Clear lookup lists (universities, subjects, etc.)
cache_clear("lookups")

# Clear everything
cache_clear("all")

Configuring Cache TTL

You can also adjust time-to-live settings or disable caching entirely. TTL values are in seconds. NULL means entries persist for the entire session.

# 2-hour search cache, 1-week detail cache
cache_config(
  search_ttl = 7200,
  detail_ttl = 604800
)

# Disable caching entirely (every call hits the server)
cache_config(enable = FALSE)

# Re-enable with defaults
cache_config(enable = TRUE, search_ttl = 3600, detail_ttl = NULL)

Request Configuration

tezr identifies itself with a package-specific user agent by default. You can override that value for an institutional network policy or a portal compatibility issue.

request_config(user_agent = "[email protected]")
request_config(reset = TRUE)

Set request_config(verbose = FALSE) or TEZR_VERBOSE=false to silence informational progress messages. Warnings and errors are still shown.

The package applies a two-second request delay, retries requests up to three times, refreshes sessions after 50 logical requests or 20 minutes, and uses in-memory caches for repeated searches, range chunks, details, and lookups. Detail requests fetch uncached records in bounded parallel batches.

Working with Results

Search results are returned in tibbles, so they work directly with dplyr and other tidyverse tools.

climate_change <- search_basic("climate change")

# Count by year
climate_change |>
  dplyr::count(year)

# Filter recent PhDs
climate_change |>
  dplyr::filter(thesis_type_tr == "Doktora", year >= 2020) |>
  dplyr::select(author, year, title_original, university)

# Most common subjects
climate_change |>
  dplyr::count(subject_tr, sort = TRUE) |>
  dplyr::slice_head(n = 10)

See the Analysis Examples vignette for complete analysis workflows with visualizations.

Limitations and Best Practices

Technical Limitations

  • No official API. tezr scrapes <tez.yok.gov.tr> by simulating browser requests and parsing HTML/JavaScript responses. Any change to the portal’s page structure, form parameters, or JavaScript patterns will break the package until updated. This is the primary fragility risk.
  • Single-year overflow. Auto-pagination splits year ranges to work around the 2000-result server cap, but cannot split below a single calendar year. If a query matches more than 2000 theses in one year, the package retrieves only the first 2000 for that year and issues a warning. You should narrow your search with additional filters (thesis type, university, subject) to avoid this.
  • In-memory cache only. All cached data (searches, details, lookups) is stored in R environment objects and lost when the session ends. You can save results to disk with readr::write_rds() or saveRDS() for persistence across sessions.
  • SSL verification disabled. The YÖK server has certificate issues, so SSL peer verification is turned off (ssl_verifypeer = FALSE). This is a security trade-off required for the package to function.
  • Fixed rate limiting. Requests use a built-in 2-second rate limit that is not user-configurable, so fetching large datasets still takes time.
  • Vector parameter expansion. search_detailed() expands vector-valued parameters (e.g., multiple universities, subjects) into separate API calls via cartesian product. Passing many multi-valued parameters can generate a large number of requests (e.g., 10 universities × 5 subjects = 50 searches).
  • Lookup matching. The list_*() lookup functions use exact and substring matching. Typos or minor name variations (for example, missing diacritics) will not match. Use list_universities() and related lookup functions to confirm exact names before searching.
  • Metadata only. The package retrieves thesis metadata. PDF URLs are included in detail records but full-text files are not downloaded. You can use URLs to download the PDFs.

Best Practices

  • Cache and save results. Run large queries once and save locally with readr::write_rds(), readr::write_csv(), or similar functions. Then reload from disk in later sessions.
  • Filter before paginating. Add year ranges, thesis types, and institutional filters to keep result sets manageable before setting max_search_results = Inf.
  • Minimize server load. Use cached results when possible. Avoid repeating identical queries.
  • Validate data quality. Metadata may have inconsistencies (missing fields, encoding issues). Clean and validate before analysis.

Citation

Use citation("tezr") for the preferred package citation. Also cite the National Thesis Center or Council of Higher Education as the source of thesis metadata and include the date you retrieved the data.