Skip to content

Read immune repertoire files into ImmunData

Source code

Description

read_repertoires() is the main function for importing AIRR-seq data. It reads one or more repertoire files, defines biological receptors, adds sample information from an optional manifest, and returns an ImmunData object.

The function saves the processed data in output_folder. This lets you work with large datasets without loading everything into memory and reopen the result later with read_immundata().

Usage

read_repertoires(
  path,
  schema,
  manifest = NULL,
  barcode_col = NULL,
  count_col = NULL,
  locus_col = NULL,
  umi_col = NULL,
  preprocess = make_default_preprocessing(),
  postprocess = make_default_postprocessing(),
  rename_columns = imd_rename_cols("10x"),
  enforce_schema = TRUE,
  manifest_file_col = "file",
  output_folder = NULL,
  repertoire_schema = "<auto>",
  verbose = getOption("immundata.verbose", TRUE),
  prematerialize = TRUE,
  prematerialize_folder = NULL
)

Arguments

path One or more repertoire file paths, or a glob pattern such as “/path/to/data/\*.tsv.gz”. Supported formats are Parquet, CSV, TSV, and gzipped CSV or TSV. All input files must have the same file type. Use “\” to take file paths from manifest instead. In that case, manifest is required.
schema Definition of receptor identity. Supply either:
  • A character vector naming the features that must match, such as c(“v_call”, “j_call”, “junction_aa”).
  • An object created by make_receptor_schema() to select one locus or pair two loci from the same cell.
Use column names as they appear after rename_columns is applied. For example, if the input columns are CDR3.aa and V.name, use rename_columns = c(cdr3_aa = “CDR3.aa”, v_call = “V.name”) together with schema = c(“cdr3_aa”, “v_call”).
manifest An optional data frame with one row per repertoire file and columns containing sample, donor, tissue, treatment, or other information. Use read_manifest() to read and validate a manifest file. Manifest paths must be unique. When path = “\”, the column named by manifest_file_col supplies the repertoire file paths. The default is NULL.
barcode_col Name of the column containing cell barcodes. Supplying it selects single-cell processing, requires umi_col, and prevents use of count_col. Use the column name after renaming. The default is NULL.
count_col Name of the column containing non-negative abundance values for bulk repertoire data. It cannot be used with barcode_col. Use the column name after renaming. The default is NULL.
locus_col Name of the column containing receptor loci such as “TRA”, “TRB”, “IGH”, “IGK”, or “IGL”. It is required when schema selects or pairs chains. Use the column name after renaming. The default is NULL.
umi_col Name of the column containing per-chain UMI or read counts. It is required whenever barcode_col is supplied and is used to choose one chain when a cell contains several chains from the same locus. Use the column name after renaming. The default is NULL.
preprocess A named list of functions applied in order before receptors are defined. Each function must accept a duckplyr table as its first argument and return a duckplyr table. By default, make_default_preprocessing() removes selected technical columns and keeps productive sequences when a productive column is available. Use NULL or list() to disable preprocessing.
postprocess A named list of functions applied in order after receptors are defined and manifest information is added. Each function must accept and return a duckplyr table. By default, make_default_postprocessing() prefixes cell barcodes when the manifest contains a Prefix column. Use NULL or list() to disable postprocessing.
rename_columns An optional named character vector in the form c(new_name = “old_name”). Renaming occurs before preprocessing and receptor definition. The default, imd_rename_cols(“10x”), standardizes common 10x names such as v_gene to v_call and chain to locus when those source columns are present. Use NULL to preserve all input names.
enforce_schema Whether multiple input files must have the same columns and column types. The default is TRUE. If FALSE, columns are combined by name and missing values are added where necessary. This is slower and can require more memory.
manifest_file_col Name of the manifest column containing repertoire file paths when path = “\”. The default is “file”. Use the same name passed as file_col to read_manifest() when it is not “file”.
output_folder Directory in which to write annotations.parquet and metadata.json. These files are the persistent backing storage for the returned object. If NULL, a folder beginning with immundata- is created beside the first input file. Supplying an existing folder replaces its annotations.parquet and metadata.json. The default is NULL.
repertoire_schema Definition of repertoires. Supply one of:
  • A character vector naming columns that define one repertoire, such as c(“donor”, “timepoint”).
  • “\”, the default. This creates one repertoire per input file, or one per manifest row when path = “\”.
  • “\”, which uses all manifest columns when a manifest is available, or the input filename otherwise.
  • NULL to leave repertoires undefined.
verbose Whether to print progress and summary messages. Defaults to getOption(“immundata.verbose”, TRUE).
prematerialize Whether CSV, TSV, and compressed text inputs should be combined into a temporary Parquet file before receptor processing. This avoids repeatedly scanning text input during downstream lazy queries. Existing Parquet input is used directly. The default is TRUE.
prematerialize_folder Directory in which to create the temporary combined Parquet file. If NULL, the default, tempdir() is used. The directory is created when necessary. The temporary file is deleted when read_repertoires() exits, including after an error.

Details

The required arguments depend on how receptor observations are represented in the input files.

Value

A disk-backed ImmunData object containing the retained chain rows, receptor definitions, manifest annotations, and ingestion provenance. If repertoire_schema is not NULL, it also contains repertoire definitions and summary statistics calculated by agg_repertoires().

Choose arguments for your data

  • Uncounted repertoire table: Supply schema. Leave barcode_col and count_col as NULL. Each retained row represents one observed chain.
  • Bulk repertoire with abundance: Supply schema and count_col. The abundance values are preserved for later repertoire statistics.
  • Single-cell, one selected chain: Use make_receptor_schema() with one chain and supply barcode_col, locus_col, and umi_col.
  • Single-cell, paired chains: Use make_receptor_schema() with two chains and supply barcode_col, locus_col, and umi_col. Only cells containing both requested chains are retained.
  • Single-cell, relaxed paired chains: Use a schema such as chains = c(“IGH”, “IGL|IGK”) with barcode_col, locus_col, and umi_col. This accepts either an IGH-IGL or IGH-IGK receptor.

In single-cell data, the chain with the highest umi_col value is retained when a cell contains several chains from the same locus.

What happens by default

Unless you override the relevant arguments, read_repertoires():

  • temporarily combines text input into Parquet before processing;
  • standardizes common 10x column names;
  • removes selected technical columns;
  • keeps productive sequences when productivity information is present;
  • prefixes barcodes when a manifest Prefix column is present;
  • creates repertoires automatically; and
  • writes the completed dataset to disk.

Set rename_columns, preprocess, postprocess, or repertoire_schema to NULL to disable the corresponding behavior.

Processing order

The function:

  1. finds and reads the input files as one duckplyr table;
  2. temporarily combines non-Parquet input into one Parquet file when prematerialize = TRUE;
  3. renames columns;
  4. applies preprocessing;
  5. defines receptors using schema;
  6. adds manifest information;
  7. applies postprocessing;
  8. defines repertoires when requested; and
  9. writes and reopens the completed ImmunData dataset.

Manifests and repertoires

A manifest annotates each input file with biological information. The repertoire_schema argument chooses which annotation columns define a repertoire and therefore determine receptor counts and proportions.

With path = “\<manifest>” and the default repertoire_schema = “\<auto>”, all manifest columns are used and each manifest row becomes one repertoire. With an explicit file path or vector of paths, “\<auto>” creates one repertoire per input file.

Output storage

The output folder is not a temporary cache. The returned object reads its receptor annotations from annotations.parquet, while metadata.json stores its schemas, repertoire summaries, and provenance. Keep this folder for as long as you need the object, or reopen it later with read_immundata().

Important: Reusing the same output_folder replaces the existing annotations.parquet and metadata.json without creating a new version.

See Also

read_manifest(), make_receptor_schema(), agg_receptors(), agg_repertoires(), make_default_preprocessing(), make_default_postprocessing(), read_immundata(), write_immundata(), ImmunData

Examples

library("immundata")

library(immundata)
library(dplyr)

options(immundata.verbose = FALSE)

# Read one bulk AIRR file and preserve its abundance column
bulk_file <- system.file(
  "extdata/tsv",
  "sample_0_1k.tsv",
  package = "immundata"
)

bulk_idata <- read_repertoires(
  path = bulk_file,
  schema = c("cdr3_aa", "v_call"),
  count_col = "counts",
  output_folder = tempfile("immundata-bulk-")
)

tibble(
  n_records = bulk_idata |> count() |> pull(n),
  n_receptors = bulk_idata$receptors |> count() |> collect() |> pull(n),
  n_repertoires = nrow(bulk_idata$repertoires)
)

#> # A tibble: 1 × 3
#>   n_records n_receptors n_repertoires
#>       <int>       <int>         <int>
#> 1       955         871             1

# Expected result:
#   n_records n_receptors n_repertoires
#         955         871             1

# Read multiple files and their sample information from a manifest
manifest_path <- system.file(
  "extdata/tsv",
  "manifest.csv",
  package = "immundata"
)
manifest <- read_manifest(manifest_path)

manifest_idata <- read_repertoires(
  path = "<manifest>",
  manifest = manifest,
  schema = c("cdr3_aa", "v_call"),
  count_col = "counts",
  output_folder = tempfile("immundata-manifest-")
)

manifest_idata$repertoires |>
  select(Therapy, Response, n_barcodes, n_receptors) |>
  arrange(Response)

#> # A tibble: 2 × 4
#>   Therapy Response n_barcodes n_receptors
#> * <chr>   <chr>         <int>       <int>
#> 1 ICI     FR             4725         871
#> 2 CAR-T   PR             4758         867

# Expected result:
#   Therapy Response n_barcodes n_receptors
#   ICI     FR             4725         871
#   CAR-T   PR             4758         867

# Read paired TRA-TRB receptors from a small single-cell table
paired_input <- tibble(
  cell_id = c("cell1", "cell1", "cell2", "cell2", "cell3"),
  locus = c("TRA", "TRB", "TRA", "TRB", "TRA"),
  v_call = c("TRAV1", "TRBV1", "TRAV1", "TRBV1", "TRAV2"),
  j_call = c("TRAJ1", "TRBJ1", "TRAJ1", "TRBJ1", "TRAJ2"),
  junction_aa = c("CAVA", "CASSB", "CAVA", "CASSB", "CAVC"),
  umi_count = c(10L, 8L, 12L, 9L, 7L)
)
paired_file <- tempfile(fileext = ".tsv")
readr::write_tsv(paired_input, paired_file)

paired_idata <- read_repertoires(
  path = paired_file,
  schema = make_receptor_schema(
    features = c("v_call", "j_call", "junction_aa"),
    chains = c("TRA", "TRB")
  ),
  barcode_col = "cell_id",
  locus_col = "locus",
  umi_col = "umi_count",
  repertoire_schema = NULL,
  output_folder = tempfile("immundata-paired-")
)

tibble(
  n_chains = paired_idata |> count() |> pull(n),
  n_cells = paired_idata |> collect() |> distinct(imd_barcode) |> nrow(),
  n_receptors = paired_idata$receptors |> count() |> collect() |> pull(n)
)

#> # A tibble: 1 × 3
#>   n_chains n_cells n_receptors
#>      <int>   <int>       <int>
#> 1        4       2           1

# Expected result:
#   n_chains n_cells n_receptors
#          4       2           1