data-in-r-packages

# Data in R Packages ## Overview R packages can include data in several forms: exported datasets for users, internal data for package functions, raw data files, and dynamic package state. This skill covers all data types, documentation requirements, and CRAN restrictions. ## Four Types of Package Data ### 1. Exported Data (data/) User-accessible datasets loaded with `data()` or direct reference. **Location**: `data/` directory **Format**: `.rda` or `.RData` files **Access**: `data(dataset_name)` or direct reference (if LazyData: true) **Documentation**: Required in `R/data.R` ### 2. Internal Data (R/sysdata.rda) Data used by package functions, not accessible to users. **Location**: `R/sysdata.rda` **Format**: Single `.rda` file containing multiple objects **Access**: Direct reference in package code **Documentation**: Not required (internal only) ### 3. Raw Data Files (inst/extdata/) Non-R data files (CSV, JSON, images, etc.) for users to access. **Location**: `inst/extdata/` **Format**: Any file format **Access**: `system.file("extdata", "file.ext", package = "pkg")` **Documentation**: Optional, usually in vignettes/examples ### 4. Dynamic Package State (environments) Runtime state stored in package environments. **Location**: Environment created in R code **Format**: In-memory R objects **Access**: Getter/setter functions **Documentation**: Document the getter/setter functions ## Exported Data (data/) ### Creating Exported Data ```r # Prepare your data: my_dataset <- data.frame( id = 1:100, value = rnorm(100), category = sample(LETTERS[1:3], 100, replace = TRUE) ) # Save to data/: usethis::use_data(my_dataset, overwrite = TRUE) ``` This creates `data/my_dataset.rda`. ### Multiple Datasets ```r # Save multiple datasets: dataset1 <- mtcars[1:10, ] dataset2 <- iris[1:50, ] usethis::use_data(dataset1, dataset2, overwrite = TRUE) # Creates data/dataset1.rda and data/dataset2.rda ``` ### Compression Options ```r # Default compression (gzip): usethis::use_data(my_dataset) # Maximum compression (xz - slowest, smallest): usethis::use_data(my_dataset, compress = "xz") # Faster compression (bzip2): usethis::use_data(my_dataset, compress = "bzip2") # No compression (largest): usethis::use_data(my_dataset, compress = FALSE) ``` **CRAN recommendation**: Use `compress = "xz"` for data >1MB. ### LazyData Add to DESCRIPTION to make data available without `data()` call: ```dcf LazyData: true ``` ```r # Without LazyData: library(mypackage) data(my_dataset) # Required head(my_dataset) # With LazyData: library(mypackage) head(my_dataset) # Direct access ``` **Note**: LazyData loads datasets into namespace but keeps them on disk until accessed (lazy loading). ## Internal Data (R/sysdata.rda) ### Creating Internal Data Internal data is for package functions only, not exported to users. ```r # Create internal lookup tables, constants, etc.: internal_lookup <- list( codes = c(A = 1, B = 2, C = 3), thresholds = c(low = 0.05, high = 0.95) ) internal_constants <- list( api_version = "v2", default_timeout = 30 ) # Save to R/sysdata.rda: usethis::use_data( internal_lookup, internal_constants, internal = TRUE, overwrite = TRUE ) ``` **All objects saved with `internal = TRUE` go into a single file**: `R/sysdata.rda` ### Using Internal Data ```r # In package functions, reference directly: my_function <- function(code) { value <- internal_lookup$codes[code] # ... use value ... } # No need for pkg:::sysdata syntax # Objects are available in package namespace ``` ### When to Use Internal Data **Good uses:** - Lookup tables - Large constants - Pre-computed values (avoid recomputation) - Default configurations **Avoid:** - Data that changes (use environments instead) - User-facing data (use data/ instead) - Very large objects (consider lazy loading strategies) ## Raw Data Files (inst/extdata/) ### Adding Raw Data Files ```r # Create inst/extdata/ directory: dir.create("inst/extdata", recursive = TRUE) # Add files manually or: usethis::use_directory("inst/extdata") # Then copy files into inst/extdata/ ``` **Common file types:** - CSV, TSV, Excel files - JSON, XML, YAML - Images (PNG, JPEG) - Shapefiles, GeoJSON - Text files, logs - Binary formats ### Accessing Raw Data Files ```r # In package code: get_example_file <- function(filename) { system.file("extdata", filename, package = "mypackage") } # Usage: csv_path <- system.file("extdata", "example.csv", package = "mypackage") data <- read.csv(csv_path) # Or in exported function: #' @examples #' file <- system.file("extdata", "example.csv", package = "mypackage") #' data <- read_my_data(file) read_my_data <- function(file) { # ... } ``` ### inst/ vs data/ ``` inst/extdata/ # Raw files, any format ├── example.csv # Access with system.file() ├── sample.json └── image.png data/ # R objects only ├── dataset1.rda # Access with data() or direct reference └── dataset2.rda ``` **Use inst/extdata/ when:** - Non-R formats (CSV, JSON, etc.) - Files users need paths to - Multiple related files - Files for examples/vignettes **Use data/ when:** - R objects for analysis - Data ready to use in R - Common datasets for package functions ## Documenting Data ### Documenting Exported Data Create `R/data.R` to document all datasets: ```r # R/data.R #' World Health Organization TB data #' #' A subset of data from the World Health Organization Global Tuberculosis #' Report, containing TB cases by country, year, age, and sex. #' #' @format A data frame with 7,240 rows and 60 columns: #' \describe{ #' \item{country}{Character. Country name} #' \item{iso2}{Character. 2-letter ISO country code} #' \item{iso3}{Character. 3-letter ISO country code} #' \item{year}{Integer. Year of observation (1995-2013)} #' \item{new_sp_m014}{Integer. New smear-positive cases in males aged 0-14} #' \item{new_sp_m1524}{Integer. New smear-positive cases in males aged 15-24} #' } #' #' @source World Health Organization Global Tuberculosis Report #' \url{https://www.who.int/teams/global-tuberculosis-programme/data} #' #' @examples #' head(who) #' summary(who$year) #' table(who$country) "who" ``` **Required tags:** - `@format` - describe structure and columns - Title and description (always) **Recommended tags:** - `@source` - where data came from - `@examples` - how to use the data ### Data Documentation Templates #### Data Frame ```r #' Customer transaction data #' #' Sample transaction data for 1,000 customers over one year, #' including purchase amounts, dates, and categories. #' #' @format A data frame with 1,000 rows and 5 columns: #' \describe{ #' \item{customer_id}{Character. Unique customer identifier} #' \item{transaction_date}{Date. Date of transaction} #' \item{amount}{Numeric. Transaction amount in USD} #' \item{category}{Factor. Product category (Electronics, Clothing, Food)} #' \item{region}{Character. Customer region (North, South, East, West)} #' } #' #' @details #' Data was generated synthetically to represent typical e-commerce #' transaction patterns. Amounts range from $5 to $500. #' #' @source Generated using simulation based on real e-commerce patterns #' #' @examples #' head(transactions) #' #' # Summary by category #' aggregate(amount ~ category, data = transactions, FUN = mean) #' #' # Transactions over time #' plot(transactions$transaction_date, transactions$amount) "transactions" ``` #### List ```r #' Configuration defaults #' #' Default configuration settings for the package. #' #' @format A list with components: #' \describe{ #' \item{api}{List. API configuration:} #' \itemize{ #' \item \code{endpoint}: Character. Base API URL #' \item \code{timeout}: Numeric. Request timeout in seconds #' \item \code{retries}: Integer. Number of retry attempts #' } #' \item{cache}{List. Cache settings:} #' \itemize{ #' \item \code{enabled}: Logical. Whether caching is enabled #' \item \code{max_size}: Numeric. Maximum cache size in MB #' } #' } #' #' @examples #' config_defaults$api$endpoint #' config_defaults$cache$enabled "config_defaults" ``` #### Vector ```r #' Built-in color palette #' #' A vector of hex color codes for data visualization. #' #' @format A named character vector of length 12: #' \describe{ #' \item{Names}{Color names (red, blue, green, etc.)} #' \item{Values}{Hex color codes} #' } #' #' @examples #' palette_colors #' palette_colors["blue"] #' #' # Use in plot #' plot(1:12, col = palette_colors, pch = 16, cex = 2) "palette_colors" ``` #### Matrix ```r #' Correlation matrix example #' #' Sample correlation matrix for demonstration purposes. #' #' @format A 10x10 numeric matrix with row and column names #' representing variables v1 through v10. Values are correlations #' ranging from -1 to 1. #' #' @examples #' correlation_matrix #' diag(correlation_matrix) # All 1s (self-correlation) "correlation_matrix" ``` ## data-raw/ Workflow Keep data preparation scripts separate from package code. ### Setup ```r # Create data-raw/ directory and template script: usethis::use_data_raw("dataset_name") ``` This creates: - `data-raw/` directory - `data-raw/dataset_name.R` script - Adds `^data-raw$` to `.Rbuildignore` ### Data Preparation Script ```r # data-raw/customer_data.R ## Code to prepare `customer_data` dataset library(dplyr) library(lubridate) # Read raw data: raw_data <- read.csv("~/Downloads/raw_customer_data.csv") # Clean and process: customer_data <- raw_data %>% # Clean column names: janitor::clean_names() %>% # Parse dates: mutate( transaction_date = ymd(transaction_date), signup_date = ymd(signup_date) ) %>% # Filter to relevant period: filter( transaction_date >= "2020-01-01", transaction_date <= "2023-12-31" ) %>% # Select and rename: select( customer_id = id, transaction_date, amount = transaction_amount, category = product_category, region = customer_region ) %>% # Remove duplicates: distinct() %>% # Sort: arrange(transaction_date) # Save to package: usethis::use_data(customer_data, overwrite = TRUE, compress = "xz") ``` ### Benefits of data-raw/ - **Reproducibility**: Anyone can recreate the data