Builds a Data Governance File (DGF) from a dataset: one row per variable, carrying the metadata, summary statistics, and graphics data needed to render a Research Guide. Writes a `.csv` and an `.xlsx`, and returns the DGF data frame.
Usage
create_dgf(
df,
vtypes = NULL,
use.df.types = FALSE,
read_as_text = TRUE,
guess.factors = TRUE,
guess.dates = FALSE,
max_factor_levels = 50,
dd = NULL,
var_name = NULL,
dd_vlabel = NULL,
dd_vdesc = NULL,
stable_data_type = NULL,
desired_data_import_rule = NULL,
stable_data_format = NULL,
dd_vname_alias = NULL,
keep.dd.cols = NULL,
preview_dd = F,
preview_dp = F,
dir = ".",
file = "DGF",
open = interactive(),
workers = getOption("datagoodr.workers", 1L),
guess_seed = getOption("datagoodr.guess_seed", 1L),
guess_n_sample = getOption("datagoodr.guess_n_sample", 300L),
preflight = c("warn", "error", "off")
)Arguments
- df
A data frame to document, or a path to a `.csv` file to read.
- vtypes, guess.dates, dd, keep.dd.cols, preview_dd, preview_dp
Accepted but currently ignored; the features they were intended to steer (explicit type overrides, date/time detection, and data-dictionary merging) are not implemented yet. See `dev/TO-DO.md`.
- use.df.types
Logical; keep the data frame's existing column types instead of re-reading the data with `readr` to re-infer them. Defaults to `FALSE`.
- read_as_text
Logical; read every column as character to preserve the source representation exactly - avoiding `readr`'s inference dropping leading zeros (`06037` -> `6037`) or casting long-integer IDs to lossy doubles - then promote only the columns that are safe to store as numeric (see internal `.safe_numeric`). Defaults to `TRUE`. Set `FALSE` for the legacy `readr`-inference behaviour.
- guess.factors
Logical; treat low-cardinality columns as factors. Defaults to `TRUE`.
- max_factor_levels
Integer; the maximum number of factor/logical levels stored in the `dd_f_levels` dictionary. High-cardinality factors are capped to the most frequent `max_factor_levels` levels, matching the frequency-ranked FACTOR LEVELS table in the Research Guide (and keeping the cell under Excel's 32,767-character limit). Defaults to `50`.
- var_name, stable_data_type
Accepted but ignored: `var_name` is always the column names of `df`, and `stable_data_type` is computed from the data. Both are legacy arguments slated for removal (see `dev/TO-DO.md`).
- dd_vlabel, dd_vdesc
Optional character vectors of per-variable short labels and long descriptions, each the same length and order as the columns of `df`. `dd_vlabel` defaults to the column names, `dd_vdesc` to blank.
- desired_data_import_rule
Optional character vector of type-conversion function names (e.g. `"read_yyyy"`), one per column, applied to the data before profiling.
- stable_data_format
Optional character vector of display-formatting function names (e.g. `"dollarize"`), one per column, applied to previews and graphics.
- dd_vname_alias
Optional character vector of per-variable alternate names. Defaults to blank.
- dir
Directory to write the DGF into. Created if it does not exist. Defaults to `"."`. Ignored when `file` is an absolute path.
- file
Output path stem (without extension) for the written `.csv` and `.xlsx`. Defaults to `"DGF"`. Resolved relative to `dir` unless absolute.
- open
Logical; open the new DGF in Excel once created. Defaults to [interactive()] — on at the console, off in scripts, tests, and CI.
- workers
Integer; how many processes to spread the per-column type guessing over. Type guessing is the dominant cost of a DGF build (measured at 63 `rg_*` profiles), it is flat in the number of rows, and it is independent per column — so columns, not rows, are what parallelises. Defaults to `1` (in-process); worth raising for wide frames, where the pool's start-up cost is amortised over more columns. Also settable with `options( datagoodr.workers = )`.
- guess_seed
Integer or `NULL`; seed for the 100-value sample `formex::guess_data_type()` draws internally. Fixed by default so repeated builds of the same data produce the same DGF — left to the session RNG, the guessed types genuinely vary between runs. `NULL` restores that older behaviour. Also settable with `options( datagoodr.guess_seed = )`.
- guess_n_sample
Integer or `NULL`; how many values per column `formex::guess_data_type()` samples when guessing its type. Defaults to `300`; `NULL` hands control back to formex, whose own default is 100.
formex samples from a column's **distinct** values, so its reported confidence is stable exactly when `guess_n_sample >= n_distinct` — below that line it is a sample statistic, and a column whose confidence lands near the acceptance threshold is decided by which values were drawn. Measured on `bb.csv`'s `birthState` (210 distinct), confidence spans 0.631-0.905 at 100 and is a flat 0.813 at 300, where the column resolves to a geographic categorical instead of falling back to text. 300 exhaustively covers most real columns; raising it further costs time on high-cardinality ones, where the draw is merely reproducible (see `guess_seed`) rather than complete. Also settable with `options( datagoodr.guess_n_sample = )`.
- preflight
What to do with the findings of [preflight_dgf()], which runs (far faster than a build) *before* any profiling and looks for degenerate columns (single-valued, all-missing, over-long values, a flag spelled several ways, duplicate columns, ...). `"warn"` (the default) prints a summary and goes on; `"error"` stops if any column is flagged as a warning or an error, for pipelines that must not proceed on questionable data; `"off"` skips the check. The findings are kept on the result: `attr(dgf, "preflight")`.
Value
The DGF as a data frame, one row per variable. Also writes `<file>.csv` and `<file>.xlsx` as a side effect. The [preflight_dgf()] findings are attached as `attr(, "preflight")` (`NULL` when `preflight = "off"`).
Details
Column types are inferred from the data (see `vtypes` / `use.df.types` / `guess.factors` to steer that), then each variable is profiled into the `rg_*` columns. The result is meant to be curated by hand - in Excel or via the `v*` arguments - and fed to [create_rg()].
A timestamp column stores one graphic row per distinct value, which for a column that is nearly unique would make the DGF enormous. Above `getOption("datagoodr.temporal.max_values", 10000)` distinct values, ISO timestamps are stored at the finest of minute, hour or day that fits (see [preflight_dgf()], rule `high_cardinality_temporal`).
If profiling a variable fails, the error names the variable and the step.