Choose and Compute a Correlation — smart

Detects the types of x and y, selects a suitable correlation method, and returns the estimate with its confidence interval and p-value.

Usage

smart_cor(
  x,
  y,
  x_type = NULL,
  y_type = NULL,
  method = NULL,
  assume_latent_normal = "auto",
  ordinal_threshold = 10,
  ignore_na = TRUE,
  conf_level = 0.95,
  alpha = 0.05,
  bootstrap = "auto",
  n_boot = 500,
  n_perm = 500,
  seed = NULL,
  polyserial_ml = FALSE,
  verbose = TRUE
)

Arguments

x, y: Vectors of the same length. May be numeric, factor, character, or logical.
x_type, y_type: Optional character strings ("continuous", "count", "binary", "ordinal", "categorical") to override automatic type detection.
method: Optional character string to force a specific method. One of "pearson", "spearman", "kendall", "point_biserial", "biserial", "rank_biserial", "phi", "tetrachoric", "yules_q", "polychoric", "polyserial", "gamma", "eta_squared", "kw_eta_squared", "cramers_v", "freemans_theta", "theils_u", or "tschuprows_t". See available_methods().
assume_latent_normal: "auto" (default), logical, or NULL. "auto" runs a likelihood-ratio chi-square test (via test_bivariate_normality()) for the latent-normality node; a saturated 2 x 2 table gives phi. TRUE forces the node to "yes" (biserial / polyserial / polychoric / tetrachoric); FALSE forces it to "no" (point-biserial / Kendall or Spearman / rank-biserial / Yule's Q). The outlier and Jarque-Bera nodes still run under TRUE and FALSE. NULL asks interactively (defaults to TRUE in non-interactive sessions).
ordinal_threshold: Integer passed to detect_type(). Default: 10.
ignore_na: Logical. If TRUE (default), rows with missing values in either x or y are silently removed (pairwise complete observations). If FALSE, the function errors when NA values are present.
conf_level: Numeric between 0 and 1. Confidence level for intervals. Default: 0.95.
alpha: Significance level for every diagnostic test (Mardia, Jarque-Bera, the LR latent-normality test, and test_linearity()). Default: 0.05.
bootstrap: Character or logical. Controls bootstrap inference: "auto" (default) bootstraps when analytic CIs are unavailable or the linearity rule recommends robust/bootstrap Pearson inference; TRUE always replaces the analytic CI with a percentile-bootstrap CI; FALSE never bootstraps. P-values remain analytic (or permutation-based for Theil's U and Freeman's theta) under every setting.
n_boot: Integer. Number of bootstrap replications when bootstrapping. Default: 500.
n_perm: Integer. Number of shuffles for the permutation p-values of Theil's U and Freeman's theta. Default: 500.
seed: Optional integer seed for the bootstrap and permutation draws. The global .Random.seed is saved and restored around a seeded call, so the global RNG state is restored. Seeded results are reproducible across R sessions but do not match the Python package, whose numpy generator draws differently.
polyserial_ml: Logical. How polyserial and biserial correlations are fitted. FALSE (default) calls polycor::polyserial() with its own defaults, the quick two-step estimator (adding only std.err = TRUE for the interval and p-value). We recommend TRUE, which fits the full maximum-likelihood estimate (the correlation and the thresholds together) with a stricter convergence tolerance, and retries from a safe starting value if the first fit fails.
verbose: Logical. If TRUE (default), prints messages about the detection and selection process.

Value

An object of class "smartcor": a named list with elements estimate, method, method_label, x_type, y_type, n, n_complete, statistic, p.value, ci_lower, ci_upper, conf_level, ci_method, ci_source, p_method, null_hypothesis, p_interpretation, alternatives, rationale, normality_test (the LR latent-normality test, when it ran), diagnostics, outlier_flag, normality_p, latent_p, linearity_p, and linearity_test (when the residual diagnostic rule ran). diagnostics is a list with outliers (a detect_outliers() result, or a list of two for a continuous-continuous pair), normality (Mardia or Jarque-Bera), latent (the LR test), and linearity (the test_linearity() result); entries that were not computed are NULL. The four summary fields are NA when not computed. Has custom print.smartcor() and tidy.smartcor() methods.

Details

The method selection follows Harshvardhan, M. and Ranjan, P. (2026), "smartcor: Intelligent Correlation Method Selection for Mixed Variable Types" (arXiv:2607.22285, doi:10.48550/arXiv.2607.22285 ); the package vignettes (browseVignettes("smartcor")) cover the same material in more depth:

Variable types	Default method	Alternative
Correlation methods (signed, from -1 to +1)
continuous + continuous	Pearson (linear, no outliers)	Spearman (outliers or nonlinear), Kendall's tau
continuous + binary	Biserial (if latent normality holds)	Point-biserial; rank-biserial (outliers or non-normal)
binary + binary	Pearson = phi	Tetrachoric (latent normal), Yule's Q (no latent continuum)
continuous + ordinal	Polyserial (if latent normality holds)	Kendall's tau (m <= 5) or Spearman
ordinal + ordinal	Polychoric (if latent normality holds)	Kendall's tau (m <= 5) or Spearman; Goodman-Kruskal gamma
binary + ordinal	Polychoric (if latent normality holds)	Rank-biserial; Kendall's tau, Spearman
Association measures (unsigned, from 0 to 1)
continuous + categorical	Correlation ratio \(\eta^2\)	Kruskal-Wallis \(\eta^2_H\) (outliers or non-normal); Cramer's V (binned)
binary + categorical	Cramer's V	–
ordinal + categorical	Freeman's theta	Cramer's V
categorical + categorical	Cramer's V	Theil's U, Tschuprow's T

Count variables are treated as continuous, so each count combination follows the corresponding continuous row. m is the number of categories of the ordinal variable (the smaller of the two for an ordinal pair).

The selection starts from diagnostics run at significance level alpha:

The outer-fence rule (detect_outliers(), Q1 - 3 IQR and Q3 + 3 IQR) is applied to each continuous variable of a continuous-continuous pair, to the continuous variable of a continuous-ordinal pair, and to the residuals around the group means for continuous-binary and continuous-categorical pairs. It is never applied to ordinal variables.
Normality: Mardia's test (MVN::mardia()) for two continuous variables; the Jarque-Bera test (moments::jarque.test()) on the continuous variable (continuous-ordinal) or on the within-group residuals (continuous-binary, continuous-categorical).
Latent normality: the LR chi-square test of test_bivariate_normality() on the (binned) contingency table.
Linearity: test_linearity() for two continuous variables.

A continuous-continuous pair gets Spearman when the outer fence flags either variable, and otherwise follows test_linearity(), which runs its diagnostics in both directions. When Pearson is selected and Mardia's test rejects, the rationale says the interval and p-value are approximate. For continuous-binary, continuous-ordinal, and continuous-categorical pairs, a fence flag or a Jarque-Bera rejection selects the rank-based method (rank-biserial, Kendall/Spearman, Kruskal-Wallis \(\eta^2_H\)). Otherwise, and for binary-ordinal and ordinal pairs, the latent-normality node decides between the latent-variable method and the observed-scale one. A binary-binary pair under "auto" gets phi: its 2 x 2 table is saturated (df = 0), so latent normality cannot be tested. Inconclusive tests keep Pearson and report the reason. The linearity decision does not depend on the order of x and y. An explicit method bypasses all diagnostics. The tests that ran are returned in diagnostics.

When a latent-variable estimator fails numerically (singular Hessian, sparse table), the function falls back to Kendall's tau, Spearman, or point-biserial and the returned method, method_label, and rationale name the method actually used, not the one that failed.

One caveat on the point-biserial interval: it reuses the Fisher-z variance 1/(n - 3), which assumes bivariate normality; a binary margin cannot satisfy that exactly, so treat the interval as approximate.

Theil's U is asymmetric: method = "theils_u" computes U(y|x), the proportion of the entropy in y explained by x. Swap the arguments for the other direction. Its p-value, like that of Freeman's theta, comes from a permutation test (n_perm shuffles), not from the bootstrap. Kruskal-Wallis \(\eta^2_H\), Freeman's theta, and Theil's U have no analytic interval and get a percentile bootstrap under bootstrap = "auto".

Examples

path = system.file("extdata", "gss_2024_casestudy.csv", package = "smartcor")
gss = read.csv(path)

# Continuous and binary variables: the diagnostics pick the method.
smart_cor(gss$coninc, gss$sex, verbose = FALSE)
#> 
#> ── Smart Correlation ───────────────────────────────────────────────────────────
#> Estimate: -0.1336
#> Method: Rank-Biserial Correlation
#> Variables: gss$coninc ("continuous") x gss$sex ("binary")
#> N: 3000
#> p-value: < 0.001
#> Test: Wald test on Fisher-z scale
#> H0: delta = 0
#> Small p-values (e.g., p < 0.05) indicate evidence against H0.
#> 95% CI: [-0.1743, -0.0925]
#> Source: Cliff (1996)
#> 
#> Diagnostics
#> Outliers (outer fence): none
#> Normality (Jarque-Bera): p < 2e-16 (rejected)
#> 
#> The Jarque-Bera test rejects normality of the within-group residuals (p <
#> 2e-16); Rank-Biserial Correlation selected.
#> 
#> ℹ Alternatives: "point_biserial" and "biserial" (pass `method = "..."` to use)

# Two ordinal variables use Kendall's tau when latent normality is not assumed.
smart_cor(gss$degree, gss$happy,
          assume_latent_normal = FALSE, verbose = FALSE)
#> 
#> ── Smart Correlation ───────────────────────────────────────────────────────────
#> Estimate: -0.0795
#> Method: Kendall's Tau-b
#> Variables: gss$degree ("ordinal") x gss$happy ("ordinal")
#> N: 3000
#> p-value: < 0.001
#> Test: asymptotic normal (cor.test, exact = FALSE)
#> H0: tau = 0
#> Small p-values (e.g., p < 0.05) indicate evidence against H0.
#> 95% CI: [-0.1030, -0.0559]
#> Source: Fieller, Hartley, and Pearson (1957)
#> 
#> Both variables are ordinal. Latent normality not assumed (assume_latent_normal
#> = FALSE); Kendall's Tau-b selected (the smaller variable has 3 categories).
#> 
#> ℹ Alternatives: "polychoric", "spearman", and "gamma" (pass `method = "..."` to use)

# Force a method when the analysis calls for it.
smart_cor(gss$coninc, gss$age, method = "spearman", verbose = FALSE)
#> 
#> ── Smart Correlation ───────────────────────────────────────────────────────────
#> Estimate: 0.0369
#> Method: Spearman Rank Correlation
#> Variables: gss$coninc ("continuous") x gss$age ("count")
#> N: 3000
#> p-value: 0.0432
#> Test: t approximation (cor.test, exact = FALSE)
#> H0: rho_S = 0
#> Small p-values (e.g., p < 0.05) indicate evidence against H0.
#> 95% CI: [0.0011, 0.0726]
#> Source: Bonett and Wright (2000)
#> 
#> Method selected by user: spearman