The decision framework

This page sets out how smart_cor() chooses a method, one variable-type pair at a time. It follows Figure 4 of Harshvardhan and Ranjan (2026). Every choice starts from a few diagnostics (Step 0), and each pair then has a short rule that turns the diagnostics into a method. The theory behind each method is in vignette("theory").

The examples use the bundled extract of the 2024 General Social Survey, plus simulated data where the survey does not reach a branch.

library(smartcor)

gss = read.csv(system.file("extdata", "gss_2024_casestudy.csv", package = "smartcor"))
vapply(gss, detect_type, character(1))
#>           age        coninc        degree         happy           sex 
#>       "count"  "continuous"     "ordinal"     "ordinal"      "binary" 
#>       marital        region 
#>      "binary" "categorical"

age is a count, so it follows the continuous rules. coninc (family income) is continuous, degree and happy are ordinal, sex and marital are binary, and region is categorical.

Step 0: diagnostics

All tests use alpha = 0.05 unless you pass another value to smart_cor(), smart_cormat(), smart_cor_df(), or compare_methods(). The outlier rule has no significance level.

Outer-fence rule

detect_outliers() flags values strictly below $Q_1 - 3\,\mathrm{IQR}$ or strictly above $Q_3 + 3\,\mathrm{IQR}$ . It is applied to each variable of a continuous-continuous pair, to the continuous variable of a continuous-ordinal pair, and to the within-group residuals of continuous-binary and continuous-categorical pairs. It is never applied to an ordinal variable.

detect_outliers(gss$coninc)
#> No outliers outside [-140750, 223792]
detect_outliers(gss$age)
#> No outliers outside [-53, 150]

Neither variable has values beyond the outer fences.

Mardia’s test (two continuous variables)

For two continuous variables, test_bivariate_normality() runs Mardia’s skewness and kurtosis tests. The pair is rejected if either part rejects.

test_bivariate_normality(gss$age, gss$coninc)
#> 
#> -- Bivariate Normality Test (Mardia) --
#> 
#> H0: x and y are bivariate normal
#> N = 3000
#> Mardia skewness: 666.3715, df = 4, p = < 2.2e-16
#> Mardia kurtosis: -6.6499, p = 2.934e-11
#> Conclusion: reject. Evidence against H0 at alpha = 0.05 (Mardia skewness,
#> Mardia kurtosis).

Mardia’s test does not change the selection. If Pearson is selected while Mardia rejects, the rationale says that the interval and p-value are approximate.

Jarque-Bera test (one continuous variable)

For a continuous-ordinal pair the Jarque-Bera test is applied to the continuous variable. For a continuous-binary pair it is applied to the residuals around the group means. test_bivariate_normality() reports it together with the $G^2$ test below.

test_bivariate_normality(gss$coninc, gss$degree)
#> 
#> -- Bivariate Normality Test (Jarque-Bera and LR chi-square) --
#> 
#> H0: the continuous variable is normal, and the table arises from a bivariate
#> normal threshold model
#> N = 3000
#> Table dimensions: 9 x 5; polychoric rho: 0.4872
#> Jarque-Bera: 621.0658, df = 2, p = < 2.2e-16
#> LR chi-square: 51.2561, df = 31, p = 0.0125
#> Conclusion: reject. Evidence against H0 at alpha = 0.05 (Jarque-Bera, LR
#> chi-square).
test_bivariate_normality(gss$coninc, gss$sex)
#> 
#> -- Bivariate Normality Test (Jarque-Bera and LR chi-square) --
#> 
#> H0: the within-group residuals of the continuous variable are normal, and the
#> table arises from a bivariate normal threshold model
#> N = 3000
#> Table dimensions: 9 x 2; polychoric rho: -0.1472
#> Jarque-Bera: 623.2718, df = 2, p = < 2.2e-16
#> LR chi-square: 5.5053, df = 7, p = 0.5985
#> Conclusion: reject. Evidence against H0 at alpha = 0.05 (Jarque-Bera).

For a continuous-categorical pair, test_bivariate_normality() does not apply, but smart_cor() runs the same test on the within-group residuals and stores it in diagnostics:

res = smart_cor(gss$coninc, gss$region, bootstrap = FALSE, verbose = FALSE)
res$diagnostics$normality
#> 
#> -- Bivariate Normality Test (Jarque-Bera) --
#> 
#> H0: the within-group residuals are normal
#> N = 3000
#> Jarque-Bera: 609.8113, df = 2, p = < 2.2e-16
#> Conclusion: reject. Evidence against H0 at alpha = 0.05 (Jarque-Bera).

$G^2$ test of the threshold model

The latent-variable methods assume that the categories come from cutting a bivariate normal distribution. The likelihood-ratio test compares the observed contingency table with the table that model implies. A continuous variable is binned into at most 10 quantile groups first.

test_bivariate_normality(gss$degree, gss$happy)
#> 
#> -- Bivariate Normality Test (LR chi-square) --
#> 
#> H0: the table arises from a bivariate normal threshold model
#> N = 3000
#> Table dimensions: 5 x 3; polychoric rho: -0.1032
#> LR chi-square: 23.9731, df = 7, p = 0.001152
#> Conclusion: reject. Evidence against H0 at alpha = 0.05 (LR chi-square).

A 2 x 2 table is saturated, so two binary variables cannot be tested:

test_bivariate_normality(gss$sex, gss$marital)
#> 
#> -- Bivariate Normality Test (LR chi-square) --
#> 
#> H0: the table arises from a bivariate normal threshold model
#> N = 3000
#> i The model is saturated for a 2 x 2 table (df = 0); no test is possible.

Linearity

test_linearity() regresses each variable on the other and orders the residuals by the regressor, averaging residuals at tied values. A direction rejects linearity when a residual-structure test (Box-Pierce, Ljung-Box, or BDS) and RESET both reject, and the pair counts as nonlinear only when both directions reject.

test_linearity(gss$age, gss$coninc)
#> 
#> -- Linearity Test (sorted residual diagnostics) --
#> 
#> H0: the relationship between x and y is linear
#> N = 3000, alpha = 0.05
#> y on x p = 2.143e-09, x on y p = 0.0008253
#> Components (y on x):
#> Box-Pierce p = 1.491e-05, Ljung-Box p = 9.848e-06, BDS p (min) = 2.143e-09
#> Breusch-Pagan p = 0.9839, RESET p = < 2.2e-16
#> Combined p = 0.0008253 (reject)
#> Decision: Spearman/Kendall (nonlinear)
#> Both directions reject linearity at alpha = 0.05, so the relationship is
#> treated as nonlinear.

Both directions reject here, so the result is the same with the arguments swapped:

lin = test_linearity(gss$coninc, gss$age)
c(lin$p_y_on_x, lin$p_x_on_y, lin$p.value)
#> [1] 8.253145e-04 2.142887e-09 8.253145e-04
lin$decision
#> [1] "Spearman/Kendall (nonlinear)"

Rules by variable-type pair

Each subsection gives the rule as an ordered list: the first condition that holds decides. TRUE and FALSE refer to assume_latent_normal, which overrides the $G^2$ node but not the outlier or Jarque-Bera nodes. $m$ is the number of categories of the ordinal variable (the smaller of the two for an ordinal pair).

(a) Continuous + continuous

The outer fence flags either variable: Spearman.
test_linearity() rejects linearity: Spearman.
Otherwise: Pearson. If Mardia’s test rejects, the rationale says the interval and p-value are approximate.

Kendall’s tau is the alternative. In the survey, age and income have no outliers, but the linearity test rejects:

smart_cor(gss$age, gss$coninc, verbose = FALSE)
#> 
#> -- Smart Correlation -----------------------------------------------------------
#> Estimate: 0.0369
#> Method: Spearman Rank Correlation (count treated as continuous)
#> Variables: gss$age ("count") x gss$coninc ("continuous")
#> N: 3000
#> p-value: 0.0432
#> Test: t approximation (cor.test, exact = FALSE)
#> H0: rho_S = 0
#> Small p-values (e.g., p < 0.05) indicate evidence against H0.
#> 95% CI: [0.0011, 0.0726]
#> Source: Bonett and Wright (2000)
#> 
#> Diagnostics
#> Outliers (outer fence): none
#> Normality (Mardia): p < 2e-16 (rejected)
#> Linearity: p = 0.000825 (nonlinear)
#> 
#> Sorted residual diagnostics (y on x): Spearman/Kendall (nonlinear)
#> 
#> i Alternatives: "pearson" and "kendall" (pass `method = "..."` to use)

On simulated bivariate normal data the same rule keeps Pearson:

set.seed(42)
n = 300
z1 = rnorm(n)
z2 = 0.5 * z1 + sqrt(0.75) * rnorm(n)
smart_cor(z1, z2, verbose = FALSE)$rationale
#> [1] "Sorted residual diagnostics (y on x): Pearson (standard t-test)"

(b) Continuous + binary

The outer fence flags the within-group residuals, or Jarque-Bera rejects their normality: rank-biserial.
The $G^2$ test (binned continuous variable against the binary one) does not reject, or TRUE: biserial.
Otherwise, or FALSE: point-biserial.

Income residuals within each sex are far from normal, so the survey pair gets rank-biserial:

smart_cor(gss$coninc, gss$sex, verbose = FALSE)
#> 
#> -- Smart Correlation -----------------------------------------------------------
#> Estimate: -0.1336
#> Method: Rank-Biserial Correlation
#> Variables: gss$coninc ("continuous") x gss$sex ("binary")
#> N: 3000
#> p-value: < 0.001
#> Test: Wald test on Fisher-z scale
#> H0: delta = 0
#> Small p-values (e.g., p < 0.05) indicate evidence against H0.
#> 95% CI: [-0.1743, -0.0925]
#> Source: Cliff (1996)
#> 
#> Diagnostics
#> Outliers (outer fence): none
#> Normality (Jarque-Bera): p < 2e-16 (rejected)
#> 
#> The Jarque-Bera test rejects normality of the within-group residuals (p <
#> 2e-16); Rank-Biserial Correlation selected.
#> 
#> i Alternatives: "point_biserial" and "biserial" (pass `method = "..."` to use)

A binary variable made by cutting a normal latent variable gets biserial:

smart_cor(z1, as.integer(z2 > 0.3), verbose = FALSE)$rationale
#> [1] "No outliers and no evidence against normality of the within-group residuals. The LR test does not reject latent bivariate normality (p = 0.462); Biserial Correlation selected."

(c) Continuous + ordinal

The outer fence flags the continuous variable, or Jarque-Bera rejects its normality: Kendall’s tau if $m \le 5$ , else Spearman.
The $G^2$ test does not reject, or TRUE: polyserial.
Otherwise, or FALSE: Kendall’s tau if $m \le 5$ , else Spearman.

degree has five categories, so the survey pair gets Kendall’s tau:

smart_cor(gss$coninc, gss$degree, verbose = FALSE)
#> 
#> -- Smart Correlation -----------------------------------------------------------
#> Estimate: 0.3569
#> Method: Kendall's Tau-b
#> Variables: gss$coninc ("continuous") x gss$degree ("ordinal")
#> N: 3000
#> p-value: < 0.001
#> Test: asymptotic normal (cor.test, exact = FALSE)
#> H0: tau = 0
#> Small p-values (e.g., p < 0.05) indicate evidence against H0.
#> 95% CI: [0.3361, 0.3774]
#> Source: Fieller, Hartley, and Pearson (1957)
#> 
#> Diagnostics
#> Outliers (outer fence): none
#> Normality (Jarque-Bera): p < 2e-16 (rejected)
#> 
#> The Jarque-Bera test rejects normality of the continuous variable (p < 2e-16);
#> Kendall's Tau-b selected (the ordinal variable has 5 categories).
#> 
#> i Alternatives: "polyserial" and "spearman" (pass `method = "..."` to use)

With a normal continuous variable and a discretised normal latent variable, polyserial is selected:

z2_ord = cut(z2, c(-Inf, -1, 0, 1, Inf), labels = FALSE)
smart_cor(z1, z2_ord, verbose = FALSE)$rationale
#> [1] "No outliers and no evidence against normality of the continuous variable. The LR test does not reject latent bivariate normality (p = 0.383); Polyserial Correlation selected."

(d) Continuous + categorical

The outer fence flags the within-group residuals, or Jarque-Bera rejects their normality: Kruskal-Wallis $\eta^2_H$ .
Otherwise: correlation ratio $\eta^2$ .

Cramer’s V on the binned continuous variable is the further alternative. The survey pair fails the Jarque-Bera check:

smart_cor(gss$coninc, gss$region, n_boot = 200, seed = 1, verbose = FALSE)
#> 
#> -- Smart Correlation -----------------------------------------------------------
#> Estimate: 0.0193
#> Method: Kruskal-Wallis eta-squared (H)
#> Variables: gss$coninc ("continuous") x gss$region ("categorical")
#> N: 3000
#> p-value: < 0.001
#> Test: Kruskal-Wallis test
#> H0: eta^2_H = 0 (identical group distributions)
#> Small p-values (e.g., p < 0.05) indicate evidence against H0.
#> 95% CI (bootstrap): [0.0112, 0.0327]
#> Source: Percentile bootstrap
#> 
#> Diagnostics
#> Outliers (outer fence): none
#> Normality (Jarque-Bera): p < 2e-16 (rejected)
#> 
#> The Jarque-Bera test rejects normality of the within-group residuals (p <
#> 2e-16); Kruskal-Wallis eta-squared (H) selected.
#> 
#> i Alternatives: "eta_squared" and "cramers_v" (pass `method = "..."` to use)

Normal data with shifted group means get $\eta^2$ :

group = factor(sample(c("a", "b", "c"), n, replace = TRUE))
shift = c(a = 0, b = 0.5, c = 1)[as.character(group)]
smart_cor(z1 + shift, group, verbose = FALSE)$rationale
#> [1] "No outliers and no evidence against normality of the within-group residuals; Correlation Ratio (eta-squared) selected."

(e) Binary + binary

"auto": phi (the 2 x 2 table is saturated, so the $G^2$ test cannot run).
TRUE: tetrachoric.
FALSE: Yule’s Q.

smart_cor(gss$sex, gss$marital, verbose = FALSE)
#> 
#> -- Smart Correlation -----------------------------------------------------------
#> Estimate: -0.0902
#> Method: Phi Coefficient (= Pearson for 0/1)
#> Variables: gss$sex ("binary") x gss$marital ("binary")
#> N: 3000
#> p-value: < 0.001
#> Test: Pearson chi-square test of independence
#> H0: variables are independent
#> Small p-values indicate evidence that the variables are not independent.
#> 95% CI: [-0.1260, -0.0544]
#> Source: Noncentral chi-square pivot
#> 
#> Diagnostics
#> Latent normality: saturated 2 x 2 table (df = 0), untestable
#> 
#> Both variables are binary. The 2 x 2 table is saturated (df = 0), so latent
#> normality cannot be tested; Phi Coefficient (= Pearson for 0/1) selected.
#> 
#> i Alternatives: "tetrachoric" and "yules_q" (pass `method = "..."` to use)
vapply(list(auto = "auto", true = TRUE, false = FALSE), function(a) {
  smart_cor(gss$sex, gss$marital, assume_latent_normal = a, verbose = FALSE)$method
}, character(1))
#>          auto          true         false 
#>         "phi" "tetrachoric"     "yules_q"

(f) Binary + ordinal

The $G^2$ test (df = $m - 2$ ) does not reject, or TRUE: polychoric.
Otherwise, or FALSE: rank-biserial.

Kendall’s tau and Spearman are alternatives.

smart_cor(gss$sex, gss$degree, verbose = FALSE)
#> 
#> -- Smart Correlation -----------------------------------------------------------
#> Estimate: 0.0154
#> Method: Rank-Biserial Correlation
#> Variables: gss$sex ("binary") x gss$degree ("ordinal")
#> N: 3000
#> p-value: 0.4404
#> Test: Wald test on Fisher-z scale
#> H0: delta = 0
#> Small p-values (e.g., p < 0.05) indicate evidence against H0.
#> 95% CI: [-0.0236, 0.0543]
#> Source: Cliff (1996)
#> 
#> Diagnostics
#> Latent normality (LR chi-square = 12.49, df = 3): p = 0.00589 (rejected)
#> 
#> One variable is binary and the other is ordinal. The LR test rejects latent
#> bivariate normality (p = 0.00589); Rank-Biserial Correlation selected.
#> 
#> i Alternatives: "polychoric", "kendall", and "spearman" (pass `method = "..."` to use)

When both variables come from cutting a bivariate normal distribution, polychoric is selected:

smart_cor(as.integer(z1 > 0), z2_ord, verbose = FALSE)$rationale
#> [1] "One variable is binary and the other is ordinal. The LR test does not reject latent bivariate normality (p = 0.131); Polychoric Correlation selected."

(g) Ordinal + ordinal

The $G^2$ test does not reject, or TRUE: polychoric.
Otherwise, or FALSE: Kendall’s tau if $m \le 5$ , else Spearman.

Goodman-Kruskal gamma is a further alternative. The threshold model does not fit degree and happy, and happy has three categories:

smart_cor(gss$degree, gss$happy, verbose = FALSE)
#> 
#> -- Smart Correlation -----------------------------------------------------------
#> Estimate: -0.0795
#> Method: Kendall's Tau-b
#> Variables: gss$degree ("ordinal") x gss$happy ("ordinal")
#> N: 3000
#> p-value: < 0.001
#> Test: asymptotic normal (cor.test, exact = FALSE)
#> H0: tau = 0
#> Small p-values (e.g., p < 0.05) indicate evidence against H0.
#> 95% CI: [-0.1030, -0.0559]
#> Source: Fieller, Hartley, and Pearson (1957)
#> 
#> Diagnostics
#> Latent normality (LR chi-square = 23.97, df = 7): p = 0.00115 (rejected)
#> 
#> Both variables are ordinal. The LR test rejects latent bivariate normality (p =
#> 0.00115); Kendall's Tau-b selected (the smaller variable has 3 categories).
#> 
#> i Alternatives: "polychoric", "spearman", and "gamma" (pass `method = "..."` to use)

(h) Binary + categorical

Always: Cramer’s V, treating the binary variable as a nominal variable with two levels.

smart_cor(gss$sex, gss$region, verbose = FALSE)
#> 
#> -- Smart Correlation -----------------------------------------------------------
#> Estimate: 0.0634
#> Method: Cramer's V
#> Variables: gss$sex ("binary") x gss$region ("categorical")
#> N: 3000
#> p-value: 0.0071
#> Test: Pearson chi-square test of independence
#> H0: variables are independent
#> Small p-values indicate evidence that the variables are not independent.
#> 95% CI: [0.0181, 0.0949]
#> Source: Noncentral chi-square pivot
#> 
#> One variable is binary and the other is categorical. Cramer's V selected
#> (treating binary as a 2-level nominal variable).

(i) Ordinal + categorical

Always: Freeman’s theta, which keeps the order of the ordinal variable.

Cramer’s V is the alternative.

smart_cor(gss$degree, gss$region, n_perm = 200, n_boot = 200, seed = 1, verbose = FALSE)
#> 
#> -- Smart Correlation -----------------------------------------------------------
#> Estimate: 0.0430
#> Method: Freeman's Theta
#> Variables: gss$degree ("ordinal") x gss$region ("categorical")
#> N: 3000
#> p-value: 0.0398
#> Test: Permutation test (B = 200, one-sided)
#> H0: theta = 0 (ordinal variable independent of group)
#> Small p-values (e.g., p < 0.05) indicate evidence against H0.
#> 95% CI (bootstrap): [0.0235, 0.0801]
#> Source: Percentile bootstrap
#> 
#> One variable is ordinal and the other is categorical. Freeman's theta selected
#> (it keeps the ordering of the ordinal variable).
#> 
#> i Alternatives: "cramers_v" (pass `method = "..."` to use)

(j) Categorical + categorical

Always: Cramer’s V.

Theil’s U and Tschuprow’s T are alternatives. The survey extract has only one categorical variable, so this example is simulated:

colour = factor(sample(c("red", "blue", "green"), n, replace = TRUE))
shape = factor(sample(c("circle", "square", "triangle"), n, replace = TRUE))
smart_cor(colour, shape, verbose = FALSE)
#> 
#> -- Smart Correlation -----------------------------------------------------------
#> Estimate: 0.1041
#> Method: Cramer's V
#> Variables: colour ("categorical") x shape ("categorical")
#> N: 300
#> p-value: 0.1646
#> Test: Pearson chi-square test of independence
#> H0: variables are independent
#> Small p-values indicate evidence that the variables are not independent.
#> 95% CI: [0.0000, 0.1648]
#> Source: Noncentral chi-square pivot
#> 
#> Both variables are categorical. Cramer's V selected as the standard symmetric
#> measure of nominal association.
#> 
#> i Alternatives: "theils_u" and "tschuprows_t" (pass `method = "..."` to use)

Summary

Variable types	Default method	Alternative
Correlation methods (signed, from -1 to +1)
Continuous + continuous	Pearson (linear, no outliers)	Spearman (outliers or nonlinear), Kendall’s tau
Continuous + binary	Biserial (if latent normality holds)	Point-biserial; rank-biserial (outliers or non-normal)
Binary + binary	Pearson = phi	Tetrachoric (latent normal), Yule’s Q (no latent continuum)
Continuous + ordinal	Polyserial (if latent normality holds)	Kendall’s tau ( $m \le 5$ ) or Spearman
Ordinal + ordinal	Polychoric (if latent normality holds)	Kendall’s tau ( $m \le 5$ ) or Spearman; Goodman-Kruskal gamma
Binary + ordinal	Polychoric (if latent normality holds)	Rank-biserial; Kendall’s tau, Spearman
Association measures (unsigned, from 0 to 1)
Continuous + categorical	Correlation ratio $\eta^2$	Kruskal-Wallis $\eta^2_H$ (outliers or non-normal); Cramer’s V (binned)
Binary + categorical	Cramer’s V	–
Ordinal + categorical	Freeman’s theta	Cramer’s V
Categorical + categorical	Cramer’s V	Theil’s U, Tschuprow’s T

Count variables are treated as continuous, so each count combination follows the corresponding continuous row. An explicit method skips every diagnostic, and compare_methods() reports all the methods available for a pair side by side.

Reference

Harshvardhan, M. and Ranjan, P. (2026). smartcor: Intelligent Correlation Method Selection for Mixed Variable Types. arXiv preprint arXiv:2607.22285. https://doi.org/10.48550/arXiv.2607.22285