Getting Started with smartcor

Why smartcor?

Researchers routinely compute correlations between variables, but the “right” correlation method depends on what kinds of variables are being compared. Pearson correlation is the default in most software, yet it is only optimal for continuous-continuous pairs. When variables are binary, ordinal, or categorical, specialised methods (point-biserial, phi, polychoric, polyserial, Cramer’s V, Kendall’s tau, and others) are more appropriate.

The scale of the problem is visible in published research. For the accompanying paper (Harshvardhan and Ranjan, 2026), we traced correlation usage through the replication supplements of published economics articles: across 197 papers issuing at least one correlation command, 92.3% of the calls were Pearson, and 88% of the papers used Pearson and nothing else, whatever the variable types involved. Recomputing 3,322 variable pairs from those supplements shows when that default is harmless and when it is not: the type-appropriate method diverges from the authors’ choice in 8.1% of pairs where no variable is ordinal, but in every single one of the 732 pairs involving an ordinal variable. Where the methods diverge the median change in the coefficient is small ($|\Delta r| \approx 0.017$), but the distribution has a long tail (reaching $0.47$), and in 36 pairs the divergence moves the result across the conventional $p < 0.05$ threshold.

smartcor automates this process: it detects variable types, selects the best method, and explains its reasoning.

Installation

# from a local source:
install.packages("smartcor", repos = NULL, type = "source")

# or use devtools from the package directory:
devtools::install("path/to/smartcor")

Quick Start

library(smartcor)

Pairwise Correlation

The main function is smart_cor(). Pass two vectors and it will figure out the rest:

# continuous + continuous → Pearson
smart_cor(mtcars$mpg, mtcars$wt)
#> 
#> ── Variable types ──
#> 
#> ℹ Detected as continuous (numeric with 25 unique values).
#> mtcars$mpg: "continuous"
#> ℹ Detected as continuous (numeric with 29 unique values).
#> mtcars$wt: "continuous"
#> 
#> ── Method selection ──
#> 
#> ✔ Selected: Pearson Correlation
#> 
#> ── Smart Correlation ───────────────────────────────────────────────────────────
#> Estimate: -0.8677
#> Method: Pearson Correlation
#> Variables: mtcars$mpg ("continuous") × mtcars$wt ("continuous")
#> N: 32
#> p-value: < 0.001
#> Test: t-test on r (cor.test)
#> H0: rho = 0
#> Small p-values (e.g., p < 0.05) indicate evidence against H0.
#> 95% CI: [-0.9338, -0.7441]
#> Source: Fisher z (cor.test)
#> 
#> Both variables are continuous; Pearson correlation selected.
#> 
#> ℹ Alternatives: "spearman" and "kendall" (pass `method = "..."` to use)

# continuous + binary → point-biserial (= Pearson)
smart_cor(mtcars$mpg, mtcars$vs)
#> 
#> ── Variable types ──
#> 
#> ℹ Detected as continuous (numeric with 25 unique values).
#> mtcars$mpg: "continuous"
#> ℹ Detected as binary (numeric with 2 unique values: 0, 1).
#> mtcars$vs: "binary"
#> 
#> ── Method selection ──
#> 
#> ✔ Selected: Point-Biserial Correlation (= Pearson)
#> 
#> ── Smart Correlation ───────────────────────────────────────────────────────────
#> Estimate: 0.6640
#> Method: Point-Biserial Correlation (= Pearson)
#> Variables: mtcars$mpg ("continuous") × mtcars$vs ("binary")
#> N: 32
#> p-value: < 0.001
#> Test: t-test on r (cor.test)
#> H0: rho_pb = 0
#> Small p-values (e.g., p < 0.05) indicate evidence against H0.
#> 95% CI: [0.4104, 0.8223]
#> Source: Fisher z (cor.test); = Pearson
#> 
#> One variable is continuous and the other is binary. Point-biserial correlation
#> is mathematically identical to Pearson when the binary variable is coded 0/1.

Ordinal Variables

When both variables have a small number of unique values, smartcor detects them as ordinal:

# numeric variables with ≤10 unique values are treated as ordinal by default;
# assume_latent_normal controls whether polychoric or Kendall is used
smart_cor(mtcars$gear, mtcars$carb, assume_latent_normal = FALSE)
#> 
#> ── Variable types ──
#> 
#> ℹ Detected as ordinal (numeric with 3 unique values, at or below threshold of 10).
#> mtcars$gear: "ordinal"
#> ℹ Detected as ordinal (numeric with 6 unique values, at or below threshold of 10).
#> mtcars$carb: "ordinal"
#> 
#> ── Method selection ──
#> 
#> ✔ Selected: Kendall's Tau-b
#> 
#> ── Smart Correlation ───────────────────────────────────────────────────────────
#> Estimate: 0.0980
#> Method: Kendall's Tau-b
#> Variables: mtcars$gear ("ordinal") × mtcars$carb ("ordinal")
#> N: 32
#> p-value: 0.5302
#> Test: asymptotic normal (cor.test, exact = FALSE)
#> H0: tau = 0
#> Small p-values (e.g., p < 0.05) indicate evidence against H0.
#> 95% CI: [-0.1455, 0.3303]
#> Source: Fieller, Hartley, and Pearson (1957)
#> 
#> Both variables are ordinal. Kendall's tau selected (no latent normality
#> assumption; stable bias across distributions).
#> 
#> ℹ Alternatives: "polychoric", "spearman", and "gamma" (pass `method = "..."` to use)

Categorical Variables

# some categorical data
colour = factor(sample(c("red", "blue", "green"), 100, replace = TRUE))
shape  = factor(sample(c("circle", "square", "triangle"), 100, replace = TRUE))
smart_cor(colour, shape)
#> 
#> ── Variable types ──
#> 
#> ℹ Detected as categorical (factor/character with 3 levels).
#> colour: "categorical"
#> ℹ Detected as categorical (factor/character with 3 levels).
#> shape: "categorical"
#> 
#> ── Method selection ──
#> 
#> ✔ Selected: Cramer's V
#> 
#> ── Smart Correlation ───────────────────────────────────────────────────────────
#> Estimate: 0.1689
#> Method: Cramer's V
#> Variables: colour ("categorical") × shape ("categorical")
#> N: 100
#> p-value: 0.2220
#> Test: Pearson chi-square test of independence
#> H0: variables are independent
#> Small p-values indicate evidence that the variables are not independent.
#> 95% CI: [0.0000, 0.2721]
#> Source: Noncentral chi-square pivot
#> 
#> Both variables are categorical. Cramer's V selected as the standard symmetric
#> measure of nominal association.
#> 
#> ℹ Alternatives: "theils_u" and "tschuprows_t" (pass `method = "..."` to use)

Override Type Detection

If the automatic detection gets it wrong, you can override:

# force cyl to be treated as continuous
smart_cor(mtcars$mpg, mtcars$cyl, y_type = "continuous")
#> 
#> ── Variable types ──
#> 
#> ℹ Detected as continuous (numeric with 25 unique values).
#> mtcars$mpg: "continuous"
#> mtcars$cyl: "continuous" (user-specified)
#> 
#> ── Method selection ──
#> 
#> ✔ Selected: Pearson Correlation
#> 
#> ── Smart Correlation ───────────────────────────────────────────────────────────
#> Estimate: -0.8522
#> Method: Pearson Correlation
#> Variables: mtcars$mpg ("continuous") × mtcars$cyl ("continuous")
#> N: 32
#> p-value: < 0.001
#> Test: t-test on r (cor.test)
#> H0: rho = 0
#> Small p-values (e.g., p < 0.05) indicate evidence against H0.
#> 95% CI: [-0.9258, -0.7163]
#> Source: Fisher z (cor.test)
#> 
#> Both variables are continuous; Pearson correlation selected.
#> 
#> ℹ Alternatives: "spearman" and "kendall" (pass `method = "..."` to use)

Force a Specific Method

smart_cor(mtcars$mpg, mtcars$wt, method = "spearman")
#> 
#> ── Variable types ──
#> 
#> ℹ Detected as continuous (numeric with 25 unique values).
#> mtcars$mpg: "continuous"
#> ℹ Detected as continuous (numeric with 29 unique values).
#> mtcars$wt: "continuous"
#> 
#> ── Method ──
#> 
#> ℹ User specified method: "spearman"
#> 
#> ── Smart Correlation ───────────────────────────────────────────────────────────
#> Estimate: -0.8864
#> Method: Spearman Rank Correlation
#> Variables: mtcars$mpg ("continuous") × mtcars$wt ("continuous")
#> N: 32
#> p-value: < 0.001
#> Test: t approximation (cor.test, exact = FALSE)
#> H0: rho_S = 0
#> Small p-values (e.g., p < 0.05) indicate evidence against H0.
#> 95% CI: [-0.9503, -0.7511]
#> Source: Bonett and Wright (2000)
#> 
#> Method selected by user: spearman

Correlation Matrix

smart_cormat() computes all pairwise correlations for a data frame, using the best method for each pair:

mat = smart_cormat(
  mtcars[, c("mpg", "cyl", "vs", "gear")],
  assume_latent_normal = FALSE
)
#> 
#> ── Smart Correlation Matrix (4 variables) ──────────────────────────────────────
#> 
#> ── Detected variable types ──
#> 
#> mpg: "continuous"
#> cyl: "ordinal"
#> vs: "binary"
#> gear: "ordinal"
#> 
#> ── Computing correlations ──
#> 
#> ℹ Processing 6 variable pairs...
#> ✔ Done. 6 pairs computed.
mat
#> 
#> ── Smart Correlation Matrix ────────────────────────────────────────────────────
#> 
#> ── Variable types ──
#> 
#> mpg: "continuous"
#> cyl: "ordinal"
#> vs: "binary"
#> gear: "ordinal"
#> 
#> ── Correlations ──
#> 
#>        mpg   cyl    vs  gear
#> mpg   1.00 -0.91  0.66  0.54
#> cyl  -0.91  1.00 -0.88 -0.51
#> vs    0.66 -0.88  1.00  0.30
#> gear  0.54 -0.51  0.30  1.00
#> ── Methods used ──
#>      mpg cyl vs gear
#> mpg  Pr  Sp  PB Sp  
#> cyl  Sp  Kn  RB Kn  
#> vs   PB  RB  Ph RB  
#> gear Sp  Kn  RB Kn
#> 
#> ℹ Legend:
#> "Pr" = Pearson Correlation
#> "Sp" = Spearman Rank Correlation
#> "PB" = Point-Biserial Correlation (= Pearson)
#> "Kn" = Kendall's Tau-b
#> "RB" = Rank-Biserial Correlation
#> "Ph" = Phi Coefficient (= Pearson for 0/1)

Tidy Output

Both smart_cor() and smart_cormat() support tidy() for integration with the tidyverse:

res = smart_cor(mtcars$mpg, mtcars$wt, verbose = FALSE)
tidy(res)
#> # A tibble: 1 × 20
#>   var_x     var_y x_type y_type estimate method method_label rationale statistic
#>   <chr>     <chr> <chr>  <chr>     <dbl> <chr>  <chr>        <chr>         <dbl>
#> 1 mtcars$m… mtca… conti… conti…   -0.868 pears… Pearson Cor… Both var…     -9.56
#> # ℹ 11 more variables: p.value <dbl>, p_method <chr>, null_hypothesis <chr>,
#> #   p_interpretation <chr>, ci_lower <dbl>, ci_upper <dbl>, conf_level <dbl>,
#> #   ci_method <chr>, ci_source <chr>, n <int>, n_complete <int>

tidy(mat)
#> # A tibble: 6 × 20
#>   var_x var_y x_type     y_type estimate method method_label rationale statistic
#>   <chr> <chr> <chr>      <chr>     <dbl> <chr>  <chr>        <chr>         <dbl>
#> 1 mpg   cyl   continuous ordin…   -0.911 spear… Spearman Ra… One vari…  10425.  
#> 2 mpg   vs    continuous binary    0.664 point… Point-Biser… One vari…      4.86
#> 3 mpg   gear  continuous ordin…    0.543 spear… Spearman Ra… One vari…   2495.  
#> 4 cyl   vs    ordinal    binary   -0.881 rank_… Rank-Biseri… One vari…     -3.91
#> 5 cyl   gear  ordinal    ordin…   -0.513 kenda… Kendall's T… Both var…     -3.16
#> 6 vs    gear  binary     ordin…    0.302 rank_… Rank-Biseri… One vari…      1.43
#> # ℹ 11 more variables: p.value <dbl>, p_method <chr>, null_hypothesis <chr>,
#> #   p_interpretation <chr>, ci_lower <dbl>, ci_upper <dbl>, conf_level <dbl>,
#> #   ci_method <chr>, ci_source <chr>, n <int>, n_complete <int>

Available Methods

Use available_methods() to see what methods are available for a given variable-type pair:

available_methods("ordinal", "ordinal")
#> # A tibble: 4 × 2
#>   method     label                    
#>   <chr>      <chr>                    
#> 1 polychoric Polychoric Correlation   
#> 2 kendall    Kendall's Tau-b          
#> 3 spearman   Spearman Rank Correlation
#> 4 gamma      Goodman-Kruskal's Gamma
available_methods("continuous", "binary")
#> # A tibble: 2 × 2
#>   method         label                                 
#>   <chr>          <chr>                                 
#> 1 pearson        Pearson Correlation                   
#> 2 point_biserial Point-Biserial Correlation (= Pearson)
available_methods("categorical", "categorical")
#> # A tibble: 3 × 2
#>   method       label                              
#>   <chr>        <chr>                              
#> 1 cramers_v    Cramer's V                         
#> 2 theils_u     Theil's U (Uncertainty Coefficient)
#> 3 tschuprows_t Tschuprow's T

Method Selection Logic

The method selection is based on the following decision matrix, drawn from simulation studies comparing correlation methods (Harshvardhan and Ranjan, 2026):

Variable types	Default method	Alternative
Correlation methods (signed, from -1 to +1)
Continuous + continuous	Pearson	Spearman (suggested, especially if nonlinear)
Continuous + binary	Pearson = point-biserial	—
Binary + binary	Pearson = phi	Tetrachoric (preferred, if latent normality holds)
Continuous + ordinal	Spearman/Kendall	Polyserial (preferred, if latent normality holds)
Ordinal + ordinal	Kendall’s tau	Polychoric (preferred, if latent normality holds)
Binary + ordinal	Rank-biserial	Spearman
Association measures (unsigned, from 0 to 1)
Continuous + categorical	Cramer’s V (binned)	—
Binary + categorical	Cramer’s V	—
Ordinal + categorical	Cramer’s V	—
Categorical + categorical	Cramer’s V	Theil’s U, Tschuprow’s T

Count variables are treated as numeric continuous variables, so each count combination follows the corresponding continuous row.

Key findings from the simulation studies:

1. Pearson = Point-Biserial = Phi for 0/1-coded binary variables (exact mathematical equivalence).
2. Polychoric/polyserial are unbiased when the latent normality assumption holds, but can be biased when it is violated.
3. Kendall’s tau is the most robust measure across different latent distributions, making it a safe default when normality is uncertain.
4. More ordinal categories = less attenuation, with the biggest improvement from 2 to 3 categories.

The Latent-Normality Decision

When the decision matters (ordinal or binary variables), the default assume_latent_normal = "auto" runs a likelihood-ratio chi-square test on the pair via test_bivariate_normality() and lets the data decide:

# runs the LR test and picks polychoric or Kendall accordingly:
smart_cor(mtcars$gear, mtcars$carb)

A binary-binary pair is the exception: its 2 x 2 table is saturated (df = 0), the test cannot run, so smartcor uses phi and leaves tetrachoric to an explicit assume_latent_normal = TRUE or method = "tetrachoric". Set assume_latent_normal = TRUE or FALSE to skip the test, or NULL to be asked interactively (non-interactive sessions then default to TRUE).

Dependencies

The latent-variable methods rely on polycor (polychoric, polyserial, and tetrachoric estimates, with their standard errors). It is listed under Imports, so it installs with smartcor.

When a latent-variable estimator fails numerically on a particular dataset, such as a singular Hessian or a sparse contingency table, smartcor does not simply error. For the polyserial correlation it first retries the fit from a safe starting value, since polycor’s default start for the correlation can land outside $(-1, 1)$ on strongly correlated pairs and abort the optimiser before it takes a step; this rescues most failures. Only if no fit can be obtained does smartcor fall back to Kendall’s tau (for polychoric) or Spearman (for polyserial); the returned method, method_label, and rationale then name the method actually used, so the substitution is visible in the output.