The Determinant Criterion β€” Coding Gain

The Second Design Objective: Maximise the Minimum Determinant

The rank criterion (Β§4) tells us to ensure rank(Ξ”)=nt\mathrm{rank} (\boldsymbol{\Delta}) = n_t for every codeword pair β€” this locks in the maximum diversity order ntnrn_t n_r and dictates the slope of the high-SNR PEP curve. Within the class of full-rank codes, the rank criterion is silent about which code is better.

The finer question is: among full-rank codes, which one has the smallest PEP intercept at practical SNRs? The PEP upper bound at high SNR is P(Xβ†’X^)β€…β€Šβ‰²β€…β€Š(SNR4nt)βˆ’ntnrdet⁑(ΔΔH)βˆ’nr.P(\mathbf{X} \to \hat{\mathbf{X}}) \;\lesssim\; \left( \frac{\text{SNR}}{4 n_t} \right)^{-n_t n_r} \det(\boldsymbol{\Delta} \boldsymbol{\Delta}^H)^{-n_r}. The intercept is det⁑(ΔΔH)βˆ’nr\det(\boldsymbol{\Delta}\boldsymbol{\Delta}^H)^{-n_r} β€” so the PEP is smallest when the determinant is largest. The determinant criterion says exactly this: among full-rank codes, maximise the minimum determinant over all codeword pairs.

This is the designer's second objective: after securing full rank for diversity, maximise the minimum determinant for coding gain. The rank criterion controls the slope; the determinant criterion controls the horizontal offset β€” equivalently, it quantifies how many dB of SNR a better code saves at a given BER.

,

Definition:

Coding Gain of a Full-Rank Space-Time Code

Let C\mathcal{C} be a full-rank space-time code (every pair Xβ‰ X^\mathbf{X} \ne \hat{\mathbf{X}} gives rank(Ξ”)=nt\mathrm{rank}(\boldsymbol{\Delta}) = n_t). The coding gain of C\mathcal{C} is Ξ³cβ€…β€Šβ‰œβ€…β€Š[min⁑Xβ‰ X^∈Cdet⁑(ΔΔH)]1/nt,\gamma_c \;\triangleq\; \left[ \min_{\mathbf{X} \ne \hat{\mathbf{X}} \in \mathcal{C}} \det(\boldsymbol{\Delta}\boldsymbol{\Delta}^H) \right]^{1/n_t}, the ntn_t-th root of the minimum determinant over codeword pairs. The minimum determinant itself is det⁑min⁑(C)β€…β€Šβ‰œβ€…β€Šmin⁑Xβ‰ X^det⁑(ΔΔH).\det_{\min}(\mathcal{C}) \;\triangleq\; \min_{\mathbf{X} \ne \hat{\mathbf{X}}} \det(\boldsymbol{\Delta}\boldsymbol{\Delta}^H).

Several normalisations appear in the literature. The TSC 1998 paper uses ∏i=1ntλi\prod_{i=1}^{n_t}\lambda_i (which equals det⁑\det for a full-rank matrix) as "the coding advantage". Tse-Viswanath (2005 §3.3.3) use γc=det⁑min⁑1/nt\gamma_c = \det_{\min}^{1/n_t}. The two differ by a power of ntn_t; both shift the PEP curve horizontally. We prefer the Tse-Viswanath definition because it normalises to an SNR scale in dB: doubling γc\gamma_c shifts the BER curve by 3nr/nt3n_r/n_t dB (see Theorem TDeterminant Criterion (Tarokh-Seshadri-Calderbank 1998), interpretation).

,

Theorem: Determinant Criterion (Tarokh-Seshadri-Calderbank 1998)

Let C\mathcal{C} be a full-rank space-time code transmitted over an i.i.d. Rayleigh block-fading MIMO channel with ntn_t transmit antennas, nrn_r receive antennas, CSIR, and ML decoding. Then at high SNR: Pe(SNR;C)β€…β€Šβ‰β€…β€Š(SNR)βˆ’ntnrβ‹…det⁑min⁑(C)βˆ’nr,P_e(\text{SNR}; \mathcal{C}) \;\doteq\; (\text{SNR})^{-n_t n_r} \cdot \det_{\min}(\mathcal{C})^{-n_r}, with det⁑min⁑(C)=min⁑Xβ‰ X^det⁑(ΔΔH)\det_{\min}(\mathcal{C}) = \min_{\mathbf{X}\ne\hat{\mathbf{X}}} \det(\boldsymbol{\Delta}\boldsymbol{\Delta}^H). In particular, among all full-rank codes of equal rate, the one maximising det⁑min⁑(C)\det_{\min}(\mathcal{C}) achieves the lowest error probability in the high-SNR regime.

Equivalently, doubling det⁑min⁑\det_{\min} shifts the high-SNR BER curve left by Ξ”SNRβ€…β€Š=β€…β€Š10log⁑102ntnrβ‹…nrβ€…β€Š=β€…β€Š3nrntnrβ‹…nrβ€…β€Š=β€…β€Š3nrntdB.\Delta \text{SNR} \;=\; \frac{10 \log_{10} 2}{n_t n_r} \cdot n_r \;=\; \frac{3 n_r}{n_t n_r} \cdot n_r \;=\; \frac{3 n_r}{n_t} \quad \text{dB}. This is the determinant-criterion design rule: maximise the minimum determinant.

The rank criterion locks in the high-SNR slope at ntnrn_t n_r; the determinant then controls the intercept. Specifically, the PEP of the worst codeword pair is (SNR/4nt)βˆ’ntnrdet⁑(ΔΔH)βˆ’nr(\text{SNR}/4n_t)^{-n_t n_r} \det(\boldsymbol{\Delta}\boldsymbol{\Delta}^H)^{-n_r}; among all pairs, PeP_e is dominated by the pair with smallest det⁑(ΔΔH)\det (\boldsymbol{\Delta}\boldsymbol{\Delta}^H) β€” i.e., by det⁑min⁑\det_{\min}.

Maximising det⁑min⁑\det_{\min} is equivalent to making the error matrix "as well-conditioned as possible" for the worst codeword pair: a diagonal ΔΔH\boldsymbol{\Delta}\boldsymbol{\Delta}^H with equal eigenvalues gives the largest determinant for a fixed Frobenius norm (AM-GM), and is the "geometric-mean-optimal" structure that Alamouti and the OSTBCs of Ch. 11 achieve by construction.

, ,

Error Matrix Ξ”\boldsymbol{\Delta}: Rank + Determinant Shape the PEP

Animated 2D slice: the error matrix Ξ”βˆˆC2Γ—2\boldsymbol{\Delta} \in \mathbb{C}^{2\times 2} and its eigenstructure as we sweep from a rank-11 (rank-deficient) codeword pair to a rank-22 (full-rank) pair. The eigenvectors of ΔΔH\boldsymbol{\Delta}\boldsymbol{\Delta}^H are drawn as orthogonal axes; the eigenvalues Ξ»1,Ξ»2\lambda_1, \lambda_2 are shown as ellipse radii; the product Ξ»1Ξ»2=det⁑(ΔΔH)\lambda_1 \lambda_2 = \det (\boldsymbol{\Delta}\boldsymbol{\Delta}^H) β€” the determinant β€” is the ellipse area. The animation shows how the rank criterion "turns on" the second dimension and the determinant criterion "grows" the ellipse area.
Geometric interpretation. Rank deficiency collapses the eigen- ellipse to a segment (one eigenvalue zero, area zero, diversity nrn_r only). Full rank opens the ellipse to two dimensions (diversity 2nr2 n_r); the ellipse area is det⁑(ΔΔH)\det(\boldsymbol{\Delta} \boldsymbol{\Delta}^H) and is what the determinant criterion maximises.

Key Takeaway

The designer's SECOND objective is maximum minimum determinant. After securing full rank (Β§4), find the codebook geometry that keeps det⁑(ΔΔH)\det(\boldsymbol{\Delta}\boldsymbol{\Delta}^H) as large as possible for the worst codeword pair. At finite SNR, doubling det⁑min⁑\det_{\min} shifts the BER curve left by 3nr/nt3n_r/n_t dB β€” the coding gain. This is precisely why full-rank codes with equal ΔΔH\boldsymbol{\Delta}\boldsymbol{\Delta}^H eigenvalues (AM-GM-optimal for a fixed Frobenius norm) are favoured: Alamouti is not only full- rank but constant-det⁑\det on every one-symbol-error pair.

Example: Alamouti's Minimum Determinant: ∣e1∣2+∣e2∣2|e_1|^2 + |e_2|^2

Continue with Alamouti's 2Γ—22\times 2 code from Example EAlamouti Code: Verify Full Rank Over All Symbol Pairs. Compute det⁑(ΔΔH)\det(\boldsymbol{\Delta} \boldsymbol{\Delta}^H) for an arbitrary codeword pair in terms of the symbol errors e1=s1βˆ’s^1e_1 = s_1 - \hat s_1, e2=s2βˆ’s^2e_2 = s_2 - \hat s_2, and identify the minimum over QPSK symbol pairs.

,

Full-Rank vs. Rank-Deficient Space-Time Codes

PropertyFull-rank STCRank-deficient STC
Rank criterionSatisfied (rmin⁑=ntr_{\min} = n_t)Fails (rmin⁑<ntr_{\min} < n_t)
Diversity orderntnrn_t n_r (maximum)rmin⁑nr<ntnrr_{\min} n_r < n_t n_r
High-SNR PEP slopeSNRβˆ’ntnr\text{SNR}^{-n_t n_r}SNRβˆ’rmin⁑nr\text{SNR}^{-r_{\min} n_r}
Coding-gain knobRelevant (det⁑min⁑\det_{\min})Irrelevant β€” diversity is the binding constraint
Examples (Ch. 11)Alamouti, OSTBCs, Golden CodeSpatial repetition, plain uncoded single-stream
Matches outage exponent d⋆(0)=ntnrd^\star(0) = n_t n_r?YesNo β€” code is asymptotically suboptimal
Design ruleMaximise det⁑min⁑\det_{\min} (determinant criterion)Redesign for full rank first (no point optimising coding gain)

Diversity Gain vs. Coding Gain

AspectDiversity gainCoding gain
ControlsSlope of BER vs. SNR curveHorizontal offset (intercept)
Design criterionRank criterion (Β§4)Determinant criterion (Β§5)
Formula at high SNRd=rmin⁑nrd = r_{\min} n_rγc=det⁑min⁑1/nt\gamma_c = \det_{\min}^{1/n_t}
Ceilingntnrn_t n_r (channel-limited)Only the power / constellation constraint limits it
Physical meaningNumber of independent fading samples seenGeometric efficiency of the codebook
Visible at low SNR?No β€” asymptotic onlyPartially β€” shifts the curve at all SNRs
Binding first?Yes β€” fix rank, then determinantNo β€” only after rank is secured

Common Mistake: Rank Dominates at High SNR; Determinant Matters at Finite SNR

Mistake:

A student reads that "the determinant criterion maximises coding gain" and concludes that a rank-11 code with a huge "determinant" (in some loose sense β€” perhaps the largest nonzero eigenvalue) is preferable to a full-rank code with a modest minimum determinant.

Correction:

Rank dominates at high SNR. Rank determines the slope of the PEP curve on a log-log BER plot, which at sufficiently high SNR is always more important than the intercept. A rank-11 code (d=nrd = n_r) is eventually outperformed by any full-rank code (d=ntnrd = n_t n_r) β€” the slopes diverge.

Determinant matters at finite SNR. Among full-rank codes, the determinant shifts the curve horizontally by 3nr/nt3n_r/n_t dB per doubling. At operationally relevant SNRs (1010–3030 dB for most wireless systems), this shift is measurable and matters. At very low SNR (below 00 dB), the PEP bound itself is loose and neither criterion is a reliable guide β€” one must simulate.

Rule of thumb. Always fix rank first (satisfy the rank criterion), then optimise det⁑min⁑\det_{\min}. Never optimise det⁑min⁑\det_{\min} at the cost of losing rank β€” you're trading a permanent slope advantage for a transient dB advantage.

,
⚠️Engineering Note

5G NR Codebook-Based Precoding and the Rank-Determinant Criteria

5G NR's downlink codebook-based precoding (3GPP TS 38.211) specifies precoder matrices W∈CntΓ—Ξ½\mathbf{W} \in \mathbb{C}^{n_t \times \nu} parametrised by a triplet (i1,i2,Ξ½)(i_1, i_2, \nu), where:

  • ν∈{1,…,8}\nu \in \{1, \ldots, 8\} is the transmission rank, i.e., the number of simultaneously transmitted layers.
  • i1,i2i_1, i_2 are codebook indices (wideband / subband) specifying the DFT-based precoder.
  • The UE reports (i1,i2,Ξ½)(i_1, i_2, \nu) via the PMI/RI on the PUCCH.

The codebook is rank-preserving: every codebook entry has rank(W)=Ξ½\mathrm{rank}(\mathbf{W}) = \nu. This enforces the rank criterion at the layer level β€” the effective channel HW\mathbf{H}\mathbf{W} has rank Ξ½\nu for every realisation of the wireless channel, and the Ξ½\nu-layer transmission is full-rank by construction.

The codebook entries are further chosen to have good minimum determinant properties across subband selection (antenna selection + DFT-based precoder): the 5G design group specifically selected DFT codebooks because they attain near-optimal det⁑min⁑\det_{\min} for equi- powered rank-ν\nu precoders. This is the engineering realisation of Tarokh-Seshadri-Calderbank 1998: rank first, determinant second.

Massive MIMO base stations (nt=64,128n_t = 64, 128) push this further by deploying Type II CSI feedback, where the UE reports coefficients of a subspace decomposition rather than a codebook index β€” the resulting beamforming is near-MRT (matched filter) and effectively achieves full-rank precoding over any measured channel, with near- optimal sub-space-dependent coding gain.

Practical Constraints
  • β€’

    Codebook sizes: Ξ½=1\nu = 1 has 16 entries for nt=2n_t = 2, 256 for nt=32n_t = 32; Ξ½=2\nu = 2 has fewer (rank constraint tightens the set)

  • β€’

    RI reporting occurs every 20 ms; sub-band PMI (i2i_2) at ∼\sim 1 ms; wideband PMI (i1i_1) every 10 ms

  • β€’

    UE-side constraint: complexity of choosing (i1,i2,Ξ½)(i_1, i_2, \nu) is O(∣i1βˆ£β‹…βˆ£i2βˆ£β‹…Ξ½max⁑)O(|i_1| \cdot |i_2| \cdot \nu_{\max}) SVD-like operations per measurement

πŸ“‹ Ref: 3GPP TS 38.211 Β§6.3.1.5; 3GPP TS 38.214 Β§5.2.2
,

Quick Check

A 4Γ—24 \times 2 full-rank space-time code (n_t = 4, n_r = 2) is replaced by another full-rank code with four times the minimum determinant. By approximately how many dB does the BER curve shift left at high SNR?

0.750.75 dB

1.51.5 dB

33 dB

66 dB

Quick Check

A designer has two candidate space-time codes at the same rate for a 2Γ—22 \times 2 i.i.d. Rayleigh MIMO channel: Code A is rank-11 with minimum determinant 100100 (in some relative unit); Code B is rank-22 (full-rank) with minimum determinant 44. Which code has lower BER at sufficiently high SNR?

Code A, because it has 25Γ—25\times the determinant

Code B, because it satisfies the rank criterion

They are equivalent since 100=4β‹…25100 = 4 \cdot 25 and 25=2rankdifference25 = 2^{rank difference}

Cannot determine without explicit Monte-Carlo

Why This Matters: Forward to OSTBCs (Ch. 11), DMT (Ch. 12), and CDA Codes (Ch. 13)

The rank and determinant criteria of this chapter set up the entire rest of Part III:

  • Ch. 11 (Space-Time Block Codes): The Alamouti scheme and the orthogonal STBCs of Tarokh-Jafarkhani-Calderbank 1999 are engineered to satisfy the rank criterion by construction (every Ξ”\boldsymbol{\Delta} is full-rank) and to have a clean determinant formula that is readily optimised over constellation choice. The Hassibi-Hochwald 2002 linear dispersion codes are the most general linear STC structure β€” full-rank by design, with det⁑min⁑\det_{\min} optimised via the LDC parameter matrices.

  • Ch. 12 (Diversity-Multiplexing Tradeoff): Zheng-Tse 2003 generalises the rank-determinant analysis from fixed-rate (r=0r = 0, maximum diversity ntnrn_t n_r) to the entire rate-diversity tradeoff curve d⋆(r)d^\star(r) for r∈[0,min⁑(nt,nr)]r \in [0, \min(n_t, n_r)]. The rank criterion of Β§4 is the r=0r = 0 endpoint of this curve. DMT-optimal codes (codes achieving d⋆(r)d^\star(r) for all rr) generalise full-rank + large-det⁑min⁑\det_{\min} constructions.

  • Ch. 13 (CDA Codes): The Elia-Kumar-Pawar-Kumar-Caire 2006 CommIT contribution constructs approximately universal codes via cyclic division algebras β€” structurally optimal full-rank codes whose minimum determinant is bounded away from zero by a number-theoretic invariant (the non-vanishing determinant property). The Golden Code (Belfiore-Rekaya-Viterbo 2005) is the prototype 2Γ—22\times 2 CDA code.

  • Ch. 17 (LAST Codes): The El Gamal-Caire-Damen 2004 CommIT contribution extends the rank-determinant framework to lattice space-time codes with MMSE-GDFE decoding, achieving the entire DMT curve.

The vocabulary β€” rank criterion, determinant criterion, diversity order, coding gain β€” is the shared language of all these chapters.

See full treatment in The Alamouti Scheme

Diversity Order

The high-SNR slope of a code's error probability on a log-log plot: d=βˆ’lim⁑SNRβ†’βˆžlog⁑Pe(SNR)/log⁑SNRd = -\lim_{\text{SNR}\to\infty} \log P_e(\text{SNR}) / \log \text{SNR}. For a space-time code on i.i.d. Rayleigh MIMO, d=rmin⁑nrd = r_{\min} n_r where rmin⁑r_{\min} is the minimum rank of the error matrix over all codeword pairs (Theorem TRank Criterion (Tarokh-Seshadri-Calderbank 1998)).

Coding Gain

The horizontal shift (in SNR dB) between two codes of equal diversity order and rate on a log-log BER plot. For a full-rank STC, the coding gain is γc=det⁑min⁑(C)1/nt\gamma_c = \det_{\min}(\mathcal{C})^{1/n_t} (Definition DCoding Gain of a Full-Rank Space-Time Code); doubling it shifts the BER curve left by 3nr/nt3n_r/n_t dB.

Error Matrix

For two space-time codewords X,X^∈CntΓ—T\mathbf{X}, \hat{\mathbf{X}} \in \mathbb{C}^{n_t\times T}, the error matrix is Ξ”=Xβˆ’X^\boldsymbol{\Delta} = \mathbf{X} - \hat{\mathbf{X}}. Its rank controls the diversity order (rank criterion, Β§4); its determinant controls the coding gain (determinant criterion, Β§5).

Outage Capacity

The Ο΅\epsilon-quantile of the random conditional capacity C(H)C(\mathbf{H}) on a block-fading channel: CΟ΅=sup⁑{R:Pr⁑[C(H)<R]≀ϡ}C_\epsilon = \sup\{R : \Pr[C(\mathbf{H}) < R] \le \epsilon\}. It is the operational benchmark for code design on quasi-static fading channels, in contrast to the ergodic capacity that applies only under fast fading.

Block Fading (Quasi-Static Fading)

A fading model where the channel matrix H\mathbf{H} is drawn from a random distribution and held constant across the TT symbols of a codeword; between codewords, H\mathbf{H} is redrawn independently. It is the operationally relevant middle ground between fast fading (ergodic) and AWGN (deterministic) for most wireless systems.

Rank Criterion

The design rule (Tarokh-Seshadri-Calderbank 1998) stating that a space-time code achieves maximum diversity ntnrn_t n_r if and only if the error matrix Ξ”=Xβˆ’X^\boldsymbol{\Delta} = \mathbf{X} - \hat{\mathbf{X}} is full-rank for every codeword pair Xβ‰ X^\mathbf{X}\ne\hat{\mathbf{X}}.

Determinant Criterion

The second design rule (Tarokh-Seshadri-Calderbank 1998), applicable among full-rank codes: to maximise coding gain, maximise det⁑min⁑(C)=min⁑Xβ‰ X^det⁑(ΔΔH)\det_{\min}(\mathcal{C}) = \min_{\mathbf{X}\ne\hat{\mathbf{X}}} \det(\boldsymbol{\Delta}\boldsymbol{\Delta}^H) over all codeword pairs. Doubling det⁑min⁑\det_{\min} shifts the BER curve left by 3nr/nt3n_r/n_t dB.