From "they move together" to covariance
Start — the everyday picture. Think of two things that tend to change together: the taller a kid is, the bigger their shoe size; the hotter the day, the more ice cream a shop sells. Knowing one gives you a hint about the other. Some pairs move opposite ways (more rain, fewer beach visitors), and some are unrelated (your shoe size and today's lottery number). A joint distribution is just the full picture of how two random quantities, $X$ and $Y$, behave together — not only on their own.
Build — putting a number on it. To measure "do they move together?", look at each outcome and ask: is $X$ above or below its average at the same moment $Y$ is? Covariance multiplies those two gaps and averages the result: $\text{Cov}(X,Y)=E[(X-\mu_X)(Y-\mu_Y)]$. When $X$ and $Y$ are usually both above (or both below) their means together, the products are positive and covariance is positive; when one is high while the other is low, it turns negative. Because covariance carries the units of $X$ times $Y$, we rescale it into the unitless correlation $\rho=\dfrac{\text{Cov}(X,Y)}{\sigma_X\sigma_Y}$, which always lands between $-1$ and $+1$. Worked number: if $\sigma_X=\sigma_Y=1$ and $\rho=0.8$, then $\text{Cov}=\rho\,\sigma_X\sigma_Y=0.8$.
Deepen — the precise statements. The variance of a sum picks up a cross term: $\text{Var}(X+Y)=\text{Var}(X)+\text{Var}(Y)+2\,\text{Cov}(X,Y)$ — for the case above that is $1+1+1.6=3.6$, well over the $2$ you would get if the two were independent. For a bivariate normal, learning $X=x$ reshapes $Y$ into a narrower bell, $Y\,|\,X{=}x\sim N\!\left(\mu_Y+\rho\tfrac{\sigma_Y}{\sigma_X}(x-\mu_X),\;\sigma_Y^2(1-\rho^2)\right)$. The leftover variance $\sigma_Y^2(1-\rho^2)$ shrinks by exactly the $\rho^2$ fraction — the same $R^2$ you meet in regression. The sliders map straight onto these symbols: rho sets $\rho$, sigma_X and sigma_Y set the two spreads, n the sample size, and X query the $x$ you condition on.
1) Set rho to 0 — the scatter cloud becomes round and the dashed best-fit line flattens out. 2) Drag rho toward +0.9 and the cloud tilts up along the diagonal; push it to −0.9 and it tilts the other way. 3) Open the Conditional tab and slide X query: the teal bell stays narrower than the gray marginal — that is variance reduced by $\rho^2$. 4) Open the Independence Test tab to see $Y=X^2$, where $\rho\approx 0$ yet $Y$ is completely determined by $X$ — proof that zero correlation is not the same as independence.
Bivariate Normal & Covariance
$$\text{Cov}(X,Y)=E[(X-\mu_X)(Y-\mu_Y)]=\rho\sigma_X\sigma_Y$$
$$\text{Var}(X+Y)=\text{Var}(X)+\text{Var}(Y)+2\text{Cov}(X,Y)$$
Conditional distribution: $Y|X=x\sim N\!\left(\mu_Y+\rho\frac{\sigma_Y}{\sigma_X}(x-\mu_X),\;\sigma_Y^2(1-\rho^2)\right)$
For Bivariate Normal only: rho=0 implies independence. In general: independence implies rho=0, but rho=0 does NOT imply independence. Example: Y=X^2 where X~N(0,1) — Cov(X,Y)=0 but Y is determined by X.
$\text{Var}(aX+bY)=a^2\text{Var}(X)+b^2\text{Var}(Y)+2ab\,\text{Cov}(X,Y)$. Portfolio variance: risk depends on correlations between assets!
Zero correlation does NOT mean independence.
Correlation only measures LINEAR dependence. Y=X^2 has rho=0 but perfect dependence. For bivariate normal ONLY, rho=0 implies independence. Also: Var(X+Y) = Var(X)+Var(Y) requires independence (or at least zero covariance); when correlated, add 2Cov(X,Y).
Example: ignoring correlation between asset returns leads to underestimated portfolio risk.
Let $X$ and $Y$ have means 0, standard deviations 1 and correlation $\rho=0.6$. Given $X=2$, the conditional distribution is $Y\mid X=2\sim N(\rho\cdot2,\;1-\rho^2)=N(1.2,\,0.64)$, so the conditional mean is 1.2 and the standard deviation is 0.8. Then $P(Y>1\mid X=2)=P\left(Z>\frac{1-1.2}{0.8}\right)=P(Z>-0.25)\approx0.599$. Also $\text{Var}(X+Y)=1+1+2(0.6)=3.2$, larger than the 2 you would get if they were independent.
Put half of your money in each of two assets with standard deviations 0.2 and 0.3. If the correlation is $\rho=0.5$, the portfolio variance is $0.25(0.04)+0.25(0.09)+2(0.25)(0.5)(0.06)=0.0475$, a standard deviation of about 0.218. If $\rho=0$ it is $0.0325$, or about 0.180. Positive correlation raises risk; this is the whole case for diversification.
Set $\rho=0.6$ and move the conditional query slider to $x=2$: the Conditional tab should show a centre near 1.2. Slide $\rho$ toward 0.99 and the cloud collapses onto a line; toward $-0.99$ it flips direction. Use the Independence Test tab with $\rho=0$ to see why uncorrelated bivariate normal data are independent, then remember the warning above: for other distributions, zero correlation does not mean independence (the standard example is $Y=X^2$).
1. If $\sigma_X=2$, $\sigma_Y=3$ and $\rho=0.5$, what is $\text{Cov}(X,Y)$? 2. In that case, what is $\text{Var}(X+Y)$? 3. For $\rho=0.8$ and unit standard deviations, what is the conditional variance of $Y$ given any $x$?
Answers. 1. $\rho\sigma_X\sigma_Y=0.5\cdot2\cdot3=3$. 2. $4+9+2\cdot3=19$. 3. $1-\rho^2=0.36$, which does not depend on $x$.