When we take a sample and calculate its mean and standard deviation, this is treated as a random variable for the population mean/standard deviation.
We treat each observation as a random variable.
The random variables \(X_{1},\dots,X_{n}\) form a random sample of size \(n\) if:
- The \(X_{i}\) are independent.
- Every \(X_{i}\) has the same probability distribution.
Note that the second is not true when sampling without replacement. We treat it as a good approximation, though, if \(n/N\le0.05\)
The above two conditions are the same as saying the \(X_{i}\) are independent and identically distributed (iid).
Normal Distribution
For a normal distribution, the expected value of the sample mean is:
\begin{equation*}
E(S)=\sqrt{\frac{2}{n-1}}\Gamma\left(\frac{n}{2}\right)\frac{\sigma}{\Gamma\left(\frac{1}{2}(n-1)\right)}
\end{equation*}