Logistic

$Y_i \sim Ber(\pi_i)$ , $f(y_i)=\pi_i^{y_i} (1-\pi_i)^{1-y_i}$

\begin{align*} & g(\mu)=g(\pi)= \ln{\frac{\pi_i}{1-\pi_i}} = \mathbb{X}_i \boldsymbol{\beta} \\\\\\ \Rightarrow \ & \pi_i = \frac{\texttt{exp}(\mathbb{X} \boldsymbol{\beta}) }{1+\texttt{exp}(\mathbb{X} \boldsymbol{\beta})} \end{align*}

\begin{align*} L & =\prod_{i=1}^n \pi_i^{y_i} (1-\pi_i)^{1-y_i} \\\\\\ & = \prod_{i=1}^n \left(\frac{\texttt{exp}(\mathbb{X}_i \boldsymbol{\beta}) }{1+\texttt{exp}(\mathbb{X}_i \boldsymbol{\beta})} \right)^{y_i} \left(\frac{1 }{1+\texttt{exp}(\mathbb{X}_i \boldsymbol{\beta})} \right)^{1-y_i} \end{align*}

\sum_{i=1}^n \left[ y_i \mathbb{X}_i \boldsymbol{\beta} -\ln(1+\texttt{exp}(\mathbb{X}_i \boldsymbol{\beta})) \right]

MLE of $\boldsymbol{\beta}$ doesn’t has closed form.

Sex	Disease	No disease	Probability of disease	Odds
male	217	162	$\frac{217}{217+162} = 0.573$	$\frac{0.573}{1-0.573} = 1.342$
female	105	136	$\frac{105}{105+136} = 0.436$	$\frac{0.436}{1-0.436} = 0.773$

\begin{align*} \frac{\text{odds(disease for male)}}{\text{odds(disease for female)}} = \frac{1.342}{0.773} = 1.74>1 \end{align*}

代表男性患疾病的機率比女性來的高

\begin{align*} -\frac{1}{n} \sum_{i=1}^n \left[ y_i \ln p_i + (1 - y_i) \ln (1 - p_i) \right], \quad y_i = 0 \text{ or } 1 \end{align*}

\begin{align*} -\frac{1}{n} \sum_{i=1}^n \sum_{c=1}^k y_{i,c} \ln p_{i,c} \end{align*}

假如現在有3個類別

\begin{align*} \left( -\ln 0.7 - \ln 0.6 - \ln 0.5 \right) \times \frac{1}{3} \approx 0.52 \end{align*}

\begin{align*} -\frac{1}{n} \sum_{i=1}^n \left| y_i-\hat{y}_i \right| \end{align*}

\begin{align*} -\frac{1}{n} \sum_{i=1}^n \left( y_i-\hat{y}_i \right)^2 \end{align*}

\begin{align*} -\frac{1}{n} \sum_{i=1}^n \sqrt{\left( y_i-\hat{y}_i \right)^2 } \end{align*}