ARMA(1,1)

$$y_t={\varphi }_0+{\varphi }_1{y}_{t-1}+{\theta }_1{ϵ}_{t-1}+{ϵ}_t$$其中，${ϵ}_t\stackrel{i.i.d.}{\sim }N(0,{\sigma }_{ϵ}^2)$

极大似然估计

假设观测数据集为 { y 1 , … , y T } \{y_1,\ldots,y_T\} {y1​,…,yT​}，令$\theta =({\varphi }_0,{\varphi }_1,{\theta }_1,{\sigma }_{ϵ}^2{)}^{\prime }$，
则有似然函数：
$$\begin{array}{rl}L(\theta )& =f_{\theta }(y_1,\dots ,y_T)\\ & =\prod _{t=2}^T{f}_{\theta }(y_t|y_{t-1})f_{\theta }(y_1)\end{array}$$其中，$y_1\sim N(\frac{{\varphi }_0}{1-{\varphi }_1},\frac{({\theta }_1^2+1){\sigma }_{ϵ}^2}{1-{\varphi }_1^2})$

1. $y_1$服从正态分布？
   ARMA(1,1)平稳，则$|{\varphi }_1|<1$
   $$\begin{array}{rl}{y}_t& ={\varphi }_0+{\varphi }_1{y}_{t-1}+{\theta }_1{ϵ}_{t-1}+{ϵ}_t\\ & ={\varphi }_0+{\varphi }_1{\varphi }_0+{\varphi }_1^2{y}_{t-n}+{\varphi }_1{\theta }_1{ϵ}_{t-2}+{\varphi }_1{ϵ}_{t-1}+{\theta }_1{ϵ}_{t-1}+{ϵ}_t\\ & =\dots \\ & ={\varphi }_0\sum _{i=0}^{n-1}{\varphi }_1^i+{\varphi }_1^n{y}_{t-n}+\sum _{i=2}^n{\varphi }_1^{i-1}{\theta }_1{ϵ}_{t-i}+\sum _{i=1}^{n-1}{\varphi }_1^i{ϵ}_{t-i}+{\theta }_1{ϵ}_{t-1}+{ϵ}_t\end{array}$$当$n\to \infty$时，${\varphi }_1^n{y}_{t-2}\to 0$，故$y_t$服从正态分布，即有$y_1$服从正态分布
2. $E{y}_1=\frac{{\varphi }_0}{1-{\varphi }_1}$
   令$\mu =E{y}_t$
   $$\begin{array}{rl}\mu & ={\varphi }_0+{\varphi }_1\mu +0+0\\ \mu & =\frac{{\varphi }_0}{1-{\varphi }_1}\end{array}$$
3. $Var(y_y)=\frac{({\theta }_1^2+1){\sigma }_{ϵ}^2}{1-{\varphi }_1^2}$
   令${\sigma }^2=Var(y_t)$
   $$\begin{array}{rl}{\sigma }^2& =0+{\varphi }_1^2{\sigma }^2+{\theta }_1^2{\sigma }_{ϵ}^2+{\sigma }_{ϵ}^2\\ {\sigma }^2& =\frac{({\theta }_1^2+1){\sigma }_{ϵ}^2}{1-{\varphi }_1^2}\end{array}$$

$y_t|y_{t-1}\sim N({\varphi }_0+{\varphi }_1{y}_{t-1},({\theta }_1^2+1){\sigma }_{ϵ}^2)$

故似然函数便可写成下式：
$$\begin{array}{rl}L(\theta )& =\prod _{t=2}^T\frac{1}{\sqrt{2\pi ({\theta }_1^2+1){\sigma }_{ϵ}^2}}exp\left(-\frac{(y_t-{\varphi }_0-{\varphi }_1{y}_{t-1}{)}^2}{2({\theta }_1^2+1){\sigma }_{ϵ}^2}\right)\frac{1}{\sqrt{2\pi {\varphi }_0/(1-{\varphi }_1)}}exp\left(-\frac{(y_1-{\varphi }_0/(1-{\varphi }_1){)}^2}{2({\theta }_1^2+1){\sigma }_{ϵ}^2/(1-{\varphi }_1^2)}\right)\end{array}$$对$-lnL(\theta )$利用梯度下降求解便可得参数估计

条件似然估计

在给定$y_0$和${ϵ}_0$的条件下，我们可以逐步推出${ϵ}_1,\dots ,{ϵ}_{T-1}$，即${ϵ}_1,\dots ,{ϵ}_{T-1}$也给定了

${ϵ}_1=y_1-({\varphi }_0+{\varphi }_1{y}_0+{\theta }_1{ϵ}_0)$
$\dots$
${ϵ}_{T-1}=y_{T-1}-({\varphi }_0+{\varphi }_1{y}_{T-1}+{\theta }_1{ϵ}_{T-2})$

故似然函数如下:
$$\begin{array}{rl}L(\theta )& =f_{\theta }(y_1,\dots ,y_T|y_0,{ϵ}_0)\\ & =f_{\theta }(y_1,\dots ,y_T|y_0,{ϵ}_0,\dots ,{ϵ}_{T-1})\\ & =\prod _{t=1}^T{f}_{\theta }(y_t|y_{t-1},{ϵ}_{t-1})\end{array}$$其中，$y_t|y_{t-1},{ϵ}_{t-1}\sim N({\varphi }_0+{\varphi }_1{y}_{t-1}+{\theta }_1{ϵ}_{t-1},{\sigma }_{ϵ}^2)$
故似然函数可推导如下：
$$L(\theta )=\prod _{t=1}^T\frac{1}{\sqrt{2\pi {\sigma }_{ϵ}^2}}exp\left(-\frac{(y_t-{\varphi }_0-{\varphi }_1{y}_{t-1}-{\theta }_1{ϵ}_{t-1}{)}^2}{2{\sigma }_{ϵ}^2}\right)$$

最终对$-lnL(\theta )$利用梯队下降便可解得参数估计。