Gumbel 重参数化相关性质证明

Gumbel 的采样过程：

z = a r g m a x i { g i + l o g ( π i ) } , g i = ? l o g ( ? l o g ( u i ) ) , u i ～ U ( 0 , 1 ) z=argmax_i \{g_i + log(\pi_i)\}, g_i = -log(-log(u_i)),u_i\sim U(0, 1) z=argmaxi?{gi?+log(πi?)},gi?=?log(?log(ui?)),ui?～U(0,1)

采样得到的随机变量满足一下分布：

$$g_i～Gumble(0,1)(1)$$

$$h_i=g_i+log({\pi }_i)～Gumble(log({\pi }_i),1)(2)$$

证明过程：

$$P(u)=P(U\le u)=u,u\in (0,1)$$

$$G=?log(?log(U)),u\in (0,1)$$

$$P(g)=P(G\le g)=P(?log(?log(U))\le g)$$

$$=P(U\le exp(?exp(?g)))$$

$$=exp(?exp(?g))$$

$$P(g)=exp(?exp(?g))$$

$$g_i～Gumble(0,1)$$

$$h_i=g_i+log({\pi }_i)～Gumble(log({\pi }_i),1)$$

$$P(Z=z)={\pi }_i(3)$$

证明过程：

$$P(Z=z|U_z=u_z)=\prod _{i\ne z}P(H_i<g_z+log({\pi }_z))$$

$$=\prod _{i\ne z}P(G_i+log({\pi }_i)<g_z+log({\pi }_z))$$

$$=\prod _{i\ne z}P(U_i<u_z^{p_i/p_z})$$

$$=\prod _{i\ne z}{u}_z^{p_i/p_z}=u_z^{1/p_z?1}$$

$$P(Z=z)={\int }_0^{1}P(Z=z|U_z=u_z)P(U_z=u_z)d{u}_z$$

$$={\int }_0^1{u}_z^{1/p_z?1}?1?d{u}_z$$

$$=\frac{1}{1/p_z}{u}_z^{1/p_z}{|}_0^1$$

$$=p_z$$