查看“︁转移熵”︁的源代码

'''转移熵'''（{{lang-en|transfer entropy}}）是测量两个[[随机过程]]之间有向（时间不对称）[[信息]]转移量的一种[[無母數統計|非参数统计量]]。<ref>{{Cite journal |last=Schreiber |first=Thomas |date=1 July 2000 |title=Measuring information transfer |journal=Physical Review Letters |volume=85 |issue=2 |page=461–464 |arxiv=nlin/0001042 |bibcode=2000PhRvL..85..461S |doi=10.1103/PhysRevLett.85.461 |pmid=10991308 |s2cid=7411376}}</ref><ref name="Scholarpedia">{{Cite journal |last=Seth |first=Anil |year=2007 |title=Granger causality |journal=[[Scholarpedia]] |volume=2 |issue=7 |page=1667 |bibcode=2007SchpJ...2.1667S |doi=10.4249/scholarpedia.1667 |doi-access=free}}</ref><ref name="Schindler07">{{Cite journal |last=Hlaváčková-Schindler |first=Katerina |last2=Palus, M |last3=Vejmelka, M |last4=Bhattacharya, J |date=1 March 2007 |title=Causality detection based on information-theoretic approaches in time series analysis |journal=Physics Reports |volume=441 |issue=1 |page=1–46 |bibcode=2007PhR...441....1H |citeseerx=10.1.1.183.1617 |doi=10.1016/j.physrep.2006.12.004}}</ref>过程''X''到另一个过程''Y''的转移熵可定义为：在已知''Y''过去值的情况下，了解''X''的过去值所能减少''Y''未来值不确定性的程度。更具体地说，假定<math> X_t </math>和<math> Y_t </math>（<math> t\in \mathbb{N} </math>）表示两个随机过程，并用[[熵 (信息论)|香农熵]]来度量信息量，则转移熵可定义为：

: <math>
T_{X\rightarrow Y} = H\left( Y_t \mid Y_{t-1:t-L}\right) - H\left( Y_t \mid Y_{t-1:t-L}, X_{t-1:t-L}\right),
</math>

其中''H'' ( ''X'' ) 表示''X''的香农熵。此外，还可以使用其他类型的[[熵 (信息论)|熵]]度量（例如{{le|雷尼熵|Rényi entropy}}）对上述定义进行扩展。<ref name="Schindler07">{{Cite journal |last=Hlaváčková-Schindler |first=Katerina |last2=Palus, M |last3=Vejmelka, M |last4=Bhattacharya, J |date=1 March 2007 |title=Causality detection based on information-theoretic approaches in time series analysis |journal=Physics Reports |volume=441 |issue=1 |page=1–46 |bibcode=2007PhR...441....1H |citeseerx=10.1.1.183.1617 |doi=10.1016/j.physrep.2006.12.004}}</ref><ref>{{Cite journal |last=Jizba |first=Petr |last2=Kleinert |first2=Hagen |last3=Shefaat |first3=Mohammad |date=2012-05-15 |title=Rényi's information transfer between financial time series |journal=Physica A: Statistical Mechanics and Its Applications |language=en |volume=391 |issue=10 |page=2971–2989 |arxiv=1106.5913 |bibcode=2012PhyA..391.2971J |doi=10.1016/j.physa.2011.12.064 |issn=0378-4371 |s2cid=51789622}}</ref>

转移熵可看作一种{{le|条件互信息|Conditional mutual information}}<ref name="Wyner1978">{{Cite journal |last=Wyner |first=A. D. |year=1978 |title=A definition of conditional mutual information for arbitrary ensembles |journal=Information and Control |volume=38 |issue=1 |page=51–59 |doi=10.1016/s0019-9958(78)90026-8 |doi-access=free}}</ref><ref name="Dobrushin1959">{{Cite journal |last=Dobrushin |first=R. L. |year=1959 |title=General formulation of Shannon's main theorem in information theory |journal=Uspekhi Mat. Nauk |volume=14 |page=3–104}}</ref>，其条件为受影响变量的历史值<math>Y_{t-1:t-L}</math>：

: <math>
T_{X\rightarrow Y} = I(Y_t ; X_{t-1:t-L} \mid Y_{t-1:t-L}).
</math>

对[[自我迴歸模型|向量自回归过程]]而言，转移熵可简化为[[格蘭傑因果關係|格兰杰因果关系]]。<ref name="Equal">{{Cite journal |last=Barnett |first=Lionel |date=1 December 2009 |title=Granger Causality and Transfer Entropy Are Equivalent for Gaussian Variables |journal=Physical Review Letters |volume=103 |issue=23 |page=238701 |arxiv=0910.4514 |bibcode=2009PhRvL.103w8701B |doi=10.1103/PhysRevLett.103.238701 |pmid=20366183 |s2cid=1266025}}</ref> 因而，转移熵适用于[[非线性回归|非线性信号]]分析等格兰杰因果关系的模型假设不成立的场合。<ref name="Greg">{{Cite conference |last=Ver Steeg |first=Greg |last2=Galstyan |first2=Aram |year=2012 |title=Information transfer in social media |publisher=[[Association for Computing Machinery|ACM]] |pages=509–518 |arxiv=1110.2724 |bibcode=2011arXiv1110.2724V |booktitle=Proceedings of the 21st international conference on World Wide Web (WWW '12)}}</ref><ref>{{Cite journal |last=Lungarella |first=M. |last2=Ishiguro, K. |last3=Kuniyoshi, Y. |last4=Otsu, N. |date=1 March 2007 |title=Methods for quantifying the causal structure of bivariate time series |journal=International Journal of Bifurcation and Chaos |volume=17 |issue=3 |page=903–921 |bibcode=2007IJBC...17..903L |citeseerx=10.1.1.67.3585 |doi=10.1142/S0218127407017628}}</ref>然而，它通常需要更多的样本才能进行准确估计。<ref>{{Cite journal |last=Pereda |first=E |last2=Quiroga, RQ |last3=Bhattacharya, J |date=Sep–Oct 2005 |title=Nonlinear multivariate analysis of neurophysiological signals. |journal=Progress in Neurobiology |volume=77 |issue=1–2 |page=1–37 |arxiv=nlin/0510077 |bibcode=2005nlin.....10077P |doi=10.1016/j.pneurobio.2005.10.003 |pmid=16289760 |s2cid=9529656}}</ref>熵公式中的概率可以使用分箱、最近邻等不用方法来估计，或为了降低复杂性而使用非均匀嵌入方法。<ref>{{Cite journal |last=Montalto |first=A |last2=Faes, L |last3=Marinazzo, D |date=Oct 2014 |title=MuTE: A MATLAB Toolbox to Compare Established and Novel Estimators of the Multivariate Transfer Entropy. |journal=PLOS ONE |volume=9 |issue=10 |page=e109462 |bibcode=2014PLoSO...9j9462M |doi=10.1371/journal.pone.0109462 |pmc=4196918 |pmid=25314003 |doi-access=free}}</ref>虽然转移熵的原始定义是建立在{{le|双变量分析|Bivariate analysis}}基础上的，但后来也扩展到[[多元变量统计|多变量]]分析中。这种扩展可以以其他潜在源变量为条件<ref>{{Cite journal |last=Lizier |first=Joseph |last2=Prokopenko, Mikhail |last3=Zomaya, Albert |year=2008 |title=Local information transfer as a spatiotemporal filter for complex systems |journal=Physical Review E |volume=77 |issue=2 |page=026110 |arxiv=0809.3275 |bibcode=2008PhRvE..77b6110L |doi=10.1103/PhysRevE.77.026110 |pmid=18352093 |s2cid=15634881}}</ref> ，或考虑从一组源进行转移<ref name="Lizier2011">{{Cite journal |last=Lizier |first=Joseph |last2=Heinzle, Jakob |last3=Horstmann, Annette |last4=Haynes, John-Dylan |last5=Prokopenko, Mikhail |year=2011 |title=Multivariate information-theoretic measures reveal directed information structure and task relevant changes in fMRI connectivity |journal=Journal of Computational Neuroscience |volume=30 |issue=1 |page=85–107 |doi=10.1007/s10827-010-0271-2 |pmid=20799057 |s2cid=3012713}}</ref>，不过这些都需要更多的样本。

转移熵被用于估计[[神经元]]的[[静息态|功能连接]]<ref name="Lizier2011" /><ref>{{Cite journal |last=Vicente |first=Raul |last2=Wibral, Michael |last3=Lindner, Michael |last4=Pipa, Gordon |date=February 2011 |title=Transfer entropy—a model-free measure of effective connectivity for the neurosciences |journal=Journal of Computational Neuroscience |volume=30 |issue=1 |page=45–67 |doi=10.1007/s10827-010-0262-3 |pmc=3040354 |pmid=20706781}}</ref><ref name="Shimono2014">{{Cite journal |last=Shimono |first=Masanori |last2=Beggs, John |date=October 2014 |title=Functional clusters, hubs, and communities in the cortical microconnectome |url= |journal=Cerebral Cortex |volume=25 |issue=10 |page=3743–57 |doi=10.1093/cercor/bhu252 |pmc=4585513 |pmid=25336598}}</ref>、[[社会网络|社交网络]]中的社会影响<ref name="Greg" />以及武装冲突事件之间的[[因果推斷|统计因果关系]]等。<ref>{{Cite journal |last=Kushwaha |first=Niraj |last2=Lee |first2=Edward D |date=July 2023 |title=Discovering the mesoscale for chains of conflict |url=https://doi.org/10.1093/pnasnexus/pgad228 |journal=PNAS Nexus |volume=2 |issue=7 |doi=10.1093/pnasnexus/pgad228 |issn=2752-6542 |pmc=10392960 |pmid=37533894}}</ref>转移熵是{{le|有向信息|Directed information}}的有限形式，于1990年由{{le|詹姆斯·马西|James Massey}}<ref>{{Cite journal |last=Massey |first=James |date=1990 |title=Causality, Feedback And Directed Information |issue=ISITA |citeseerx=10.1.1.36.5688}}</ref>定义为<math>I(X^n\to Y^n) =\sum_{i=1}^n I(X^i;Y_i|Y^{i-1})</math>，其中<math>X^n</math>表示向量<math>X_1,X_2,...,X_n</math>，<math>Y^n</math>则表示<math>Y_1,Y_2,...,Y_n</math> 。有向信息在描述具有或没有反馈的通信信道的基本极限（[[信道容量]]）中起着关键作用。<ref>{{Cite journal |last=Permuter |first=Haim Henry |last2=Weissman |first2=Tsachy |last3=Goldsmith |first3=Andrea J. |date=February 2009 |title=Finite State Channels With Time-Invariant Deterministic Feedback |journal=IEEE Transactions on Information Theory |volume=55 |issue=2 |page=644–662 |arxiv=cs/0608070 |doi=10.1109/TIT.2008.2009849 |s2cid=13178}}</ref><ref>{{Cite journal |last=Kramer |first=G. |date=January 2003 |title=Capacity results for the discrete memoryless network |journal=IEEE Transactions on Information Theory |volume=49 |issue=1 |page=4–21 |doi=10.1109/TIT.2002.806135}}</ref>

== 参见 ==
* [[互信息]]
* [[因果关系]]
* [[潜在结果模型]]

== 参考文献 ==
{{reflist|30em}}

[[Category:信息學熵]]
[[Category:非参数统计]]
[[Category:因果律]]
[[Category:回归分析]]
[[Category:时间序列]]