查看“︁Word2vec”︁的源代码

{{机器学习导航栏}}
{{RoughTranslation}}
'''Word2vec'''是一群用來產生[[詞嵌入|詞向量]]的相關模型。這些模型為淺层雙層的[[神經網路]]，用來訓練以重新建構語言學之詞文本。網路以詞表現，並且需猜測相鄰位置的輸入詞，在word2vec中[[词袋模型]]假設下，詞的順序是不重要的。 

訓練完成之後，word2vec模型可以把每個詞映射到一個向量，來表示詞与詞之間的關係。該向量為神經網路的隱藏層<ref name="mikolov">{{cite web|first=Tomas|last=Mikolov|title=Efficient Estimation of Word Representations in Vector Space|url=http://arxiv.org/pdf/1301.3781.pdf|accessdate=2015-08-14|display-authors=etal|archive-date=2022-05-09|archive-url=https://web.archive.org/web/20220509180219/https://arxiv.org/pdf/1301.3781.pdf}}</ref>。

Word2vec依賴skip-grams或[[詞袋模型|連續詞袋]]（CBOW）來建立神經詞嵌入。Word2vec為托馬斯·米科洛夫（Tomas Mikolov）在[[Google]]帶領的研究團隊創造。該演算法漸漸被其他人所分析和解釋<ref name="explain">{{cite web |first1=Yoav |last1=Goldberg |first2=Omar |last2=Levy |title=word2vec Explained: Deriving Mikolov et al.’s Negative-Sampling Word-Embedding Method |url=http://arxiv.org/pdf/1402.3722v1.pdf |accessdate=2015-08-14 |archive-date=2022-01-22 |archive-url=https://web.archive.org/web/20220122002315/http://arxiv.org/pdf/1402.3722v1.pdf }}</ref><ref name="extensions">{{cite AV media|first=Radim|last=Řehůřek|title=Word2vec and friends|medium=Youtube video|url=https://www.youtube.com/watch?v=wTp3P2UnTfQ|accessdate=2015-08-14|archive-date=2020-05-22|archive-url=https://web.archive.org/web/20200522072228/https://www.youtube.com/watch?v=wTp3P2UnTfQ&gl=US&hl=en}}</ref>。

==Skip-grams和CBOW ==
CBOW把一個詞從詞窗剔除。在CBOW下給定{{mvar|n}}詞圍繞著詞{{mvar|w}}，word2vec預測一個句子中其中一個缺漏的詞{{mvar|c}}，即以機率<math>p(c|w)</math>來表示。相反地，Skip-gram給定詞窗中的文本，預測當前的詞<math>p(w|c)</math>。

==延伸==
Word2vec用來建構整份文件（而分獨立的詞）的延伸應用已被提出<ref name="doc2vec">{{cite web|first=Quoc|last=Le|title=Distributed Representations of Sentences and Documents.|url=http://arxiv.org/pdf/1405.4053v2.pdf|accessdate=2016-02-18|display-authors=etal|archive-date=2021-11-23|archive-url=https://web.archive.org/web/20211123033159/https://arxiv.org/pdf/1405.4053v2.pdf}}</ref>，
該延伸稱為paragraph2vec或doc2vec，並且用C、Python<ref name="doc2vec_python">{{cite web|title=Doc2Vec tutorial using Gensim|url=https://medium.com/@klintcho/doc2vec-tutorial-using-gensim-ab3ac03d3a1|accessdate=2015-08-02|display-authors=etal|archive-date=2021-01-23|archive-url=https://web.archive.org/web/20210123122255/https://medium.com/@klintcho/doc2vec-tutorial-using-gensim-ab3ac03d3a1}}</ref><ref name="doc2vec_imdb">{{cite web|title=Doc2vec for IMDB sentiment analysis|url=https://github.com/piskvorky/gensim/blob/develop/docs/notebooks/doc2vec-IMDB.ipynb|accessdate=2016-02-18|display-authors=etal|archive-date=2020-01-07|archive-url=https://web.archive.org/web/20200107172512/https://github.com/RaRe-Technologies/gensim/blob/develop/docs/notebooks/doc2vec-IMDB.ipynb|dead-url=no}}</ref>和 Java/Scala<ref name="doc2vec_java">{{cite web|title=Doc2Vec and Paragraph Vectors for Classification|url=http://deeplearning4j.org/doc2vec.html|accessdate=2016-01-13|display-authors=etal|archive-url=https://web.archive.org/web/20151231073758/http://deeplearning4j.org/doc2vec.html|archive-date=2015-12-31|dead-url=yes}}</ref>實做成工具（參考下方）。Java和Python也支援推斷文件嵌入於未觀測的文件。

==分析==
對word2vec框架為何做[[词嵌入]]如此成功知之甚少，約阿夫·哥德堡（Yoav Goldberg）和歐莫·列維（Omer Levy）指出word2vec的功能導致相似文本擁有相似的嵌入（用[[余弦相似性]]計算）並且和[[約翰·魯伯特·弗斯]]的{{link-en|分佈語義|Distributional semantics|分佈假說}}有關。

==實作==
* [https://code.google.com/p/word2vec/ C] {{Wayback|url=https://code.google.com/p/word2vec/ |date=20160310194120 }}
* [https://web.archive.org/web/20160305131924/http://deeplearning4j.org/word2vec.html Java/Scala]
* [https://github.com/tensorflow/tensorflow/blob/r1.1/tensorflow/examples/tutorials/word2vec/word2vec_basic.py Python] {{Wayback|url=https://github.com/tensorflow/tensorflow/blob/r1.1/tensorflow/examples/tutorials/word2vec/word2vec_basic.py |date=20200324104847 }}
* [http://radimrehurek.com/gensim/models/word2vec.html Python] {{Wayback|url=http://radimrehurek.com/gensim/models/word2vec.html |date=20201030161741 }}

== 參見 ==
* [[向量空間模型]]

== 参考文献 ==
{{Reflist|30em}}

{{自然语言处理}}
{{Differentiable computing}}

[[Category:自由软件]]
[[Category:自然语言处理]]
[[Category:人工神经网络]]
[[Category:机器学习]]