Hiroki Naganuma
Reference
Answer
Distributed representation, n-gram
-
[Word2Vecとは 分散表現・Skip-gram法](https://ledge.ai/word2vec/)
初期化
Pretrainind
What is a Pre-trained Model? Simply put, a pre-trained model is a model created by some one else to solve a similar problem. Instead of building a model from scratch to solve a similar problem, you use the model trained on other problem as a starting point. For example, if you want to build a self learning car. You can spend years to build a decent image recognition algorithm from scratch or you can take inception model (a pre-trained model) from Google which was built on ImageNet data to identify images in those pictures. A pre-trained model may not be 100% accurate in your application, but it saves huge efforts required to re-invent the wheel.
事前学習済みモデルとは何ですか？ 簡単に言うと、似たような問題を解決するために他の人が作ったモデルのことです。同じような問題を解決するために一からモデルを作るのではなく、他の問題で訓練されたモデルを出発点として使うのです。例えば、自己学習する自動車を作りたい場合。何年もかけてまともな画像認識アルゴリズムを一から作ることもできるし、GoogleのImageNetデータで作られたインセプションモデル（事前学習済みモデル）を使って、それらの写真に写っている画像を識別することもできる。学習済みのモデルは、あなたのアプリケーションで100％正確ではないかもしれませんが、車輪を再発明するのに必要な膨大な労力を節約することができます。
-
Representation Learning (1) — Greedy Layer-Wise Unsupervised Pretraining
-
Greedy Layer-Wise Training of Deep Networks
Greedy layer-wise supervised training A reasonable question to ask is whether the fact that each layer is trained in an unsupervised way is critical or not. An alternative algorithm is supervised, greedy and layer-wise: train each new hidden layer as the hidden layer of a one-hidden layer supervised neural network NN (taking as input the output of the last of previously trained layers), and then throw away the output layer of NN and use the parameters of the hidden layer of NN as pre-training initialization of the new top layer of the deep net, to map the output of the previous layers to a hopefully better representation.