貪心科技cv計算機視覺訓練營project1-image captioning

(ImageAcatlyingona aacatlyingonaCNN+LSTM+aacatlyingonaCNN+LSTM+應用領域文字嵌入

遷移學習(transferLSTM(遞歸神經(jīng)網(wǎng)絡文字嵌入

遷移學習(transferLSTM(遞歸神經(jīng)網(wǎng)絡文字嵌入

遷移學習(transferLSTM(遞歸神經(jīng)網(wǎng)絡文字嵌入

遷移學習(transferLSTM(遞歸神經(jīng)網(wǎng)絡文字嵌入

遷移學習(transferLSTM(遞歸神經(jīng)網(wǎng)絡絡adogisrunningthroughthe絡文字嵌入

遷移學習(transferLSTM(遞歸神經(jīng)網(wǎng)絡PythonAnaconda安裝包64windows:https://

condainstall-cconda-forgeKerascondainstall-cconda-forgenltkcondainstall-canacondaPillowcondainstall-canacondaOpenCVcondainstall-cconda-forgecondainstallpytorch=0.4.1-c數(shù)據(jù)集(FramingImageDescriptionasaRankingTaskDataModelsandEvaluationMetrics2013.)8000圖像每幅圖5個標題分為3個集合6000個訓練圖像1000個開發(fā)圖像1000個測試圖Achildinapinkdressisclimbingupasetofstairsinanentryway.AgirlgoingintoawoodenbuildingAlittlegirlclimbingintoawoodenyhouse.Alittlegirlclimbingthestairstoheryhouse.Alittlegirlinapinkdressgoingintoawoodencabin

AchildinapinkdressisclimbingupasetofstairsinanentrywayAgirlgoingintoawoodenbuilding.Alittlegirlclimbingintoa yhouse.Alittlegirlclimbingthestairsto yhouse.Alittlegirlinapinkdressgoingintoawoodencabin衡量兩個句子的相似度一個句子與其他幾個句子的相似度(CorpusBLEU:BilingualEvaluationUnderstudy(雙語評估替換BLEU，全稱為BilingualEvaluationUnderstudy（雙語評估替具體內(nèi)容可以參考: 或者sequencefeature)圖像sequencefeature)圖像單詞sequencesequencefeature)單詞從圖像(image)到圖像特征(image[0.235,0.457,0.5685,...,VGG16(VeryDeepConvolutionalNetworksforLarge-ScaleVisualRecognition)Pre-trainedmodel:OxfordVisualGeometryGroup贏得2014用于圖像分類將輸入圖像分為1000遷移學習(transferVGG16CNN原本的目標是分類基于ImageNet數(shù)據(jù)集進行訓練,VGG16的最后一層是將倒數(shù)第二層 單詞嵌入(WordLSTM的輸入是數(shù)值,單詞需要轉(zhuǎn)換為數(shù)值才能使用LSTM,最簡單的方式是將單詞轉(zhuǎn)化為整數(shù),每個單詞都對應于一個整數(shù)但是這樣的方式無法有效的表達單詞直接的相關性,例如:國王,王后,男,女如果整數(shù)10,11,200,300表示的話,無法體現(xiàn)出:國王–男+女≈王一個整數(shù)來表達一個單詞.向量提供了更大的信息量,里面可以嵌入單詞之間的關系,更好的表達一個單詞.步驟初始化圖像標題為循環(huán)如下步驟將圖像特征和當前的圖像標題作為輸入,預測標題的下一個單詞,假設單詞為將word1如果word1的值為”endseq”,或者當前標題的長度達到了標題最大長度,退出循任務一:使 創(chuàng) 定義 N網(wǎng)絡結(jié)任務二將flicker8k的圖像文件轉(zhuǎn)為圖像特征保存從給定的VGG16網(wǎng)絡結(jié)構(gòu)文件和網(wǎng)絡權(quán)值文件創(chuàng)建VGG16 ,key為文件名(不帶.jpg后綴),value為一個網(wǎng)絡的輸出單詞任務三完成create_tokenizer單詞sequencesequencefeature)單詞任務三完成create_tokenizer單詞sequencesequencefeature)任務三完成create_input_data_for_one_image函為了訓練LSTM訓練數(shù)據(jù)中的每一個圖像的每一個標題都需要被重新拆分為輸入和輸出部分如果標題為”acatsitsonthetable”,需要添加起始和結(jié)束標志,變?yōu)椤畇tartseqacatsitsonthetableendseq’,再從它astartseqstartseqastartseqacatstartseqacatsitsstartseqacatsitsonstartseqacatsitsonthe另外需要預處理單詞,‘s和一些不需要的標點符號sequencefeature)任務四:構(gòu)建自動產(chǎn)生圖像標題的網(wǎng)絡模型sequencefeature)單詞單詞使用交叉驗證cross_validation來衡量不同結(jié)構(gòu)的優(yōu)astartseqstartseqastartseqacat[0.234,0.1124,…,startseqacatsitsstartseqacatsitsonstartseqacatsitsonthesequencefeature)sequencefeature)單