深度學(xué)習(xí)案例教程 課件6.6實踐任務(wù)新聞主題分類

第六節(jié)

實踐任務(wù)-新聞主題分類本章的實踐任務(wù)是關(guān)于新聞主題的分類，以一段新聞報道中的文本描述內(nèi)容為輸入，使用模型幫助我們判斷它最有可能屬于哪一種類型的新聞，這是典型的文本分類問題，我們這里假定每種類型是互斥的，即文本描述有且只有一種類型。實踐任務(wù)第六節(jié)

實踐任務(wù)-新聞主題分類AG_NEWS數(shù)據(jù)集是一個包含了100多萬篇文章的集合。學(xué)術(shù)社區(qū)為了數(shù)據(jù)挖掘、信息檢索、數(shù)據(jù)壓縮等方面的學(xué)術(shù)研究整理了這個數(shù)據(jù)集。AG_NEWS數(shù)據(jù)集有4個標(biāo)簽，即模型中的數(shù)據(jù)分類數(shù)量為4，這4個標(biāo)簽為：World、Sports、Business和Sci/Tec。Torchtext庫提供了一些原始數(shù)據(jù)迭代器，這些迭代器可以查看原始的字符串?dāng)?shù)據(jù)。例如，AG_NEWS數(shù)據(jù)集迭代器可以生成包含文本和標(biāo)簽的元組。實踐1數(shù)據(jù)處理importtorchfromtorchtext.datasetsimportAG_NEWStrain_iter=iter(AG_NEWS(split='train'))print(next(train_iter))代碼實現(xiàn)實踐任務(wù)1-數(shù)據(jù)處理【結(jié)果】(3,"WallSt.BearsClawBackIntotheBlack(Reuters)Reuters-Short-sellers,WallStreet'sdwindling\\bandofultra-cynics,areseeinggreen

again.")實踐任務(wù)1-數(shù)據(jù)處理tokenizer=get_tokenizer('basic_english') train_iter=AG_NEWS(split='train') #分詞生成器defyield_tokens(data_iter):for_,textindata_iter:yieldtokenizer(text)vocab=build_vocab_from_iterator(yield_tokens(train_iter),specials=["<unk>"])vocab.set_default_index(vocab["<unk>"])print(vocab(['here','is','an','example']))

#結(jié)果：[475,21,30,5297]代碼實現(xiàn)實踐任務(wù)1-數(shù)據(jù)處理tokenizer=get_tokenizer('basic_english') train_iter=AG_NEWS(split='train') #分詞生成器defyield_tokens(data_iter):for_,textindata_iter:yieldtokenizer(text)vocab=build_vocab_from_iterator(yield_tokens(train_iter),specials=["<unk>"])vocab.set_default_index(vocab["<unk>"])print(vocab(['here','is','an','example']))

#結(jié)果：[475,21,30,5297]代碼實現(xiàn)實踐任務(wù)1-數(shù)據(jù)處理tokenizer=get_tokenizer('basic_english') train_iter=AG_NEWS(split='train') #分詞生成器defyield_tokens(data_iter):for_,textindata_iter:yieldtokenizer(text)vocab=build_vocab_from_iterator(yield_tokens(train_iter),specials=["<unk>"])vocab.set_default_index(vocab["<unk>"])print(vocab(['here','is','an','example']))

#結(jié)果：[475,21,30,5297]代碼實現(xiàn)實踐任務(wù)1-數(shù)據(jù)處理tokenizer=get_tokenizer('basic_english') train_iter=AG_NEWS(split='train') #分詞生成器defyield_tokens(data_iter):for_,textindata_iter:yieldtokenizer(text)vocab=build_vocab_from_iterator(yield_tokens(train_iter),specials=["<unk>"])vocab.set_default_index(vocab["<unk>"])print(vocab(['here','is','an','example']))

#結(jié)果：[475,21,30,5297]代碼實現(xiàn)實踐任務(wù)1-數(shù)據(jù)處理tokenizer=get_tokenizer('basic_english') train_iter=AG_NEWS(split='train') #分詞生成器defyield_tokens(data_iter):for_,textindata_iter:yieldtokenizer(text)vocab=build_vocab_from_iterator(yield_tokens(train_iter),specials=["<unk>"])vocab.set_default_index(vocab["<unk>"])print(vocab(['here','is','an','example']))

#結(jié)果：[475,21,30,5297]代碼實現(xiàn)實踐任務(wù)1-數(shù)據(jù)處理tokenizer=get_tokenizer('basic_english') train_iter=AG_NEWS(split='train') #分詞生成器defyield_tokens(data_iter):for_,textindata_iter:yieldtokenizer(text)vocab=build_vocab_from_iterator(yield_tokens(train_iter),specials=["<unk>"])vocab.set_default_index(vocab["<unk>"])print(vocab(['here','is','an','example']))

#結(jié)果：[475,21,30,5297]代碼實現(xiàn)實踐任務(wù)-新聞主題分類文本分類模型是由PyTorch庫中的nn.EmbeddingBag層和一個用于分類的線性連接層組成。nn.EmbeddingBag在默認(rèn)狀態(tài)下計算所有詞向量的平均值。雖然，文本文章由不同的長度，但是nn.EmbeddingBag模型不需要填充，因為文本的長度保存在偏移量中了。實踐2模型搭建實踐任務(wù)-新聞主題分類classTextClassificationModel(nn.Module):def__init__(self,vocab_size,embed_dim,num_class):super(TextClassificationModel,self).__init__()self.embedding=nn.EmbeddingBag(vocab_size,embed_dim,sparse=True)self.fc=nn.Linear(embed_dim,num_class)self.init_weights()definit_weights(self):initrange=0.5self.embedding.weight.data.uniform_(-initrange,initrange)self.fc.weight.data.uniform_(-initrange,initrange)self.fc.bias.data.zero_()代碼實現(xiàn)實踐任務(wù)-新聞主題分類classTextClassificationModel(nn.Module):def__init__(self,vocab_size,embed_dim,num_class):super(TextClassificationModel,self).__init__()self.embedding=nn.EmbeddingBag(vocab_size,embed_dim,sparse=True)self.fc=nn.Linear(embed_dim,num_class)self.init_weights()definit_weights(self):initrange=0.5self.embedding.weight.data.uniform_(-initrange,initrange)self.fc.weight.data.uniform_(-initrange,initrange)self.fc.bias.data.zero_()代碼實現(xiàn)實踐任務(wù)-新聞主題分類classTextClassificationModel(nn.Module):def__init__(self,vocab_size,embed_dim,num_class):super(TextClassificationModel,self).__init__()self.embedding=nn.EmbeddingBag(vocab_size,embed_dim,sparse=True)self.fc=nn.Linear(embed_dim,num_class)self.init_weights()definit_weights(self):initrange=0.5self.embedding.weight.data.uniform_(-initrange,initrange)self.fc.weight.data.uniform_(-initrange,initrange)self.fc.bias.data.zero_()代碼實現(xiàn)實踐任務(wù)-新聞主題分類defforward(self,text,offsets):embedded=self.embedding(text,offsets)returnself.fc(embedded)deftrain(dataloader):model.train()total_acc,total_count=0,0log_interval=500start_time=time.time()代碼實現(xiàn)實踐任務(wù)-新聞主題分類defforward(self,text,offsets):embedded=self.embedding(text,offsets)returnself.fc(embedded)deftrain(dataloader):model.train()total_acc,total_count=0,0log_interval=500start_time=time.time()代碼實現(xiàn)實踐任務(wù)-新聞主題分類foridx,(label,text,offsets)inenumerate(dataloader):optimizer.zero_grad()predicted_label=model(text,offsets)loss=criterion(predicted_label,label)loss.backward()torch.nn.utils.clip_grad_norm_(model.parameters(),0.1)optimizer.step()total_acc+=(predicted_label.argmax(1)==label).sum().item()total_count+=

label.size(0)ifidx%log_interval==0andidx>0:elapsed=time.time()-start_timeprint('|epoch{:3d}|{:5d}/{:5d}batches''|accuracy{:8.3f}'.format(epoch,idx,len(dataloader),total_acc/total_count))total_acc,total_count=0,0start_time=time.time()代碼實現(xiàn)實踐任務(wù)-新聞主題分類foridx,(label,text,offsets)inenumerate(dataloader):optimizer.zero_grad()predicted_label=model(text,offsets)loss=criterion(predicted_label,label)loss.backward()torch.nn.utils.clip_grad_norm_(model.parameters(),0.1)optimizer.step()total_acc+=(predicted_label.argmax(1)==label).sum().item()total_count+=

label.size(0)ifidx%log_interval==0andidx>0:elapsed=time.time()-start_timeprint('|epoch{:3d}|{:5d}/{:5d}batches''|accuracy{:8.3f}'.format(epoch,idx,len(dataloader),total_acc/total_count))total_acc,total_count=0,0start_time=time.time()代碼實現(xiàn)實踐任務(wù)-新聞主題分類foridx,(label,text,offsets)inenumerate(dataloader):optimizer.zero_grad()predicted_label=model(text,offsets)loss=criterion(predicted_label,label)loss.backward()torch.nn.utils.clip_grad_norm_(model.parameters(),0.1)optimizer.step()total_acc+=(predicted_label.argmax(1)==label).sum().item()total_count+=

label.size(0)ifidx%log_interval==0andidx>0:elapsed=time.time()-start_timeprint('|epoch{:3d}|{:5d}/{:5d}batches''|accuracy{:8.3f}'.format(epoch,idx,len(dataloader),total_acc/total_count))total_acc,total_count=0,0start_time=time.time()代碼實現(xiàn)實踐任務(wù)-新聞主題分類foridx,(label,text,offsets)inenumerate(dataloader):optimizer.zero_grad()predicted_label=model(text,offsets)loss=criterion(predicted_label,label)loss.backward()torch.nn.utils.clip_grad_norm_(model.parameters(),0.1)optimizer.step()total_acc+=(predicted_label.argmax(1)==label).sum().item()total_count+=

label.size(0)ifidx%log_interval==0andidx>0:elapsed=time.time()-start_timeprint('|epoch{:3d}|{:5d}/{:5d}batches''|accuracy{:8.3f}'.format(epoch,idx,len(dataloader),total_acc/total_count))total_acc,total_count=0,0start_time=time.time()代碼實現(xiàn)實踐任務(wù)-新聞主題分類foridx,(label,text,offsets)inenumerate(dataloader):optimizer.zero_grad()predicted_label=model(text,offsets)loss=criterion(predicted_label,label)loss.backward()torch.nn.utils.clip_grad_norm_(model.parameters(),0.1)optimizer.step()total_acc+=(predicted_label.argmax(1)==label).sum().item()total_count+=

label.size(0)ifidx%log_interval==0andidx>0:elapsed=time.time()-start_timeprint('|epoch{:3d}|{:5d}/{:5d}batches''|accuracy{:8.3f}'.format(epoch,idx,len(dataloader),total_acc/total_count))total_acc,total_count=0,0start_time=time.time()代碼實現(xiàn)實踐任務(wù)-新聞主題分類foridx,(label,text,offsets)inenumerate(dataloader):optimizer.zero_grad()predicted_label=model(text,offsets)loss=criterion(predicted_label,label)loss.backward()torch.nn.utils.clip_grad_norm_(model.parameters(),0.1)optimizer.step()total_acc+=(predicted_label.argmax(1)==label).sum().item()total_count+=

label.size(0)ifidx%log_interval==0andidx>0:elapsed=time.time()-start_timeprint('|epoch{:3d}|{:5d}/{:5d}batches''|accuracy{:8.3f}'.format(epoch,idx,len(dataloader),total_acc/total_count))total_acc,total_count=0,0start_time=time.time()代碼實現(xiàn)實踐任務(wù)-新聞主題分類foridx,(label,text,offsets)inenumerate(dataloader):optimizer.zero_grad()predicted_label=model(text,offsets)loss=criterion(predicted_label,label)loss.backward()torch.nn.utils.clip_grad_norm_(model.parameters(),0.1)optimizer.step()total_acc+=(predicted_label.argmax(1)==label).sum().item()total_count+=

label.size(0)ifidx%log_interval==0andidx>0:elapsed=time.time()-start_timeprint('|epoch{:3d}|{:5d}/{:5d}batches''|accuracy{:8.3f}'.format(epoch,idx,len(dataloader),total_acc/total_count))total_acc,total_count=0,0start_time=time.time()代碼實現(xiàn)實踐任務(wù)-新聞主題分類defevaluate(dataloader):model.eval()total_acc,total_count=0,0withtorch.no_grad():foridx,(label,text,offsets)inenumerate(dataloader):predicted_label=model(text,offsets)loss=criterion(predicted_label,label)total_acc+=(predicted_label.argmax(1)==label).sum().item()total_count+=label.size(0)returntotal_acc/total_count實踐任務(wù)-新聞主題分類defevaluate(dataloader):model.eval()total_acc,total_count=0,0withtorch.no_grad():foridx,(label,text,offsets)inenumerate(dataloader):predicted_label=model(text,offsets)loss=criterion(predicted_label,label)total_acc+=(predicted_label.argmax(1)==label).sum().item()total_count+=label.size(0)returntotal_acc/total_count實踐任務(wù)-新聞主題分類defevaluate(dataloader):model.eval()total_acc,total_count=0,0withtorch.no_grad():foridx,(label,text,offsets)inenumerate(dataloader):predicted_label=model(text,offsets)loss=criterion(predicted_label,label)total_acc+=(predicted_label.argmax(1)==label).sum().item()total_count+=label.size(0)returntotal_acc/total_count實踐任務(wù)-新聞主題分類因為原始的AG_NEWS沒有測試數(shù)據(jù)集，我們將數(shù)據(jù)集拆分為訓(xùn)練集和測試集，其中訓(xùn)練集占比為95%，測試集占5%。這里使用PyTorch核心庫中torch.utils.data.dataset.random_split來拆分?jǐn)?shù)據(jù)。CrossEntropyLoss在一個類中結(jié)合了nn.LogSoftmax和nn.NLLLoss，當(dāng)想訓(xùn)練一個有多個類別的分類模型的時候非常實用。SGD作為優(yōu)化器實現(xiàn)了隨機(jī)梯度下降。初始的學(xué)習(xí)率為5，使用StepLR在學(xué)習(xí)的過程中調(diào)整學(xué)習(xí)率。實踐3模型訓(xùn)練實踐任務(wù)-新聞主題分類#超參數(shù)EPOCHS=10LR=5BATCH_SIZE=64criterion=torch.nn.CrossEntropyLoss()optimizer=torch.optim.SGD(model.parameters(),lr=LR)scheduler=torch.optim.lr_scheduler.StepLR(optimizer,1.0,gamma=0.1)total_accu=Nonetrain_iter,test_iter=AG_NEWS()train_dataset=to_map_style_dataset(train_iter)test_dataset=to_map_style_dataset(test_iter)num_train=int(len(train_dataset)*0.95)split_train_,split_valid_=\random_split(train_dataset,[num_train,len(train_dataset)-num_train])代碼實現(xiàn)實踐任務(wù)-新聞主題分類#超參數(shù)EPOCHS=10LR=5BATCH_SIZE=64criterion=torch.nn.CrossEntropyLoss()optimizer=torch.optim.SGD(model.parameters(),lr=LR)scheduler=torch.optim.lr_scheduler.StepLR(optimizer,1.0,gamma=0.1)total_accu=Nonetrain_iter,test_iter=AG_NEWS()train_dataset=to_map_style_dataset(train_iter)test_dataset=to_map_style_dataset(test_iter)num_train=int(len(train_dataset)*0.95)split_train_,split_valid_=\random_split(train_dataset,[num_train,len(train_dataset)-num_train])代碼實現(xiàn)實踐任務(wù)-新聞主題分類#超參數(shù)EPOCHS=10LR=5BATCH_SIZE=64criterion=torch.nn.CrossEntropyLoss()optimizer=torch.optim.SGD(model.parameters(),lr=LR)scheduler=torch.optim.lr_scheduler.StepLR(optimizer,1.0,gamma=0.1)total_accu=Nonetrain_iter,test_iter=AG_NEWS()train_dataset=to_map_style_dataset(train_iter)test_dataset=to_map_style_dataset(test_iter)num_train=int(len(train_dataset)*0.95)split_train_,split_valid_=\random_split(train_dataset,[num_train,len(train_dataset)-num_train])代碼實現(xiàn)實踐任務(wù)-新聞主題分類#超參數(shù)EPOCHS=10LR=5BATCH_SIZE=64criterion=torch.nn.CrossEntropyLoss()optimizer=torch.optim.SGD(model.parameters(),lr=LR)scheduler=torch.optim.lr_scheduler.StepLR(optimizer,1.0,gamma=0.1)total_accu=Nonetrain_iter,test_iter=AG_NEWS()train_dataset=to_map_style_dataset(train_iter)test_dataset=to_map_style_dataset(test_iter)num_train=int(len(train_dataset)*0.95)split_train_,split_valid_=\random_split(train_dataset,[num_train,len(train_dataset)-num_train])代碼實現(xiàn)實踐任務(wù)-新聞主題分類train_dataloader=DataLoader(split_train_,batch_size=BATCH_SIZE,shuffle=True,collate_fn=collate_batch)valid_dataloader=DataLoader(split_valid_,batch_size=BATCH_SIZE,shuffle=True,collate_fn=collate_batch)test_dataloader=DataLoader(test_dataset,batch_size=BATCH_SIZE,shuffle=True,collate_fn=collate_batch)forepochinrange(1,EPOCHS+1):epoch_start_time=time.time()train(train_dataloader)accu_val=evaluate(valid_dataloader)iftotal_accuisnotNoneandtotal_accu>accu_val:scheduler.step()else:total_accu=accu_val代碼實現(xiàn)實踐任務(wù)-新聞主題分類train_dataloader=DataLoader(split_train_,batch_size=BATCH_SIZE,shuffle=True,collate_fn=collate_batch)valid_dataloader=DataLoader(split_valid_,batch_size=BATCH_SIZE,shuffle=True,collate_fn=collate_batch)test_dataloader=DataLoader(test_dataset,batch_size=BATCH_SIZE,shuffle=True,collate_fn=collate_batch)forepochinrange(1,EPOCHS+1):epoch_start_time=time.time()train(train_dataloader)accu_val=evaluate(valid_dataloader)iftotal_accuisnotNoneandtotal_accu>accu_val:scheduler.step()else:total_accu=accu_val代碼實現(xiàn)實踐任務(wù)-新聞主題分類在前面已經(jīng)定義了評估模型準(zhǔn)確率的函數(shù)evaluate，下面使用模型來測試一則新聞，代碼如下：ag_news_label={1:"World",2:"Sports",3:"