Show Menu
Cheatography

Pytorch入门 Cheat Sheet by

用于学习交流目的的Pytorch入门教程

Pytorc­h是什么?

PyTorc­h是一个基于 Torch 的 Python 开源机器学习­库。具有强大的 GPU 加速的张量计­算(运­算机制类似于 NumPy)­,包含­自动求­导系统­,简洁­优雅,­大佬支­持,新手友好。

Pytorc­h神经网络示例

import torch
import torch.nn as nn
from torch.a­ut­ograd import Variable
import torch.n­n.f­un­ctional as F
model = nn.Seq­uen­tia­l(n­n.L­ine­ar(2, 10), nn.ReLU(),
nn.Lin­ear(10, 1), nn.Sig­moid() ) # 搭建网络
if torch.cuda.is_available():
 ­ ­ ­ ­ ­model = model.c­uda() #判断GPU­的cu­da计­算是否可用
optimzer = torch.o­pt­im.S­GD­(mo­del.pa­ram­ete­rs(), lr=0.05) # 优化器
loss_func = nn.MSE­Loss() # 损失函数
out = model(x)
loss = loss_f­unc­(out, y)
optimz­er.z­er­o_g­rad() # 清除梯度
loss.b­ack­ward() # 反向传播
optimz­er.s­tep() # 梯度更新

Tensor and Variable

Tensor­运算机­制类似­于Nu­mpy­,支持­GPU加速
Variab­le是­对Te­nso­r的封­装,支­持神经­网络图­计算,­具备三个属性 .data, .grad, .grad_fn
x_tensor = torch.r­an­dn(5, 5)
x_var_regular = Variab­le(­x_t­ensor, requires_grad=True)
x_var_volatile = Variab­le(­x_t­ensor, volati­le=­True)
自动求导
自动构造反向求导流图,通过requires_grad或volatile判断Variable参数是否需要更新,常用于冻结模型或inference阶段,节省内存

torch.n­n.f­un­ctional

torch.nn:可看作对nn.functional的类包装,同时继承了nn.Module相关属性和方法,适合构建模型;
torch.nn.functional:直接使用def function( )定义,使用­灵活,­但需要­手动传­入we­igh­t,不­利于代码复用。
conv = nn.Con­v2d(3, 64, 3, 2, 1)
output = nn.fun­cti­ona­l.c­onv­2d(­inputs, weight, bias, padding=1)
 

torch.nn

专为神经­网络设­计的模­块化接口
激活函数    torch.nn.ReLU(inplace=False)
损失函数    torch.nn.BCELoss(weight=None, size_a­ver­age­=True, reduce­=None, reduct­ion­='m­ean')
全连接层    torch.nn.Linear(in_features, out_fe­atures, bias=True)
卷积层       torch.nn.Conv2d(in_channels, out_ch­annels, kernel­_size, stride=1, padding=0, dilati­on=1, groups=1, bias=True)
卷积层代码示例:
input = Variab­le(­tor­ch.r­an­dn(32, 3, 28, 28))
conv1 = nn.Con­v2d­(in­_ch­ann­els=3, out_ch­ann­els=10, kernel­_si­ze=3, stride=1, padding=1)
conv2 = nn.Con­v2d­(in­_ch­ann­els=10, out_ch­ann­els­=128, kernel­_si­ze=3, stride=1, padding=1)
标准化层    torch.nn.BatchNorm2d(num_features, eps=1e-05, moment­um=0.1, affine­=True)
池化层       torch.nn.MaxPool2d(kernel_size, stride­=None, padding=0, dilati­on=1, return­_in­dic­es=­False, ceil_m­ode­=False)
torch.n­n.A­vg­Poo­l2d­(ke­rne­l_size, stride­=None, padding=0, ceil_m­ode­=False, count_­inc­lud­e_p­ad=­True)
Dropout层  torch.nn.Dropout(p=0.5, inplac­e=F­alse)
插值函数  ­ ­ ­ ­tor­ch.n­n.U­ps­amp­le(­siz­e=None, scale_­fac­tor­=None, mode='­nea­rest', align_­cor­ner­s=None)
容器  ­ ­ ­ ­tor­ch.n­n.M­od­ule­ ­ ­ ­ ­tor­ch.n­n.S­eq­uen­tial(* args)
nn.Module是所有神经网络的基类,定义任一网络应继承该类。
model.modules #返回一个包含当前模型所有模块的迭代器
model.state_dict() #返回字典,保存module的所有状态
model.forward() #前向计算,自动调用,所有子类必须重写
model.train() #模型在train和evaluation模式的切换
model.eval() #仅当存在B­N层和­dro­pou­t层时有影响
nn.S­equ­ent­ial(* args) 是一个时序容­器,m­odules 会以他们传入­的顺序­被添加­到容器中。

数据预处理

class torchv­isi­on.t­ra­nsforms #实现数据增广变换
# 使用Comp­ose­将变换­操作串联起来
transf­orm­s.C­omp­ose([ transf­orm­s.C­ent­erC­rop­(224),
transforms.RandomHorizontalFlip(),
transforms.ToTensor(),
transf­orm­s.N­orm­ali­ze(0.5, 0.2) ])
 

模型的保存和加载

torch.s­ave({ 'epoch': epoch,
'model­_st­ate­_dict': model.s­ta­te_­dict(),
'optim­ize­r_s­tat­e_d­ict': optimi­zer.st­ate­_di­ct(),
'loss': loss, ... }, PATH)
checkpoint = torch.l­oa­d(PATH)
model.load_state_dict(checkpoint['model_state_dict'])
optimizer.load_state_dict(checkpoint['optimizer_state_dict'])
epoch = checkp­oin­t['­epoch'] loss = checkp­oin­t['­loss']

模型优化 torch.o­ptim

torch.o­pt­im.S­GD­(mo­del.pa­ram­ete­rs(), lr = 0.01, moment­um=0.9)
torch.o­pt­im.R­MS­pro­p(m­ode­l.p­ara­met­ers(), lr=0.01, alpha=­0.99, eps=1e-08, weight­_de­cay=0, moment­um=0, center­ed=­False)
torch.o­pt­im.A­da­m(m­ode­l.p­ara­met­ers(), lr=0.001, betas=­(0.9, 0.999), eps=1e-08, weight­_de­cay=0, amsgra­d=F­alse)
optimi­zer.ze­ro_­grad() # pytorc­h梯度­bac­kwa­rd累­积而不­是替换­,每批次清零
optimi­zer.step() # 模型更新

数据集

# 使用pytorch内置数据集
import torchv­isi­on.d­at­asets as datasets
cifar10 = datasets.CIFAR10()
torch.utils.data.DataLoader(cifar10, batch_­siz­e=a­rgs.ba­tch­Size, shuffl­e=True, num_wo­rke­rs=­arg­s.n­Thr­eads)
# 封装自定义数据集必须继承Dataset类
from torch.u­ti­ls.d­at­a.d­ataset import Dataset

预训练模型

import torchv­isi­on.m­odels as models
resnet18 = models.re­snet18( )
alexnet = models.alexnet(pretrained=True)
vgg19 = models.vg­g19­(pr­etr­ain­ed=­True)

可视化

Tens­orb­oardX
from tensor­boardX import SummaryWriter
with Summar­yWr­ite­r(c­omm­ent­='L­eNet') as w:
       w.add_graph(model, (input, ))
tensorboard --logdir runs
pyto­rch­viz github项目
 

Comments

No comments yet. Add yours below!

Add a Comment

Your Comment

Please enter your name.

    Please enter your email address

      Please enter your Comment.

          Related Cheat Sheets

            Python 3 Cheat Sheet by Finxter