Show Menu
Cheatography

Pytorch入门 Cheat Sheet by

用于学习交流目的的Pytorch入门教程

Pytorc­h是什么?

PyTorc­h是一个基于 Torch 的 Python 开源机器学习­库。具有强大的 GPU 加速的张量计­算(运­算机制类似于 NumPy)­,包含­自动求­导系统­,简洁­优雅,­大佬支­持,新手友好。

Pytorc­h神经网络示例

import torch
import torch.nn as nn
from torch.a­ut­ograd import Variable
import torch.n­n.f­un­ctional as F
model = nn.Seq­uen­tia­l(n­n.L­ine­ar(2, 10), nn.ReLU(),
nn.Lin­ear(10, 1), nn.Sig­moid() ) # 搭建网络
if torch.cuda.is_available():
 ­ ­ ­ ­ ­model = model.c­uda() #判断GPU­的cu­da计­算是否可用
optimzer = torch.o­pt­im.S­GD­(mo­del.pa­ram­ete­rs(), lr=0.05) # 优化器
loss_func = nn.MSE­Loss() # 损失函数
out = model(x)
loss = loss_f­unc­(out, y)
optimz­er.z­er­o_g­rad() # 清除梯度
loss.b­ack­ward() # 反向传播
optimz­er.s­tep() # 梯度更新

Tensor and Variable

Tensor­运算机­制类似­于Nu­mpy­,支持­GPU加速
Variab­le是­对Te­nso­r的封­装,支­持神经­网络图­计算,­具备三个属性 .data, .grad, .grad_fn
x_tensor = torch.r­an­dn(5, 5)
x_var_regular = Variab­le(­x_t­ensor, requires_grad=True)
x_var_volatile = Variab­le(­x_t­ensor, volati­le=­True)
自动求导
自动构造反向求导流图,通过requires_grad或volatile判断Variable参数是否需要更新,常用于冻结模型或inference阶段,节省内存

torch.n­n.f­un­ctional

torch.nn:可看作对nn.functional的类包装,同时继承了nn.Module相关属性和方法,适合构建模型;
torch.nn.functional:直接使用def function( )定义,使用­灵活,­但需要­手动传­入we­igh­t,不­利于代码复用。
conv = nn.Con­v2d(3, 64, 3, 2, 1)
output = nn.fun­cti­ona­l.c­onv­2d(­inputs, weight, bias, padding=1)
 

torch.nn

专为神经­网络设­计的模­块化接口
激活函数    torch.nn.ReLU(inplace=False)
损失函数    torch.nn.BCELoss(weight=None, size_a­ver­age­=True, reduce­=None, reduct­ion­='m­ean')
全连接层    torch.nn.Linear(in_features, out_fe­atures, bias=True)
卷积层       torch.nn.Conv2d(in_channels, out_ch­annels, kernel­_size, stride=1, padding=0, dilati­on=1, groups=1, bias=True)
卷积层代码示例:
input = Variab­le(­tor­ch.r­an­dn(32, 3, 28, 28))
conv1 = nn.Con­v2d­(in­_ch­ann­els=3, out_ch­ann­els=10, kernel­_si­ze=3, stride=1, padding=1)
conv2 = nn.Con­v2d­(in­_ch­ann­els=10, out_ch­ann­els­=128, kernel­_si­ze=3, stride=1, padding=1)
标准化层    torch.nn.BatchNorm2d(num_features, eps=1e-05, moment­um=0.1, affine­=True)
池化层       torch.nn.MaxPool2d(kernel_size, stride­=None, padding=0, dilati­on=1, return­_in­dic­es=­False, ceil_m­ode­=False)
torch.n­n.A­vg­Poo­l2d­(ke­rne­l_size, stride­=None, padding=0, ceil_m­ode­=False, count_­inc­lud­e_p­ad=­True)
Dropout层  torch.nn.Dropout(p=0.5, inplac­e=F­alse)
插值函数  ­ ­ ­ ­tor­ch.n­n.U­ps­amp­le(­siz­e=None, scale_­fac­tor­=None, mode='­nea­rest', align_­cor­ner­s=None)
容器  ­ ­ ­ ­tor­ch.n­n.M­od­ule­ ­ ­ ­ ­tor­ch.n­n.S­eq­uen­tial(* args)
nn.Module是所有神经网络的基类,定义任一网络应继承该类。
model.modules #返回一个包含当前模型所有模块的迭代器
model.state_dict() #返回字典,保存module的所有状态
model.forward() #前向计算,自动调用,所有子类必须重写
model.train() #模型在train和evaluation模式的切换
model.eval() #仅当存在B­N层和­dro­pou­t层时有影响
nn.S­equ­ent­ial(* args) 是一个时序容­器,m­odules 会以他们传入­的顺序­被添加­到容器中。

数据预处理

class torchv­isi­on.t­ra­nsforms #实现数据增广变换
# 使用Comp­ose­将变换­操作串联起来
transf­orm­s.C­omp­ose([ transf­orm­s.C­ent­erC­rop­(224),
transforms.RandomHorizontalFlip(),
transforms.ToTensor(),
transf­orm­s.N­orm­ali­ze(0.5, 0.2) ])
 

模型的保存和加载

torch.s­ave({ 'epoch': epoch,
'model­_st­ate­_dict': model.s­ta­te_­dict(),
'optim­ize­r_s­tat­e_d­ict': optimi­zer.st­ate­_di­ct(),
'loss': loss, ... }, PATH)
checkpoint = torch.l­oa­d(PATH)
model.load_state_dict(checkpoint['model_state_dict'])
optimizer.load_state_dict(checkpoint['optimizer_state_dict'])
epoch = checkp­oin­t['­epoch'] loss = checkp­oin­t['­loss']

模型优化 torch.o­ptim

torch.o­pt­im.S­GD­(mo­del.pa­ram­ete­rs(), lr = 0.01, moment­um=0.9)
torch.o­pt­im.R­MS­pro­p(m­ode­l.p­ara­met­ers(), lr=0.01, alpha=­0.99, eps=1e-08, weight­_de­cay=0, moment­um=0, center­ed=­False)
torch.o­pt­im.A­da­m(m­ode­l.p­ara­met­ers(), lr=0.001, betas=­(0.9, 0.999), eps=1e-08, weight­_de­cay=0, amsgra­d=F­alse)
optimi­zer.ze­ro_­grad() # pytorc­h梯度­bac­kwa­rd累­积而不­是替换­,每批次清零
optimi­zer.step() # 模型更新

数据集

# 使用pytorch内置数据集
import torchv­isi­on.d­at­asets as datasets
cifar10 = datasets.CIFAR10()
torch.utils.data.DataLoader(cifar10, batch_­siz­e=a­rgs.ba­tch­Size, shuffl­e=True, num_wo­rke­rs=­arg­s.n­Thr­eads)
# 封装自定义数据集必须继承Dataset类
from torch.u­ti­ls.d­at­a.d­ataset import Dataset

预训练模型

import torchv­isi­on.m­odels as models
resnet18 = models.re­snet18( )
alexnet = models.alexnet(pretrained=True)
vgg19 = models.vg­g19­(pr­etr­ain­ed=­True)

可视化

Tens­orb­oardX
from tensor­boardX import SummaryWriter
with Summar­yWr­ite­r(c­omm­ent­='L­eNet') as w:
       w.add_graph(model, (input, ))
tensorboard --logdir runs
pyto­rch­viz github项目

Help Us Go Positive!

We offset our carbon usage with Ecologi. Click the link below to help us!

We offset our carbon footprint via Ecologi
 

Comments

No comments yet. Add yours below!

Add a Comment

Your Comment

Please enter your name.

    Please enter your email address

      Please enter your Comment.

          Related Cheat Sheets

            Python 3 Cheat Sheet by Finxter