gll89的个人博客分享 http://blog.sciencenet.cn/u/gll89

博文

【Pytorch】GN在3DResNet中的实现

已有 2129 次阅读 2020-7-20 20:19 |个人分类:DeepLearning|系统分类:科研笔记

  1.  official website https://pytorch.org/docs/master/generated/torch.nn.GroupNorm.html#groupnorm

       class torch.nn.GroupNorm(num_groups: intnum_channels: inteps: float = 1e-05affine: bool = True)

    params:  

    • num_groups (int) – number of groups to separate the channels into

    • num_channels (int) – number of channels expected in input

    • eps – a value added to the denominator for numerical stability. Default: 1e-5

    • affine – a boolean value that when set to True, this module has learnable per-channel affine parameters initialized to ones (for weights) and zeros (for biases). Default: True.

      =================

      Input_shape: (N, C, *)  where C=num_groups

      Output_shape: (N, C, *) same shape as the input_shape

      由以上可知,输入params只需输入num_groups与num_channels。

    • 由原文知,num_groups对值不敏感

    • num_channels 可由原来model获取  (具体如下)

    (N, 

2. 将3D ResNet中的BN转为GN:1)找到BN layer;2)代替BN为GN

 

   def convert_BN2GN(model):  
   for name, module in model._modules.items():  #遍历layer
      if len(list(module.children())) > 0:  #对每个block内部的layers,调用原函数进行遍历
        self.convert_BN2GN(module)
      elif isinstance(module, torch.nn.modules.batchnorm.BatchNorm3d): #对每一个BN layer进行操作
        module_tmp = nn.GroupNorm(32, module.num_features)  
        #其中,32为num_groups; model.num_features为num_channels
        model._modules[name] = module_tmp
    return model

Ref: https://www.kaggle.com/c/aptos2019-blindness-detection/discussion/104686

https://github.com/ppwwyyxx/GroupNorm-reproduce/blob/master/ImageNet-ResNet-PyTorch/resnet_gn.py







http://blog.sciencenet.cn/blog-1969089-1242863.html

上一篇:vocabularies for writing
下一篇:Latex知识点

0

该博文允许注册用户评论 请点击登录 评论 (0 个评论)

数据加载中...
扫一扫,分享此博文

Archiver|手机版|科学网 ( 京ICP备07017567号-12 )

GMT+8, 2021-10-25 14:27

Powered by ScienceNet.cn

Copyright © 2007- 中国科学报社

返回顶部