# 【Pytorch】GN在3DResNet中的实现

1.  official website https://pytorch.org/docs/master/generated/torch.nn.GroupNorm.html#groupnorm

class torch.nn.GroupNorm(num_groups: intnum_channels: inteps: float = 1e-05affine: bool = True)

params:

• num_groups (int) – number of groups to separate the channels into

• num_channels (int) – number of channels expected in input

• eps – a value added to the denominator for numerical stability. Default: 1e-5

• affine – a boolean value that when set to True, this module has learnable per-channel affine parameters initialized to ones (for weights) and zeros (for biases). Default: True.

=================

Input_shape: (N, C, *)  where C=num_groups

Output_shape: (N, C, *) same shape as the input_shape

由以上可知，输入params只需输入num_groups与num_channels。

• 由原文知，num_groups对值不敏感

• num_channels 可由原来model获取  （具体如下）

(N,

2. 将3D ResNet中的BN转为GN：1）找到BN layer；2）代替BN为GN

   def convert_BN2GN(model):
for name, module in model._modules.items():  #遍历layer
if len(list(module.children())) > 0:  #对每个block内部的layers，调用原函数进行遍历
self.convert_BN2GN(module)
elif isinstance(module, torch.nn.modules.batchnorm.BatchNorm3d): #对每一个BN layer进行操作
module_tmp = nn.GroupNorm(32, module.num_features)
#其中，32为num_groups; model.num_features为num_channels
model._modules[name] = module_tmp
return model

https://github.com/ppwwyyxx/GroupNorm-reproduce/blob/master/ImageNet-ResNet-PyTorch/resnet_gn.py

https://blog.sciencenet.cn/blog-1969089-1242863.html

## 全部精选博文导读

GMT+8, 2021-12-6 10:44