jingweimo的个人博客分享 http://blog.sciencenet.cn/u/jingweimo

博文

Nonlinearity and loss function in multi-tasking problems

已有 753 次阅读 2018-6-13 21:35 |系统分类:科研笔记

 1. Multi-class vs Multi-label

What is the difference between a multiclass problem and a multilabel problem?

(https://stats.stackexchange.com/questions/11859/what-is-the-difference-between-multiclass-and-multilabel-problem)

I suspect the difference is that in multi-class problems the classes are mutually exclusive, whereas for multi-label problems each label represents a different classification task, but the tasks are somehow related (so there is a benefit in tackling them together rather than separately).  For example, in the famous leptograspus crabs dataset there are examples of males and females of two colour forms of crab.  You could approach this as a multi-class problem with four classes (male-blue, female-blue, male-orange, female-orange) or as a multi-label problem, where one label would be male/female and the other blue/orange.  Essentially in multi-label problems a pattern can belong to more than one class.

  • in the multilabel case, one sample might be assigned more than one class.

  • in the multiclass case, there are more than 2 classes in total.

As a side note, nothing prevents you from having a multilabel-multiclass classification problem.

See also:

https://stats.stackexchange.com/questions/319874/multi-label-or-multi-class-or-both


 2. Binary_crossentropy vs Categorical_crossentropy

Should I use a categorical cross entropy or binary cross entropy loss for binary predictions?

1.png

Binary cross-entropy is used for binary or multi-class, multi-label classifications, with sigmoid activation (in the last layer); whereas categorical cross entropy is for multi-class classification (where each example belongs to a single class) with softmax activation (in the last layer).

See also:

https://stackoverflow.com/questions/42081257/keras-binary-crossentropy-vs-categorical-crossentropy-performance

https://stackoverflow.com/questions/47877083/keras-binary-crossentropy-categorical-crossentropy-confusion


3. How to choose cross-entropy loss ?

https://stackoverflow.com/questions/47034888/how-to-choose-cross-entropy-loss-in-tensorflow

In functional sense, the sigmoid is a partial case of the softmax function, when the number of classes equals 2. Both of them do the same operation: transform the logits to probabilities.

In simple binary classification, there's no big difference between the two, however in case of multinomial classification, sigmoid allows to deal with non-exclusive labels (a.k.a. multi-labels), while softmax deals with exclusive classes.

See also:

http://ml-cheatsheet.readthedocs.io/en/latest/loss_functions.html




http://blog.sciencenet.cn/blog-578676-1118819.html

上一篇:Updating Theano 0.9.0 to 1.02
下一篇:[转载]The Optimum Hyperspectral Design for Your Applications

0

该博文允许注册用户评论 请点击登录 评论 (0 个评论)

数据加载中...
扫一扫,分享此博文

Archiver|手机版|科学网 ( 京ICP备14006957 )

GMT+8, 2018-8-20 07:37

Powered by ScienceNet.cn

Copyright © 2007- 中国科学报社

返回顶部