https://utsavdarlami.github.io/blogs/tags/entropy/Recent content in Entropy on felladogHugo -- gohugo.ioenSat, 26 Feb 2022 15:34:05 +0545https://utsavdarlami.github.io/blogs/notes/2021-06-03--11-01-27z--decision_tree/Thu, 03 Jun 2021 16:46:00 +0545https://utsavdarlami.github.io/blogs/notes/2021-06-03--11-01-27z--decision_tree/References : Reading : Tom Mitchell Lectures slides of chapter 3 Tom Mitchell, Machine Learning Chapter 3 ID3 algorithm complete solution Reddit Post Link Drive Pdf Link To Read : Chapter 3 — Decision Tree Learning — Part 1 Questions : Approximating discrete-valued fucntions robust to noisy data capable of learning disjunctive expressions search a completely expressive hypothesis space and thus avoid the difficulties of restricted hypothesis spaces. Representation Decision trees classify instances by sorting them down the tree from the root to some leaf node each leaf node assigns a classification each interal node specifies a test of some attributes of the instances each branch descending from a node corresponds to one of the possible values of the attribute represented by that node.https://utsavdarlami.github.io/blogs/notes/2021-03-06--03-56-06z--entropy/Sat, 06 Mar 2021 09:41:00 +0545https://utsavdarlami.github.io/blogs/notes/2021-03-06--03-56-06z--entropy/References : https://www.wikiwand.com/en/Entropy%5F(information%5Ftheory) Entropy in image, scikit-image [3] https://bricaud.github.io/personal-blog/entropy-in-decision-trees/ Questions : What is the point of entropy in decision tree? Entropy is a measure of disorder. A high entropy is essentially saying that the data is scattered around while a low entropy means that nearly all the data is the same. In information theory, information entropy is the log-base-2 of the number of possible outcomes for a message. \(- \log_{2} p\) Based on our dataset we can say