savoy082

Pt En In this post I will introduce Markov Decision Processes, a common tool used in Reinforcement Learning, a branch of Machine Learning. By the end of the post you will be able to make some sense of the figure above! I will couple the formal details, definitions and maths with an intuitive example that will accompany us throughout this post. In later posts we will make our example more complete and use other examples to explain other properties and characteristics of the MDPs. Let me introduce the context of the example: From a simplistic point of view, I only have two moods: " hungry " and " thirsty ". Thankfully, my parents taught me how to eat and how to drink, so that I can fulfill the needs I mentioned earlier. Of course that eating when I am hungry makes me happy, just as drinking when I am thirsty makes me happy! Not only that, but eating when I am hungry usually satisfies me, much like drinking when I am thirsty usually satisfies me. Suppose that, g...

Pt En It has become quite common for online services to provide some form of 2-factor authentication when logging in from unknown devices. For example, whenever I try to access my Gmail account from a computer I never used, I get a text message with a one-time use 6-digit code. One day I was using that same service to log in into my email, when I noticed that one of the digits in the security code appeared twice, like the $1$ in $315641$. But when I read the other text messages from Google, I noticed that there were plenty more security codes with repeated digits than security codes that had six different digits. I found that weird and then decided to compute the probabilities of these events, just to check whether my intuition was tricking me or not... We are about to compute some probabilities regarding these $6$-digit codes - which I will start calling PINs for the sake of brevity - with the rather intuitive formula $$P(\text{some property}\ A) = \frac{\text{# PINs that satis...

Pt En This post has the purpose of presenting a result that may seem counterintuitive and that can provide a really nice excuse for a wager between you and one or more of your friends. For this post, when I talk about a birthdate I am only referring to the day and month of birth, and not the year. What is the probability that you and your best friend have the same birth day and month? Even without an exact number one knows that you are much more likely to have different birthdates than having equal birthdates. Assuming all $366$ days are equally likely, the probability that two people have the same birthdate is $\frac{1}{366} \approx 0.27\%$ and the probability that the birthdate is different is $\frac{365}{366} \approx 99.73\%$. How many people do you need so that the probability of existing at least two sharing the birthdate is higher than the probability of everyone having different birthdates? What would your guess be? It only takes $23$ people. If you have a group of $23$...