Correlation vs Mutual Information

This post is based on a (very small) part of the (dense and technical) paper Fooled by Correlation by N.N. Taleb, found at (1)

Notes on the main ideas in this post are available from Universidad de Cantabria, found at (2)

The aims of this post are to 1) introduce mutual information as a measure of similarity and 2) to show the nonlinear relationship between correlation and information my means of a relatively simple example

Introduction

A significant part of Statistical analysis is understanding how random variables are related – how much knowledge about the value of one variable tells us about the value of another. This post will consider this issue in the context of Gaussian random variables. More specifically, we will compare- and discuss the relationship between- correlation and mutual information.

Mutual Information

The Mutual Information between 2 random variables is the amount of information that one gains about a random variable by observing the value of the other.…

By | March 28th, 2020|English, Level: intermediate, Uncategorized|0 Comments

The (Central) Cauchy distribution

The core of this post comes from Mathematical Statistics and Data Analysis by John A. Rice which is a useful resource for subjects such as UCT’s STA2004F.

Introduction

The Cauchy distribution has a number of interesting properties and is considered a pathological (badly behaved) distribution. What is interesting about it is that it is a distribution that we can think about in a number of different ways*, and we can formulate the probability density function these ways. This post will handle the derivation of the Cauchy distribution as a ratio of independent standard normals and as a special case of the Student’s t distribution.

Like the normal- and t-distributions, the standard form is centred on, and symmetric about 0. But unlike these distributions, it is known for its very heavy (fat) tails. Whereas you are unlikely to see values that are significantly larger or smaller than 0 coming from a normal distribution, this is just not the case when it comes to the Cauchy distribution.…

By | September 17th, 2019|English, Level: intermediate|0 Comments

p-values (part 3): meta distribution of p-values

Introduction

So far we have discussed what p-values are and how they are calculated, as well as how bad experiments can lead to artificially small p-values. The next thing that we will look at comes from a paper by N.N. Taleb (1), in which he derives the meta-distribution of p-values i.e. what ranges of p-values we might expect if we repeatedly did an experiment where we sampled from the same underlying distribution.

The derivations are pretty in depth and this content and the implications of the results are pretty new to me, so any discrepancies/misinterpretations found should be pointed out and/or discussed.

Thankfully, in this video (2) there is an explanation that covers some of what the paper says as well as some Monte-Carlo simulations. My discussion will focus on some simulations of my own that are based on those that are done in the video.

What we are talking about

We have already discussed what p-values mean and how they can go wrong.…

By | September 5th, 2019|English, Level: intermediate|0 Comments

Integrals with sec and tan when the power of tan is odd

We went through an example in class today which was

 

\int tan^6\theta \sec^4\theta d\theta

 

In this case we took out two powers of sec and then converted all the other \sec into $latex\ tan$, which left a function of tan times sec^2\theta d\theta. We wanted to do this because the derivative of \tan is \sec^2 and so we can do a simple substitution. If we have an odd power of \tan, we can employ a different trick. Let’s look at:

 

I=\int \tan^5\theta\sec^7\theta d\theta.

 

Here, sec is an odd power and so we can’t employ the same trick as before. Now we want to convert everything to a function of \sec and have only a factor which is the derivative of \sec left over. The derivative of \sec is \sec\tan, so let’s try and take this out:

 

I=\int \tan^5\theta\sec^7\theta d\theta=\int \tan^4\theta\sec^6\theta (\sec\theta\tan\theta)d\theta.

 

Now convert the \tan into \sec by \tan^2\theta=\sec^2\theta-1:

 

I=\int (\sec^2\theta-1)^2\sec^6\theta (\sec\theta\tan\theta)d\theta=\int (\sec^{10}\theta-2\sec^8\theta+\sec^6\theta) (\sec\theta\tan\theta)d\theta

 

where here we have just expanded out the bracket and multiplied everything out.…

Fundamental theorem of calculus example

We did an example today in class which I wanted to go through again here. The question was to calculate

 

\frac{d}{dx}\int_a^{x^4}\sec t dt

 

We spot the pattern immediately that it’s an FTC part 1 type question, but it’s not quite there yet. In the FTC part 1, the upper limit of the integral is just x, and not x^4. A question that we would be able to answer is:

 

\frac{d}{dx}\int_a^{x}\sec t dt

 

This would just be \sec x. Or, of course, we can show that in exactly the same way:

 

\frac{d}{du}\int_a^{u}\sec t dt=\sec u

 

That’s just changing the names of the variables, which is fine, right? But that’s not quite the question. So, how can we convert from x^4 to u? Well, how about a substitution? How about letting x^4=u and seeing what happens. This is actually just a chain rule. It’s like if I asked you to calculate:

 

\frac{d}{dx} g(x^4).

 

You would just say: Let x^4=u and then we have:

 

\frac{d}{dx} g(x^4)=\frac{du}{dx}\frac{d}{du}g(u)=4x^3 g'(u).…

PDE: Physics, Math and Common Sense. Part I: Conservation Law

wing-flow
Source: CFDIinside blog

INTRODUCTION

The course of Partial differential equations (PDEs) usually is a tough one. There is a number of factors contributing to this toughness:

  • PDE course combines the knowledge from calculus, algebra, ordinary differential equations (ODEs), complex analysis and functional analysis. Simply put, there is a lot that you need to know about!
  • PDE methods often (or should I say, mostly?) come from physics, but this aspect is not always emphasized and, as a result, the intuition is lost.
  • There is lots of abstraction in the PDE course material: characteristics, generalized functions (distributions), eigenfunctions, convolutions and etc. Many of these concepts actually have simple interpretations, but again, this is not emphasized.
  • PDEs themselves are tough. In contrast to ODEs, there are no general methods for all kinds of PDEs. The field is young and a bit messy.

This series of posts aims to demystify PDEs and show some general way of handling PDE problems by combining physical intuition and mathematical methods.…

By | May 21st, 2018|Level: intermediate, Uncategorized|0 Comments

Using integration to calculate the volume of a solid with a known cross-sectional area.

Hi there again, I have not written a post in while, here goes my second post.

I would like us to discuss one of the important applications of integration. We have seen how integration can be used to solve the area problem, in this post we are going to see how we can use a similar idea to solve the volume problem. I suggest that we start by looking at the solids whose volume we know very well. You should be able to calculate the volumes of the cylinders below (yes,  they are all cylinders.)

 

circular cylinder                                 rectangular cylinder                triangular cylinder

Cylinders are nice, we only need to multiply the cross-sectional area by the height/length to find the volume. This is because they have two identical flat ends and the same cross-section from one end to the other. Unfortunately, not all the solid figures that we come across everyday are cylinders. The figures below are not cylinders.…

An integral expression for n!

I gave a challenge question at the end of class a week or so ago. Here I will give the solution and show that it gives us something rather strange and surprisingly useful.

I wrote down the following, and asked you to prove it:

 

\int_0^\infty e^{-t} t^N dt=N!

 

For N\ge 0, N\in \mathbb{Z}. Now, N! can be thought of as the number of different orderings of pulling N objects out of a bag (without replacement) when they are all different. If you have N things in a bag, then there are N possible things that you can pull out first. There are then N-1 ways of pulling out the next object, N-2 ways of pulling out the next, etc. and finally, when you’ve pulled out N-1 objects there’s only a single possibility of pulling out the last. So:

 

N!=N(N-1)(N-2)(N-3)...3.2.1

 

And the number of ways of pulling no objects out of a bag is 1, because you just don’t pull anything out.…

The confusion about discontinuity

Whilst reading Mícheál Ó Searcóid’s book Metric Spaces, I found out about a nuance in the definition of continuity that I was not previously aware of, and something which may be taught incorrectly at high schools. M. Searcóid states that a function such as tan(x) is continuous (read the page here). The definition of continuity at a point is based on the fact that the function has a value at that point (if a function is continuous at x = a, then f(a) has a value in the expression |f(x)-f(a)|<ϵ). However, following M. Searcóid’s line of thought about continuous functions, it does not make sense to consider points at which a function is not defined. If we were asked to prove that the function is ‘discontinuous’ at a point, we would need to show that the condition for continuity at that point is false. And the negation* of the condition for continuity would not make sense at a point where the function value is not defined.…

By | July 21st, 2016|Courses, Level: intermediate|2 Comments

On Convergent Sequences and Prime Numbers

Ever since Euclid first proved that there are infinitely many prime numbers, mathematicians have found ever more creative ways to prove the same result, and also various stronger theorems that imply it. Dirichlet’s Theorem, for example, states that ifm and n are relatively prime integers, then there are infinitely many prime numbers of the form mk + n for some integer k. It is also known that the sum of the reciprocals of the prime numbers diverges, that the sum

\displaystyle \sum_{\substack{p \leq n \\ p \text{ prime}}} \frac{1}{p} \sim \log(\log(n))

and that the number of prime numbers less than n is asymptotically equal to \displaystyle \frac{n}{\log(n)}. In this blog post, we will continue this proud tradition by proving that there are infinitely many prime numbers which have your phone number somewhere in their digits, and which simultaneously have a prime number of digits.

To do so, we will look at the convergence of two different sums: that of the reciprocals of the primes with a prime number of digits, and that of the reciprocals of the natural numbers which do not contain your phone number amongst their digits.…

By | May 22nd, 2016|Level: intermediate, Uncategorized|2 Comments