Automatic Differentiation

Much of this content is based on lecture slides from slides from Professor David Barber at University College London: resources relating to this can be found at: www.cs.ucl.ac.uk/staff/D.Barber/brml

What is Autodiff?

Autodiff, or Automatic Differentiation, is a method of determining the exact derivative of a function with respect to its inputs. It is widely used in machine learning- in this post I will give an overview of what autodiff is and why it is a useful tool.

The above is not a very helpful definition, so we can compare autodiff first to symbolic differentiation and numerical approximations before going into how it works.

Symbolic differentiation is what we do when we calculate derivatives when we do it by hand, i.e. given a function f, we find a new function f'. This is really good when we want to know how functions behave across all inputs. For example if we had f(x) = x^2 + 3x + 1 we can find the derivative as f'(x) = 2x + 3 and then we can find the derivative of the function for all values of x.…

By | October 23rd, 2019|English, Uncategorized|0 Comments

K-means: Intuitions, Maths and Percy Tau

Much of this content is based on lecture slides from slides from Professor David Barber at University College London: resources relating to this can be found at: www.cs.ucl.ac.uk/staff/D.Barber/brml

The K-means algorithm

The K-means algorithm is one of the simplest unsupervised learning* algorithms. The aim of the K-means algorithm is, given a set of observations \mathbf{x}_1, \mathbf{x}_2, \dots \mathbf{x}_n, to group these observations into K different groups in the best way possible (‘best way’ here refers to minimising a loss/cost/objective function).

This is a clustering algorithm, where we want to assign each observation to a group that has other similar observations in it. This could be useful, for example, to split Facebook users into groups that will each be shown a different advertisement.

* unsupervised learning is performed on data without labels, i.e. we have a group of data points x_1, \dots, x_n (scalar or vector) and we want to find something out about how this data is structured.…

By | September 7th, 2019|English|0 Comments

Use your machine learning powers to solve the stock market on numer.ai

Home-grown South African mathematics, statistics and computer science have come together to give us numer.ai, founded by Richard Craib. This site which has come up with a seemingly brilliant idea, allowing anyone free access to otherwise very expensive data, but in such an encrypted form that you don’t know what the data means, but its patterns are preserved. This data is stock market data which you can use to make predictions. The predictions on their own don’t mean anything, so you send these predictions back to numer.ai, and they can apply it to the unencrypted data and make purchases on the stock market based on the most accurate models on their test data.

It’s simple but brilliant. They give you something very expensive for free, and you give them something very valuable for free. The brightest minds in machine learning can then potentially earn big money which would be impossible if it weren’t for the beauty of homomorphic encryption.…

By | December 10th, 2015|Uncategorized|1 Comment