Numerai : a network of data scientists to predict the stock market
Dec 26, 2016
3 minute read

Numerai is a website about Machine Learning. Well, no surprises here, the topic is sky-rocketing in the IT world nowadays.


But here, it is about the Stock Market, a dream for a lot of data scientists : many features, predictions that matter (the more accurate you predict, the more money you earn) and, without hiding ourselves, being able to tell your step-parents when they ask about your job Well I’m predicting the stock market you know.

A tale about the Stock Market

Once upon a time, a man wondered what made predictions in Machine Learning so accurate on some topics. He noticed that everything has been a matter of publishing the data. Nothing beats the world-wide community on a dataset, best predictions only come from global participation.

However, the stock market is somewhat special. Datasets are highly valuable so each financial company is protecting its data, and if you manage to build a really strong model, you will certainly keep it for yourself ! So this man was facing a huge issue : how to make the world work on the stock market dataset without giving it to everybody ?

This clever guy found a solution: Cryptography. Recently, encryption tools which conserve additions and multiplications have been discovered, so the idea of an encrypted dataset came, that would still be able to gather meaningful features and give rise to relevant predictions. Everybody was now able to take part without the real and sensitive dataset.

Let’s go

Nothing else was needed to launch Numerai, a place where the best data scientists are working together on improving predictions about the stock market. The organization uses stacking to combine predictions from everybody and aims to build the strongest model possible.

Stacking is about having a meta model which takes in input everybody’s predictions and gives them a weight according to the improvement they bring to the global model. For example, if I submit a model that is, in a good way, radically different from others, I may improve Numerai’s model a lot.


Why predict the stock market ?

There is an obvious financial aspect behind predicting the financial field, but it still can be more than that. Everybody wins when the stock market is predictable, because resources will be allocated perfectly forever. Then comes the ethic : Is the South going to benefit from this ? Will a financial company buy the organization ? This needs to become a problematic people should really think about.

Money money money

As I presented it, predictions of the stock market can be highly valuable, and Numerai is not hiding its meta model predictions. They sell them to financial companies and distribute the money they earn amongst the best models : the more your model is helping the meta model to get closer to reality, the more you will be rewarded.

A leaderboard is also available on the front page : Numerai, if you want to know more about it.

There is currently no limitation to signing up, so give it a shot, because even if your vision is totally different from everyone else’s, you could still improve what they’ve achieved so far. I am going to try my hand at it, attempting to elaborate strong models, but I know how challenging it could be …

See ya !

comments powered by Disqus