Predicting stock or foreign exchange prices with ML models

WorldQuant University have a free masters (have to score 70%+ in their maths entrance exam) in Financial Engineering but it is 2 years so may be too long for what you are looking for. WorldQuant have a community with learning resources called Brain: Brain - WorldQuant . I have signed up but I haven’t utilised any of it yet so I cannot really provide feedback on that.
NOTE: WorldQuant University was made by WorldQuant but they are treated as separate entitites.

1 Like

This is a very interesting problem, and I think a lot of us have considered if this is possible, and there are absolutely people spending millions to try develop a system like this that works. One of the things I have considered a lot is that there are a lot of people including large brokerages and hedge funds using AI trading models, it is possible to develop a system to take advanage of knowing how they will trade? It would be interesting to try and correlate a model you created with particular large traders’ transactions.

There are some good technical suggestions already here as well as many practical ones. I don’t have much to add to what has already been said, except that I have seen traders who have used much more primitive tools with EMAs and such and seemed to make a lot of money. The issue quite often is sorting out if they were just lucky or if the system really worked. Kind of like all the people I know that continually win money playing slot machines when you know that the numerical eventuality is that you will lose. On the other hand, 60% correct would make a KILLING in the highly leveraged world of currency trading, but you would need a massive amount of data before investing substantial money. And then some world event not even considered in your model could put you so upside down you never recover.

About 10 years ago I thought I have a pretty good system figured out for stock trading using options and for a while I was killing it, making thousands of dollars a day, then one day it didn’t work and I lost about $50k. After making some of it back, then losing more I realized that I was so heavily leveraged that all my “winnings” were really coming from the large amount of leverage applied, and I was just digging deep into that pool – until it came time to pay up. I dumped the “trading” and used what I had left for “investing” which isn’t as exciting but has done well for me.

I think that what you are doing has value, but would best be combined with other indicators that influence pricing, rather than trying to predict a value, that is nearly random using just these indicators. I think they would be good indicators of momentum and such when considering events that influence pricing and perhaps be a good indicator how strongly a particular event might effect the market. Even that would have a lot of value.

The financial engineering sounds fascinating, but this isn’t my real job even though I used to be a full time software engineer. I would love to hear more about it.

1 Like

Hello,

If this issue has viable solutions, ARIMA models and LSTM could potentially yield marginally superior outcomes. I attained a 79% success rate in predicting the SPY direction using three years of data. However, this impressive result does not hold up under the cross-validation method, displaying only a slight improvement over 50%. Additionally, as demonstrated in another article, simple statistical methods possess greater predictive capabilities. In summary, as further comments suggest, the challenge is remarkably tough. Financial time series are predominantly composed of noise.

Regards
SLaignel

1 Like

Thanks @Andrew_Wylde and @Laignel_Sebastien for your wise advice!

I’ve been thinking about experimenting with a reversion to the mean strategy by following these steps:

  1. take a series of bollinger bands for the eur_usd exchange rate, including bollinger bands values for each of the last 28 days.

  2. add a series of simple moving averages over each of the last 28 days.

  3. run ARIMA and LTSM models to predict the direction of the next tick mark.

My hypothesis is that the reversion to the mean happens and that the likelihood of it happening is higher the more days the sma is less than the lower band or higher than the upper band.

What do you think about testing this strategy? It seems so simple that I wonder if doing it will yield anything interesting.

1 Like

“which notifications users would click or which apps users would install after seeing ads.”
I think, It’s more the case of neuromarketing, than a future prediction

1 Like

It’s actually a bit of both. Ultimately, the advertisers only cared if we could accurately predict which users would end up being “whale” customers. But yes, I guess you could say that we were using neuromarketing.

1 Like

This is more likely a Reinforcement Learning problem. I studied this subject a few years ago, I hope this material I selected helps you: GitHub - Draichi/Portfolio-Management-list: 📓 List of portfolio management resources, using Reinforcement Learning.

2 Likes

Thanks @L_D1! I found your list very fascinating. I’ve also been interested in trying reinforcement learning. Have you actually tried reinforcement learning before?

1 Like

Thanks! Yes, I tried before, but just for fun, without a formal understanding of the subject, and I ended up creating a customized RL environment for the crypto market (GitHub - Draichi/T-1000: ⚡ 𝘋𝘦𝘦𝘱 𝘙𝘓 𝘈𝘭𝘨𝘰𝘵𝘳𝘢𝘥𝘪𝘯𝘨 𝘸𝘪𝘵𝘩 𝘙𝘢𝘺 𝘈𝘗𝘐). Now I’m doing a reinforcement specialization on Coursera to better understand the subject

1 Like

Hello @David_Park why don’t do a system on the stock long term prices? Deciding only on the historical data if something will go up/down in a particular moment is speculation and this has high variance because it can also something not affected by reasonable factors. Building a system that ingest balance company data and gives you price projection could be more effective for my point of view. SO I would use ML more for investing proposal than speculation.

1 Like

I am also interesting in doing this project , any books recommended ? to start with ? I was reading this book Machine Learning for Algorithmic Trading by Stefan Jansen, I am not sure if any one used any concepts from that book , I started and then got busy with other courses ? probably we should try reading it ??

1 Like

You should check out Numerai if your interested in Quant Finance, they provide obfuscated data and you submit models similar to Kaggle.

1 Like

Use time series concept and apply/train statistical models like FBprophet …

1 Like

@daniele_ing and @Sanjay_Gautam1 : check out these resources for applying ML to trades.

I found these in a Udacity course on Reinforcement Learning, and they seem pretty useful for understanding how to apply ML to financial markets. I’m interested in trying out the code in the QLearning/Trading github repo to try to replicate the author’s results.

1 Like

I too had this in my mind but little different approach, what about predicting for sort time. Example say market starts at 10:00 AM we took 15 mins of data 10:00-10:15 and predict for next 15 mins.
Hope is:

  1. We will be able to train with Supervised Learning on past data.
  2. As interval is short hence under lying pattern will be same. Like training on 15 mins of Data and predict next 15 mins.
    Any thoughts.

@David_Park

1 Like

I think that’s it in a nutshell. Suggests to me the old joke, “What’s the best way to have a $1 million portfolio? Start with a $2 million portfolio and make investment decisions using a homegrown price prediction model.” As suggested by others above, I do think it’s critical to identify what type of trading you want to do. If it is intraday, the modeling might be less complex, but to make money you’re going to need millisecond level data latency, and no one who has that is giving it away. If you’re looking at long positions, none of the literature I have read focuses (exclusively) on individual asset price. Rather it’s all about correlation and risk management. Correlation blows up the parameters but their are techniques for mitigating that problem. Personally, I’m testing out risk parity models for portfolio construction/ optimization with infrequent rebalancing (month(s)), but I’m strictly a long position investor.

Here’s a paper on risk parity portfolio construction that I see referenced frequently in the stuff I’m reading…

http://www.thierry-roncalli.com/download/erc.pdf

I think this is the correct attribution

S. Maillard, T. Roncalli and J. Teiletche, “On the Properties of Equally-Weighted Risk Contributions Portfolios,” The Journal of Portfolio Management, Vol. 36, No. 4, 2010, pp. 60-70. http://dx.doi.org/10.3905/jpm.2010.36.4.060

Cheers

1 Like

@Sanjay_Gautam1 all the research I did I posted in this github repo GitHub - Draichi/Portfolio-Management-list: 📓 List of portfolio management resources, using Reinforcement Learning.

hope it helps

I’m also interested in this topic, espcially for the application of reinforcement learning in trading.
I have studied the knowledge of reinforcement learning from several courses and I also studied the paper " MACHINE LEARNING FOR TRADING" from GORDON RITTER, which is the best for this topic ( I think).

Next step I’d like to do some projects for RL in trading and Thanks to this post, I found on Github that there are two projects which are quite interesting:
TensorTrade: GitHub - tensortrade-org/tensortrade: An open source reinforcement learning framework for training, evaluating, and deploying robust trading agents.
TradeMaster. GitHub - TradeMaster-NTU/TradeMaster: TradeMaster is an open-source platform for quantitative trading empowered by reinforcement learning ⚡ 🌈

I’d like to start from one of the above two. My target is to have an environment which can allow to use my own data, customize features and develop customized RL algorithem

Can anyone give some idea about which one is better out of the above two projects on Github(or any other alternative)?

Any other comments are also welcome.

Thanks a lot.