Forward trading some AI

Cont.
By the way, I plan my datasources such that any datapoint of any source is deterministic and immutable, which allows me to pretty easily store every byte of data that comes into the system in a large (going on 80GB now) database which I can source from when backtesting (Which conveniently just means I made a datasource that reads from the stored past data from other datasources). I recommend this as well, because it gives you precise market/other datasource replay at any point in time for accurate (more accurate than any online service you'll find) market reconstruction and replay for backtesting at any point in time.

looking in to their .net as half of my code is c#

Very relevant to my interests. My longterm goal is to train an agent on multiple markets so it would make sense to do this.

Update us along the way user, I'd love to follow your progress.

Will do, thanks for the input

>I'm sure needs a lot more time on realtime, real life market data.
Yes?

Also, I do store my state,action,reward Tuples currently, may be worthwile to expand on that So I could backtest without my SOMs having to descritize the space every time for every iteration.

off topic tech shit for babbys first bot - show us your real money trades on live data or keep it in Jow Forums where it belongs

I'm asking the best places for 1000tick live data though.. I'm not showing off and I could get absurdly good results in backrest. Not what I'm hafter

Hmm, seems like you can reconstruct your state action reward from the input data by feeding the input data back into the strategy as if it were coming in live. That data should produce the same input layer, and the same model should take the same action and lead to the same S' and R.

Have you found Futures to be easier to learn, or any nuance there? Curious if you've noticed anything interesting compared to typical instruments.

Oh you want live data?
Really, stream trades and depth indefinitely, and fuck candlestick charts - You can reconstruct one second candles from the raw trades if you want - Trades and orderbook events is all you need to replay any point in time down to the millisecond.

Deep learning for HFT is hardly babby's first bot. Have you seen Goldman HFT equity curves? It's a straight fucking line every time, super interesting

Similar to how mine is set up, but that are separate, as the ability to add dimensions to the brokers data feed isn't in the docs, and because I'm using tick data instead of time data I haven't nailed now a grade way to progress. As to futures, I chose a market with high participation under the assumption that any manipulation would take longer and have less effect

My current input is buy volume ,sell volume, spread, and time to complete. Never sees a chart, but I'd love some ideas on input

get these cringe lords off this board now mods do your jobs

Attached: 1553434355152.jpg (351x351, 22K)

Yeah what I'm getting at is that you can store nothing but the stream of trades from the broker, and you can use math to reconstruct any price action or volume action that you want from that during runtime - volume is just the sum of trade quantities - candlesticks are just the min, max, start, and end prices of trade prices during a time interval - SMA is just the average of candlesticks - so-on and so-on.

The other necessity is depth - you can reconstruct spread from depth, and use depth and trades to get high accuracy backtesting

so all you need to stream and store is trades and depth - everything else is a transformation into whatever features you want for the input layer

>hating on a thread discussing literally what the top tier financial institutions are doing for the lion's share of their trading profits
come on, user, don't be a Jow Forumslet

That makes total sense now. I'm currently using last and bid/ask to determine buy/sell. Switching to a tape reading method as you describe, is there something I would have to change?

you can actually keep your tick based method, just aggregate:

Let's say you want minute-wide ticks, and you need the past 10 ticks' volume and spread, for your input:

Find the last 10 minutes worth of trades and depth updates
Aggregate every 1 minute worth of trades into a single tick by summing volume etc. You should have 10 ticks now.
Pull out the volumes for your input layer
Compound every 1 minute worth of depth updates into a running depth snapshot.
At every minute boundry, pull out the lowest ask and highest bid for your input layer.

Now you can reconstruct ticks from streams.
Or, also have a tape reading strategy on the side - you can do both now.

I've got work to do! Thanks user

What book?

Trading I. The zone, volume price analysis, and some Richard wyckoff stuff. I was simplifying. The argument is that the tape is no longre human readable, so I figured I had an answer for that

Do you really think humans are better? I can't wait for the machines to take over and rid the universe of humans.

Attached: bleached.jpg (474x266, 22K)

>60% of hedge funds underperformed the s&p 500 for the past two decades
>top tier financial institutions

the problem with deeplearning shit is you dont know what it's doing so there is no guarantee it will work in the future. come up with a theory, develop it, backtest, then implement.

I would argue that of you don't know what's going on it's your fault. I have pretty granular control of mine,
but that's understanding the maths. There is a black box of sorts but its definitely not a set and forget

Back from dinner
Good luck user, hope you make more threads in the future.

It's an interesting thought experiment to assert that:
If you can prove mathematically that an algorithm is capable of converging on a series of nonlinear regressions that maximizes a value function, then:
you DO know what's going on in there - you know that it's maximizing value, and you might as well be just as confident in its ability as if you handcrafted the solution yourself.
You just don't know the specific logic it's encoding.

Basically, you can handcraft c = a+b. But if you write a linear regression that converges on c = a+b, then you should be equally confident in that piece of software.
I get your point, though - Mine is just an academic argument for the shits and giggles.

yip that's about it.

Did you write your own alg?

Yes, but algorithms didn't really do it justice. There are many