Any advancement on compiled vector oriented languages

[prikolno]

> What are the issues with using large numbers in calculations in Python?



Speed. The runtime scales with ~n^1.585 for two numbers with n-digits. There's asymptotically faster algorithms that improve on the exponent but they're slower for small n. If you're doing this very frequently and with large numbers (most often in cryptography), then it makes a huge difference to use a more exotic algorithm. No impact for finance where numbers can be represented in 64 bits.

[svisstack]

microservices, deploy on k8s and forget? doing job -> yes. dont touch.

you want golang for something ok, other thing for something else it's also ok.



i love matlab but's it's has terrible OO which dont make any sense if you can code in something legit like c++, C#, java. worth to mention you can host the matlab code as rest service or convert to c++ (i never used it actually)

[jslade]

@sviss: Matlab OO is completely worthless, and nobody should ever write code this way. Better namespaces would help a lot. Their OO thing fails at being better namespaces, and is a third rate student implementation of OO using eval.



@rickyvik: you do realize that Octave exists right? Most of the low level Matlab packages have been more or less cloned, and the IDE is almost as good. FWIIW matlab had this RtOS target called Xpctarget at one point; I built an inertial navigation gizmo with it. Supposedly you can compile to verilog as well.



I use J for developing new ML and statistical/signal processing algorithms. One of the killer apps; linking to arbitrary C libraries is literally a one-liner. I also use the database for a data tee-off on a feed handler (it's been up for a year and a half now -the EC2 instance is less stable), and a liquidity finding gizmo. It takes a year or two to be useful in it, but it is the highest productivity language I have ever used. I haven't been using it much in the last year and a half, but it's my jackhammer and stradivarius. It's a fairly antisocial programming language, but I'm a misanthrope and prefer not to talk to anyone else anyway.



@prikolno -I see where you are coming from now. I suppose the only use for a repl in your world is running stats and looking at charts -and those are probably fixed enough you'd want them in a static report.



@espresso -I'll let you in on a secret: silicon valley engineers are mostly yeast life. They do things like slap together "cloud" crap in containers not because it's a sound practice, or interesting engineering, but for the same reason they all wear hoodies well into their 30s. They're mostly powerless lemmings, and the main reason "the cloud" exists as a 'go-to' is because accountants find calculating depreciation too difficult.

[rickyvic]

J is interesting, will never find a developer though....

Octave is not good imo, nothing good about it

[prikolno]

> I see where you are coming from now. I suppose the only use for a repl in your world is running stats and looking at charts



Yes, glad we're synced. And we do a lot of the modeling and exploration work outside of a REPL.



> linking to arbitrary C libraries is literally a one-liner.



That's actually pretty cool.

[Maggette]

Everybody is different...but taht is IMHO not the advantage of an REPL.



Working with a REPL is kind of "poor mens test driven development". Lots of code I write is already tested (at least in a trivial way) even before I write a unit test for it.



That has vastly improved my development speed.



Also it is quite great to explore new APIs and language features.

[rickyvic]

Basically the two solutions I found acceptable for my workflow and to avoid going insane are:

1)

a)matlab for research offline and hdf5 for db, which so far worked out ok.

b) An api/gateway connection which will push data to a shared memory and keep a book of real time quotes up to date (this is being developed in C++).

c) Then all I need to do is to write the real time algo in matlab and then compile to C or any low level machine code (to be studied and tested).

2)

doing everything in a language like J but not sure how it works in realtime. It means a year from now so will be version 2.

Probably will need to interface to C.



This is for an algorithmic trading system

[bullero]

@rickyvic - Why don't you just write everything in C/C++? If you really really really want to mix languages you can make them talk via ZeroMQ for example. Now, if you need to solve some math systems quickly you have libs like Blaze, alglib and xTensor for that so no need for matlab or any other matrix language. And finally whats up with the shared (main?) memory, its quite slow.

[rickyvic]

@bullero

thanks for your input. There is no reason to make the research environment in C++, it would take ages and we need rapid prototyping. All we need to do is writing params to a db which will be loaded by the trading algos.

For prod it would be good to have a fast vector oriented language to port the code quickly. Shared memory, I meant a process that reads data and pushes them via zeromq and the algo reads them.

Otherwise the idea would be using a shared memory so all algorithms can read from the same table in parallel.



But I guess that would be slow... so your suggestion is to read the raw data, process them (so rebuild OB and record transactions) in memory and run the algo on the dataset?

Then replicating the same for all strats?

[bullero]

@rickyvic - Difficult to give any specific suggestions because (in my opinion (whatever its worth )) the implementation is dependent on your trading logic and trading/investment time scale. However, I think its clever to do the research 'backtesting' using the same system you would use for live action. Again, depends on your strategy and time scale. For a traditional 'style investing' approach this is not crucial issue - for a latency critical stuff more so. (I do not mean you should not worry about data granularity)



Now, for intra-day system research I would do the following: I would replay a trading day by sending those market data messages to my system as if it was going live. After consuming the market data I send my actions to (a) hypothetical exchange(s) which will then send me back simulated fills in case such an event happens. The exact simulation of fills is quite complex in practice but you can try to get some sort of statistical estimates. The statistical behavior of this hypothetical exchange may be replicated (to a point) using information distilled from the same market data that I am sending to my box. Btw, there is a large body of literature out there giving you the mathematical framework for building this.