Scalalab Fslab, functional/quasi functional languages

rickyvic · Post by **rickyvic** » Tue Sep 15, 2015 10:46 pm

Any idea if I can in part replace Matlab/R with any of those?

The idea is to use them for quick time to production.

In any case R or Matlab can be called if needed as I understood.

Logic would be to avoid rewriting the code in a lower level language.

Hansi · Post by **Hansi** » Tue Sep 15, 2015 11:10 pm

I don't understand why the question brings up Scala on one hand and F# on the other, what are you using in production today?

Scalalab doesn't seem to have much traction. And I don't use Scala so I'll reserve commenting on it.

I don't get Fslab... what does it actually add? I've used all of those libraries and most of them are great but I see no point in using FsLab, it doesn't seem to add anything.

Using the R Provider does give flexibility but you're still using R which is slow anyway so you gain pretty much nothing other then ease of interop.

Our current data analytics app is a C# ASP.NET app with a separate in-house built R Engine which keeps multiple R sessions running on the server ready to do number crunching with all packages etc already loaded and whatever data possible cached and primed. It works out okay, not optimal but okay. I didn't use R.NET when building it because R.NET was pretty early stages and buggy when I built this a few years ago.

Going pure F# (+ libs referenced by FsLab) or Scala is most likely most adventitious. I'd go with F# today if we didn't have 130K+ LOC in our R library and trying to get a dozen people to shift languages is not feasible at this point.

Maggette · Post by **Maggette** » Wed Sep 16, 2015 12:20 am

Hi,

maybe a little bit off topic.

I started to use Scala a lot lateley, because at the moment I use SPARK (non finance application) on a Hadoop/Yarn cluster.

It is quite elegant and I like to code in it..but it comes attached with annoying Maven Shit and SBFUCK typical for JVM world.

As an aside: I hope MSFT will come up with a powerfull and distributed in memory/spill over big data framework and use F# and C# for it ...haaa...dreaming of Linq to Amazon -S3 ${L_SMILIES_SMILE}$ ....

rickyvic · Post by **rickyvic** » Wed Sep 16, 2015 1:14 am

In production today I use matlab as in research mode, feeds are written in java or directly matlab with COM interface.

You clarified a few points, in fact probably going with matlab and writing a mix of matlab and scala in production could be a solution.

The real deal would be using one fast language for both prod and research.

rickyvic · Post by **rickyvic** » Wed Sep 16, 2015 9:39 am

Admittedly I am not a great developer "annoying Maven Shit and SBFUCK typical for JVM world" would you be able to expand this comment, thanks for your help.

About scala how is performance compared to java?

Anybody used Jane street's Ocaml libraries? They should be good after all these years.

Maggette · Post by **Maggette** » Wed Sep 16, 2015 11:24 pm

Hi,

if you are not familiar with the Java/JVM world (and I am not used to Java), you sometimes have problems doing even the most trivial things (stuff you don't event think about when you are using matlab). Dependency conflicts, Java still has no useable native DateTime class and and and ...

If you are coming from a nice and neat framework like the .NET or write most of your applications/scripts in Matlab, R or Python (for things that they are good for) you sometimes get anoyed by this stuff.

Performance Scala vs Java...depends on the problem. In the SPARK universe I am living at the moment, scala is IMO the way to go (SPARK is written in scala).

rickyvic · Post by **rickyvic** » Thu Sep 17, 2015 4:57 pm

Understood thx, it is not trivial. In your experience .net would be a much more hassle free experience?

Interesting Spark btw.

Maggette · Post by **Maggette** » Fri Sep 18, 2015 2:56 am

Imho .NET is by far the better framework. But other people might have different opinions.

MadMax · Post by **MadMax** » Fri Sep 18, 2015 12:19 pm

1) I would say .NET is a better framework but if you are going to run on Linux then you will go with JVM.

2) JVM-based "ecosystems" for distributed computing are far more developed.

rickyvic · Post by **rickyvic** » Fri Sep 18, 2015 1:15 pm

In this specific case the decision is based on

1) pain in converting matlab code to the target language

2) pain in fixing bugs that are hard to find

3) performance vs C especially in real time

4) ease of integration with external apis

5) time to production

If performance vs C++ is enormously far (say >3 times) then it cannot be a candidate.

Off topic: I have been told that armadillo C++ could be an alternative to functional languages, used together with matlab coder.