shared market database

ComteZero · Post by **ComteZero** » Sat Jun 02, 2012 12:42 pm

i feel everybody is reinventing the wheel when it comes to market parameters datawharehouse. do you know if non-commercial exists ? if not, how about setting one ?

granchio · Post by **granchio** » Mon Jun 04, 2012 2:39 pm

if by "market parameters" you mean things like implied volatilities, then it's hard to do, as the meaning of the numbers depend on the pricing models you use.

ComteZero · Post by **ComteZero** » Mon Jun 04, 2012 7:34 pm

it can be discussed, and raw data can still be available.

iasmath · Post by **iasmath** » Sun Jul 15, 2012 5:32 am

I feel like the same - reinventing the wheel... But how feasible it is to set one database? Some few relevant points:

- Tons of data! Where to store? What are the costs? I have a SQL database on a website host but do not know if size would be enough

- Who can edit? Imagine someone execute a huge delete... Perhaps we should assign responsibilities (e.g. user "xpto1" loads prices for currencies ccy1, ccy2 etc, user "xpto2" loads retail equity prices ....)

- Interday data only or intraday too?

I think the most important is trustable raw data with correct split/dividend infos and not shit like yahoo finance. Then implied vols can be calculated by different (documented) methodologies/models.

granchio · Post by **granchio** » Mon Jul 16, 2012 5:09 pm

Agreed raw data is what matter.

Strange · Post by **Strange** » Tue Jul 17, 2012 1:20 am

I think it makes a lot of sense from the perspective of sharing initial data, storage and maintenance costs. On the other hand, data and data manipulation is frequently a big part of the edge.

iasmath · Post by **iasmath** » Tue Jul 17, 2012 2:29 am

Agreed on edge. And that is one of the reasons I liked the idea...

- Data manipulation does not need to be stored. Focus on raw data...

- I suppose the edge is based on publicly infos. So it is already available to all of us and, if database is large enough, it would still not be obvious to anyone that there is an edge on using a given info

- Data would be yours and not of your shop (humm this is an ugly statement! I am feeling bad)

Strange · Post by **Strange** » Tue Jul 17, 2012 4:42 am

What if my shop is literally my shop? ${L_SMILIES_SMILE}$

Personally, i think vols/options is the last place to go for something like this. I'd start at daily data for futures (there is no free source AFAIK) and higher resolution data like 1 min bars.

ComteZero · Post by **ComteZero** » Mon Jul 23, 2012 10:08 am

i guess raw data is important, but processed data can also be available if manipulations are well documented. thus every user can decide if he agrees on what has been done, or if he wants to use raw data from scratch.

MadMax · Post by **MadMax** » Thu Jul 26, 2012 4:02 pm

what about raw data + "views"/scripting facilities allowing users to apply their own manipulations.