Collaboration infrastructure

Scotty · Post by **Scotty** » Mon Jul 02, 2012 6:30 am

Suppose I was working with someone in a different part of the world and wanted to share things like:

- market data (gigabytes)

- pdfs and documents

- code

I need it to be simple to access and somewhat secure. Some options are (but I'm a bit clueless):

Google cloud

Amazon ec2

dropbox

a VPN

Any experiences or advice to share?

Tradenator · Post by **Tradenator** » Mon Jul 02, 2012 6:34 am

Dropbox has worked well for me, but I am a bit apprehensive on the security around all these things. I, too, would like to hear more.

YukaRedux · Post by **YukaRedux** » Mon Jul 02, 2012 6:53 am

I've found Dropbox to be outstanding in terms of value & uptime - being able to access my docs from the iPhone has also saved my ass on more than a few occasions. Super easy to set up & use.

Security-wise, they're OK - they've had some problems in the past. My feeling is that anything of critical importance and secrecy that you shoot into any cloud-based provider should be separately encrypted & password protected.

Trev · Post by **Trev** » Mon Jul 02, 2012 7:06 am

SpiderOak is very solid and has a better reputation vis a vis security versus Dropbox. It also has an iPhone/Android app. I've been using SpiderOak and Dropbox both for about a year now and really like them both: I added SpiderOak after reading about some about security issues with Dropbox (i.e. ability for on-site personal to see passwords and other info) and added SpiderOak due to security implementation.

goldorak · Post by **goldorak** » Mon Jul 02, 2012 8:22 am

If it is all about sharing documents, you may want to have a look at amazon S3 rather than amazon EC2. On Amazon S3 you can have your documents encrypted "on-the-fly".

s3sync is a pretty useful tool in that case.

weismat2 · Post by **weismat2** » Mon Jul 02, 2012 10:59 am

Depending on the quality of the internet access, you may also consider buying synology boxes and use sftp and/or rsync to sync the boxes.

You might need such a box anyway as a datastore locally.

The software bundled with the boxes has a nice GUI to setup Unix services for non-Unix persons.

moonvest · Post by **moonvest** » Mon Jul 02, 2012 8:24 pm

Has anyone had experience with Huddle?

I'm trying to find a more secure alternative to DropBox as well.

granchio · Post by **granchio** » Mon Jul 02, 2012 11:09 pm

to share code, we use SVN with the main repository on a dedicated server. we use it also for important docs, sheets etc so we get the versioning on those too.

jslade · Post by **jslade** » Mon Jul 02, 2012 11:34 pm

I have used dropbox and google docs, and found them helpful for sharing data. I have no idea how secure they are.

One thing I have found helpful for remotely managing small projects is basecamp. It's free for "single projects" which can be as big as you want them to be. You can build up task lists with associated documentation, links and so on. I think you can upload files up to 2Gb.

Another commercial thing I've used is projectlocker; the SVN and wiki can be helpful if you're using SVN to share code.

DrGrumpy · Post by **DrGrumpy** » Tue Jul 03, 2012 8:01 am

I guess you need to specify what "somewhat secure" means to you Scotty.

You can have perhaps two of: secure (think encrypted), easy to access and easy to setup/maintain.

My view is that any machine that you don't have physical control over is somewhere you would want to think hard before storing any of your special sauce...

Dropbox and Spideroak are fine for a lot of things. Pogoplug is an interesting option where you have your own drives at your own various locations. My concern with these is that you don't control your own data. Pogoplug in particular feels vulnerable - you need their licenced software to access *your* data on *your* drives. Dropbox (or Ubuntu One for example) is not encrypted, so someone hacks their server, or gets your password and you're screwed. Recall you need to think about whether your communications with the servers are encrypted.

The other option is to host things yourself. Get a cheap box, put in a raid, use vpn (yuck!) or ssh in to access your data. Software-wise, use git and git-annex to remove the svn repository single point of failure. git-annex looks an awesome way to manage large data over various machines. I'm using it for movies, music, my paper archive and data sets without a complaint as yet.

An alternative that I've been told about is Tahoe-LAFS. Distributed; encrypted; error encoding (i.e. k-of-n encoding means you can lose (n-k) machines and still have all of your data); read/write, read only or verify only access; open source. Sounds an interesting option. Not sure how easy it is to use...