I'm looking to for a service that is similar to Amazon S3, a simple service to store and retrieve arbitrary data (and meta-data), but one that runs locally in your own data center. Strictly speaking, I'm not sure whether you would call this a CDN or a lightweight CMS.
It must be horizontally scalable (both for storage and bandwidth) and fault tolerable. It must also support REST, preferably WS too, with a pluggable authentication and authorization system. Something built with Java EE would be preferable for more convenient integration and extensibility, but this is just a personal preference, and it not a requirement.
Suggestions?
Here are a few open source solutions I have come across that deserve further research:
- Apache Sling (JCR based CMS (JSR170, JSR283), RESTful interface).
- Apache Hadoop (Java based distributed data-store, map reduce functionality).
- HBase (built on top of Hadoop, provding Google Bigtable-like capabilities).
- CouchDB (Erlang based key/value DB with Map/Reduce functionality, RESTful interface).
- Dynomite (Erlang based, Amazon dynamo clone).
- Voldemort (Distributed key-value storage system).
- Cassandra (highly scalable, eventually consistent, distributed, structured key-value store).
- MongoDB (highly scalable, JSON document based storage).
Walrus project (mostly s3 api compatible) . . .
http://open.eucalyptus.com/wiki/EucalyptusStorage_v1.4
Park place is an S3 clone in Ruby.
In addition to Park Place, the only other big player against S3 right now is Nirvanix.
Nirvanix