Seattle Conference on Scalability: Scalable Wikipedia with E
August 29th, 2009 | by admin |
Google Tech Talks
June 14, 2008
ABSTRACT
IGlobal online services at Amazon, eBay, Myspace, YouTube, or Google serve millions of customers with tens of thousands of servers located throughout the world. At this scale, components fail continuously and it is difficult to maintain a consistent state while hiding failures from the application. Peer-to-peer protocols provide availability by replicating services among peers, but they are mostly limited to write-once/read-many data sharing. To extend them beyond the typical file sharing, the support of fast transactions on distributed hash tables (DHTs) is an important yet missing feature.
We will present a distributed key/value store based on a DHT that supports consistent writes. Our system comprises three layers:
- a DHT layer for scalable, reliable access to replicated data,
- a transaction layer to ensure data consistency in the face of concurrent write operations,
- an application layer with an extremely high access rate.
For the application layer, we selected a distributed, scalable Wiki with full transaction support. We will show that our Wiki outperforms the public Wikipedia in terms of served page requests per second and
we will discuss how the development of the distributed code benefited from the use of Erlang.
This is joint work of Zuse Institute Berlin and onScale solutions GmbH.
Speaker: Thorsten Schuett, Zuse Institute Berlin
Thorsten Schütt is a senior researcher with the Zuse Institute Berlin (ZIB) and a co-founder of onScale solutions GmbH. He received a CS diploma with distinction in 2002 from the Technical University Berlin. Since then he works as a research staff member in the Computer Science Research Department at ZIB and participates in several EU projects like GridLab, XtreemOS and Selfman. He is the principal system architect of the scalable, transactional key/value store at ZIB. His research interests include distributed data management, scalable grid systems, p2p algorithms and self-managing transactional
storage systems.
Slides for this talk are available at http://groups.google.com/group/seattle-scalability-conference
Duration : 0:26:31
[youtube pW339qR7DvU]
7 Responses to “Seattle Conference on Scalability: Scalable Wikipedia with E”
By marstein on Aug 29, 2009 | Reply
If you give a talk …
If you give a talk in the future, make sure that the microphone does not rub against your shirt.
By DeletedaccountOx on Aug 29, 2009 | Reply
Awake you of Israel …
Awake you of Israel, come learn the truth! This white man is nothing but wicked! GMS is rousing the House of Jacob come and sing with us! youtube Lastdaysofedom and set yourself free, you so-called blacks, native americans and latins!
By scalability on Aug 29, 2009 | Reply
The slides can be …
The slides can be found on the onscale de website.
By mux4fun on Aug 29, 2009 | Reply
User marstein said …
User marstein said it already. This talk is nearly worthless without the slides… but it’s so interesting! Please could anybody fix that? Is there someone out there who has the slides and can load them up/make a youtube presentation — one could view this flv and the new one side by side… thanks in advance!
By marstein on Aug 29, 2009 | Reply
The slides are …
The slides are overexposed unfortunately. And they are essential to understanding the subject. dang.
By Jack9C on Aug 29, 2009 | Reply
Where can we get …
Where can we get the slides? These are too distorted for me to understand.
By alexandredulaunoy on Aug 29, 2009 | Reply
Chord seems nice on …
Chord seems nice on paper… but I’m always wondering why git cannot be used. Chord is storing the change sets that needs to be propagated (in a very simplified way). Why not using git where only blobs are exchanged while commits are just ref to blobs and trees.. Git merging/rebasing must be accommodated but that’s also the case for Chord. Git has a major advantage the space efficiency of the date store.