跳至主要内容

博文

目前显示的是 十月, 2012的博文

[译]MapReduce 模板,算法,使用案例

原文地址: http://highlyscalable.wordpress.com/2012/02/01/mapreduce-patterns/ MapReduce 模板,算法,使用案例   Posted on February 1, 2012 24 In this article I digested a number of MapReduce patterns and algorithms to give a systematic view of the different techniques that can be found on the web or scientific articles. Several practical case studies are also provided. All descriptions and code snippets use the standard Hadoop's MapReduce model with Mappers, Reduces, Combiners, Partitioners, and sorting. This framework is depicted in the figure below. 在文章中,我摘取一些 MapReduce 的模板何算法,对网上和学术论文仲的不同技术给出一个系统的观点,同时也提供一些实际的案例。所有各处的描述和代码片段都是根据标准的 Hadoop 中的 MapReduce 模型完成,其中包括Mappers,Reduces,Combiners, Paritioners,Sorting 模块。框架的描述如下: MapReduce Framework Basic MapReduce Patterns 基础的 MapReduce 模板 Counting and Summing 计数器和累加器 Problem Statement:  There is a number of documents where each document is a set of terms. It is required to calculate a total number of occurrences of each term in all do...