Home > Article > Backend Development > Writing MapReduce jobs using Python
mrjob allows you to write MapReduce jobs in Python 2.5+ and run them on multiple different platforms. You can:
Write multi-step MapReduce jobs using pure Python
Test on your local machine
On a Hadoop cluster Run
Use Amazon Elastic MapReduce (EMR) to run on the cloud
The installation method of pip is very simple, no configuration is required, run directly: pip install mrjob
Code example:
from mrjob.job import MRJob class MRWordCounter(MRJob): def mapper(self, key, line): for word in line.split(): yield word, 1 def reducer(self, word, occurrences): yield word, sum(occurrences) if __name__ == '__main__': MRWordCounter.run()