learning_spark.pdf

(6557 KB) Pobierz
Spark
LIGHTNING-FAST DATA ANALYSIS
Learning
Holden Karau, Andy Konwinski,
Patrick Wendell & Matei Zaharia
Learning Spark
Data in all domains is getting bigger. How can you work with it efficiently?
This book introduces Apache Spark, the open source cluster computing
system that makes data analytics fast to write and fast to run. With Spark,
you can tackle big datasets quickly through simple APIs in Python, Java,
and Scala.
Written by the developers of Spark, this book will have data scientists and
engineers up and running in no time. You’ll learn how to express parallel
jobs with just a few lines of code, and cover applications from simple batch
jobs to stream processing and machine learning.
■
Learning Spark is at the
“
top of my list for anyone
needing a gentle guide
to the most popular
framework for building
big data applications.
Chief Data Scientist, O’Reilly Media
—Ben Lorica
”
Quickly dive into Spark capabilities such as distributed
datasets, in-memory caching, and the interactive shell
Leverage Spark’s powerful built-in libraries, including Spark
SQL, Spark Streaming, and MLlib
Use one programming paradigm instead of mixing and
matching tools like Hive, Hadoop, Mahout, and Storm
Learn how to deploy interactive, batch, and streaming
applications
Connect to data sources including HDFS, Hive, JSON, and S3
Master advanced topics like data partitioning and shared
variables
■
■
■
■
■
Holden Karau,
a software development engineer at Databricks, is active in open
source and the author of
Fast Data Processing with Spark
(Packt Publishing).
Andy Konwinski,
co-founder of Databricks, is a committer on Apache Spark and
co-creator of the Apache Mesos project.
Patrick Wendell
is a co-founder of Databricks and a committer on Apache Spark.
He also maintains several subsystems of Spark’s core engine.
Matei Zaharia,
CTO at Databricks, is the creator of Apache Spark and serves as
its Vice President at Apache.
PROGR AMMING L ANGUAGES/SPARK
Twitter: @oreillymedia
facebook.com/oreilly
US $39.99
CAN $45.99
ISBN: 978-1-449-35862-4
Learning Spark
Holden Karau, Andy Konwinski, Patrick Wendell, and
Matei Zaharia
Learning Spark
by Holden Karau, Andy Konwinski, Patrick Wendell, and Matei Zaharia
Copyright © 2015 Databricks. All rights reserved.
Printed in the United States of America.
Published by O’Reilly Media, Inc., 1005 Gravenstein Highway North, Sebastopol, CA 95472.
O’Reilly books may be purchased for educational, business, or sales promotional use. Online editions are
also available for most titles (http://safaribooksonline.com). For more information, contact our corporate/
institutional sales department: 800-998-9938 or
corporate@oreilly.com.
Editors:
Ann Spencer and Marie Beaugureau
Production Editor:
Kara Ebrahim
Copyeditor:
Rachel Monaghan
Proofreader:
Charles Roumeliotis
Indexer:
Ellen Troutman
Interior Designer:
David Futato
Cover Designer:
Ellie Volckhausen
Illustrator:
Rebecca Demarest
February 2015:
First Edition
Revision History for the First Edition
2015-01-26:
2015-03-27:
2015-05-08:
First Release
Second Release
Third Release
See
http://oreilly.com/catalog/errata.csp?isbn=9781449358624
for release details.
The O’Reilly logo is a registered trademark of O’Reilly Media, Inc.
Learning Spark,
the cover image of a
small-spotted catshark, and related trade dress are trademarks of O’Reilly Media, Inc.
While the publisher and the authors have used good faith efforts to ensure that the information and
instructions contained in this work are accurate, the publisher and the authors disclaim all responsibility
for errors or omissions, including without limitation responsibility for damages resulting from the use of
or reliance on this work. Use of the information and instructions contained in this work is at your own
risk. If any code samples or other technology this work contains or describes is subject to open source
licenses or the intellectual property rights of others, it is your responsibility to ensure that your use
thereof complies with such licenses and/or rights.
978-1-449-35862-4
[LSI]
Table of Contents
Foreword. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix
Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi
1. Introduction to Data Analysis with Spark. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
What Is Apache Spark?
A Unified Stack
Spark Core
Spark SQL
Spark Streaming
MLlib
GraphX
Cluster Managers
Who Uses Spark, and for What?
Data Science Tasks
Data Processing Applications
A Brief History of Spark
Spark Versions and Releases
Storage Layers for Spark
1
2
3
3
3
4
4
4
4
5
6
6
7
7
2. Downloading Spark and Getting Started. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
Downloading Spark
Introduction to Spark’s Python and Scala Shells
Introduction to Core Spark Concepts
Standalone Applications
Initializing a SparkContext
Building Standalone Applications
Conclusion
9
11
14
17
17
18
21
iii

Plik z chomika:

Inne pliki z tego folderu:

Inne foldery tego chomika:

Zgłoś jeśli naruszono regulamin