Description
Gain the key language concepts and programming techniques of Scala in the context of big data analytics and Apache Spark. The book begins by introducing you to Scala and establishes a firm contextual understanding of why you should learn this language, how it stands in comparison to Java, and how Scala is related to Apache Spark for big data analytics. Next, youll set up the Scala environment ready for examining your first Scala programs. This is followed by sections on Scala fundamentals including mutable/immutable variables, the type hierarchy system, control flow expressions and code blocks. The author discusses functions at length and highlights a number of associated concepts such as functional programming and anonymous functions. The book then delves deeper into Scals powerful collections system because many of Apache Sparks APIs bear a strong resemblance to Scala collections. Along the way youll see the development life cycle of a Scala program. This involves compiling and building programs using the industry-standard Scala Build Tool (SBT). Youll cover guidelines related to dependency management using SBT as this is critical for building large Apache Spark applications. Scala Programming for Big Data Analytics concludes by demonstrating how you can make use of the concepts to write programs that run on the Apache Spark framework. These programs will provide distributed and parallel computing, which is critical for big data analytics. What You Will Learn See the fundamentals of Scala as a general-purpose programming language Understand functional programming and object-oriented programming constructs in Scala Use Scala collections and functions Develop, package and run Apache Spark applications for big data analytics Who This Book Is For Data scientists, data analysts and data engineers who intend to use Apache Spark for large-scale analytics. Irfan Elahi is a senior consultant in Deloitte Australia specializing in big data and machine learning. His primary focus lies in using big data and machine learning to support business growth with multifaceted and strong ties to the telecommunications, energy, retail and media industries. He has worked on a number of projects in Australia to design, prototype, develop, and deploy production-grade big data solutions in Amazon Web Services (AWS) and Azure to support a number of use-cases ranging from enterprise data warehousing, ETL offloading, analytics, batch processing and stream processing while employing leading commercial Hadoop solutions such as Cloudera and Hortonworks. He has worked closely with clients’ systems and software engineering teams using DevOps to enhance the continuous integration and continuous deployment (CICD) processes and manage a Hadoop cluster’s operations and security. In addition to his technology competencies, Irfan has recently presented at the DataWorks Summit in Sydney on the subject of in-memory big data technologies and in a number of meetups all around the world. He also remains involved delivering knowledge-transfer sessions, training and workshops about big data and machine learning, both within his firm and at clients. He also has launched Udemy courses on Apache Spark for big data analytics and R programming for data science with more than 18,000 students from 145 countries enrolled. . Chapter 1 – Scala Language o Introduction o Getting to know Scala o Why learn Scala o Scala and Java o Interoperability with Java Libraries o Verbosity – Scala and Java o Scala – A statically typed language o Apache Spark and Scala o Scala performance benefits o Learning Apache Spark . Chapter 2 – Installing Scala o Introduction o Checking Scala installation status in your system o Verifying Java development kit (JDK) installation status o Installing Scala in Windows o Verifying Scala installation status o Exercise . Chapter 3 – Using Scala Shell o Introduction o Getting help in Scala shell o Hello world in Scala REPL o Understanding hello world in Scala REPL step by step o Real life example: Usefulness of Scala REPL’s data type highlighting feature o Paste mode in Scala REPL o Retrieving history in Scala REPL o Auto-completion feature of Scala REPL o Exiting from Scala REPL o Exercise . Chapter 4 – Variables o Introduction o Immutability of objects in Scala o Defining variables (Mutable and Immutable) in Scala o Why Immutability Is so emphasized in Scala? o Mutability and type-safety caveats o Specifying types for variables and type inference o Exercise . Chapter 5 – Data Types o Introduction o Exercise – Data types o Boolean type o Exercises – Boolean type o String type o Exercise – string types o Special types in Scala o Type casting in Scala o Exercise – special types . Chapter 6 – Conditional Statements o Introduction o Caveats – Using {} after if/else o Nested If-else statements o If else as ternary operator o Pattern matching o Exercise . Chapter 7 – Code Blocks o Introduction o Caveats – Code Blocks o Code Blocks and if/else statements o Exercise . Chapter 8 – Functions o Introduction o Why use Functions at all? o Intuitive understanding of Functions o Invoking a Function o Caveats – Function definition o Functions With multiple parameters o Positional parameters o Default value of parameters in Functions o Function with no arguments aka 0 Parity o Single line functions o When To actually use return statements o Passing function as arguments o Anonymous Functions . Chapter 9 – Scala collections o Introduction o Real life and intuitive examples of collections o Lists o Indexing list elements o What can you store in lists? o Widely used lists operations o Iterating over list o Using Map Function for iterating over lists o Getting to know Functional Programming concepts o Using foreach on lists o Using filter on lists o Reduce operation on lists o List equality check o Alternative ways to create lists o Exercise – lists o Sets o Map collections o Indexing a map o Alternative ways lo create map collections o Manipulating maps o Iterating through Maps in Functional Style o Tuples o Indexing tuples o Iterating over tuples o Alternative ways to create tuples o Mutable collections o Implications related to mutable collections o Mutable maps o Nested collections . Chapter 10 – Loops o Introduction o Types of loops in Scala o Guards in for Loop o While Loop o Comparison of for and while Loop: Which one suits well in what scenarios? . Chapter 11 – Using classes and packages o Introduction o Classes and objects in Scala o Mutating attribute values and caveats o Singleton objects o Classes and packages o Importing packages o Exercise . Chapter 12 – Exception Handling o Introduction o Fundamentals of Exception Handling in Scala o Implications in Type Inference and Exception Handling o Exercise – Exception Handling . Chapter 13 – Development Lifecycle in Scala o Introduction o Fundamentals of development Lifecycle o Introducing Scala Build Tool (SBT) o Directory Structure for SBT o Dependency Management via SBT o Compiling via SBT o Building and Packaging via SBT o Exercise – Packaging Apache Spark Applications . Chapter 14 – Hello World in Apache Spark o Development Environment for Apache Spark Development o Instantiating Spark Session and Context Object Using OOP Concepts o Using Spark Context Object’s Functions to Create Spark-Native Data Structure (RDD) o Using RDD’s Transformations Employing Functional Programming and Scala Collection Concepts o Employing Scala Functions Concepts in Spark RDD’s Transformations
