All topics
Data · Learning hub

PySpark notes for developers

Master PySpark with a curated set of 2 developer notes — core concepts, patterns, and interview prep. Maintained by the DevRecall team.

Save this stack to your DevRecallTest yourself — PySpark quizMore Data notes
01 · PySpark

RDDs, SparkSession & Basic Operations

RDDs, SparkSession & Basic Operations Getting Started from pyspark.sql import SparkSession spark = SparkSession.builder.appName('MyApp').getOrCreate() sc = spark.sparkContext # underlying SparkContext, used for RDDs/broa…

Read
02 · PySpark

Broadcast Variables, Accumulators & Performance

Broadcast Variables, Accumulators & Performance Broadcast Variables & Accumulators # Broadcast — cached ONCE on every worker, instead of re-shipped per task country_codes = sc.broadcast({'US': 'United States', 'UK': 'Uni…

Read

Related stacks

AirflowApache KafkaApache SparkBigQueryHadoopJupyterJupyterLabKerasKeras TunerKibanaLogstashMachine Learning
Browse all stacks →

Keep your PySpark knowledge sharp.

Save this stack to your personal DevRecall — add your own notes, track what you're learning, and share what you know with the community.

Get started — free forever