Pyspark

Pyspark commands for managing data analysis, manipulation, and visualization

15 commands
Search & Copy
Save Favorites
15
Total Commands
100%
Free Access
⚡
Lightning Fast

Available Commands

Click on any command to copy it instantly

15 commands available
15 of 15
pyspark

Initialize Spark

Create SparkSession to start PySpark

from pyspark.sql import SparkSession
pyspark
init
spark
pyspark

Create DataFrame

Create DataFrame from list or RDD

spark.createDataFrame()
pyspark
dataframe
create
pyspark

Read CSV

Load CSV file into DataFrame

spark.read.csv()
pyspark
read
csv
pyspark

Show DataFrame

Display rows of DataFrame

df.show()
pyspark
dataframe
view
pyspark

Print Schema

Print schema of DataFrame

df.printSchema()
pyspark
dataframe
schema
pyspark

Select Columns

Select specific columns from DataFrame

df.select("col1","col2")
pyspark
dataframe
select
pyspark

Filter Rows

Filter DataFrame based on condition

df.filter()
pyspark
dataframe
filter
pyspark

Where

Alternative to filter()

df.where()
pyspark
dataframe
where
pyspark

Group By

Group DataFrame by column(s)

df.groupBy("col")
pyspark
dataframe
aggregate
pyspark

Aggregation

Aggregate with functions like sum, avg, max

df.agg()
pyspark
dataframe
aggregate
pyspark

Order By

Sort DataFrame by column(s)

df.orderBy()
pyspark
dataframe
sort
pyspark

Join DataFrames

Join two DataFrames

df1.join(df2, on="col", how="inner")
pyspark
dataframe
join
pyspark

Drop Column

Drop column(s) from DataFrame

df.drop("col")
pyspark
dataframe
drop
pyspark

Run SQL

Execute SQL queries

spark.sql()
pyspark
sql
query
pyspark

Save DataFrame

Write DataFrame to storage

df.write.format("csv").save("path")
pyspark
dataframe
save

Missing a command?

Help us improve this collection by suggesting commands that should be added. Your contributions help the entire developer community!