Related

Select Columns

pyspark

Select specific columns from DataFrame

Drop Column

pyspark

Drop column(s) from DataFrame

Show DataFrame

pyspark

Display rows of DataFrame

pyspark/create-df

Create DataFrame

Create DataFrame from list or RDD

pyspark
dataframe
create

Command

spark.createDataFrame()

Explanation

PySpark DataFrames are distributed collections of data.

Examples

Create DataFrame from list

df = spark.createDataFrame([(1,"A"),(2,"B")], ["id","name"])

More

Print Schema

pyspark

Print schema of DataFrame

Filter Rows

pyspark

Filter DataFrame based on condition

Group By

pyspark

Group DataFrame by column(s)