Pergunta de entrevista da empresa HCLTech

Explain Spark Context in detail

Resposta da entrevista

Sigiloso

12 de mai. de 2025

Its the gateway/entrypoint to all spark functionalities> its an object in the Spark Core API that allows your Spark application to access the cluster resources and perform operations like: Creating RDDs (Resilient Distributed Datasets) Loading external data Configuring Spark settings Managing job execution # Create a SparkContext sc = SparkContext("local", "MyApp") # Create an RDD from a list data = [1, 2, 3, 4, 5] rdd = sc.parallelize(data) # Perform a transformation and action squared = rdd.map(lambda x: x * x) print(squared