Ndtyjky

import findspark

findspark.init('C:spark')

from pyspark.sql import SparkSession



spark = SparkSession.builder.getOrCreate()



a = 

i=1880

while i<2018:

    a.append(str(i)+'/'+str(i)+'verr.csv')

    i = i+1



dataset1 = spark.read.format('csv').option('header','true').load('C://venq/uyh/'+ a)

i run code and i get the error;
dataset1 = spark.read.format('csv').option('header','true').load('C://venq/uyh/'+ a)
TypeError: can only concatenate str (not "list") to str

i have a "C:venquyh18801880verr.csv" format a loop.
I have csv files in nested folders. I want to read them all with spark. however I get the following error. How can I solve this? thanks

variable 'a' is a list of files.

dataset1 = spark.read.format('csv').option('header','true').load('C://venq/uyh/'+ a)

Here you are trying to concatenate a string 'C://venq/uyh/' with 'a' which is a list - which throws the error. Try

root = r"C://venq/uyh/"



while i<2018:

    a.append(root + str(i)+'/'+ str(i)+'verr.csv')

    i = i+1

and then use a directly

dataset1 = spark.read.format('csv').option('header','true').load(a)