I am trying to write a script that takes multiple .csv files and stitches them together. Does anyone know how I can take hundreds of files in a folder and put them in a list? (would be the list called filenames in this) import csv filenames=['qwe.csv', 'asd.csv', 'zxc.csv']
for z in range(1,len(filenames)):
fields = [] rows = []
with open(filenames[z], 'rU') as csvfile: csvReader = csv.reader(csvfile)
for row in csvReader: rows.append(row)
with open(filenames[0], 'a') as csvMainfile: csvWriter = csv.writer(csvMainfile)
for x in range(1, len(rows)): csvWriter.writerow(rows[x])
if you wanted to not do it in python you could shell script it: touch allMyCSV.csv; for FILE in `ls -1v theDirOfYourCSVs`; do cat $FILE >> allMyCSV.csv; done or something like that. probably faster than python. but you seem to be using some sort of "csv.writer" class which sounds like there's a real python package for handling csv, so use that, not hacky shit like shellscripting
Benjamin Barnes
I'm an environmental engineer and only took matlab in school. How is this terrible?
Evan Jenkins
this is a typical Jow Forums larper who probably will suggest "the only pure, minimalistic way to do this is to implement it in C and then optimize the algorithm in the assembly. because python taking 1MB of memory is MUH BLOAT"
there are plenty of this kind of retard around here. for a simple task like this, i commend you for using python, the best scripting language.
Chase Murphy
>thinking the retard is just a larper pretending to be retarded Actually user, I'm just a retard that learned to use python last week
Adam Collins
yes, glob.glob() returns a python list with the format ['fileA', 'fileB', 'fileC', ... in normal shell-style alphabetical order
Mason Miller
No, I mean the script is not pythonic (eg. using range() instead of iterating directly over the elements with an optional enumerate()) and storing all rows in memory before writing. This should have O(1) memory complexity if you use iterators properly.
just be a man and don't download a special package for every simple task. that way you'll learn to code.
already there are two DIY solutions here
Angel Clark
powershell is the future my man
Dominic Scott
>using python for something bash does in 2 lines
Pajeet pls go
Jose Moore
Just started using python, wanted to learn the hard way
Colton Reed
python is unironically the easy way, not the hard way. just compare my one-line answer: (not two line like dr. asshat says) with your almost-there answer (just needs a glob.glob inserted) the bash one-liner looks like fucking nonsense with a crapload of $'s and backticks (`) and weird command-line options, whereas the python is actually readable -- and it works cross-platform
stick with python and ignore the Jow Forums larpers
Ryder Nguyen
thanks for the advice. This will sound completely retarded but I dont really understand how to terminal on mac (and what bash even means). Just starting to learn coding for practical application so I'm probably coming across as an idiot, appreciate the advice
David Richardson
on mac you can use spotlight to search for "Terminal"
that opens up a Terminal.app window which, by default, gives you a "bash shell", which means it gives you a command-line interface for running commands. (and bash uses vanilla "shell language" instead of alternatives you sometimes find like "c-shell" language in e.g. the tcsh shell.)
once you open terminal you can try the commands "ls" and "cd" to navigate through directories; then once you find your directory of CSVs you could try the one-liner here:
Jose Adams
thanks a lot, that was actually really helpful man
