I am working through reading and writing from text and csv files.
The script below successfully reads input.txt into a string and echoes it to the screen. It then writes the what was read in to an output.txt file.
It also reads in a third file called input.csv into a Str[] and echoes the first first three rows to the screen delaminated correctly.
It is the last four lines of the block below where I try to read in the same input.csv file into a 2D Str[][] that I get this error:
Unknown slot 'sys::File.CsvInStream'
I realize I am using the util pod and not the sys pod and using CsvInStream and not InStream (in). I read the docs, examples, and the source in the docs....
I have a feeling this is a syntax issue but I cannot figure it out. It is my second day learning Fantom and I need some help. I really enjoy Fantom...what an elegant language with very intelligent syntactic sugar.
Respectfully, Space Ghost
class ReadString
{
public static Void main (Str[] args)
{
Uri fileuri := `./src/input.txt`
echo(fileuri)
echo(fileuri.typeof)
Str contents := File(fileuri).readAllStr
echo ("contents: $contents")
Uri fileuri2 := `./src/output.txt`
fileuri2.toFile.out(false).printLine(contents).close
Uri fileuri3 := `./src/input.csv`
Str[] contentscsv := File(fileuri3).in.readAllLines
echo; echo(contentscsv[0..2])
Uri fileuri4 := `./src/input.csv`
Str[][] contentscsv2 := File(fileuri4).CsvInStream.readAllRows
echo;echo
echo(contentscsv2[4])
}
}
HenrySat 17 Dec 2022
Hi Space Ghost!
To Construct the CsvInStream class you need to call the make constructor with an InStream, so it's not possible to create it inline using the dot operator as you've shown here.
It's only a small Syntactical change to get this working, try something like:
Uri fileuri4 := `./src/input.csv`
Str[][] contentscsv2 := CsvInStream(File(fileuri4).in).readAllRows
echo;echo
echo(contentscsv2[4])
For some context on the compilation error you're seeing, it's because of the dot operator, it's looking for a method or field on File called CsvInStream (Which doesn't exist!)
Hope that helps!
SpaceGhostSat 17 Dec 2022
Hi Henry, thank you so much for the reply. Your explanation makes sense and below is the code change. It does not work , rather this provides another compilation error.
I used the dot operator .readAllRows at the end of line(12) which is a slot method for the CsvInStream class to read in the wrapped stream.
The (File(fileuri2)in) in line(12) is the InStream that the CsvInStream should use make to wrap it and make an instance (object) of a CsvInStream that we can read in with the appropriate method....right? BTW, lines (6), (7), and (8) work perfectly but they are part of the sys::File abstract const class and not the util::CsvInStream class
Here is the compilation error which I commented in above the suspect line (12) of code as well:
I tried a half dozen variations of this with no luck??? I read the source multiple times and this all looks like it should work.... hope someone can help.
1 class ReadCsv
2 {
3 public static Void main (Str[] args)
4 {
5 // the two lines below work and echo csv contents as Str[]
6 Uri fileuri := `./src/input.csv`
7 Str[] contents := File(fileuri).readAllLines // convenience for .in.readAllLines, guarenteed close of stream
8 echo(contents)
9
10 Uri fileuri2 := `./src/input.csv`
11 // the line below produce this error: Unknown method 'readcsv_0::ReadCsv.CsvInStream' ERROR: cannot compile script
12 Str[][] contents2 := CsvInStream(File(fileuri2).in).readAllRows
13 echo(contents2)
14 }
15 }
SlimerDudeSat 17 Dec 2022
Hi SpaceGhost,
By default, the Fantom compiler only knows about classes in the sys pod. But CsvInStream is a class in the util pod - so we have to tell the compiler where to find it. Do this with a using statement:
using util::CsvInStream
class ReadCsv {
public static Void main (Str[] args) {
...
...
}
}
One new question below is (5). The rest is a thank you and showing what worked to benefit future readers of this thread.
First off, thank you to SlimerDude and Henry! Your both really helped me get over the hump...and with a little more sweat-and-reading of the doc library source and reviewing the * Examples * it is all sorted out. For my next step, I am going to do binary read/write next, LOL....so may run into some questions again!
For the record - I spent most my life as (in order) a veteran of: BASIC, PASCAL, FORTRAN, C, D, and a touch of NIM -- all Procedural only, i.e. Java / Fantom is my first Object Oriented experience...interesting and I am going between WTH! and This is Cool! I started with Java three weeks ago and said wow, this OO is complex and then found Fantom a few days ago from SlimerDude's couple videos that read my mind on Java which has the gold standard SDK and runtime (JVM) but is a beast.
Second, I am going to post a long block below summarizing my learning, but also as a backdrop to one additional question in (5) below.
You will see below that I was able to read/write my Str[][] list using the util:CsvIn(Out)Stream class and methods. To write the Str[][] back to disk I used a for loop with iterator and cycled through the size of the Str[][].
My question is there a way or pod::Class.method that I am missing to do this in one "gulp"?, i.e. without using a for loop at the end of the .fan file.
For below: I will first paste the:
(a) input.txt file which is of course the same as the input.csv file you see the src opening up steams to,
(b) then paste the .fan file,
(c) then I will paste in the output.txt, and
(d) outputcsv.txt files so you can see that the former (with sys::Uri.toFile.out.printLine) done by line has blanks (between comma deliminator) for the cells that were missing in the original input data files (I did this on purpose to see how the methods handled them and they did great), and the latter (with util::CsvOutStream.out.writeRow) inserts the blanks as "" in the disk file.
Respectfully, SpaceGhost
(a) INPUT.TXT (INPUT.CSV is the same, I just changed the file suffix) FILES
Regarding your question, The CsvOutStream class unfortunately doesn't contain a method like writeRows or something that would be able to write the csv wholesale, but I would instead recommend that you first construct an instance of CsvOutStream with the File.out and then simply loop over your list of contentscsv2 using the List.each method, calling writeRow with each item in the list.
This way you save the requirement of calculating the size of the list to loop over (as each simply iterates over each item in the list) and also means you will only have to create and open/close the one instance of the CsvOutStream for writing!
One other tiny think is that generally, unless you need the Uri itself for something else, when working with File IO, I'd recommend just constructing the File instances inline with the uri (but that's only really a preference for code cleanliness!)
There will be some OutStream subclasses that will allow a full write of rows, though they tend to primarily appear in the haystack classes for SkySpark, where they make heavy use of Grids, and more List-oriented objects. But in essence what they will do behind the scenes is just the same as calling an each on a list of items and writing them one by one.
SpaceGhostSat 17 Dec 2022
Henry, you are so right. With the open and close for a large CSV the script took 73 seconds. By moving it outside the for-each or for-next loop we dropped below 3 seconds!. FYI, the .size read for row and column on took literally 1ms more time.
I just posted #2882 (https://fantom.org/forum/topic/2882)with some very cool results using your suggestion and then I did the same with a for-next in Fantom. I compared these results with a huge csv to Basic (and indirectly with C,D,and Fortran which were slightly slower than Basic). Take a look...pretty neat.
SpaceGhost Sat 17 Dec 2022
Henry Sat 17 Dec 2022
Hi Space Ghost!
To Construct the CsvInStream class you need to call the
make
constructor with anInStream
, so it's not possible to create it inline using the dot operator as you've shown here.It's only a small Syntactical change to get this working, try something like:
For some context on the compilation error you're seeing, it's because of the dot operator, it's looking for a method or field on
File
calledCsvInStream
(Which doesn't exist!)Hope that helps!
SpaceGhost Sat 17 Dec 2022
Hi Henry, thank you so much for the reply. Your explanation makes sense and below is the code change. It does not work , rather this provides another compilation error.
.readAllRows
at the end of line(12) which is a slot method for theCsvInStream
class to read in the wrapped stream.(File(fileuri2)in)
in line(12) is the InStream that theCsvInStream
should usemake
to wrap it and make an instance (object) of aCsvInStream
that we can read in with the appropriate method....right? BTW, lines (6), (7), and (8) work perfectly but they are part of thesys::File
abstract const class and not theutil::CsvInStream
classSlimerDude Sat 17 Dec 2022
Hi
SpaceGhost
,By default, the Fantom compiler only knows about classes in the
sys
pod. ButCsvInStream
is a class in theutil
pod - so we have to tell the compiler where to find it. Do this with ausing
statement:See Using in CompilationUnits for more details.
SpaceGhost Sat 17 Dec 2022
One new question below is (5). The rest is a thank you and showing what worked to benefit future readers of this thread.
SlimerDude
andHenry
! Your both really helped me get over the hump...and with a little more sweat-and-reading of the doc library source and reviewing the * Examples * it is all sorted out. For my next step, I am going to do binary read/write next, LOL....so may run into some questions again!Str[][]
list using the util:CsvIn(Out)Stream class and methods. To write theStr[][]
back to disk I used a for loop with iterator and cycled through the size of theStr[][]
.input.txt
file which is of course the same as theinput.csv
file you see the src opening up steams to,output.txt
, andoutputcsv.txt
files so you can see that the former (withsys::Uri.toFile.out.printLine
) done by line has blanks (between comma deliminator) for the cells that were missing in the original input data files (I did this on purpose to see how the methods handled them and they did great), and the latter (withutil::CsvOutStream.out.writeRow
) inserts the blanks as "" in the disk file.Respectfully, SpaceGhost
(a) INPUT.TXT (INPUT.CSV is the same, I just changed the file suffix) FILES
(b) .fan FILE
(c) output.txt
(d) outputcsv.txt
End of Reply
Henry Sat 17 Dec 2022
Glad to see it's working!
Regarding your question, The CsvOutStream class unfortunately doesn't contain a method like
writeRows
or something that would be able to write the csv wholesale, but I would instead recommend that you first construct an instance ofCsvOutStream
with theFile.out
and then simply loop over your list ofcontentscsv2
using the List.each method, callingwriteRow
with each item in the list.Something along the lines of:
This way you save the requirement of calculating the size of the list to loop over (as
each
simply iterates over each item in the list) and also means you will only have to create and open/close the one instance of theCsvOutStream
for writing!One other tiny think is that generally, unless you need the
Uri
itself for something else, when working with File IO, I'd recommend just constructing theFile
instances inline with the uri (but that's only really a preference for code cleanliness!)There will be some
OutStream
subclasses that will allow a full write of rows, though they tend to primarily appear in the haystack classes for SkySpark, where they make heavy use of Grids, and more List-oriented objects. But in essence what they will do behind the scenes is just the same as calling an each on a list of items and writing them one by one.SpaceGhost Sat 17 Dec 2022
Henry, you are so right. With the open and close for a large CSV the script took 73 seconds. By moving it outside the for-each or for-next loop we dropped below 3 seconds!. FYI, the .size read for row and column on took literally 1ms more time.
I just posted #2882 (https://fantom.org/forum/topic/2882)with some very cool results using your suggestion and then I did the same with a for-next in Fantom. I compared these results with a huge csv to Basic (and indirectly with C,D,and Fortran which were slightly slower than Basic). Take a look...pretty neat.
Cheers!