How do I write effectively to a file in F#?

f#

Solution

I came up with the following solution:

namespace FSharpBasics

module Program2 =

    open System
    open System.IO
    open System.Diagnostics

    let seqTest count : seq<string> =
        let template = "<author>\
                    <name>Name {0}</name>\
                    <age>{0}</age>\
                    <books>\
                    <book>Book {0}</book>\
                    </books>\
                    </author>"

        let row (i: int) =
            String.Format (template, i)

        seq {
            yield "<authors>"
            for x in [ 1..count ] do
                yield row x
            yield "</authors>"
        }

    [<EntryPoint>]
    let main argv =
        printfn "File will be written now"

        let stopwatch = Stopwatch.StartNew()
        File.WriteAllLines (@".\test.xml", seqTest 10000) |> ignore
        stopwatch.Stop()

        printf "Ended, took %f seconds" stopwatch.Elapsed.TotalSeconds

        System.Console.ReadKey() |> ignore

        0

It takes less than 90 milliseconds on my laptop to create a well-formed test.xml file with 10,000 authors.

Problem

I want to generate large xml files for testing purpose but the code I ended up with is really slow, the time grows exponentially with the number of rows I write to the file. Th example below shows that it takes milliseconds to write 100 rows, but it takes over 20 seconds to write 1000 rows (on my machine). I really can't figured out what is making this slow, since I think that writing 1000 rows shouldn't take that long. Also, writing 200 rows takes about 4 times as long as writing 100 rows which is not good. To run the code you might want to change the path for the `StreamWriter`. ``` open System.IO open System.Diagnostics let xmlSeq = Seq.initInfinite (fun index -> sprintf "<author><name>name%d</name><age>%d</age><books><book>book%d</book></books></author>" index index index) let createFile (seq: string seq) numberToTake fileName = use streamWriter = new StreamWriter("C:\\tmp\\FSharpXmlTest\\FSharpXmlTest\\" + fileName, false) streamWriter.WriteLine("<startTag>") let rec internalWriter (seq: string seq) (sw:StreamWriter) i (endTag:string) = match i with | 0 -> (sw.WriteLine(Seq.head seq); sw.WriteLine(endTag)) | _ -> (sw.WriteLine(Seq.head seq); internalWriter (Seq.skip 1 seq) sw (i-1) endTag) internalWriter seq streamWriter numberToTake "</startTag>" let funcTimer fn = let stopWatch = Stopwatch.StartNew() printfn "Timing started" fn() stopWatch.Stop() printfn "Time elased: %A" stopWatch.Elapsed (funcTimer (fun () -> createFile xmlSeq 100 "file100.xml")) (funcTimer (fun () -> createFile xmlSeq 1000 "file1000.xml")) ```

Original source

Related problems