parsing a string in python: how to split newlines while ignoring newline inside quotes

parsing, python, regex

Solution

Here's a much easier solution.

Match groups of `(?:"[^"]*"|.)+`. Namely, "things in quotes or things that aren't newlines".

Example:

import re
re.findall('(?:"[^"]*"|.)+', text)

NOTE: This coalesces several newlines into one, as blank lines are ignored. To avoid that, give a null case as well: `(?:"[^"]*"|.)+|(?!\Z)`.

The `(?!\Z)` is a confusing way to say "not the end of a string". The `(?!` `)` is negative lookahead; the `\Z` is the "end of a string" part.

Tests:

import re

texts = (
    'text',
    '"text"',
    'text\ntext',
    '"text\ntext"',
    'text"text\ntext"text',
    'text"text\n"\ntext"text"',
    '"\n"\ntext"text"',
    '"\n"\n"\n"\n\n\n""\n"\n"'
)

line_matcher = re.compile('(?:"[^"]*"|.)+')

for text in texts:
    print("{:>27} → {}".format(
        text.replace("\n", "\\n"),
        " [LINE] ".join(line_matcher.findall(text)).replace("\n", "\\n")
    ))

#>>>                        text → text
#>>>                      "text" → "text"
#>>>                  text\ntext → text [LINE] text
#>>>                "text\ntext" → "text\ntext"
#>>>        text"text\ntext"text → text"text\ntext"text
#>>>    text"text\n"\ntext"text" → text"text\n" [LINE] text"text"
#>>>            "\n"\ntext"text" → "\n" [LINE] text"text"
#>>>    "\n"\n"\n"\n\n\n""\n"\n" → "\n" [LINE] "\n" [LINE] "" [LINE] "\n"

Problem

I have a text that i need to parse in python. It is a string where i would like to split it to a list of lines, however, if the newlines (\n) is inside quotes then we should ignore it. for example: ``` abcd efgh ijk\n1234 567"qqqq\n---" 890\n ``` should be parsed into a list of the following lines: ``` abcd efgh ijk 1234 567"qqqq\n---" 890 ``` I've tried to it with `split('\n')`, but i don't know how to ignore the quotes. Any idea? Thanks!

Original source

Related problems