How-to remove non-ascii characters and append a space in the field where the non-ascii characters were using a Perl one-liner?

Question

Hi Stack Overflow community, I have the following problem. I got this file called `bad`, with the following contents: ``` SPAM EATER PO BOX 5555 FAKE STREET FOO BAR Ã¬PO BOX 1234 LOLLERCOASTER VILLAGE LOL MAN PO BOX 9876 NEXT DOOR ``` I want to remove the non-ascii character from it (at the start of the second column of the second record), in order to get a file free of strange characters and with all its columns aligned. Plus, there's this one requirement to achieve this using a Perl one-liner - so, no `awk`, `sed`, or alike commands can be used. I tried the following, but got short by one space in the third column: ``` $ perl -plne 's/[^[:ascii:]]//g' bad > bad.clean $ cat bad.clean SPAM EATER PO BOX 5555 FAKE STREET FOO BAR PO BOX 1234 LOLLERCOASTER VILLAGE LOL MAN PO BOX 9876 NEXT DOOR ``` I also tried using the same one-liner, but this time replacing the non-ascii character by a space. In this case, the record ended up with two extra spaces in the second column, and one extra spa…

Accepted Answer

Take out 2 non-ascii, add one space after field. Uses non-ascii and 3 spaces as delimiter pairs.

```
 #  s/[^[:ascii:]]{2}(.*?[ ]{3})/$1 /g

[^[:ascii:]]{2} 
 ( .*? [ ]{3} )
```

Perl test case

```
$/ = undef;
$str = <DATA>;
$str =~ s/[^[:ascii:]]{2}(.*?[ ]{3})/$1 /g;
print $str;

__DATA__
SPAM EATER       PO BOX 5555          FAKE STREET
FOO BAR          Ã¬PO BOX 1234         LOLLERCOASTER VILLAGE
LOL MAN          PO BOX 9876          NEXT DOOR
```

Output >>

```
SPAM EATER       PO BOX 5555          FAKE STREET
FOO BAR          PO BOX 1234          LOLLERCOASTER VILLAGE
LOL MAN          PO BOX 9876          NEXT DOOR
```

How-to remove non-ascii characters and append a space in the field where the non-ascii characters were using a Perl one-liner?

Solution

Problem