Lex: a small program to count words in the input

compilation, compiler-construction, lex, yacc

Solution

You need a rule to deal with 'uninteresting' characters; you still need to count them.

You don't want to reject newlines.

You don't need the trailing context on the definition of `word`. You should probably include capital letters as `character`.

This seems to work:

%{
#include <stdio.h>
#include <stdlib.h>
int cno = 0, wno = 0, lno = 0; /*counts of characters, words and lines */
%}

character [a-zA-Z]
digit [0-9]
word ({character}|{digit})+
line \n

%%

{line} { cno++; lno++; }
{word} { wno++; cno += strlen(yytext); }
. { cno++; }

%%

int main(void)
{
    yylex();
    printf("Number of characters: %d; ", cno);
    printf("Number of words:      %d; ", wno);
    printf("Number of lines:      %d\n", lno);
    return 0;
}

When run on its own source code, the output was:

Number of characters: 463; Number of words:      65; Number of lines:      27

The standard `wc` command (which has a different definition of 'word') yields:

  27      73     463 xyz.l

This agrees on the number of lines and characters.

Problem

I'm extremely new to Lex and the complete requirement of this problem is as follows: Write a Lex input file that will produce a program that counts characters, words, and lines in a text file and reports the counts. Define a word to be any sequence of letters and/or digits, without punctuation or spaces. Punctuation and white space do not count as words. Now I've written down the code: ``` %{ #include <stdio.h> #include <stdlib.h> int cno = 0, wno = 0, lno = 0; /*counts of characters, words and lines */ %} character [a-z] digit [0-9] word ({character}|{digit})+[^({character}|{digit})] line \n %% {line} { lno++; REJECT; } {word} { wno++; REJECT; } {character} { cno++; } %% void main() { yylex(); fprintf(stderr, "Number of characters: %d; Number of words: %d; Number of lines: %d\n", cno, wno, lno); return; } ``` I tested it with the text file: ``` this is line #1 line #2 is here !@#$%^&*() haha hey hey ``` And I got the output ``` #1 #2 !@#$%^&*() Number of characters: 30; Number of words: 45; Number of lines: 4 ``` But the correct output should be ``` Number of characters: 30; Number of words: 11; Number of lines: 4 ``` I guess the error of "number of words" should be somehow due to every count of characters, so how should I modify my program to tackle with this? Also, there're some unnecessary output coming out (those punctuations). How should I modify my program to avoid them? Thank you very much.

Original source