NLTK. Find if a sentence is in a questioning form
nlp, nltk, python
Solution
One simple way to do this is to parse a sentence and look for the tag assigned to it. For example, parsing the sentence "Is there any way to do this?" with Stanford parser will return:
(ROOT
(SQ (VBZ Is)
(NP (EX there))
(NP
(NP (DT any) (JJ other) (NN way))
(S
(VP (TO to)
(VP (VB do)
(NP (DT this))))))
(. ?)))
where `SQ` denotes "Inverted yes/no question, or main clause of a wh-question, following the wh-phrase in SBARQ". Another example:
(ROOT
(SBARQ
(WHNP (WP What))
(SQ (VBZ is)
(NP
(NP (DT the) (NN capital))
(PP (IN of)
(NP (NNP Scotland)))))
(. ?)))
where SBARQ denotes "Direct question introduced by a wh-word or a wh-phrase". It's pretty straightforward to call an external parser from Python and process its output, for example check this Python interface to Stanford NLP tools.
Problem
I am trying to detect if a sentence is a question or a statement. Apart from looking for a question mark at the end of the sentence, is there another way to detect this? I am processing Twitter posts and people are not necessarily following good practises like question marks on Twitter. Reference to other libraries is also ok with me if nltk does now work.