Detecting Sudoku Square in an Image

I would like to ask another question related to How to remove convexity defects in a Sudoku square? I implemented a Sudoku Solver in OpenCV-Python and it work pretty good. But at this time, i made an assumption which will simplify my problem. Assumption is : `Sudoku boundary(square) is the biggest blob in the input image` Eg: The red block is the detected square. You can see it covers the major part of the image. The Problem: Problem is nothing but the assumption. `If sudoku square has another square around it, method fails. Or if the image has another bigger blob than the sudoku square, again method fails.` For eg, take this image. (I don't want to upload original image here, it is pretty big, also i want to upload the result anyway) I tried several following methods to find sudoku square in this image: 1) `Find the biggest blob` Then I got the red coloured region. `Method Failed`. (Image is resized from original image to reduce size) 2) `Find only square regions` You can see a lot of candidates are there, especially `KING CROSSWORD` which is all way similar to Sudoku. `Method again fails` `In short, this image has everything, to fail me.` Question: `How to detect a sudoku square in an image, especially in the test image i gave? Is there any better algorithm for this?` UPDATE: After reading some answers and comments, i think i should give an update. Look at the below image: This image has a question sudoku and previous sudoku's answer. Both are alike. I think searching for sub-blocks or OCR testing won't work here.