Triangle - Square intersection test in 2d
c++, collision-detection, geometry, intersection, math
Solution
Compare your rectangle (or square) against each edge of the triangle, by taking the triangle's vertices and building the equation of a line for each edge, with a consistent ordering (clockwise or anti-clockwise around the triangle).
If the rectangle is entirely outside of the triangle on any edge, it does not intersect.
Test again with the rectangle's edges against the triangle.
There is potentially a boost to performance by knowing the rectangle is axis-aligned, as you can work out which corner is most likely to be inside the triangle, and test only that one, rather than testing all four corners.
Whether or not that is a win depends on implementation. Sometimes it can be faster to blindly check four coordinates rather than actually calculating the best one.
Checking the triangle against the rectangle should be easier as the line equations are simple tests against x or y when the rectangle is axis aligned.
This is a generalised form of separating axis test - finding a line or plane which separates the two objects, thus proving that they can't intersect. If you want more performance, you can find the nearest features of the two objects to work out the most appropriate line/plane to use, rather than trying all of them.
Problem
How can I test if a triangle and a square is intersecting each other? Is there any way of optimizing this, when we know it's a square instead of rectangle? Also, the square is axis aligned, so that should give some more boost to performance? Or should I just break the square into triangles, and make triangle-triangle intersection check twice? Edit: To clarify: I'm trying to check if those two shapes overlap each other in any way. So the triangle can be inside square, and square can be inside triangle, and it should return true for that too.