I am looking for a good algorithm to find an axis-aligned rectangle inside a (not necessarily convex) polygon. A maximal rectangle would be nice, but is not necessary - any algorithm that can find a "fairly good" rectangle would be fine.
The polygon may also have holes, but any pointers to algorithms that only work for convex or simple polygons would be helpful too.
In my implementation, intersection testing for sides is fairly cheap, but "point in polygon" tests are expensive, so ideally should be minimised.
What about using ear clipping? You could find the maximum axis-aligned rectangle in each triangle. Then you could try to join triangles and recompute your rectangles.
http://cgm.cs.mcgill.ca/~athens/cs507/Projects/2003/DanielSud/
Has an algorithm for convex, the references might be worth a look.
not sure if it could be extended to non-convex though.
Just for reference, so far all I have is brute force: make a grid, and for points on the grid, if they are inside the polygon, make a series of rectangles by expanding each corner or side in turn until it hits a side. Then just pick the largest.
The simplest (and very effective) optimisation is only to test whether a grid point is in the polygon once you have checked that it is not contained in one of the rectangles already constructed, as 'point in rectangle' checking is blazing fast.
For obvious reasons, this is fairly slow and inexact, not to mention inelegant :)
One solution is to split the concave polygon into convex segments then use cobbal's link.
Since you really have two different fundamental problems, have you considered other alternatives to the hit test problem, such as using a BSP tree? You can speed that up further by laying a grid over the poly and constructing a BSP tree for each grid square. Or a kd-tree with at most one edge in each leaf?
Edit: I'll ellaborate on the kd-tree (out of boredom, even if it might be of any use to anyone):
kd-trees have the following properties:
To use this for the polygon hit detection, construct the tree as follows:
If the split vertex is chosen appropriately, the tree should have depth close to log(N), where N is the number of vertices. Each leaf node will have at most one edge going through it. To do the hit detection: