What do the status icons mean, and can I just make it green and red?

By default, Simon uses time-sensitive status icons to indicate whether tests are successful, a failure, or recovered.

These icons start off as gray upwards-facing triangles, which turn bright green () when a change occurs, and gradually fade through shades of green back to gray over time if no further changes occur. If a failure occurs, a red downwards-facing triangle () is used. When the test recovers from the failure, an orange upwards-facing triangle () is used, which also slowly fades back to gray over time.

These icons provide a lot of useful information just from their colors. But if you'd prefer to keep things simple, you can do so. Go to the General Preferences, and choose the Use simple status icons option. That will make Simon display a green icon when the test is successful, or the bright red icon when it is failing.