The thermostat slows down the flow of water through the radiator, to give it a chance to cool down before it returns to the engine. Removing the thermostat will cause a free flow of hot water through the radiator, which will return to the engine at nearly the same temperature and cause the engine to overheat in a short amount of time.
Without a thermostat the water flows so fast thru the engine that the heat transfer to the water is reduced significantly, I worry about the cylinder temps whenever I see a motor without the thermostat installed.
That should be correct, spring side toward the engine. You can pull up a parts list on most of the Honda sites on the internet and it has a pretty good drawing. The trick is finding what section the part you are looking for is hiding in, I found your thermostat along with the fuel pump drawings.