There are a couple of different ideas that come to mind.
The first would be to use a metal block or dowel large enough to cover the CPU and stick out of the case slightly. Use a good thermal epoxy to bond it to the CPU and silicone RTV to seal the opening in the case around the block or dowel. The purpose if this is to move heat out of the case, so copper would be good because it conducts heat very well. Then you attached a large heatsink to the outside end of the block or dowel, something with a lot of surface area so that you won't need a fan. Aluminum would work well for that because it radiates heat better than copper. Use thermal epoxy to bond the heatsink to the block/dowel.
Another option would be to use a heat-pipe
type cooler. Some desktop computer, and many laptop coolers use heat-pipes to quickly move the heat away from the CPU to a heatsink located elsewhere. These can be extremely effective, but they are not something you could easily make yourself, so you would have to find an existing one that would fit your needs. Sealing the case around the heat-pipes could also prove challenging.
Suggestions above about using a Pi-Zero are also worth considering. If the software you are running doesn't require the power of a Pi3, a Zero will generate less heat. Of course it also only has a single USB port (which would be used for a WiFi dongle).
My password is the last 8 digits of Pi.