Personally, I wouldn’t worry about it. Many of us run our RPis without fans and without issues (though I can’t say that no one has had issues). Don’t let that stop you if you just want to do it for the heck of it, though.
The bigger concern is power outages. If you don’t already have one, I recommend getting a UPS.