0

I have a simple python flask server that is started by start-stop-daemon. It was running well last week, and this monday I see that it is not serving anymore (curl http://localhost/mypage takes forever and never returns). Last log is from friday.

The strange thing is that sudo service myService status is "running" and I see that the process is indeed running (ps aux | grep $(cat /var/run/myservice.pid))

Is there a way to know what could have happened? I have no idea where to go next from here.

Anthon
  • 79,293
Thomas
  • 893
  • Your question is not specific. – Ruban Savvy Dec 09 '13 at 08:51
  • I know, but I knew what was going on / had more details I would give them. maybe it will ring a bell to someone more skilled than me no ? – Thomas Dec 09 '13 at 09:08
  • Restart the service. That it is running doesn't mean it the process is actually listing to port 80, just that there is a PID in a file that corresponds to a running process (depending on the way it is checked it does not have to be flask, might even be another process having that number, but your ps aux.. rules that out) – Anthon Dec 09 '13 at 09:28
  • Does the service write any logs? Read them. – doug65536 Dec 09 '13 at 09:32
  • ok, I added some more logs here and there and I hope the problem will appear again, I don't like when things stop working over the night. thanks – Thomas Dec 09 '13 at 09:34
  • @Thomas check apache error logs and update here . – Rahul Patil Dec 09 '13 at 12:00
  • Thanks to all for your answers ; if you are interested in knowing what happened, see my answer below – Thomas Dec 11 '13 at 15:06

1 Answers1

0

It seems that flask (lightweight python server framework) is the culprit. I have seen many questions about why it becomes hanging.

The behaviour I was describing fits the case, as if flask entered an infinite loop or is waiting for threads to finish or for more memory or ... : the server was running but curl http://localhost... would not return but not fail either.

So nothing in the logs, no crash, process still running, hard to say what went wrong. Until I checked the number of files opened. And there were 1020 or so sockets opened, and the server couldn't open any more. That's where is the problem, still need to fix it but at least it explains the situation

Thomas
  • 893
  • after investigation : using pymongo to access mongodb and not closing properly the connection. So the too many opened sockets were internal sockets, not external ones. Problem solved http://stackoverflow.com/questions/20540129/ever-increasing-number-of-opened-sockets-on-python-web-server/20558448 – Thomas Dec 13 '13 at 14:30