media_crawler | metadata indexing and searching of video containers | Search Engine library

I wrote a program to crawl a website and because of amount of link that I have to crawl I use Python multiprocessing. when my program starts everything is fine and my exceptions log very well but after 2-3 hours 2-3 or all 4 child processes use 0% CPU and cause I didn't use async my last line of the program which is log the "Done!" String does not execute! in the target function of my process pool I wrap all the code with a try/except statement so my process shouldn't be crash and if it crash I have to see some output in nohup.log file (I run this script with nohup myscript.py & in the background!). I dont know what's happening and it's really made me angry.

I searched in the internet and see someone told that use my_pool.close() after the pool statement (cause he said child processes not necessarily close after their tasks) but it didn't work either :(

my code is about 200 lines length so I cant put them all here! I summarize it for you. if you need detail in some section just tell me