python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 8 -a ' --fifo -a 533 -l 100 ' -s datou_current_533 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 3392423 load datou : 533 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? load thcls load pdts Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec no input labels no input values updating current state to 1 we have a portfolio with more photos than limit : 414>100 please execute split_portfolio.py -i 23317092 -l 100 size over we load limit photo not treated list_input_json: {} Current got : datou_id : 533, datou_cur_ids : ['2948069'] with mtr_portfolio_ids : ['23317092'] and first list_photo_ids : [] new path : /proc/3392423/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! DataTypes for each output/input checked ! List Step Type Loaded in datou : tfhub_classification2, argmaxHTTP Error 404: Not Found can't download the photo : 1361038005 complete_path : /home/admin/workarea/git/Velours/python/temp/1748252966_3392423_1361038005_8021150ec66c24956ee33075e738d321.jpg FBHTTP Error 404: Not Found can't download the photo : 1361038004 complete_path : /home/admin/workarea/git/Velours/python/temp/1748252966_3392423_1361038004_0d48ec226515a04f46048c1fdd6cd6f9.jpg FBHTTP Error 404: Not Found can't download the photo : 1361038003 complete_path : /home/admin/workarea/git/Velours/python/temp/1748252966_3392423_1361038003_aecc23d327fdf15941ec4fbfbb89bacf.jpg FBHTTP Error 404: Not Found can't download the photo : 1361038002 complete_path : /home/admin/workarea/git/Velours/python/temp/1748252966_3392423_1361038002_a625c19dc4b2e74c5c20c4c272ab61a2.jpg FBHTTP Error 404: Not Found can't download the photo : 1361038000 complete_path : /home/admin/workarea/git/Velours/python/temp/1748252966_3392423_1361038000_17e095387a6eb2325e9f943a125ccabd.jpg FBHTTP Error 404: Not Found can't download the photo : 1361037982 complete_path : /home/admin/workarea/git/Velours/python/temp/1748252966_3392423_1361037982_7a914a128c9d304e068bb95e26471acf.jpg FBHTTP Error 404: Not Found can't download the photo : 1361037967 complete_path : /home/admin/workarea/git/Velours/python/temp/1748252966_3392423_1361037967_dee1fd2d964f3e33e323dde91988f9a8.jpg FBHTTP Error 404: Not Found can't download the photo : 1361037966 complete_path : /home/admin/workarea/git/Velours/python/temp/1748252966_3392423_1361037966_edbe1fc059e54cff1c56381e05903cdb.jpg FBHTTP Error 404: Not Found can't download the photo : 1361037965 complete_path : /home/admin/workarea/git/Velours/python/temp/1748252966_3392423_1361037965_79c9c3336a36f0fefb715a0cde09867d.jpg FBHTTP Error 404: Not Found can't download the photo : 1361037963 complete_path : /home/admin/workarea/git/Velours/python/temp/1748252966_3392423_1361037963_6e82b9e979523736c7afd2ade809c835.jpg FBHTTP Error 404: Not Found can't download the photo : 1361037944 complete_path : /home/admin/workarea/git/Velours/python/temp/1748252966_3392423_1361037944_b8d6af9c1eb096fa4505692a2b439ab4.jpg FBHTTP Error 404: Not Found can't download the photo : 1361037911 complete_path : /home/admin/workarea/git/Velours/python/temp/1748252966_3392423_1361037911_5c527628a7170cbfc0bc0d3a31ea7ea2.jpg FBHTTP Error 404: Not Found can't download the photo : 1361037909 complete_path : /home/admin/workarea/git/Velours/python/temp/1748252966_3392423_1361037909_447936e48ee01ee185cf7ff43cfcefec.jpg FBHTTP Error 404: Not Found can't download the photo : 1361037892 complete_path : /home/admin/workarea/git/Velours/python/temp/1748252966_3392423_1361037892_b2ce90374e28efb1309fbdc0089b9236.jpg FBHTTP Error 404: Not Found can't download the photo : 1361037891 complete_path : /home/admin/workarea/git/Velours/python/temp/1748252966_3392423_1361037891_1b661c0e754005ad34a379e0e98b0dcd.jpg FBHTTP Error 404: Not Found can't download the photo : 1361037874 complete_path : /home/admin/workarea/git/Velours/python/temp/1748252966_3392423_1361037874_5f9a3892f7f7c0f8ed3680250b477cf5.jpg FBHTTP Error 404: Not Found can't download the photo : 1361037872 complete_path : /home/admin/workarea/git/Velours/python/temp/1748252966_3392423_1361037872_a10f4ab65bd9a5cf1402374a04dca08a.jpg FBHTTP Error 404: Not Found can't download the photo : 1361037871 complete_path : /home/admin/workarea/git/Velours/python/temp/1748252966_3392423_1361037871_bd9484472fa858f135fdc1ff7bcc0847.jpg FBHTTP Error 404: Not Found can't download the photo : 1361037870 complete_path : /home/admin/workarea/git/Velours/python/temp/1748252966_3392423_1361037870_0ede0b07ca9b20e51b9ca5b5925da3d0.jpg FBHTTP Error 404: Not Found can't download the photo : 1361037869 complete_path : /home/admin/workarea/git/Velours/python/temp/1748252966_3392423_1361037869_0fc326fc789794a6c6ddf317817e8b17.jpg FBHTTP Error 404: Not Found can't download the photo : 1361037856 complete_path : /home/admin/workarea/git/Velours/python/temp/1748252966_3392423_1361037856_bf680bdb4563e8850c6deaad0b151437.jpg FBHTTP Error 404: Not Found can't download the photo : 1361037854 complete_path : /home/admin/workarea/git/Velours/python/temp/1748252966_3392423_1361037854_db1465948dbc755cd32e3fb2dc43aeeb.jpg FBHTTP Error 404: Not Found can't download the photo : 1361037853 complete_path : /home/admin/workarea/git/Velours/python/temp/1748252966_3392423_1361037853_3be53d248d83ff0aa83670d2d14b0cf2.jpg FBHTTP Error 404: Not Found can't download the photo : 1361037792 complete_path : /home/admin/workarea/git/Velours/python/temp/1748252966_3392423_1361037792_dc15e59b4ec0fb21c56512ba58fcba60.jpg FBHTTP Error 404: Not Found can't download the photo : 1361037791 complete_path : /home/admin/workarea/git/Velours/python/temp/1748252966_3392423_1361037791_a893e1cc4062559ebed32b1e1539cd82.jpg FBHTTP Error 404: Not Found can't download the photo : 1361037789 complete_path : /home/admin/workarea/git/Velours/python/temp/1748252966_3392423_1361037789_8c417acf79cd578fea64a33b140afe26.jpg FBHTTP Error 404: Not Found can't download the photo : 1361037788 complete_path : /home/admin/workarea/git/Velours/python/temp/1748252966_3392423_1361037788_13af02baf5da1a37e9d1db3073742f07.jpg FBHTTP Error 404: Not Found can't download the photo : 1361037771 complete_path : /home/admin/workarea/git/Velours/python/temp/1748252966_3392423_1361037771_4b19ca3a94bb642bd0b27a5375c2753d.jpg FBHTTP Error 404: Not Found can't download the photo : 1361037770 complete_path : /home/admin/workarea/git/Velours/python/temp/1748252966_3392423_1361037770_f573de54dd038fd3f8eb10900f0c8087.jpg FBHTTP Error 404: Not Found can't download the photo : 1361037767 complete_path : /home/admin/workarea/git/Velours/python/temp/1748252966_3392423_1361037767_d3620c90c9b75772053b31afcf61c848.jpg FBHTTP Error 404: Not Found can't download the photo : 1361037766 complete_path : /home/admin/workarea/git/Velours/python/temp/1748252966_3392423_1361037766_d856efc9a423505bc605101dfd595986.jpg FBHTTP Error 404: Not Found can't download the photo : 1361037763 complete_path : /home/admin/workarea/git/Velours/python/temp/1748252966_3392423_1361037763_08fc140a416faf510ecfda550e5a3854.jpg FBHTTP Error 404: Not Found can't download the photo : 1361037762 complete_path : /home/admin/workarea/git/Velours/python/temp/1748252966_3392423_1361037762_52e902013468c9ae37c0b324374e9d1f.jpg FBHTTP Error 404: Not Found can't download the photo : 1361037744 complete_path : /home/admin/workarea/git/Velours/python/temp/1748252966_3392423_1361037744_d7d9a2d09fdc2b4b31d4c64e7c01f27e.jpg FBHTTP Error 404: Not Found can't download the photo : 1361037743 complete_path : /home/admin/workarea/git/Velours/python/temp/1748252966_3392423_1361037743_4ad96109c0763dcb109260e88f18dffe.jpg FBHTTP Error 404: Not Found can't download the photo : 1361037742 complete_path : /home/admin/workarea/git/Velours/python/temp/1748252966_3392423_1361037742_527f2f95d4289bb471a47f339c00174c.jpg Fwe have missing 100 photos in the step downloads : photo missing : [1361038541, 1361038539, 1361038538, 1361038537, 1361038525, 1361038505, 1361038504, 1361038486, 1361038485, 1361038484, 1361038482, 1361038481, 1361038480, 1361038457, 1361038438, 1361038437, 1361038436, 1361038435, 1361038411, 1361038410, 1361038390, 1361038389, 1361038388, 1361038387, 1361038385, 1361038368, 1361038366, 1361038365, 1361038350, 1361038325, 1361038323, 1361038309, 1361038290, 1361038289, 1361038286, 1361038285, 1361038253, 1361038252, 1361038250, 1361038246, 1361038244, 1361038243, 1361038242, 1361038240, 1361038173, 1361038172, 1361038171, 1361038148, 1361038147, 1361038146, 1361038145, 1361038144, 1361038142, 1361038123, 1361038104, 1361038071, 1361038070, 1361038069, 1361038068, 1361038051, 1361038033, 1361038031, 1361038029, 1361038023, 1361038005, 1361038004, 1361038003, 1361038002, 1361038000, 1361037982, 1361037967, 1361037966, 1361037965, 1361037963, 1361037944, 1361037911, 1361037909, 1361037892, 1361037891, 1361037874, 1361037872, 1361037871, 1361037870, 1361037869, 1361037856, 1361037854, 1361037853, 1361037792, 1361037791, 1361037789, 1361037788, 1361037771, 1361037770, 1361037767, 1361037766, 1361037763, 1361037762, 1361037744, 1361037743, 1361037742] try to delete the photos missing in DB Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038541 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038539 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038538 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038537 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038525 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038505 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038504 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038486 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038485 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038484 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038482 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038481 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038480 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038457 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038438 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038437 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038436 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038435 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038411 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038410 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038390 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038389 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038388 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038387 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038385 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038368 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038366 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038365 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038350 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038325 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038323 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038309 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038290 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038289 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038286 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038285 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038253 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038252 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038250 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038246 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038244 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038243 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038242 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038240 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038173 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038172 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038171 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038148 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038147 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038146 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038145 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038144 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038142 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038123 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038104 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038071 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038070 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038069 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038068 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038051 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038033 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038031 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038029 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038023 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038005 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038004 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038003 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038002 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361038000 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361037982 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361037967 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361037966 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361037965 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361037963 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361037944 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361037911 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361037909 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361037892 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361037891 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361037874 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361037872 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361037871 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361037870 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361037869 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361037856 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361037854 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361037853 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361037792 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361037791 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361037789 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361037788 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361037771 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361037770 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361037767 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361037766 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361037763 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361037762 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361037744 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361037743 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361037742 length of list_filenames : 0 ; length of list_pids : 0 ; length of list_args : 0 time to download the photos : 34.29675006866455 About to test input to load empty input or other problem ret : 2 all the photo in this batch is missing ! 1.55user 0.69system 0:36.57elapsed 6%CPU (0avgtext+0avgdata 99356maxresident)k 0inputs+112outputs (56major+48786minor)pagefaults 0swaps