python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 8 -a ' --fifo -a 533 -l 100 ' -s datou_current_533 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 988606 load datou : 533 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? load thcls load pdts Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec no input labels no input values updating current state to 1 we have a portfolio with more photos than limit : 1589>100 please execute split_portfolio.py -i 23360454 -l 100 size over we load limit photo not treated list_input_json: {} Current got : datou_id : 533, datou_cur_ids : ['2960994'] with mtr_portfolio_ids : ['23360454'] and first list_photo_ids : [] new path : /proc/988606/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! DataTypes for each output/input checked ! List Step Type Loaded in datou : tfhub_classification2, argmax list_input_json : {} origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BHTTP Error 404: Not Found can't download the photo : 1361256070 complete_path : /home/admin/workarea/git/Velours/python/temp/1748355207_988606_1361256070_70e05198617ea56f8b64b913b0f14280.jpg FBHTTP Error 404: Not Found can't download the photo : 1361256065 complete_path : /home/admin/workarea/git/Velours/python/temp/1748355207_988606_1361256065_7c838dbac30aef9097cf215a59724c83.jpg FBHTTP Error 404: Not Found can't download the photo : 1361256061 complete_path : /home/admin/workarea/git/Velours/python/temp/1748355207_988606_1361256061_f827b0b3a36aa781f82e29fcfc815fde.jpg FBHTTP Error 404: Not Found can't download the photo : 1361255881 complete_path : /home/admin/workarea/git/Velours/python/temp/1748355207_988606_1361255881_542ab2af3dde550b2756356d1097515d.jpg FBHTTP Error 404: Not Found can't download the photo : 1361255828 complete_path : /home/admin/workarea/git/Velours/python/temp/1748355207_988606_1361255828_a230d06bdc87e2ff46b45ceab7e97b25.jpg FBHTTP Error 404: Not Found can't download the photo : 1361255793 complete_path : /home/admin/workarea/git/Velours/python/temp/1748355207_988606_1361255793_2f39823de836e8523df540d305ee9caf.jpg FBHTTP Error 404: Not Found can't download the photo : 1361255755 complete_path : /home/admin/workarea/git/Velours/python/temp/1748355207_988606_1361255755_4515c9730b693584937dea8fbca89561.jpg FBHTTP Error 404: Not Found can't download the photo : 1361255719 complete_path : /home/admin/workarea/git/Velours/python/temp/1748355207_988606_1361255719_081a6558ef5987017d1b5c7ad4c577ee.jpg FBHTTP Error 404: Not Found can't download the photo : 1361255574 complete_path : /home/admin/workarea/git/Velours/python/temp/1748355207_988606_1361255574_0d099f3e9eff2a76a7673d00625652a3.jpg FBHTTP Error 404: Not Found can't download the photo : 1361255537 complete_path : /home/admin/workarea/git/Velours/python/temp/1748355207_988606_1361255537_b3f15514c2c55164a5543a59051e1f7e.jpg FBHTTP Error 404: Not Found can't download the photo : 1361255458 complete_path : /home/admin/workarea/git/Velours/python/temp/1748355207_988606_1361255458_6e3f842770b83d3888d8857301145dc3.jpg FBHTTP Error 404: Not Found can't download the photo : 1361255408 complete_path : /home/admin/workarea/git/Velours/python/temp/1748355207_988606_1361255408_81e115541ff425093d387f4e66eb3505.jpg FBHTTP Error 404: Not Found can't download the photo : 1361255359 complete_path : /home/admin/workarea/git/Velours/python/temp/1748355207_988606_1361255359_9c6a572ce170e68ff6d9d9e17a834e1d.jpg FBHTTP Error 404: Not Found can't download the photo : 1361254966 complete_path : /home/admin/workarea/git/Velours/python/temp/1748355207_988606_1361254966_0a4526c44b2e2892dd5c4cb41569c392.jpg FBHTTP Error 404: Not Found can't download the photo : 1361254904 complete_path : /home/admin/workarea/git/Velours/python/temp/1748355207_988606_1361254904_785be63c77ad52a059334c89d3413c89.jpg FBHTTP Error 404: Not Found can't download the photo : 1361254546 complete_path : /home/admin/workarea/git/Velours/python/temp/1748355207_988606_1361254546_79a183bac431cb153089177a143cace9.jpg FBHTTP Error 404: Not Found can't download the photo : 1361254541 complete_path : /home/admin/workarea/git/Velours/python/temp/1748355207_988606_1361254541_7cfed437713ca2628b99de7300e53b63.jpg FBHTTP Error 404: Not Found can't download the photo : 1361254537 complete_path : /home/admin/workarea/git/Velours/python/temp/1748355207_988606_1361254537_da3b4f4c307076dea30ebefef89edc80.jpg FBHTTP Error 404: Not Found can't download the photo : 1361254531 complete_path : /home/admin/workarea/git/Velours/python/temp/1748355207_988606_1361254531_3c54aed0aa3ccebc31195b3be98adb5e.jpg FBHTTP Error 404: Not Found can't download the photo : 1361253614 complete_path : /home/admin/workarea/git/Velours/python/temp/1748355207_988606_1361253614_45002caeae6144a1d5bd1889dd44961b.jpg FBHTTP Error 404: Not Found can't download the photo : 1361252778 complete_path : /home/admin/workarea/git/Velours/python/temp/174835HTTP Error 404: Not Found can't download the photo : 1361247497 complete_path : /home/admin/workarea/git/Velours/python/temp/1748355207_988606_1361247497_cf596b8cfd24dd8c4c6851efd2c7d095.jpg FBHTTP Error 404: Not Found can't download the photo : 1361247492 complete_path : /home/admin/workarea/git/Velours/python/temp/1748355207_988606_1361247492_2a38256e1e7c34be229059cf4bd95e58.jpg FBHTTP Error 404: Not Found can't download the photo : 1361247488 complete_path : /home/admin/workarea/git/Velours/python/temp/1748355207_988606_1361247488_1f2aa89f232f3ea69ab02247997200bd.jpg FBHTTP Error 404: Not Found can't download the photo : 1361247483 complete_path : /home/admin/workarea/git/Velours/python/temp/1748355207_988606_1361247483_c720467099c682bf22852f9ce9a372e9.jpg FBHTTP Error 404: Not Found can't download the photo : 1361247478 complete_path : /home/admin/workarea/git/Velours/python/temp/1748355207_988606_1361247478_f2cbfe90d9bcfe2a46b8cbe1d18ac173.jpg FBHTTP Error 404: Not Found can't download the photo : 1361247352 complete_path : /home/admin/workarea/git/Velours/python/temp/1748355207_988606_1361247352_878e86d32e2082e698d786c5ef4c0776.jpg FBHTTP Error 404: Not Found can't download the photo : 1361247330 complete_path : /home/admin/workarea/git/Velours/python/temp/1748355207_988606_1361247330_bf4cb097a50ecb75bae02e6f0adcb978.jpg FBHTTP Error 404: Not Found can't download the photo : 1361247065 complete_path : /home/admin/workarea/git/Velours/python/temp/1748355207_988606_1361247065_dde08ab7e48f60e4105c600d7cec1c26.jpg FBHTTP Error 404: Not Found can't download the photo : 1361247028 complete_path : /home/admin/workarea/git/Velours/python/temp/1748355207_988606_1361247028_d27b0e0c51f6852a6f5be2babad761f0.jpg FBHTTP Error 404: Not Found can't download the photo : 1361246852 complete_path : /home/admin/workarea/git/Velours/python/temp/1748355207_988606_1361246852_9f311ac437c87921db1ad4381c04d08e.jpg FBHTTP Error 404: Not Found can't download the photo : 1361246816 complete_path : /home/admin/workarea/git/Velours/python/temp/1748355207_988606_1361246816_814255641ec17dec5fa9ffdad9a6e071.jpg FBHTTP Error 404: Not Found can't download the photo : 1361246758 complete_path : /home/admin/workarea/git/Velours/python/temp/1748355207_988606_1361246758_03bcd8d0a42d11e85467ee8db0644acd.jpg FBHTTP Error 404: Not Found can't download the photo : 1361246717 complete_path : /home/admin/workarea/git/Velours/python/temp/1748355207_988606_1361246717_65f5a2661906416b50e411823a09e462.jpg FBHTTP Error 404: Not Found can't download the photo : 1361246677 complete_path : /home/admin/workarea/git/Velours/python/temp/1748355207_988606_1361246677_98bfe23baeaac098717e344f49a804c9.jpg FBHTTP Error 404: Not Found can't download the photo : 1361245814 complete_path : /home/admin/workarea/git/Velours/python/temp/1748355207_988606_1361245814_a651d8b7601b9e3fae196f4e868196e7.jpg FBHTTP Error 404: Not Found can't download the photo : 1361245778 complete_path : /home/admin/workarea/git/Velours/python/temp/1748355207_988606_1361245778_3e6cb3f157463aaa80a23b6232857c58.jpg FBHTTP Error 404: Not Found can't download the photo : 1361245644 complete_path : /home/admin/workarea/git/Velours/python/temp/1748355207_988606_1361245644_aacc519f87be1e06cbd461b7d53424b0.jpg FBHTTP Error 404: Not Found can't download the photo : 1361245325 complete_path : /home/admin/workarea/git/Velours/python/temp/1748355207_988606_1361245325_4114b7e346990e8cbed6117badcaac6b.jpg FBHTTP Error 404: Not Found can't download the photo : 1361244134 complete_path : /home/admin/workarea/git/Velours/python/temp/1748355207_988606_1361244134_1bcfff520a70b820551397011c551518.jpg FBHTTP Error 404: Not Found can't download the photo : 1361244106 complete_path : /home/admin/workarea/git/Velours/python/temp/1748355207_988606_1361244106_6ea83e72c049351cb91bac04d6ce8171.jpg Fwe have missing 100 photos in the step downloads : photo missing : [1361256070, 1361256065, 1361256061, 1361255881, 1361255828, 1361255793, 1361255755, 1361255719, 1361255574, 1361255537, 1361255458, 1361255408, 1361255359, 1361254966, 1361254904, 1361254546, 1361254541, 1361254537, 1361254531, 1361253614, 1361252778, 1361252441, 1361252367, 1361252163, 1361252038, 1361251929, 1361251425, 1361251347, 1361251254, 1361250878, 1361250870, 1361250863, 1361250857, 1361250812, 1361250801, 1361250261, 1361249675, 1361249581, 1361249509, 1361249435, 1361249368, 1361249289, 1361249187, 1361249154, 1361249120, 1361249101, 1361249097, 1361249057, 1361249052, 1361249044, 1361249038, 1361249035, 1361249031, 1361248980, 1361248941, 1361248897, 1361248849, 1361248811, 1361248784, 1361248740, 1361248738, 1361248734, 1361248728, 1361248723, 1361248678, 1361248675, 1361248670, 1361248641, 1361248607, 1361248368, 1361248362, 1361248355, 1361248343, 1361248288, 1361248222, 1361247739, 1361247696, 1361247680, 1361247675, 1361247672, 1361247497, 1361247492, 1361247488, 1361247483, 1361247478, 1361247352, 1361247330, 1361247065, 1361247028, 1361246852, 1361246816, 1361246758, 1361246717, 1361246677, 1361245814, 1361245778, 1361245644, 1361245325, 1361244134, 1361244106] try to delete the photos missing in DB Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361256070 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361256065 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361256061 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361255881 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361255828 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361255793 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361255755 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361255719 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361255574 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361255537 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361255458 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361255408 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361255359 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361254966 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361254904 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361254546 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361254541 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361254537 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361254531 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361253614 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361252778 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361252441 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361252367 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361252163 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361252038 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361251929 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361251425 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361251347 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361251254 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361250878 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361250870 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361250863 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361250857 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361250812 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361250801 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361250261 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361249675 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361249581 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361249509 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361249435 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361249368 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361249289 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361249187 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361249154 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361249120 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361249101 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361249097 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361249057 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361249052 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361249044 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361249038 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361249035 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361249031 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361248980 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361248941 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361248897 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361248849 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361248811 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361248784 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361248740 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361248738 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361248734 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361248728 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361248723 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361248678 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361248675 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361248670 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361248641 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361248607 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361248368 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361248362 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361248355 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361248343 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361248288 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361248222 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361247739 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361247696 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361247680 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361247675 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361247672 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361247497 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361247492 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361247488 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361247483 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361247478 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361247352 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361247330 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361247065 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361247028 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361246852 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361246816 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361246758 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361246717 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361246677 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361245814 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361245778 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361245644 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361245325 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361244134 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1361244106 length of list_filenames : 0 ; length of list_pids : 0 ; length of list_args : 0 time to download the photos : 34.994953870773315 About to test input to load empty input or other problem ret : 2 all the photo in this batch is missing ! 1.48user 0.82system 0:37.81elapsed 6%CPU (0avgtext+0avgdata 99028maxresident)k 0inputs+104outputs (12major+48892minor)pagefaults 0swaps