python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 3318 ' -s datou_3318 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 3762721 load datou : 3318 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? [(photo_id, hashtag_id, hashtag_type, x0, x1, y0, y1, score, seg_temp, polygons), ...] was removed should we ? chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? id de la photo (peut être local ou global) was removed should we ? chemin de la photo was removed should we ? (x0, y0, x1, y1) was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? None was removed should we ? donnée sous forme de nombre was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5275, 'learn_RUBBIA_REFUS_AMIENS_23', 16384, 25088, 'learn_RUBBIA_REFUS_AMIENS_23', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2021, 4, 23, 14, 19, 39), datetime.datetime(2021, 4, 23, 14, 19, 39)) load thcls load THCL from format json or kwargs add thcl : 2847 in CacheModelConfig load pdts add pdt : 5275 in CacheModelConfig Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Date 2026-07-13 07:40:28.417508 Current got : datou_id : 3318, datou_cur_ids : ['4391771'] with mtr_portfolio_ids : ['31265155'] and first list_photo_ids : [] new path : /proc/3762721/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 7928 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 8092 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 8092 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7933 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 7935 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 7934 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 7934 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13649 velours_tree is not consistent : 2 used against 1 in the step definition ! Step 9283 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 7935 doesn't seem to be define in the database( WARNING : type of input 3 of step 7934 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 1 of step 7935 doesn't seem to be define in the database( WARNING : output 1 of step 7933 have datatype=7 whereas input 1 of step 7935 have datatype=None WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 2 of step 8092 doesn't seem to be define in the database( WARNING : type of output 3 of step 8092 doesn't seem to be define in the database( WARNING : type of input 1 of step 7933 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10917 doesn't seem to be define in the database( WARNING : type of output 2 of step 7928 doesn't seem to be define in the database( WARNING : type of input 1 of step 10918 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 7935 have datatype=10 whereas input 3 of step 10916 have datatype=6 WARNING : output 0 of step 7935 have datatype=10 whereas input 0 of step 13649 have datatype=18 WARNING : type of output 1 of step 13649 doesn't seem to be define in the database( WARNING : type of input 5 of step 10916 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, crop_condition, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, velours_tree, send_mail_cod, split_time_score over limit max, limiting to limit_max 40 list_input_json : [] origin We have 1 , BHTTP Error 401: Unauthorized can't download the photo : 1383771673 complete_path : /home/admin/workarea/git/Velours/python/temp/1783921228_3762721_1383771673_b7c13924bd8539336abaa1c95eaa7407.jpg FBHTTP Error 401: Unauthorized can't download the photo : 1383771653 complete_path : /home/admin/workarea/git/Velours/python/temp/1783921228_3762721_1383771653_705526990b2722c5090d99fd20b44249.jpg FBHTTP Error 401: Unauthorized can't download the photo : 1383771606 complete_path : /home/admin/workarea/git/Velours/python/temp/1783921228_3762721_1383771606_ee6f2899a499718edcc90ce119da6a07.jpg FBHTTP Error 401: Unauthorized can't download the photo : 1383771599 complete_path : /home/admin/workarea/git/Velours/python/temp/1783921228_3762721_1383771599_9b56ae03bc8768426f0d1f7035c3977b.jpg FBHTTP Error 401: Unauthorized can't download the photo : 1383771594 complete_path : /home/admin/workarea/git/Velours/python/temp/1783921228_3762721_1383771594_201259f43af2dc3244b22bdae6d154ce.jpg FBHTTP Error 401: Unauthorized can't download the photo : 1383771589 complete_path : /home/admin/workarea/git/Velours/python/temp/1783921228_3762721_1383771589_1c87d700bd54ef928de05cf9456d5251.jpg FBHTTP Error 401: Unauthorized can't download the photo : 1383771582 complete_path : /home/admin/workarea/git/Velours/python/temp/1783921228_3762721_1383771582_c984e3e6afee8349272966bab67a227e.jpg FBHTTP Error 401: Unauthorized can't download the photo : 1383771570 complete_path : /home/admin/workarea/git/Velours/python/temp/1783921228_3762721_1383771570_50216f4508c34c34b3c33b35dd36a920.jpg FBHTTP Error 401: Unauthorized can't download the photo : 1383771548 complete_path : /home/admin/workarea/git/Velours/python/temp/1783921228_3762721_1383771548_1b1ee2ff5c5c8674ac14769078e0ee4f.jpg FBHTTP Error 401: Unauthorized can't download the photo : 1383771546 complete_path : /home/admin/workarea/git/Velours/python/temp/1783921228_3762721_1383771546_6a017773a7f6f9a0e22124d3119350b6.jpg FBHTTP Error 401: Unauthorized can't download the photo : 1383771544 complete_path : /home/admin/workarea/git/Velours/python/temp/1783921228_3762721_1383771544_e15c8bc161ba29f91d8311eb744069a3.jpg FBHTTP Error 401: Unauthorized can't download the photo : 1383771538 complete_path : /home/admin/workarea/git/Velours/python/temp/1783921228_3762721_1383771538_2cd62f7c619af4c294058b2df97655ca.jpg FBHTTP Error 401: Unauthorized can't download the photo : 1383771531 complete_path : /home/admin/workarea/git/Velours/python/temp/1783921228_3762721_1383771531_b640b2051fe3619b9194c53a630a04c1.jpg FBHTTP Error 401: Unauthorized can't download the photo : 1383771478 complete_path : /home/admin/workarea/git/Velours/python/temp/1783921228_3762721_1383771478_b7044a1f87ae0af847ed251b8a3cdf98.jpg FBHTTP Error 401: Unauthorized can't download the photo : 1383771474 complete_path : /home/admin/workarea/git/Velours/python/temp/1783921228_3762721_1383771474_25e7397ea30f917d7decc47e47102468.jpg FBHTTP Error 401: Unauthorized can't download the photo : 1383771470 complete_path : /home/admin/workarea/git/Velours/python/temp/1783921228_3762721_1383771470_750829346e6fe8fd27481b1e4ff7ce72.jpg FBHTTP Error 401: Unauthorized can't download the photo : 1383771466 complete_path : /home/admin/workarea/git/Velours/python/temp/1783921228_3762721_1383771466_e32a5c9661ea55a6524a49e9a1a234f9.jpg FBHTTP Error 401: Unauthorized can't download the photo : 1383771461 complete_path : /home/admin/workarea/git/Velours/python/temp/1783921228_3762721_1383771461_ef4924ceeb6be15687d1c2fec6024336.jpg FBHTTP Error 401: Unauthorized can't download the photo : 1383771455 complete_path : /home/admin/workarea/git/Velours/python/temp/1783921228_3762721_1383771455_b3c13de830a59f500b26faf5bd775cf3.jpg FBHTTP Error 401: Unauthorized can't download the photo : 1383771417 complete_path : /home/admin/workarea/git/Velours/python/temp/1783921228_3762721_1383771417_ee33612632bb92f53eb7071419df6607.jpg FBHTTP Error 401: Unauthorized can't download the photo : 1383771413 complete_path : /home/admin/workarea/git/Velours/python/temp/1783921228_3762721_1383771413_f06074c4233f2888e20352e20064966b.jpg FBHTTP Error 401: Unauthorized can't download the photo : 1383771409 complete_path : /home/admin/workarea/git/Velours/python/temp/1783921228_3762721_1383771409_c22547d883f1ef5701340906725a2101.jpg FBHTTP Error 401: Unauthorized can't download the photo : 1383771394 complete_path : /home/admin/workarea/git/Velours/python/temp/1783921228_3762721_1383771394_3382912ac9b1011358d3bca0dc6f15d3.jpg Fall photo missing in this thread we have missing 23 photos in the step downloads : photo missing : [1383771673, 1383771653, 1383771606, 1383771599, 1383771594, 1383771589, 1383771582, 1383771570, 1383771548, 1383771546, 1383771544, 1383771538, 1383771531, 1383771478, 1383771474, 1383771470, 1383771466, 1383771461, 1383771455, 1383771417, 1383771413, 1383771409, 1383771394] try to delete the photos missing in DB Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383771673 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383771653 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383771606 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383771599 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383771594 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383771589 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383771582 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383771570 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383771548 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383771546 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383771544 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383771538 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383771531 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383771478 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383771474 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383771470 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383771466 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383771461 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383771455 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383771417 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383771413 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383771409 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383771394 length of list_filenames : 0 ; length of list_pids : 0 ; length of list_args : 0 time to download the photos : 12.307273864746094 About to test input to load empty input or other problem ret : 2 len(total_photo_id_missing) : 23 1.86user 0.81system 0:15.95elapsed 16%CPU (0avgtext+0avgdata 113656maxresident)k 0inputs+56outputs (11major+57573minor)pagefaults 0swaps