python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 8 -a ' -a 4871 -l 100 ' -s datou_current_4871 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 2215957 load datou : 4871 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? load thcls load pdts Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec no input labels no input values updating current state to 1 we have a portfolio with more photos than limit : 1295>100 please execute split_portfolio.py -i 23183727 -l 100 size over we load limit photo not treated list_input_json: {} Current got : datou_id : 4871, datou_cur_ids : ['2914480'] with mtr_portfolio_ids : ['23183727'] and first list_photo_ids : [] new path : /proc/2215957/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! List Step Type Loaded in datou : blur_detection over limit max, limiting to limit_max 20 list_input_json : {} origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 20 ; length of list_pids : 20 ; length of list_args : 20 time to download the photos : 2.2923743724823 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 1 step1:blur_detection Wed May 21 10:42:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec inside step blur_detection methode: ratio et variance treat image : temp/1747816947_2215957_1359994525_8545192cef5492cf72b15338aa41f425.jpg resize: (1080, 1632) 1359994525 -5.334303893335369 treat image : temp/1747816947_2215957_1359994522_8054ad34214fae1e69d6e2c83c4adafd.jpg resize: (1080, 1632) 1359994522 -2.037404670327899 treat image : temp/1747816947_2215957_1359994517_376a4c15c7541be142d67d097981c1d3.jpg resize: (1080, 1632) 1359994517 -1.4171667971637412 treat image : temp/1747816947_2215957_1359994491_552d7d6643a74d0c44f300fd326ef056.jpg resize: (1080, 1632) 1359994491 -5.176194851852663 treat image : temp/1747816947_2215957_1359994490_23fac921dab6e5c14d74fd38ff70f0e3.jpg resize: (1080, 1632) 1359994490 -5.324548664238076 treat image : temp/1747816947_2215957_1359994488_f99a3d0eaeafc231cde25a69ab2db707.jpg resize: (1080, 1632) 1359994488 -3.7155382372536705 treat image : temp/1747816947_2215957_1359994485_0b979660744a5db30e7c04f8b23be7d7.jpg resize: (1080, 1632) 1359994485 -5.271371315498357 treat image : temp/1747816947_2215957_1359994482_553e12bbe6ec34978ee43605c46ec3b9.jpg resize: (1080, 1632) 1359994482 -5.081599983619229 treat image : temp/1747816947_2215957_1359994477_aea4edfffe4071a50cf8664ced2803cf.jpg resize: (1080, 1632) 1359994477 -5.074817983878004 treat image : temp/1747816947_2215957_1359994404_f56eab8e234a1ee313779e237584ad45.jpg resize: (1080, 1632) 1359994404 -3.7214795857990595 treat image : temp/1747816947_2215957_1359994396_2349a3d4ef083af97be21bcf60d7b21c.jpg resize: (1080, 1632) 1359994396 -1.366322327466925 treat image : temp/1747816947_2215957_1359994394_57ee738e56cb71867a7224f26ee30354.jpg resize: (1080, 1632) 1359994394 -4.349929822672256 treat image : temp/1747816947_2215957_1359994383_594a500f0e82193a2e77ea8f946e6cbe.jpg resize: (1080, 1632) 1359994383 -5.483876773361901 treat image : temp/1747816947_2215957_1359994353_56eae104b2d8850500b8990a590a128a.jpg resize: (1080, 1632) 1359994353 -5.890404650219362 treat image : temp/1747816947_2215957_1359994349_dd5ec74f47182352f8e86b65a6de4417.jpg resize: (1080, 1632) 1359994349 -5.626612133376309 treat image : temp/1747816947_2215957_1359994176_b8d8c70f75097cacdf6a9fdc26d7c9a4.jpg resize: (1080, 1632) 1359994176 -2.227647714340391 treat image : temp/1747816947_2215957_1359994150_3d30b7b621447508a8356e156466926b.jpg resize: (1080, 1632) 1359994150 -1.3558846244349987 treat image : temp/1747816947_2215957_1359994124_d7380e2a8ff3b08522aadb12b46d6d07.jpg resize: (1080, 1632) 1359994124 -2.0378326930728496 treat image : temp/1747816947_2215957_1359991295_b043df087ffd62f53087f2c70072af4a.jpg resize: (1080, 1632) 1359991295 -5.470355841701436 treat image : temp/1747816947_2215957_1359991288_b30f4f84b2effa897e4532df9773801d.jpg resize: (1080, 1632) 1359991288 -5.882013116784093 Inside saveOutput : final : True verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 20 time used for this insertion : 6.80484938621521 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 20 time used for this insertion : 0.03670978546142578 begin to insert list_values into mtr_datou_result : length of list_values in save_final : 20 time used for this insertion : 0.0470280647277832 save missing photos in datou_result : time spend for datou_step_exec : 17.858771800994873 time spend to save output : 6.893903732299805 total time spend for step 1 : 24.752675533294678 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 14.52user 5.80system 0:29.32elapsed 69%CPU (0avgtext+0avgdata 137168maxresident)k 2776inputs+9064outputs (19major+257983minor)pagefaults 0swaps