python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 8 -a ' -a 4871 -l 100 ' -s datou_current_4871 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 3639193 load datou : 4871 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? load thcls load pdts Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec no input labels no input values updating current state to 1 we have a portfolio with more photos than limit : 2144>100 please execute split_portfolio.py -i 22042798 -l 100 size over we load limit photo not treated list_input_json: {} Current got : datou_id : 4871, datou_cur_ids : ['2722371'] with mtr_portfolio_ids : ['22042798'] and first list_photo_ids : [] new path : /proc/3639193/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! List Step Type Loaded in datou : blur_detection over limit max, limiting to limit_max 20 list_input_json : {} origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 20 ; length of list_pids : 20 ; length of list_args : 20 time to download the photos : 2.5295119285583496 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 1 step1:blur_detection Fri Apr 4 09:22:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec inside step blur_detection methode: ratio et variance treat image : temp/1743751347_3639193_1350008406_cb41f88f9b740ad41f29d1ff4db0be23.jpg resize: (1080, 1632) 1350008406 -5.561461649579392 treat image : temp/1743751347_3639193_1350008401_055a97167fdc44dbf86d27e966fcdd85.jpg resize: (1080, 1632) 1350008401 -4.313542050306195 treat image : temp/1743751347_3639193_1350008398_b1f932c8a9ef4683f91aec5287c8e570.jpg resize: (1080, 1632) 1350008398 -4.27453980376916 treat image : temp/1743751347_3639193_1350008395_ac2ae05b7c0ff2500bc3a5bdb798fcc0.jpg resize: (1080, 1632) 1350008395 -5.5381599300238475 treat image : temp/1743751347_3639193_1350008392_c6c64811b1d80c3b05daaf69488800a3.jpg resize: (1080, 1632) 1350008392 -6.109358762103334 treat image : temp/1743751347_3639193_1350008389_2c13affe3bf0a840609496b9e8cced21.jpg resize: (1080, 1632) 1350008389 -4.003541243978005 treat image : temp/1743751347_3639193_1350008344_215b9cfb4484ec64fe72abd00ce7b133.jpg resize: (1080, 1632) 1350008344 -6.367503010425898 treat image : temp/1743751347_3639193_1350008307_8b26b97f76e72fc90b6f1ea8f139dd5a.jpg resize: (1080, 1632) 1350008307 -6.17263110391062 treat image : temp/1743751347_3639193_1350008302_5b5a0d4786b3ce6155010c322a85e35e.jpg resize: (1080, 1632) 1350008302 -2.0504736967017703 treat image : temp/1743751347_3639193_1350008283_93bb91f804451691a48910768fc71396.jpg resize: (1080, 1632) 1350008283 -1.3751562386400689 treat image : temp/1743751347_3639193_1350008092_e46376342b78f0c17d4caf7bf9690145.jpg resize: (1080, 1632) 1350008092 -6.602523545021554 treat image : temp/1743751347_3639193_1350008068_c2257162547a36d6150c4d80af1d9ce5.jpg resize: (1080, 1632) 1350008068 -6.337067247783844 treat image : temp/1743751347_3639193_1350008061_5186df56be0cf03ebb754d98ec5ae11b.jpg resize: (1080, 1632) 1350008061 -6.103350203675182 treat image : temp/1743751347_3639193_1350008011_5f0d2dceec16a980689853a519932746.jpg resize: (1080, 1632) 1350008011 -6.4879267463343835 treat image : temp/1743751347_3639193_1350008006_c558daf0d80ae32969bd04730aa782d8.jpg resize: (1080, 1632) 1350008006 -5.922315216178817 treat image : temp/1743751347_3639193_1350008002_8c2bc3157ab9a701081488708fc4f759.jpg resize: (1080, 1632) 1350008002 -5.835126978163361 treat image : temp/1743751347_3639193_1350007815_c6b35a9c48a9f8e476c59a015f88010c.jpg resize: (1080, 1632) 1350007815 -4.556045805219328 treat image : temp/1743751347_3639193_1350007790_201881ba6d8263f47054da3707b4ccfe.jpg resize: (1080, 1632) 1350007790 -1.6859534810858192 treat image : temp/1743751347_3639193_1350007777_e40ba9fa0185ec4aec9a82f85529e3a8.jpg resize: (1080, 1632) 1350007777 -3.070733388778066 treat image : temp/1743751347_3639193_1350007737_8a25fe045953af025a431130618d4be5.jpg resize: (1080, 1632) 1350007737 -5.591663493676377 Inside saveOutput : final : True verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 20 time used for this insertion : 0.011931180953979492 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 20 time used for this insertion : 0.018022775650024414 begin to insert list_values into mtr_datou_result : length of list_values in save_final : 20 time used for this insertion : 0.015192508697509766 save missing photos in datou_result : time spend for datou_step_exec : 14.121116161346436 time spend to save output : 0.049596548080444336 total time spend for step 1 : 14.17071270942688 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 14.77user 1.92system 0:18.18elapsed 91%CPU (0avgtext+0avgdata 137556maxresident)k 144inputs+10752outputs (0major+162338minor)pagefaults 0swaps