python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 8 -a ' -a 4871 -l 100 ' -s datou_current_4871 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 3265205 load datou : 4871 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? load thcls load pdts Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec no input labels no input values updating current state to 1 we have a portfolio with more photos than limit : 1728>100 please execute split_portfolio.py -i 20234215 -l 100 size over we load limit photo not treated list_input_json: {} Current got : datou_id : 4871, datou_cur_ids : ['2547945'] with mtr_portfolio_ids : ['20234215'] and first list_photo_ids : [] new path : /proc/3265205/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! List Step Type Loaded in datou : blur_detection over limit max, limiting to limit_max 20 list_input_json : {} origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 20 ; length of list_pids : 20 ; length of list_args : 20 time to download the photos : 2.0610313415527344 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 1 step1:blur_detection Wed Feb 5 10:07:30 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec inside step blur_detection methode: ratio et variance treat image : temp/1738746448_3265205_1334873043_33ce865ebfda10a758399a711e30ed19.jpg resize: (1080, 1632) 1334873043 -0.8756850007827816 treat image : temp/1738746448_3265205_1334873042_0bb1d93a436526c480e76a7faad08503.jpg resize: (1080, 1632) 1334873042 -3.8182830400935743 treat image : temp/1738746448_3265205_1334872970_7a2a37b547c4db4b602259775813b309.jpg resize: (1080, 1632) 1334872970 -5.238223190779892 treat image : temp/1738746448_3265205_1334872968_5e87fd5f6b25e70ca007c63dec3e7d3d.jpg resize: (1080, 1632) 1334872968 -5.666571834147536 treat image : temp/1738746448_3265205_1334872963_410c26d32c75e6abcb46208d84a68099.jpg resize: (1080, 1632) 1334872963 -5.972396572459353 treat image : temp/1738746448_3265205_1334872962_173935e65cf6fddf3fd75e37f06364ec.jpg resize: (1080, 1632) 1334872962 -4.113550549341646 treat image : temp/1738746448_3265205_1334872900_c517f9579a151e1a5372f19f5d35a90f.jpg resize: (1080, 1632) 1334872900 -4.900614124086404 treat image : temp/1738746448_3265205_1334872897_af6e275c82969f506eaa77b2a435f1bf.jpg resize: (1080, 1632) 1334872897 -5.708593976821828 treat image : temp/1738746448_3265205_1334872896_ef1907ae93ef8a93f92c1bcd32aafff6.jpg resize: (1080, 1632) 1334872896 -3.9780899533171015 treat image : temp/1738746448_3265205_1334872894_9ffa14e1e33f060d26c34d332274da54.jpg resize: (1080, 1632) 1334872894 -5.649697711114836 treat image : temp/1738746448_3265205_1334872892_ccd08a5a7177e5827e652820e6ac8ef3.jpg resize: (1080, 1632) 1334872892 -5.507309745868954 treat image : temp/1738746448_3265205_1334872891_a176c3a9a4fe45bb8a54757b249d8001.jpg resize: (1080, 1632) 1334872891 -4.894305465095415 treat image : temp/1738746448_3265205_1334872815_624f82f5b72897458d132d9af697c892.jpg resize: (1080, 1632) 1334872815 -5.879874951923289 treat image : temp/1738746448_3265205_1334872813_1c4ba0036f58464dc28624266c13625b.jpg resize: (1080, 1632) 1334872813 -6.011380151564484 treat image : temp/1738746448_3265205_1334872803_75c146e3142610b79b6d2a127b6a98e3.jpg resize: (1080, 1632) 1334872803 -6.281195042217577 treat image : temp/1738746448_3265205_1334872801_abc9aa2b22c6beb59a4d4971a840373e.jpg resize: (1080, 1632) 1334872801 -5.994822717055499 treat image : temp/1738746448_3265205_1334872799_ac2130ff77063c870b46ff7efdb5e170.jpg resize: (1080, 1632) 1334872799 -6.02287137532998 treat image : temp/1738746448_3265205_1334872798_e2dbdfa0ca0a4145d27286e2586b7212.jpg resize: (1080, 1632) 1334872798 -5.713859694163434 treat image : temp/1738746448_3265205_1334872719_159b3bf94f6b1a9c3f3b0a2f8c127164.jpg resize: (1080, 1632) 1334872719 -1.8188211047130742 treat image : temp/1738746448_3265205_1334872718_23402dab28310f777fbb2336fff40a3f.jpg resize: (1080, 1632) 1334872718 -3.0599014486841405 Inside saveOutput : final : True verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 20 time used for this insertion : 0.01369166374206543 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 20 time used for this insertion : 0.012888669967651367 begin to insert list_values into mtr_datou_result : length of list_values in save_final : 20 time used for this insertion : 0.014747381210327148 save missing photos in datou_result : time spend for datou_step_exec : 13.496051549911499 time spend to save output : 0.04679584503173828 total time spend for step 1 : 13.542847394943237 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 14.48user 1.46system 0:18.23elapsed 87%CPU (0avgtext+0avgdata 138912maxresident)k 0inputs+10376outputs (3major+118901minor)pagefaults 0swaps