python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 8 -a ' -a 4871 -l 100 ' -s datou_current_4871 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 3133132 load datou : 4871 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? load thcls load pdts Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec no input labels no input values updating current state to 1 we have a portfolio with more photos than limit : 4791>100 please execute split_portfolio.py -i 19382293 -l 100 size over we load limit photo not treated list_input_json: {} Current got : datou_id : 4871, datou_cur_ids : ['2460022'] with mtr_portfolio_ids : ['19382293'] and first list_photo_ids : [] new path : /proc/3133132/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! List Step Type Loaded in datou : blur_detection over limit max, limiting to limit_max 20 list_input_json : {} origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 20 ; length of list_pids : 20 ; length of list_args : 20 time to download the photos : 3.16502046585083 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 1 step1:blur_detection Tue Mar 25 19:27:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec inside step blur_detection methode: ratio et variance treat image : temp/1742927246_3133132_1333528042_22209f65663ea38a8d3fc398eb81f21c.jpg resize: (1080, 1632) 1333528042 3.6469811193479993 treat image : temp/1742927246_3133132_1333528031_d32689d59f36d546dc0e6cef329ba129.jpg resize: (1080, 1632) 1333528031 0.3298117169138482 treat image : temp/1742927246_3133132_1333528028_c7833a8ccb8da2be5050afad55ab2b0b.jpg resize: (1080, 1632) 1333528028 -0.3429359737704701 treat image : temp/1742927246_3133132_1333528024_650cca16b63516b797d7353350bc5ade.jpg resize: (1080, 1632) 1333528024 0.7758775067061301 treat image : temp/1742927246_3133132_1333528020_c420782dd34307940d701c8243cff3a1.jpg resize: (1080, 1632) 1333528020 0.8927399723375371 treat image : temp/1742927246_3133132_1333528015_03db7b9b0016992964cf843d85759eff.jpg resize: (1080, 1632) 1333528015 0.8174849726384856 treat image : temp/1742927246_3133132_1333528010_10a3764ac4d5fdc48879e7e8e8073ac8.jpg resize: (1080, 1632) 1333528010 2.2032259811250956 treat image : temp/1742927246_3133132_1333527534_a2b5291fdfb9c88daaa82028685ad631.jpg resize: (1080, 1632) 1333527534 4.524756953934417 treat image : temp/1742927246_3133132_1333527471_266c0065ec3e31f688627bf3599aa043.jpg resize: (1080, 1632) 1333527471 4.596387435955253 treat image : temp/1742927246_3133132_1333527404_dbd64bc8bbf69655abc0c2f15fd8cbf1.jpg resize: (1080, 1632) 1333527404 3.9966672311307327 treat image : temp/1742927246_3133132_1333527334_26438b006b34bcb7166ffd3f5a61664a.jpg resize: (1080, 1632) 1333527334 5.2149471332831405 treat image : temp/1742927246_3133132_1333527272_7d110da4d2e773c98564b3fb79e14664.jpg resize: (1080, 1632) 1333527272 2.693630254123764 treat image : temp/1742927246_3133132_1333527201_2eac62a93c088917ebda9a82b4ebf849.jpg resize: (1080, 1632) 1333527201 3.8944097282850385 treat image : temp/1742927246_3133132_1333526941_43af656c3e650537fc38df22a1de9a46.jpg resize: (1080, 1632) 1333526941 7.184664645872602 treat image : temp/1742927246_3133132_1333526932_4c728c391c0c2cc27186d7e5825028e6.jpg resize: (1080, 1632) 1333526932 7.041378847773666 treat image : temp/1742927246_3133132_1333526889_429f284695dbf18fdf04b24918a600b1.jpg resize: (1080, 1632) 1333526889 7.69275724892973 treat image : temp/1742927246_3133132_1333526887_ec83b58252071097d2d6c69a1c608145.jpg resize: (1080, 1632) 1333526887 5.580519677397305 treat image : temp/1742927246_3133132_1333526885_6f59d2fda2a35754bb41d0e941fd68d7.jpg resize: (1080, 1632) 1333526885 0.09649375198764912 treat image : temp/1742927246_3133132_1333526882_899eeb8d13c29230b9772da59730aba1.jpg resize: (1080, 1632) 1333526882 1.9702894411035972 treat image : temp/1742927246_3133132_1333526695_387a98dd8331958e83be079ce29944f9.jpg resize: (1080, 1632) 1333526695 2.0566021879244336 Inside saveOutput : final : True verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 20 time used for this insertion : 0.013852119445800781 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 20 time used for this insertion : 0.1240839958190918 begin to insert list_values into mtr_datou_result : length of list_values in save_final : 20 time used for this insertion : 0.01563429832458496 save missing photos in datou_result : time spend for datou_step_exec : 15.872518062591553 time spend to save output : 0.15815496444702148 total time spend for step 1 : 16.030673027038574 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 13.93user 4.16system 0:20.70elapsed 87%CPU (0avgtext+0avgdata 138796maxresident)k 2832inputs+3264outputs (15major+250488minor)pagefaults 0swaps