python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 8 -a ' -a 4871 -l 100 ' -s datou_current_4871 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 2892582 load datou : 4871 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? load thcls load pdts Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec no input labels no input values updating current state to 1 we have a portfolio with more photos than limit : 937>100 please execute split_portfolio.py -i 20624335 -l 100 size over we load limit photo not treated list_input_json: {} Current got : datou_id : 4871, datou_cur_ids : ['2594849'] with mtr_portfolio_ids : ['20624335'] and first list_photo_ids : [] new path : /proc/2892582/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! List Step Type Loaded in datou : blur_detection over limit max, limiting to limit_max 20 list_input_json : {} origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 20 ; length of list_pids : 20 ; length of list_args : 20 time to download the photos : 2.7006418704986572 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 1 step1:blur_detection Tue Feb 18 09:37:30 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec inside step blur_detection methode: ratio et variance treat image : temp/1739867847_2892582_1338303997_4dba76fdaf3bbfc1f9efe86a6e91a0ed.jpg resize: (1080, 1632) 1338303997 -6.171975132703792 treat image : temp/1739867847_2892582_1338303987_34a86ea3d8ac2c84931cd392e47350a9.jpg resize: (1080, 1632) 1338303987 -6.092309653692536 treat image : temp/1739867847_2892582_1338303980_6caf2bf1626d00f1a4fbd197763dd126.jpg resize: (1080, 1632) 1338303980 -5.57097901121393 treat image : temp/1739867847_2892582_1338303976_9e5b77f9ef67f2e6445dcc267e8a25c8.jpg resize: (1080, 1632) 1338303976 -5.504080052287986 treat image : temp/1739867847_2892582_1338298355_3a18304e4a4c0b645817715bb6509b9c.jpg resize: (1080, 1632) 1338298355 -5.8214029272118495 treat image : temp/1739867847_2892582_1338298349_a9a3ff49ad1253151e896a837c6694f1.jpg resize: (1080, 1632) 1338298349 -7.101546237712614 treat image : temp/1739867847_2892582_1338298336_a2c2d16600efa6cb264e454a0eaf2b84.jpg resize: (1080, 1632) 1338298336 -7.209292812097674 treat image : temp/1739867847_2892582_1338298331_194d91b356bf61d563567eb71067efca.jpg resize: (1080, 1632) 1338298331 -6.7087961250551365 treat image : temp/1739867847_2892582_1338298329_9ccdd677f8b314445e261bdab8522706.jpg resize: (1080, 1632) 1338298329 -6.326815602952177 treat image : temp/1739867847_2892582_1338298327_27990334bdd19ea6c391556a32fc135e.jpg resize: (1080, 1632) 1338298327 -6.228774482711404 treat image : temp/1739867847_2892582_1338298325_71990609bbccaad684aa183dcf17b24c.jpg resize: (1080, 1632) 1338298325 -4.972784930326656 treat image : temp/1739867847_2892582_1338298322_b5019405419b43ee5e2bf088fdc23c3e.jpg resize: (1080, 1632) 1338298322 -5.506993656564476 treat image : temp/1739867847_2892582_1338298306_e9939a2f964149869e2b2259c8eecdbe.jpg resize: (1080, 1632) 1338298306 -5.866659997138441 treat image : temp/1739867847_2892582_1338298297_5653a21a75d8850c9f6eaf7831df6af8.jpg resize: (1080, 1632) 1338298297 -6.625360452312646 treat image : temp/1739867847_2892582_1338298292_0b8fbdce4c611aa2a3cd471674fa2f51.jpg resize: (1080, 1632) 1338298292 -6.580301825424543 treat image : temp/1739867847_2892582_1338298286_b7c948b5f0639bbf603e29a5ec13307d.jpg resize: (1080, 1632) 1338298286 -5.724195934030539 treat image : temp/1739867847_2892582_1338298285_f184dcaad657d321f92d8bb61017563f.jpg resize: (1080, 1632) 1338298285 -2.9949225464763005 treat image : temp/1739867847_2892582_1338298283_da4a843ecd29c2cca554c12988e74136.jpg resize: (1080, 1632) 1338298283 -1.9187867300162695 treat image : temp/1739867847_2892582_1338298277_a8443d6ad505903308b3d74010dd0eeb.jpg resize: (1080, 1632) 1338298277 -5.866659997138441 treat image : temp/1739867847_2892582_1338298269_8cfceb0fd3b3ae7347e2c9fa0d9e8cb2.jpg resize: (1080, 1632) 1338298269 -6.625360452312646 Inside saveOutput : final : True verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 20 time used for this insertion : 0.013616085052490234 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 20 time used for this insertion : 0.14886212348937988 begin to insert list_values into mtr_datou_result : length of list_values in save_final : 20 time used for this insertion : 0.4409644603729248 save missing photos in datou_result : time spend for datou_step_exec : 15.035512685775757 time spend to save output : 0.6085867881774902 total time spend for step 1 : 15.644099473953247 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 14.43user 3.18system 0:20.83elapsed 84%CPU (0avgtext+0avgdata 139996maxresident)k 2848inputs+11864outputs (32major+257757minor)pagefaults 0swaps