python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 8 -a ' -a 4871 -l 100 ' -s datou_current_4871 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 3909328 load datou : 4871 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? load thcls load pdts Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec no input labels no input values updating current state to 1 we have a portfolio with more photos than limit : 1560>100 please execute split_portfolio.py -i 21570233 -l 100 size over we load limit photo not treated list_input_json: {} Current got : datou_id : 4871, datou_cur_ids : ['2678697'] with mtr_portfolio_ids : ['21570233'] and first list_photo_ids : [] new path : /proc/3909328/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! List Step Type Loaded in datou : blur_detection over limit max, limiting to limit_max 20 list_input_json : {} origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 20 ; length of list_pids : 20 ; length of list_args : 20 time to download the photos : 2.0714809894561768 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 1 step1:blur_detection Fri Mar 21 14:17:31 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec inside step blur_detection methode: ratio et variance treat image : temp/1742563049_3909328_1346001939_466ca407e1ef6a9af2c2d541f81084c5.jpg resize: (1080, 1632) 1346001939 -3.24641687128435 treat image : temp/1742563049_3909328_1346001892_4588f061e84217a1a1aff131091df7ec.jpg resize: (1080, 1632) 1346001892 -6.283929094858885 treat image : temp/1742563049_3909328_1346001839_8190bd0fea8fe71e9652a6bdb676e6de.jpg resize: (1080, 1632) 1346001839 -4.168856440617726 treat image : temp/1742563049_3909328_1346001796_da056f3157cc976e5ac1031d7e123c92.jpg resize: (1080, 1632) 1346001796 -5.837564790733928 treat image : temp/1742563049_3909328_1346001728_191311a76cc6dafad5eb851f584933b7.jpg resize: (1080, 1632) 1346001728 -5.63008363488518 treat image : temp/1742563049_3909328_1346001692_8bf408c061bbf1828af3385c2cc1071f.jpg resize: (1080, 1632) 1346001692 -4.953700740148543 treat image : temp/1742563049_3909328_1346001566_ef8e786f891a8ccc21b804a89a3bd0ef.jpg resize: (1080, 1632) 1346001566 -6.179838918743877 treat image : temp/1742563049_3909328_1346001520_5664ab3bb767fc733c765e16b1b224ed.jpg resize: (1080, 1632) 1346001520 -6.09686368765322 treat image : temp/1742563049_3909328_1346001465_1a89bbbcaef8a0b64470470d6df58474.jpg resize: (1080, 1632) 1346001465 -6.3689163400336986 treat image : temp/1742563049_3909328_1346001435_5499fe6f48377108cbd93ce4c8bc77f8.jpg resize: (1080, 1632) 1346001435 -6.484506014576876 treat image : temp/1742563049_3909328_1346001412_a53e4a0414e0b2f4f801cd353ecc03de.jpg resize: (1080, 1632) 1346001412 -6.023963265440378 treat image : temp/1742563049_3909328_1346001395_a66c1cabe5072729889c546ab0f2c7b2.jpg resize: (1080, 1632) 1346001395 -2.51850333973491 treat image : temp/1742563049_3909328_1346001386_e9938d01373edb3eee38ee00dec585eb.jpg resize: (1080, 1632) 1346001386 -6.421972681361691 treat image : temp/1742563049_3909328_1346001381_e9667261dddf89ba56141303307a90c6.jpg resize: (1080, 1632) 1346001381 -5.766274831303495 treat image : temp/1742563049_3909328_1346001377_5bde93564ed65e02dbab715c80b87e92.jpg resize: (1080, 1632) 1346001377 -6.1073506210118245 treat image : temp/1742563049_3909328_1346001373_f739f82c36323f2139a70b1699eb988d.jpg resize: (1080, 1632) 1346001373 -6.013411786998415 treat image : temp/1742563049_3909328_1346001354_2c3acb9aecf52d07ee897276352ae7fa.jpg resize: (1080, 1632) 1346001354 -6.118323135006047 treat image : temp/1742563049_3909328_1346001331_fb2848364377c87b20a4ac340483f13f.jpg resize: (1080, 1632) 1346001331 -5.484818096706836 treat image : temp/1742563049_3909328_1346001156_35fa586431fd43321aa90187740cd233.jpg resize: (1080, 1632) 1346001156 -6.232437675818885 treat image : temp/1742563049_3909328_1346001084_e72b7aadd92e61841c8cf7123b916b15.jpg resize: (1080, 1632) 1346001084 -5.9832237721258394 Inside saveOutput : final : True verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 20 time used for this insertion : 1.2371430397033691 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 20 time used for this insertion : 0.015902996063232422 begin to insert list_values into mtr_datou_result : length of list_values in save_final : 20 time used for this insertion : 0.022202491760253906 save missing photos in datou_result : time spend for datou_step_exec : 16.62850832939148 time spend to save output : 1.2805044651031494 total time spend for step 1 : 17.90901279449463 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 14.89user 4.32system 0:22.41elapsed 85%CPU (0avgtext+0avgdata 138892maxresident)k 4008inputs+11464outputs (40major+256844minor)pagefaults 0swaps