python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 8 -a ' -a 4871 -l 100 ' -s datou_current_4871 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 1133301 load datou : 4871 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? load thcls load pdts Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec no input labels no input values updating current state to 1 we have a portfolio with more photos than limit : 957>100 please execute split_portfolio.py -i 22899558 -l 100 size over we load limit photo not treated list_input_json: {} Current got : datou_id : 4871, datou_cur_ids : ['2832080'] with mtr_portfolio_ids : ['22899558'] and first list_photo_ids : [] new path : /proc/1133301/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! List Step Type Loaded in datou : blur_detection over limit max, limiting to limit_max 20 list_input_json : {} origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 20 ; length of list_pids : 20 ; length of list_args : 20 time to download the photos : 2.2446067333221436 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 1 step1:blur_detection Tue May 13 05:22:31 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec inside step blur_detection methode: ratio et variance treat image : temp/1747106548_1133301_1357767422_6c0c7e074aa0d2841def115d71436f09.jpg resize: (1080, 1632) 1357767422 -1.8138826972794164 treat image : temp/1747106548_1133301_1357767420_997ca097d05ad4142753a243453c6ef6.jpg resize: (1080, 1632) 1357767420 -0.8434819694787654 treat image : temp/1747106548_1133301_1357767419_e3f14fb13337c57165faf9bc2b500373.jpg resize: (1080, 1632) 1357767419 -4.366197909773211 treat image : temp/1747106548_1133301_1357767418_81ec32fbd4363cc785dd0abddf72752e.jpg resize: (1080, 1632) 1357767418 -5.601164272413847 treat image : temp/1747106548_1133301_1357767415_c398886e2fc4da59238687668a2cafea.jpg resize: (1080, 1632) 1357767415 -5.2847075579390985 treat image : temp/1747106548_1133301_1357767389_f2fbf92160e9187ac2b710f2e2d23091.jpg resize: (1080, 1632) 1357767389 -3.5433306369385895 treat image : temp/1747106548_1133301_1357767387_5d38bdafbe920eb7a0310e686b4192d1.jpg resize: (1080, 1632) 1357767387 -5.659786349984503 treat image : temp/1747106548_1133301_1357767385_38b9d4de015f88a96cfe53918c88ca7c.jpg resize: (1080, 1632) 1357767385 -5.8108837844215095 treat image : temp/1747106548_1133301_1357767379_361402bc519a38ce1c50275f57393ca4.jpg resize: (1080, 1632) 1357767379 -2.368321317469564 treat image : temp/1747106548_1133301_1357767355_2de9f637ddc0a49ff4e7ddce5ef0b1d6.jpg resize: (1080, 1632) 1357767355 -6.343816414044596 treat image : temp/1747106548_1133301_1357767353_7b33b5d8eb85282fb9d203584d26f278.jpg resize: (1080, 1632) 1357767353 -5.124223143563445 treat image : temp/1747106548_1133301_1357767351_e6fa5ffd5d7e6735632ee37261348c05.jpg resize: (1080, 1632) 1357767351 -2.7794388539999315 treat image : temp/1747106548_1133301_1357767350_7fff5ca32a72e7b5077a6659839efe14.jpg resize: (1080, 1632) 1357767350 -5.736303450456584 treat image : temp/1747106548_1133301_1357767349_3d561b6b384bb720e44c86f777c014e4.jpg resize: (1080, 1632) 1357767349 -5.885106205411859 treat image : temp/1747106548_1133301_1357767346_5180fbdbac87d1fe4755a9f6c3a3accb.jpg resize: (1080, 1632) 1357767346 -5.771493569771929 treat image : temp/1747106548_1133301_1357767328_3178ec9e701522ddcd9461c2637662dc.jpg resize: (1080, 1632) 1357767328 -1.5718176341786985 treat image : temp/1747106548_1133301_1357767326_5f1411f47f153927d2f3ee3e40dcf94b.jpg resize: (1080, 1632) 1357767326 -4.582261181267702 treat image : temp/1747106548_1133301_1357767323_48b831a4a6659fe053a9cc685c319afd.jpg resize: (1080, 1632) 1357767323 -5.778854960722966 treat image : temp/1747106548_1133301_1357767320_a60d503ed354905e8a84657e8914fec6.jpg resize: (1080, 1632) 1357767320 -2.6991861861014224 treat image : temp/1747106548_1133301_1357767318_019fb555606b9703eba155255f788c95.jpg resize: (1080, 1632) 1357767318 -6.201711661021461 Inside saveOutput : final : True verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 20 time used for this insertion : 0.012757062911987305 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 20 time used for this insertion : 0.361863374710083 begin to insert list_values into mtr_datou_result : length of list_values in save_final : 20 time used for this insertion : 0.45011401176452637 save missing photos in datou_result : time spend for datou_step_exec : 19.976526498794556 time spend to save output : 0.830054521560669 total time spend for step 1 : 20.806581020355225 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 14.96user 7.07system 0:26.95elapsed 81%CPU (0avgtext+0avgdata 140076maxresident)k 3848inputs+9608outputs (20major+260368minor)pagefaults 0swaps