python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 8 -a ' -a 4871 -l 100 ' -s datou_current_4871 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 2714821 load datou : 4871 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? load thcls load pdts Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec no input labels no input values updating current state to 1 we have a portfolio with more photos than limit : 580>100 please execute split_portfolio.py -i 21478223 -l 100 size over we load limit photo not treated list_input_json: {} Current got : datou_id : 4871, datou_cur_ids : ['2674161'] with mtr_portfolio_ids : ['21478223'] and first list_photo_ids : [] new path : /proc/2714821/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! List Step Type Loaded in datou : blur_detection over limit max, limiting to limit_max 20 list_input_json : {} origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 20 ; length of list_pids : 20 ; length of list_args : 20 time to download the photos : 2.1898107528686523 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 1 step1:blur_detection Mon Mar 17 07:32:30 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec inside step blur_detection methode: ratio et variance treat image : temp/1742193148_2714821_1344661448_68459ac5f288289f63f6724949a05a0b.jpg resize: (1080, 1632) 1344661448 -6.159021604739044 treat image : temp/1742193148_2714821_1344661437_1f6c893ec26c9e49b159e88c63980d17.jpg resize: (1080, 1632) 1344661437 -6.266138171672507 treat image : temp/1742193148_2714821_1344661423_5019a56b491d0ca257e6f9b4b12be376.jpg resize: (1080, 1632) 1344661423 -4.902362429287814 treat image : temp/1742193148_2714821_1344661421_d9d4cd76e02bf7ec8964ab9b3f81a50a.jpg resize: (1080, 1632) 1344661421 -6.251387976420491 treat image : temp/1742193148_2714821_1344661419_def083e66ec5e2fbd260ee3b995d544a.jpg resize: (1080, 1632) 1344661419 -6.104800768368497 treat image : temp/1742193148_2714821_1344661416_a1ae7758d6c747ea27fdc784aef31c96.jpg resize: (1080, 1632) 1344661416 -4.8691220460045415 treat image : temp/1742193148_2714821_1344661355_2b66ec70768bef9dc84d816c439544be.jpg resize: (1080, 1632) 1344661355 -5.990863049460525 treat image : temp/1742193148_2714821_1344661335_1ea30ab078741d05623418b6ba639a14.jpg resize: (1080, 1632) 1344661335 -4.141106824762472 treat image : temp/1742193148_2714821_1344661327_922ce08d74399651ae2c33bf8588f899.jpg resize: (1080, 1632) 1344661327 -2.085820360035655 treat image : temp/1742193148_2714821_1344661314_a679b6736d4d066d81958ed9a128bcad.jpg resize: (1080, 1632) 1344661314 -4.124069262691517 treat image : temp/1742193148_2714821_1344661173_f542e0954266c9e4f6dcb3dbbbc7ca17.jpg resize: (1080, 1632) 1344661173 -1.9400413359278248 treat image : temp/1742193148_2714821_1344661114_5ea5476bd54b810c5217549b8248cdff.jpg resize: (1080, 1632) 1344661114 -6.057757455364538 treat image : temp/1742193148_2714821_1344661109_4388b7ecb9d0178885371e167f0d48df.jpg resize: (1080, 1632) 1344661109 -5.11528467171716 treat image : temp/1742193148_2714821_1344661104_33b0035a946fdb6bae64f028d514a17a.jpg resize: (1080, 1632) 1344661104 -5.610323117055241 treat image : temp/1742193148_2714821_1344661099_e811de93c33a0a9885535db8a91d9f1a.jpg resize: (1080, 1632) 1344661099 -3.79022621066154 treat image : temp/1742193148_2714821_1344661096_ea46a65f79ea4120413912c93ba5c542.jpg resize: (1080, 1632) 1344661096 -5.6166858466252965 treat image : temp/1742193148_2714821_1344661092_7a1a16dfea850c4f2ee714dcb5db56f0.jpg resize: (1080, 1632) 1344661092 -6.035400759289321 treat image : temp/1742193148_2714821_1344660896_88c38ef9acef56b1597d0765a53b4cbe.jpg resize: (1080, 1632) 1344660896 -6.071648923464256 treat image : temp/1742193148_2714821_1344660890_de1ef6782cc863ad109542b73cf7f07f.jpg resize: (1080, 1632) 1344660890 -3.8327906176277597 treat image : temp/1742193148_2714821_1344660882_1f0b2c3e9d534d449bd22784337e0569.jpg resize: (1080, 1632) 1344660882 -2.3907403454305998 Inside saveOutput : final : True verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 20 time used for this insertion : 0.013005495071411133 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 20 time used for this insertion : 0.013684749603271484 begin to insert list_values into mtr_datou_result : length of list_values in save_final : 20 time used for this insertion : 0.014960289001464844 save missing photos in datou_result : time spend for datou_step_exec : 14.477634191513062 time spend to save output : 0.04656553268432617 total time spend for step 1 : 14.524199724197388 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 13.66user 3.41system 0:20.06elapsed 85%CPU (0avgtext+0avgdata 141108maxresident)k 160inputs+10512outputs (0major+264214minor)pagefaults 0swaps