python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 8 -a ' -a 4871 -l 100 ' -s datou_current_4871 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 3740431 load datou : 4871 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? load thcls load pdts Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec no input labels no input values updating current state to 1 we have a portfolio with more photos than limit : 1665>100 please execute split_portfolio.py -i 20677012 -l 100 size over we load limit photo not treated list_input_json: {} Current got : datou_id : 4871, datou_cur_ids : ['2599675'] with mtr_portfolio_ids : ['20677012'] and first list_photo_ids : [] new path : /proc/3740431/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! List Step Type Loaded in datou : blur_detection over limit max, limiting to limit_max 20 list_input_json : {} origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 20 ; length of list_pids : 20 ; length of list_args : 20 time to download the photos : 2.1537389755249023 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 1 step1:blur_detection Thu Feb 20 22:02:31 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec inside step blur_detection methode: ratio et variance treat image : temp/1740085349_3740431_1338676605_0946cfaa81b87dd9ed55b6e8bcd25213.jpg resize: (1080, 1632) 1338676605 -5.390564376699064 treat image : temp/1740085349_3740431_1338676601_8d8cf257d0a545630f3b1f55a63ddefb.jpg resize: (1080, 1632) 1338676601 -5.865830277219208 treat image : temp/1740085349_3740431_1338676596_4fde3ccf373aa43915de4e2828dcbb3b.jpg resize: (1080, 1632) 1338676596 -5.501948627723225 treat image : temp/1740085349_3740431_1338676591_9da2ff540b4d76f02d529e33b01acedb.jpg resize: (1080, 1632) 1338676591 -5.73743392810543 treat image : temp/1740085349_3740431_1338676587_9112b7101e63056331703d9cdd4e13cc.jpg resize: (1080, 1632) 1338676587 0.334763686832466 treat image : temp/1740085349_3740431_1338676582_c9b36df39b0551734cb429e3ebb700b8.jpg resize: (1080, 1632) 1338676582 -4.152156054444681 treat image : temp/1740085349_3740431_1338676576_30a6791063ce27e85afc7ccb0b137218.jpg resize: (1080, 1632) 1338676576 -5.328052190724927 treat image : temp/1740085349_3740431_1338676575_27beea93773756b707e6c77b1e58db5f.jpg resize: (1080, 1632) 1338676575 -5.328052190724927 treat image : temp/1740085349_3740431_1338676573_aa8c9e8456cae3b0520c4f1f33099d6b.jpg resize: (1080, 1632) 1338676573 -5.127559449302439 treat image : temp/1740085349_3740431_1338676571_9bb03c9b0c5b16e90ac489af43977be0.jpg resize: (1080, 1632) 1338676571 -5.127559449302439 treat image : temp/1740085349_3740431_1338676569_ee62669975ddae2b7f958b21ffc891f2.jpg resize: (1080, 1632) 1338676569 -5.846789742205002 treat image : temp/1740085349_3740431_1338676568_3d1dd4e13a03da37e7d4cbc498d773cd.jpg resize: (1080, 1632) 1338676568 -5.846789742205002 treat image : temp/1740085349_3740431_1338676565_94159bd12ce5d7f5e95e57bd0a95a10a.jpg resize: (1080, 1632) 1338676565 -5.256333557382305 treat image : temp/1740085349_3740431_1338676564_ee9dabe4c0ecade651f2e54ecc8c17e3.jpg resize: (1080, 1632) 1338676564 -5.256333557382305 treat image : temp/1740085349_3740431_1338676563_7b1acc02d7e6508d46403e93dc7268e6.jpg resize: (1080, 1632) 1338676563 -3.927899955194779 treat image : temp/1740085349_3740431_1338676560_cf0b3da4d92e51d8cd01daea02a7431d.jpg resize: (1080, 1632) 1338676560 -3.927899955194779 treat image : temp/1740085349_3740431_1338676552_d8a9dc274c71299250588603bfb4f249.jpg resize: (1080, 1632) 1338676552 -5.284294755516348 treat image : temp/1740085349_3740431_1338676547_a719fded1011e3f9fa1344937ed0f7b2.jpg resize: (1080, 1632) 1338676547 -5.497283779375873 treat image : temp/1740085349_3740431_1338676541_7d498e60c0e677a62bd4a3f4df8186a3.jpg resize: (1080, 1632) 1338676541 -2.38013762306325 treat image : temp/1740085349_3740431_1338676536_3340da45da496587721bd5d50efc4ef6.jpg resize: (1080, 1632) 1338676536 -2.870545755355482 Inside saveOutput : final : True verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 20 time used for this insertion : 0.016947031021118164 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 20 time used for this insertion : 0.01578068733215332 begin to insert list_values into mtr_datou_result : length of list_values in save_final : 20 time used for this insertion : 0.01702141761779785 save missing photos in datou_result : time spend for datou_step_exec : 13.352890253067017 time spend to save output : 0.054218292236328125 total time spend for step 1 : 13.407108545303345 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 14.48user 1.11system 0:19.15elapsed 81%CPU (0avgtext+0avgdata 140412maxresident)k 3288inputs+10088outputs (49major+119639minor)pagefaults 0swaps