python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 8 -a ' -a 4871 -l 100 ' -s datou_current_4871 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 2950595 load datou : 4871 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? load thcls load pdts Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec no input labels no input values updating current state to 1 we have a portfolio with more photos than limit : 606>100 please execute split_portfolio.py -i 20128731 -l 100 size over we load limit photo not treated list_input_json: {} Current got : datou_id : 4871, datou_cur_ids : ['2535930'] with mtr_portfolio_ids : ['20128731'] and first list_photo_ids : [] new path : /proc/2950595/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! List Step Type Loaded in datou : blur_detection over limit max, limiting to limit_max 20 list_input_json : {} origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 20 ; length of list_pids : 20 ; length of list_args : 20 time to download the photos : 2.3577511310577393 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 1 step1:blur_detection Sat Feb 1 03:17:31 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec inside step blur_detection methode: ratio et variance treat image : temp/1738376248_2950595_1333566317_a5d1a07e60cf66e0220138cbba89de83.jpg resize: (1080, 1632) 1333566317 -4.220240064281603 treat image : temp/1738376248_2950595_1333566236_6b06ebbb247aed47a5ed40bf28a9104c.jpg resize: (1080, 1632) 1333566236 -6.0545487913479095 treat image : temp/1738376248_2950595_1333566234_6405b58c9efbb24e2c9b9cac9f749bf2.jpg resize: (1080, 1632) 1333566234 -5.323720210395274 treat image : temp/1738376248_2950595_1333566233_9e60065bfd0273c88c77209d72808e96.jpg resize: (1080, 1632) 1333566233 -5.839868161793216 treat image : temp/1738376248_2950595_1333566224_b5ccf3fd563a6d426817c58e9afe8430.jpg resize: (1080, 1632) 1333566224 -5.346566543462719 treat image : temp/1738376248_2950595_1333566223_8c775a2362cc11e2f6885d6b8d5fc1b8.jpg resize: (1080, 1632) 1333566223 -5.549743750148042 treat image : temp/1738376248_2950595_1333566085_4858852dd5ec231b8c8669ec117a0095.jpg resize: (1080, 1632) 1333566085 -6.2158664588402015 treat image : temp/1738376248_2950595_1333566078_a120944e573eb8fdbe2191ea9aa24d14.jpg resize: (1080, 1632) 1333566078 -5.559334736850789 treat image : temp/1738376248_2950595_1333566073_14cea6d119799f6497709bcb8392ad01.jpg resize: (1080, 1632) 1333566073 -5.751529184277961 treat image : temp/1738376248_2950595_1333561689_b668bbb54145164a2e2f7aca61565a63.jpg resize: (1080, 1632) 1333561689 -6.613476361381641 treat image : temp/1738376248_2950595_1333561688_889b5809bfb89a96f978172cdf0e6cb1.jpg resize: (1080, 1632) 1333561688 -2.4379558530238254 treat image : temp/1738376248_2950595_1333561686_e410cee29b47a47912777238933fe10c.jpg resize: (1080, 1632) 1333561686 -3.110312536609298 treat image : temp/1738376248_2950595_1333561681_b2c51cd0c1a6bff4ad80ba55bff34a67.jpg resize: (1080, 1632) 1333561681 -5.061282798228182 treat image : temp/1738376248_2950595_1333561680_3864ae783050f607a9a83d9786238d3a.jpg resize: (1080, 1632) 1333561680 -3.530439343481578 treat image : temp/1738376248_2950595_1333561674_e24b67c98347c89f3efd994b552acbe1.jpg resize: (1080, 1632) 1333561674 -5.399938641822614 treat image : temp/1738376248_2950595_1333561670_05ca7e38f599a821348158951e98f2ff.jpg resize: (1080, 1632) 1333561670 -6.2009181805435984 treat image : temp/1738376248_2950595_1333561666_31ee6ae83c45e528f45146223dcb2645.jpg resize: (1080, 1632) 1333561666 -5.5879934319833 treat image : temp/1738376248_2950595_1333561658_db605b0708fe5f7005991547526c4c45.jpg resize: (1080, 1632) 1333561658 -6.623351533256254 treat image : temp/1738376248_2950595_1333561593_90018b42ca0defeefe15377c9f48e2d2.jpg resize: (1080, 1632) 1333561593 -3.4348632053822903 treat image : temp/1738376248_2950595_1333561557_48fb67e740360f72b28a7e21d2607278.jpg resize: (1080, 1632) 1333561557 -3.436373857622401 Inside saveOutput : final : True verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 20 time used for this insertion : 0.015631914138793945 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 20 time used for this insertion : 0.015941143035888672 begin to insert list_values into mtr_datou_result : length of list_values in save_final : 20 time used for this insertion : 0.019916057586669922 save missing photos in datou_result : time spend for datou_step_exec : 14.712991952896118 time spend to save output : 0.056329965591430664 total time spend for step 1 : 14.769321918487549 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 14.05user 3.20system 0:21.21elapsed 81%CPU (0avgtext+0avgdata 139100maxresident)k 24inputs+10696outputs (14major+232862minor)pagefaults 0swaps