python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 8 -a ' -a 4871 -l 100 ' -s datou_current_4871 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 596729 load datou : 4871 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? load thcls load pdts Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec no input labels no input values updating current state to 1 we have a portfolio with more photos than limit : 2578>100 please execute split_portfolio.py -i 20315060 -l 100 size over we load limit photo not treated list_input_json: {} Current got : datou_id : 4871, datou_cur_ids : ['2560927'] with mtr_portfolio_ids : ['20315060'] and first list_photo_ids : [] new path : /proc/596729/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! List Step Type Loaded in datou : blur_detection over limit max, limiting to limit_max 20 list_input_json : {} origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 20 ; length of list_pids : 20 ; length of list_args : 20 time to download the photos : 2.6237382888793945 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 1 step1:blur_detection Fri Feb 7 18:37:30 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec inside step blur_detection methode: ratio et variance treat image : temp/1738949847_596729_1335571661_93b47107523e9fabaf5ee3db685d22ae.jpg resize: (1080, 1632) 1335571661 -6.044789073513322 treat image : temp/1738949847_596729_1335571658_eef9a63afbb4c67f359d664696e77163.jpg resize: (1080, 1632) 1335571658 -4.414634331103001 treat image : temp/1738949847_596729_1335571657_d79c7364a46ab090caa6a4a55cf20ab5.jpg resize: (1080, 1632) 1335571657 -6.322914069256309 treat image : temp/1738949847_596729_1335571654_6c88ad0eed5e42b042e47403de0f051d.jpg resize: (1080, 1632) 1335571654 -4.845580099247499 treat image : temp/1738949847_596729_1335569581_235b9f6457819c2b0dd1cd801239b82e.jpg resize: (1080, 1632) 1335569581 -6.103261869221853 treat image : temp/1738949847_596729_1335569557_bbfb7c929f7f15170c7960ea440d41be.jpg resize: (1080, 1632) 1335569557 -6.001390732357845 treat image : temp/1738949847_596729_1335568099_830930804fa6925aa669b6fc306ac84f.jpg resize: (1080, 1632) 1335568099 -1.0288595905142033 treat image : temp/1738949847_596729_1335568087_970d61de12bc5508f887d887d68f0929.jpg resize: (1080, 1632) 1335568087 -2.6659767716882516 treat image : temp/1738949847_596729_1335568075_1d1cb05ed05c06ac56479b6e3a416b8c.jpg resize: (1080, 1632) 1335568075 -2.510286434296617 treat image : temp/1738949847_596729_1335568065_557c21be51ac2de4f2c0c9c66cf423e9.jpg resize: (1080, 1632) 1335568065 -4.5699815314080725 treat image : temp/1738949847_596729_1335567968_b22753d7d77361750a2a45b15cffbd83.jpg resize: (1080, 1632) 1335567968 -4.793506645745709 treat image : temp/1738949847_596729_1335567965_635c8b98584d08843430b5cfe427d42e.jpg resize: (1080, 1632) 1335567965 -6.3182742559591105 treat image : temp/1738949847_596729_1335567960_b335bc97f232fba6eebda5ebf5b4c044.jpg resize: (1080, 1632) 1335567960 -6.396067473217927 treat image : temp/1738949847_596729_1335567956_64c07461fba9922bc6f836799f8dc340.jpg resize: (1080, 1632) 1335567956 -5.136719247906592 treat image : temp/1738949847_596729_1335567952_4de919d54f4f0758b6bb7516c57337fd.jpg resize: (1080, 1632) 1335567952 -6.212879884194042 treat image : temp/1738949847_596729_1335567946_ebda4accd1f6303363a7f869f6875533.jpg resize: (1080, 1632) 1335567946 -3.77495755533191 treat image : temp/1738949847_596729_1335566326_7a1bdacc37eda51f19ea86ff153dc5ef.jpg resize: (1080, 1632) 1335566326 -6.292137085982601 treat image : temp/1738949847_596729_1335566321_28cad7e7d52d8aca466eb2c8eac95e26.jpg resize: (1080, 1632) 1335566321 -5.881866291962594 treat image : temp/1738949847_596729_1335566318_af056b384cc871113a69648d703e2f4d.jpg resize: (1080, 1632) 1335566318 -5.814681473789901 treat image : temp/1738949847_596729_1335566314_458d478ca5d5d18b16c448c38e2ded64.jpg resize: (1080, 1632) 1335566314 -5.173666544052349 Inside saveOutput : final : True verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 20 time used for this insertion : 0.015882253646850586 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 20 time used for this insertion : 0.01767754554748535 begin to insert list_values into mtr_datou_result : length of list_values in save_final : 20 time used for this insertion : 0.018096208572387695 save missing photos in datou_result : time spend for datou_step_exec : 14.587106943130493 time spend to save output : 0.05603814125061035 total time spend for step 1 : 14.643145084381104 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 13.87user 2.78system 0:20.43elapsed 81%CPU (0avgtext+0avgdata 137608maxresident)k 2464inputs+10888outputs (16major+229659minor)pagefaults 0swaps