python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 8 -a ' -a 4871 -l 100 ' -s datou_current_4871 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 1947727 load datou : 4871 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? load thcls load pdts Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec no input labels no input values updating current state to 1 we have a portfolio with more photos than limit : 762>100 please execute split_portfolio.py -i 20278395 -l 100 size over we load limit photo not treated list_input_json: {} Current got : datou_id : 4871, datou_cur_ids : ['2557277'] with mtr_portfolio_ids : ['20278395'] and first list_photo_ids : [] new path : /proc/1947727/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! List Step Type Loaded in datou : blur_detection over limit max, limiting to limit_max 20 list_input_json : {} origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 20 ; length of list_pids : 20 ; length of list_args : 20 time to download the photos : 2.112821340560913 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 1 step1:blur_detection Thu Feb 6 03:17:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec inside step blur_detection methode: ratio et variance treat image : temp/1738808247_1947727_1335098011_75ff3dbec41a04448f79d6b720045827.jpg resize: (1080, 1632) 1335098011 -6.142119068752749 treat image : temp/1738808247_1947727_1335098010_a9eee5b960d17cd060d4681148756263.jpg resize: (1080, 1632) 1335098010 -6.326609307451057 treat image : temp/1738808247_1947727_1335098009_464c1240a4e38566f3354f2fe5fb3b78.jpg resize: (1080, 1632) 1335098009 -6.289393214269239 treat image : temp/1738808247_1947727_1335098008_75ba9fa8682666249be2c24580113075.jpg resize: (1080, 1632) 1335098008 -5.90658089542425 treat image : temp/1738808247_1947727_1335098007_74bd34397bf9e316d2be86029aebed36.jpg resize: (1080, 1632) 1335098007 -2.0613528648740607 treat image : temp/1738808247_1947727_1335098006_4248e0420ea63134bff0de56594c8c36.jpg resize: (1080, 1632) 1335098006 -1.1608529917621666 treat image : temp/1738808247_1947727_1335097950_9615527e833ae08c9208abcc695675eb.jpg resize: (1080, 1632) 1335097950 -5.685265432471586 treat image : temp/1738808247_1947727_1335097936_73d9e927db8dfeddf9616edb384847d6.jpg resize: (1080, 1632) 1335097936 -5.970734522127226 treat image : temp/1738808247_1947727_1335097901_27f663a5485da3d8870414dc660d14b1.jpg resize: (1080, 1632) 1335097901 -5.666837722598168 treat image : temp/1738808247_1947727_1335097846_d99886974099db8905788f098ba44a5a.jpg resize: (1080, 1632) 1335097846 -5.799346443793066 treat image : temp/1738808247_1947727_1335097834_ac7559ae15c2b89fd900adcd51fef702.jpg resize: (1080, 1632) 1335097834 -5.772331586536481 treat image : temp/1738808247_1947727_1335097833_5e2ff14914919a649395824a4963e855.jpg resize: (1080, 1632) 1335097833 -6.383058975089198 treat image : temp/1738808247_1947727_1335097829_ff4dc7e7ab50b52dab364dec125aa94b.jpg resize: (1080, 1632) 1335097829 -5.097467374396468 treat image : temp/1738808247_1947727_1335097828_675b09f2bd12c42841e2cd4db9b5ac74.jpg resize: (1080, 1632) 1335097828 -4.69594180812842 treat image : temp/1738808247_1947727_1335097827_6037bb56747cb052e54fafcea94b515a.jpg resize: (1080, 1632) 1335097827 -5.298994656435283 treat image : temp/1738808247_1947727_1335097826_0393ada8535cdae85bb3add63d857bab.jpg resize: (1080, 1632) 1335097826 -5.861027750851238 treat image : temp/1738808247_1947727_1335097824_ca0e6af3e45d859b081e8db91c741eb6.jpg resize: (1080, 1632) 1335097824 -5.454264204705368 treat image : temp/1738808247_1947727_1335097822_a5a0f326086394de2407746a1d083b80.jpg resize: (1080, 1632) 1335097822 -5.614666705151631 treat image : temp/1738808247_1947727_1335097820_9dd04883292a51ed78afb99642fe3ff9.jpg resize: (1080, 1632) 1335097820 -0.33051509556748576 treat image : temp/1738808247_1947727_1335097817_75f698863de733accc5da033a7b93a1c.jpg resize: (1080, 1632) 1335097817 -3.9506765145640745 Inside saveOutput : final : True verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 20 time used for this insertion : 0.014777421951293945 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 20 time used for this insertion : 0.016991138458251953 begin to insert list_values into mtr_datou_result : length of list_values in save_final : 20 time used for this insertion : 0.017963886260986328 save missing photos in datou_result : time spend for datou_step_exec : 13.338635206222534 time spend to save output : 0.05468392372131348 total time spend for step 1 : 13.393319129943848 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 13.86user 2.24system 0:18.39elapsed 87%CPU (0avgtext+0avgdata 140464maxresident)k 0inputs+10544outputs (12major+167042minor)pagefaults 0swaps