python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 8 -a ' -a 4871 -l 100 ' -s datou_current_4871 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 3918001 load datou : 4871 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? load thcls load pdts Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec no input labels no input values updating current state to 1 we have a portfolio with more photos than limit : 4791>100 please execute split_portfolio.py -i 19382293 -l 100 size over we load limit photo not treated list_input_json: {} Current got : datou_id : 4871, datou_cur_ids : ['2460022'] with mtr_portfolio_ids : ['19382293'] and first list_photo_ids : [] new path : /proc/3918001/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! List Step Type Loaded in datou : blur_detection over limit max, limiting to limit_max 20 list_input_json : {} origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 20 ; length of list_pids : 20 ; length of list_args : 20 time to download the photos : 3.0198514461517334 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 1 step1:blur_detection Tue Mar 25 22:52:33 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec inside step blur_detection methode: ratio et variance treat image : temp/1742939550_3918001_1333491097_5da0af6649bb27c9888c906ba0c1023c.jpg resize: (1080, 1632) 1333491097 1.7284276365782494 treat image : temp/1742939550_3918001_1333491011_feaf19cf3a870f95b45c9a49b2dbf713.jpg resize: (1080, 1632) 1333491011 2.2487786289468223 treat image : temp/1742939550_3918001_1333491006_a336019c9b63758fa9a5b6e6818f86ff.jpg resize: (1080, 1632) 1333491006 3.73265546809072 treat image : temp/1742939550_3918001_1333491004_ba22c2d996281aae614f8351cc822af6.jpg resize: (1080, 1632) 1333491004 2.209601402326102 treat image : temp/1742939550_3918001_1333491001_f5b7bae6b668227117193670d7604266.jpg resize: (1080, 1632) 1333491001 2.3570902374254543 treat image : temp/1742939550_3918001_1333490997_d9a8b7cc2f63f45ae506e5d72990bc10.jpg resize: (1080, 1632) 1333490997 0.7852373079389898 treat image : temp/1742939550_3918001_1333490992_82228acf805257df06043a766238c8ad.jpg resize: (1080, 1632) 1333490992 2.6904640027215128 treat image : temp/1742939550_3918001_1333490544_8e2d5a4ea0828e32ab9374d7d91017db.jpg resize: (1080, 1632) 1333490544 3.3461047559881147 treat image : temp/1742939550_3918001_1333490541_8cfa485fc2e025cd591fb6ed833e32db.jpg resize: (1080, 1632) 1333490541 0.6056900084024086 treat image : temp/1742939550_3918001_1333490539_30110260bb3014da7f06631a13754870.jpg resize: (1080, 1632) 1333490539 0.8108027461723064 treat image : temp/1742939550_3918001_1333490536_27c043f1b66d24dd94c7a2100dd1de17.jpg resize: (1080, 1632) 1333490536 1.7306185163430685 treat image : temp/1742939550_3918001_1333490532_bd70937ec77916fa574d567181f57e20.jpg resize: (1080, 1632) 1333490532 1.5908015651206922 treat image : temp/1742939550_3918001_1333490527_9e07f7b3886f7d16f6114925be472255.jpg resize: (1080, 1632) 1333490527 2.5189578938247776 treat image : temp/1742939550_3918001_1333489375_a411a3c8614e927ac97de2ba7c39881c.jpg resize: (1080, 1632) 1333489375 3.862047076000936 treat image : temp/1742939550_3918001_1333489372_a465c5fa257d6901166163367a15a265.jpg resize: (1080, 1632) 1333489372 5.562973370810318 treat image : temp/1742939550_3918001_1333489369_3373f6c6f371e0a885c99f9e061511a8.jpg resize: (1080, 1632) 1333489369 4.93248965039624 treat image : temp/1742939550_3918001_1333489367_a6e9602fd3477c7b58798764a08fa46d.jpg resize: (1080, 1632) 1333489367 2.026340161756736 treat image : temp/1742939550_3918001_1333489365_105803f4efd90c178856fc3f13506727.jpg resize: (1080, 1632) 1333489365 2.2326504266080494 treat image : temp/1742939550_3918001_1333489363_8e39857df419ca2b788789782b0b3819.jpg resize: (1080, 1632) 1333489363 3.7850022049417613 treat image : temp/1742939550_3918001_1333489344_dedba3b9c4290d3efac25b4b4bd96cfe.jpg resize: (1080, 1632) 1333489344 3.8547065149352724 Inside saveOutput : final : True verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 20 time used for this insertion : 0.01874709129333496 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 20 time used for this insertion : 3.222930669784546 begin to insert list_values into mtr_datou_result : length of list_values in save_final : 20 time used for this insertion : 0.01788616180419922 save missing photos in datou_result : time spend for datou_step_exec : 15.503140449523926 time spend to save output : 3.2643020153045654 total time spend for step 1 : 18.76744246482849 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 14.49user 3.67system 0:26.22elapsed 69%CPU (0avgtext+0avgdata 140416maxresident)k 680inputs+3080outputs (19major+264958minor)pagefaults 0swaps