python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 8 -a ' -a 4871 -l 100 ' -s datou_current_4871 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 2249716 load datou : 4871 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? load thcls load pdts Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec no input labels no input values updating current state to 1 we have a portfolio with more photos than limit : 2262>100 please execute split_portfolio.py -i 21962613 -l 100 size over we load limit photo not treated list_input_json: {} Current got : datou_id : 4871, datou_cur_ids : ['2714916'] with mtr_portfolio_ids : ['21962613'] and first list_photo_ids : [] new path : /proc/2249716/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! List Step Type Loaded in datou : blur_detection over limit max, limiting to limit_max 20 list_input_json : {} origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 20 ; length of list_pids : 20 ; length of list_args : 20 time to download the photos : 2.5849900245666504 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 1 step1:blur_detection Wed Apr 2 16:17:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec inside step blur_detection methode: ratio et variance treat image : temp/1743603447_2249716_1349506348_ce18111de43f06a0ff648419e2b781ca.jpg resize: (1080, 1632) 1349506348 -0.8530279939984001 treat image : temp/1743603447_2249716_1349506345_3caf8f7fa724c4f46f0974bb21214add.jpg resize: (1080, 1632) 1349506345 -4.637571696670268 treat image : temp/1743603447_2249716_1349506339_3b2460693e99491b090b1107befe1e98.jpg resize: (1080, 1632) 1349506339 -3.6564289089929174 treat image : temp/1743603447_2249716_1349506308_d9a34a58a899d6893bc1b43dc16c70f6.jpg resize: (1080, 1632) 1349506308 -6.371238601254303 treat image : temp/1743603447_2249716_1349506113_36dcf1f21f88f750f14802601e647c16.jpg resize: (1080, 1632) 1349506113 -5.638897877568657 treat image : temp/1743603447_2249716_1349506089_6c27e1264f2aab45f52b936dfef8d148.jpg resize: (1080, 1632) 1349506089 -5.642880091323465 treat image : temp/1743603447_2249716_1349506064_1442d0edef9848ef96058d0d22919c99.jpg resize: (1080, 1632) 1349506064 -5.8004379542550595 treat image : temp/1743603447_2249716_1349506038_6b720151046a5d9ac0363d49daa163d3.jpg resize: (1080, 1632) 1349506038 -6.172157630239518 treat image : temp/1743603447_2249716_1349506019_f50cabec29c9f4f80a47fadffe179483.jpg resize: (1080, 1632) 1349506019 -4.55269575900881 treat image : temp/1743603447_2249716_1349500448_f555094b1b9056493a81483eeb1d5042.jpg resize: (1080, 1632) 1349500448 -3.658731286599368 treat image : temp/1743603447_2249716_1349500445_1d9f88a22d60c54e9957bc5e1a8d6d95.jpg resize: (1080, 1632) 1349500445 -5.680097548268106 treat image : temp/1743603447_2249716_1349500410_807c20122cc40c3937f58c3d0860c930.jpg resize: (1080, 1632) 1349500410 -4.624335251149516 treat image : temp/1743603447_2249716_1349500405_987291c5d45e876b0b68a622a44f9d6e.jpg resize: (1080, 1632) 1349500405 -5.303248035719474 treat image : temp/1743603447_2249716_1349500403_03109dfab2369ba1d72c109a251ebd7d.jpg resize: (1080, 1632) 1349500403 -5.347180465077554 treat image : temp/1743603447_2249716_1349500375_bd393000b2de10300ca030e0b115770f.jpg resize: (1080, 1632) 1349500375 -4.762299885767974 treat image : temp/1743603447_2249716_1349500373_cae6ba92d6dab23495fc97101e033446.jpg resize: (1080, 1632) 1349500373 -4.90527768769691 treat image : temp/1743603447_2249716_1349500369_95a31751a5288c4731e5241ac780873f.jpg resize: (1080, 1632) 1349500369 -4.786737545195368 treat image : temp/1743603447_2249716_1349500367_055b0fceb266bf8099ab88498553f772.jpg resize: (1080, 1632) 1349500367 -3.956479701051651 treat image : temp/1743603447_2249716_1349500358_08d2085dfc01c7be055c20cf8b2a0fe4.jpg resize: (1080, 1632) 1349500358 -4.662075138030432 treat image : temp/1743603447_2249716_1349500331_5023c5d7ab76aa4d531d130cacf1c59c.jpg resize: (1080, 1632) 1349500331 -1.1253367231472122 Inside saveOutput : final : True verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 20 time used for this insertion : 0.013819456100463867 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 20 time used for this insertion : 0.013866662979125977 begin to insert list_values into mtr_datou_result : length of list_values in save_final : 20 time used for this insertion : 0.018914461135864258 save missing photos in datou_result : time spend for datou_step_exec : 15.977386474609375 time spend to save output : 0.05144238471984863 total time spend for step 1 : 16.028828859329224 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 14.79user 3.63system 0:20.96elapsed 87%CPU (0avgtext+0avgdata 137536maxresident)k 368inputs+9608outputs (9major+224625minor)pagefaults 0swaps