python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 8 -a ' -a 4871 -l 100 ' -s datou_current_4871 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 963329 load datou : 4871 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? load thcls load pdts Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec no input labels no input values updating current state to 1 we have a portfolio with more photos than limit : 3611>100 please execute split_portfolio.py -i 20234215 -l 100 size over we load limit photo not treated list_input_json: {} Current got : datou_id : 4871, datou_cur_ids : ['2547945'] with mtr_portfolio_ids : ['20234215'] and first list_photo_ids : [] new path : /proc/963329/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! List Step Type Loaded in datou : blur_detection over limit max, limiting to limit_max 20 list_input_json : {} origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 16 ; length of list_pids : 16 ; length of list_args : 16 time to download the photos : 1.605025053024292 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 1 step1:blur_detection Wed Feb 5 21:27:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec inside step blur_detection methode: ratio et variance treat image : temp/1738787248_963329_1335027067_9957430ac5a952a785cd559f91f73ba2.jpg resize: (1080, 1632) 1335027067 -3.6392500759619075 treat image : temp/1738787248_963329_1335027065_de678f9784cd3082ce6684acf348eb74.jpg resize: (1080, 1632) 1335027065 -1.9782530339533697 treat image : temp/1738787248_963329_1335027063_66dd940a9364f233ba1b58e6b3f7734b.jpg resize: (1080, 1632) 1335027063 -2.618699869239003 treat image : temp/1738787248_963329_1335027059_2ca0291c6d9f393335f5295ce1a3f120.jpg resize: (1080, 1632) 1335027059 -5.136033102511927 treat image : temp/1738787248_963329_1335027039_4034ab5ab38a34d29c801eb7108d8e30.jpg resize: (1080, 1632) 1335027039 -4.037674898688485 treat image : temp/1738787248_963329_1335027037_a8df76b999676af75bdfcd93f82ceed2.jpg resize: (1080, 1632) 1335027037 -6.069670017861027 treat image : temp/1738787248_963329_1335027033_808abe019a1507db8ddcfbb27c89ffbc.jpg resize: (1080, 1632) 1335027033 -6.561469464965654 treat image : temp/1738787248_963329_1335026921_c3f0f9b09315fa5eb6b2a1381f2ed97e.jpg resize: (1080, 1632) 1335026921 -4.284947228880893 treat image : temp/1738787248_963329_1335026882_65b249b8dc475d7eddf17e356d67b50b.jpg resize: (1080, 1632) 1335026882 -6.0423773346075595 treat image : temp/1738787248_963329_1335026800_5a4e2354628df01cf0fe1540df45edcd.jpg resize: (1080, 1632) 1335026800 -5.722793223647665 treat image : temp/1738787248_963329_1335026774_41c03752fdec6981a7a456ad4e98dcff.jpg resize: (1080, 1632) 1335026774 -2.0933379973068527 treat image : temp/1738787248_963329_1335026773_32619100ddc60c7052958b281913787f.jpg resize: (1080, 1632) 1335026773 -6.18294894192445 treat image : temp/1738787248_963329_1335026663_d8a5c7be060b6b1c41e4283210765436.jpg resize: (1080, 1632) 1335026663 -5.7582791243976175 treat image : temp/1738787248_963329_1335026553_c3c83496760a2e9bc85dfc912810673d.jpg resize: (1080, 1632) 1335026553 -5.973069221062693 treat image : temp/1738787248_963329_1335026316_0a2ff495ea49a04b4a3442ee211ddd65.jpg resize: (1080, 1632) 1335026316 -6.57789534243675 treat image : temp/1738787248_963329_1335026166_5c9054204eee8891396bc7dcfcf31e8c.jpg resize: (1080, 1632) 1335026166 -6.052249536032285 Inside saveOutput : final : True verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 16 time used for this insertion : 0.012625455856323242 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 16 time used for this insertion : 0.011938333511352539 begin to insert list_values into mtr_datou_result : length of list_values in save_final : 16 time used for this insertion : 0.013167858123779297 save missing photos in datou_result : time spend for datou_step_exec : 10.757503986358643 time spend to save output : 0.042073726654052734 total time spend for step 1 : 10.799577713012695 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 16 set_done_treatment 11.85user 1.46system 0:15.45elapsed 86%CPU (0avgtext+0avgdata 140548maxresident)k 0inputs+8328outputs (7major+106533minor)pagefaults 0swaps