python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 8 -a ' -a 4871 -l 100 ' -s datou_current_4871 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 2704263 load datou : 4871 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? load thcls load pdts Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec no input labels no input values updating current state to 1 we have a portfolio with more photos than limit : 226>100 please execute split_portfolio.py -i 20128731 -l 100 size over we load limit photo not treated list_input_json: {} Current got : datou_id : 4871, datou_cur_ids : ['2535930'] with mtr_portfolio_ids : ['20128731'] and first list_photo_ids : [] new path : /proc/2704263/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! List Step Type Loaded in datou : blur_detection over limit max, limiting to limit_max 20 list_input_json : {} origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 20 ; length of list_pids : 20 ; length of list_args : 20 time to download the photos : 2.0038788318634033 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 1 step1:blur_detection Sat Feb 1 02:32:29 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec inside step blur_detection methode: ratio et variance treat image : temp/1738373548_2704263_1333555404_0082d950cb1fc726be10c86bd2766a24.jpg resize: (1080, 1632) 1333555404 -3.6587466364471575 treat image : temp/1738373548_2704263_1333555402_f6148890834752f0d966c90ea8ba53cd.jpg resize: (1080, 1632) 1333555402 -2.377164222442041 treat image : temp/1738373548_2704263_1333555386_7d1aab2862d838ac8ff26038b5f0c154.jpg resize: (1080, 1632) 1333555386 -5.481245450198529 treat image : temp/1738373548_2704263_1333555384_d16033d7a17acd1f189c0d81169101ba.jpg resize: (1080, 1632) 1333555384 -5.149230723417907 treat image : temp/1738373548_2704263_1333555355_1a6691ee6df76d9b83cc819a46f379fb.jpg resize: (1080, 1632) 1333555355 -4.85137582100797 treat image : temp/1738373548_2704263_1333555352_99aaa6ed96400052e12415a458c7601a.jpg resize: (1080, 1632) 1333555352 -6.390964935668701 treat image : temp/1738373548_2704263_1333555249_d1fec3a8234c6a7b9d416c50de061528.jpg resize: (1080, 1632) 1333555249 -6.001928908514171 treat image : temp/1738373548_2704263_1333555232_69cb1b24936306222535b3a2a96e103a.jpg resize: (1080, 1632) 1333555232 -6.173563676024038 treat image : temp/1738373548_2704263_1333555226_d4fe7514d11cdcec163f39eaa8d018e6.jpg resize: (1080, 1632) 1333555226 -6.515328243974006 treat image : temp/1738373548_2704263_1333555224_e511a640b3f9a3aa496a6b1a1214f903.jpg resize: (1080, 1632) 1333555224 -5.923124696117145 treat image : temp/1738373548_2704263_1333555221_551d66ba5d414b61fe371e247d46f34a.jpg resize: (1080, 1632) 1333555221 -6.108657683504242 treat image : temp/1738373548_2704263_1333555203_dfdb866c5559086ec30aef4e70c73eac.jpg resize: (1080, 1632) 1333555203 -6.404176626274521 treat image : temp/1738373548_2704263_1333555198_23852a2fc30e79ad3e5b346b69c8d653.jpg resize: (1080, 1632) 1333555198 -5.595001860259832 treat image : temp/1738373548_2704263_1333555192_bdab90ac3536803f76853994e70763a9.jpg resize: (1080, 1632) 1333555192 -5.86520073529368 treat image : temp/1738373548_2704263_1333555184_426e8b8b0241552c69435851e3954e94.jpg resize: (1080, 1632) 1333555184 -6.650120548598386 treat image : temp/1738373548_2704263_1333555181_251076b420dd74ad1109b2c8992c625a.jpg resize: (1080, 1632) 1333555181 -6.3563375907558015 treat image : temp/1738373548_2704263_1333555178_8e8b1c677fda0f06282d8ef8cb4b56ba.jpg resize: (1080, 1632) 1333555178 -6.260520892354282 treat image : temp/1738373548_2704263_1333555174_eeef08d52476071e5e155febc54846bf.jpg resize: (1080, 1632) 1333555174 -2.0273047966519977 treat image : temp/1738373548_2704263_1333555167_aa529a16d68eb41c82f980679ce8aea7.jpg resize: (1080, 1632) 1333555167 -5.9611944866197355 treat image : temp/1738373548_2704263_1333555165_0e638a9a4f1eea20221ea4466859e094.jpg resize: (1080, 1632) 1333555165 -1.0322694268904964 Inside saveOutput : final : True verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 20 time used for this insertion : 0.14969921112060547 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 20 time used for this insertion : 0.015978097915649414 begin to insert list_values into mtr_datou_result : length of list_values in save_final : 20 time used for this insertion : 0.016670942306518555 save missing photos in datou_result : time spend for datou_step_exec : 18.946045637130737 time spend to save output : 0.18763041496276855 total time spend for step 1 : 19.133676052093506 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 15.77user 6.01system 0:24.25elapsed 89%CPU (0avgtext+0avgdata 139960maxresident)k 112inputs+11368outputs (1major+251977minor)pagefaults 0swaps