python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 8 -a ' -a 4871 -l 100 ' -s datou_current_4871 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 4179417 load datou : 4871 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? load thcls load pdts Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec no input labels no input values updating current state to 1 we have a portfolio with more photos than limit : 543>100 please execute split_portfolio.py -i 21674640 -l 100 size over we load limit photo not treated list_input_json: {} Current got : datou_id : 4871, datou_cur_ids : ['2687508'] with mtr_portfolio_ids : ['21674640'] and first list_photo_ids : [] new path : /proc/4179417/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! List Step Type Loaded in datou : blur_detection over limit max, limiting to limit_max 20 list_input_json : {} origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 19 ; length of list_pids : 19 ; length of list_args : 19 time to download the photos : 2.4411709308624268 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 1 step1:blur_detection Tue Mar 25 07:17:32 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec inside step blur_detection methode: ratio et variance treat image : temp/1742883450_4179417_1347102440_936ed67af557aa7a8e1ec73f739a948a.jpg resize: (1080, 1632) 1347102440 -6.358731538426349 treat image : temp/1742883450_4179417_1347102429_f308a6e84a0eb3be5b6f7ff666fc263d.jpg resize: (1080, 1632) 1347102429 -5.3366633408621995 treat image : temp/1742883450_4179417_1347102419_e75049fbca0dcc2f9ff3c6f2c591fb2b.jpg resize: (1080, 1632) 1347102419 -3.9041956539896523 treat image : temp/1742883450_4179417_1347102409_772a1fd747daa4048dc4f36ad3c3f28a.jpg resize: (1080, 1632) 1347102409 -6.061101499133441 treat image : temp/1742883450_4179417_1347102399_4263a5e6813e1f515224f46a368ad544.jpg resize: (1080, 1632) 1347102399 -6.09300323983209 treat image : temp/1742883450_4179417_1347102392_c00c0c0c29caef94ef263bb9dadb6ead.jpg resize: (1080, 1632) 1347102392 -6.357150573148954 treat image : temp/1742883450_4179417_1347102383_5512d39ad224c927e9b141100f51ece8.jpg resize: (1080, 1632) 1347102383 -5.719621271312527 treat image : temp/1742883450_4179417_1347102381_2fbd5339ad7ff0f66bbf618984f31c52.jpg resize: (1080, 1632) 1347102381 -6.29445836686314 treat image : temp/1742883450_4179417_1347102379_ff72831d956180f598e2f8ca1af65b7a.jpg resize: (1080, 1632) 1347102379 -2.603469500725655 treat image : temp/1742883450_4179417_1347102377_e76c5b25a7c055c809ddfed07531a954.jpg resize: (1080, 1632) 1347102377 -2.8551313519094976 treat image : temp/1742883450_4179417_1347102373_361cf9d02f5fc1228cb4b0c7c085d952.jpg resize: (1080, 1632) 1347102373 -6.14627822210856 treat image : temp/1742883450_4179417_1347102367_2d509edd23b64d2382baa8d76f7fe7c9.jpg resize: (1080, 1632) 1347102367 -5.822756583427045 treat image : temp/1742883450_4179417_1347102364_6208ed02bdfd25a5bd20db4eb0fe10b8.jpg resize: (1080, 1632) 1347102364 -6.234744269089941 treat image : temp/1742883450_4179417_1347102351_721a956e994a071297a632c608d53ae9.jpg resize: (1080, 1632) 1347102351 -5.996266649023966 treat image : temp/1742883450_4179417_1347102339_2ec73570b3134ab44a27e63ff8a1ee40.jpg resize: (1080, 1632) 1347102339 -4.540466029505856 treat image : temp/1742883450_4179417_1347102328_125986f90c664a72894d99081d6fa181.jpg resize: (1080, 1632) 1347102328 -6.242504530256419 treat image : temp/1742883450_4179417_1347102317_ffce527a678a728283500c5052b9438d.jpg resize: (1080, 1632) 1347102317 -5.719621271312527 treat image : temp/1742883450_4179417_1347102307_b5c609ef6a7ddcc78cc0668c8151a0f5.jpg resize: (1080, 1632) 1347102307 -6.29445836686314 treat image : temp/1742883450_4179417_1347102154_ad0b65e20a5b7d81756705aae1018361.jpg resize: (1080, 1632) 1347102154 -4.457192465203014 Inside saveOutput : final : True verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 19 time used for this insertion : 0.14255428314208984 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 19 time used for this insertion : 0.011871576309204102 begin to insert list_values into mtr_datou_result : length of list_values in save_final : 19 time used for this insertion : 0.6941254138946533 save missing photos in datou_result : time spend for datou_step_exec : 13.91258692741394 time spend to save output : 0.8533668518066406 total time spend for step 1 : 14.765953779220581 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 19 set_done_treatment 13.17user 3.13system 0:21.48elapsed 75%CPU (0avgtext+0avgdata 139848maxresident)k 112inputs+11136outputs (7major+205742minor)pagefaults 0swaps