python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 8 -a ' -a 4871 -l 100 ' -s datou_current_4871 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 2851328 load datou : 4871 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? load thcls load pdts Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec no input labels no input values updating current state to 1 we have a portfolio with more photos than limit : 3893>100 please execute split_portfolio.py -i 22109363 -l 100 size over we load limit photo not treated list_input_json: {} Current got : datou_id : 4871, datou_cur_ids : ['2728262'] with mtr_portfolio_ids : ['22109363'] and first list_photo_ids : [] new path : /proc/2851328/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! List Step Type Loaded in datou : blur_detection over limit max, limiting to limit_max 20 list_input_json : {} origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 20 ; length of list_pids : 20 ; length of list_args : 20 time to download the photos : 2.4345736503601074 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 1 step1:blur_detection Thu Apr 17 22:02:31 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec inside step blur_detection methode: ratio et variance treat image : temp/1744920149_2851328_1350854957_e647770798ffc3c4e1991e58d3463ca0.jpg resize: (1080, 1632) 1350854957 -5.266363526460054 treat image : temp/1744920149_2851328_1350854877_6669ddf27dcaf15e2e73b7c4dd1aac14.jpg resize: (1080, 1632) 1350854877 -0.7522648505345356 treat image : temp/1744920149_2851328_1350854791_a924948e224846cf3c21dab6fa583650.jpg resize: (1080, 1632) 1350854791 -3.053675173477021 treat image : temp/1744920149_2851328_1350854701_107b34d550f0cefcc9424cbb5df94758.jpg resize: (1080, 1632) 1350854701 -3.0961194976862982 treat image : temp/1744920149_2851328_1350854645_4f0852182638fa5571723073d93b6abf.jpg resize: (1080, 1632) 1350854645 -5.6291995615409816 treat image : temp/1744920149_2851328_1350854180_93405574e7d0d5a05d28275e93558d8c.jpg resize: (1080, 1632) 1350854180 -3.763832882463095 treat image : temp/1744920149_2851328_1350854130_466949cfb392ee60970241035505a45f.jpg resize: (1080, 1632) 1350854130 -5.632132268782081 treat image : temp/1744920149_2851328_1350854102_408ed10c86a0f78879aed0f306ef26e5.jpg resize: (1080, 1632) 1350854102 -5.30510986469529 treat image : temp/1744920149_2851328_1350854074_13c090c2f50fe480eb421b5628284a87.jpg resize: (1080, 1632) 1350854074 -4.31840028448955 treat image : temp/1744920149_2851328_1350854036_b34d84d9dd29eb0dcb9e0beca2a7b187.jpg resize: (1080, 1632) 1350854036 -6.063123962742834 treat image : temp/1744920149_2851328_1350853991_8c27447af68fc944173ee4a48ed356dc.jpg resize: (1080, 1632) 1350853991 -5.746627794080528 treat image : temp/1744920149_2851328_1350853678_439cb658b0e7eab91544c4d9948422c8.jpg resize: (1080, 1632) 1350853678 -1.7530707449956697 treat image : temp/1744920149_2851328_1350853620_2bbb237b31ddc03bdaa0e8f17c5a8889.jpg resize: (1080, 1632) 1350853620 -2.0428864369131756 treat image : temp/1744920149_2851328_1350853567_b9bf21dba4e34254bd00a27d8b790e1a.jpg resize: (1080, 1632) 1350853567 -3.2856883429403734 treat image : temp/1744920149_2851328_1350853521_c6f909af81f97991b5b0dd7f709ee8b5.jpg resize: (1080, 1632) 1350853521 -3.490176916416962 treat image : temp/1744920149_2851328_1350853329_6e1da9865eb2e7ff6aa22d0eba0a9515.jpg resize: (1080, 1632) 1350853329 -5.224674230152121 treat image : temp/1744920149_2851328_1350853250_a65ac3afddb4b00b573a66c0480d318f.jpg resize: (1080, 1632) 1350853250 -5.730586201905084 treat image : temp/1744920149_2851328_1350853190_95fa6b1f874b2215142615ec7686afee.jpg resize: (1080, 1632) 1350853190 -3.5738025961518427 treat image : temp/1744920149_2851328_1350853127_a8ed7e0c742bafc131cb443bd38bed83.jpg resize: (1080, 1632) 1350853127 -5.916552650549083 treat image : temp/1744920149_2851328_1350853066_7f2c0ffd2911ab1482e8e3eeb653c408.jpg resize: (1080, 1632) 1350853066 -5.768218923365487 Inside saveOutput : final : True verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 20 time used for this insertion : 0.012978315353393555 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 20 time used for this insertion : 0.014219045639038086 begin to insert list_values into mtr_datou_result : length of list_values in save_final : 20 time used for this insertion : 0.015231132507324219 save missing photos in datou_result : time spend for datou_step_exec : 13.96200442314148 time spend to save output : 0.04698514938354492 total time spend for step 1 : 14.008989572525024 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 14.87user 1.56system 0:20.00elapsed 82%CPU (0avgtext+0avgdata 139552maxresident)k 2432inputs+9520outputs (11major+125656minor)pagefaults 0swaps