python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 8 -a ' -a 4871 -l 100 ' -s datou_current_4871 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 2280443 load datou : 4871 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? load thcls load pdts Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec no input labels no input values updating current state to 1 we have a portfolio with more photos than limit : 4567>100 please execute split_portfolio.py -i 18811123 -l 100 size over we load limit photo not treated list_input_json: {} Current got : datou_id : 4871, datou_cur_ids : ['2417733'] with mtr_portfolio_ids : ['18811123'] and first list_photo_ids : [] new path : /proc/2280443/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! List Step Type Loaded in datou : blur_detection over limit max, limiting to limit_max 20 list_input_json : {} origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 20 ; length of list_pids : 20 ; length of list_args : 20 time to download the photos : 2.8172693252563477 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 1 step1:blur_detection Sat Feb 1 00:17:30 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec inside step blur_detection methode: ratio et variance treat image : temp/1738365448_2280443_1333035182_9ad003c638bcc7c16d3e7d4bbed905a2.jpg resize: (1080, 1632) 1333035182 1.4062024822737327 treat image : temp/1738365448_2280443_1333035176_c40453306a156a69657765e65e03777e.jpg resize: (1080, 1632) 1333035176 1.6107751150250207 treat image : temp/1738365448_2280443_1333035170_92158c8ff66f43f416ece40693186498.jpg resize: (1080, 1632) 1333035170 2.3251748947113806 treat image : temp/1738365448_2280443_1333035080_ab24b3fa81de5464f4278e7f5f4a68b1.jpg resize: (1080, 1632) 1333035080 1.7356805469651695 treat image : temp/1738365448_2280443_1333035050_5332f44b882358083cb6950da16ba6c3.jpg resize: (1080, 1632) 1333035050 2.521037973793896 treat image : temp/1738365448_2280443_1333035028_1da8c6edfd5f15bf989055d914b5f57d.jpg resize: (1080, 1632) 1333035028 2.4928297102051373 treat image : temp/1738365448_2280443_1333035015_e4794e3f9292ade8803d9d7c5efea4f9.jpg resize: (1080, 1632) 1333035015 2.2031710082318385 treat image : temp/1738365448_2280443_1333034988_194ca596b112cc96d75d3dfd1e7cbe53.jpg resize: (1080, 1632) 1333034988 2.6840065949619083 treat image : temp/1738365448_2280443_1333034981_916ecc04c46b6f2a66b5e213856e056d.jpg resize: (1080, 1632) 1333034981 4.0168124305075406 treat image : temp/1738365448_2280443_1333034932_48368704d7eb201b7a5d42ea251eb5d0.jpg resize: (1080, 1632) 1333034932 3.6425319660423514 treat image : temp/1738365448_2280443_1333034828_55772689843ec44214e97ee510a8c6ff.jpg resize: (1080, 1632) 1333034828 0.6354432594212849 treat image : temp/1738365448_2280443_1333034822_7e903e3e6abd4a1cff5126ffbf9e245a.jpg resize: (1080, 1632) 1333034822 0.6986822551988466 treat image : temp/1738365448_2280443_1333034820_dce7bfe0787e2b6c253b50dcdade2d7d.jpg resize: (1080, 1632) 1333034820 0.786421870675961 treat image : temp/1738365448_2280443_1333034816_2804cc2de895932fbe560e0369f425bd.jpg resize: (1080, 1632) 1333034816 0.9767573412493902 treat image : temp/1738365448_2280443_1333034812_bc21a95698aec0176f4d87b33ecc4d10.jpg resize: (1080, 1632) 1333034812 1.5156744068753314 treat image : temp/1738365448_2280443_1333034811_910db62a742aea208845cf59525733e8.jpg resize: (1080, 1632) 1333034811 3.3752436978634246 treat image : temp/1738365448_2280443_1333034739_c6675076fedcf3790b779f3118ba89d8.jpg resize: (1080, 1632) 1333034739 2.61030920694516 treat image : temp/1738365448_2280443_1333034736_1b5b248f0ac9a4a8c6ad613acbad926d.jpg resize: (1080, 1632) 1333034736 2.5285684826991686 treat image : temp/1738365448_2280443_1333034728_afb7861c43721e874585e78d9776410c.jpg resize: (1080, 1632) 1333034728 2.7771711776681283 treat image : temp/1738365448_2280443_1333034713_6a092bde515be0b3ea3aca125aa77460.jpg resize: (1080, 1632) 1333034713 2.388040517562896 Inside saveOutput : final : True verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 20 time used for this insertion : 0.01965928077697754 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 20 time used for this insertion : 0.01696038246154785 begin to insert list_values into mtr_datou_result : length of list_values in save_final : 20 time used for this insertion : 0.018840312957763672 save missing photos in datou_result : time spend for datou_step_exec : 15.210149765014648 time spend to save output : 0.060114145278930664 total time spend for step 1 : 15.270263910293579 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 14.12user 3.57system 0:19.90elapsed 88%CPU (0avgtext+0avgdata 143288maxresident)k 5000inputs+3144outputs (46major+259700minor)pagefaults 0swaps