python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 4254 ' -s datou_consolidate -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 2973802 load datou : 4254 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11635 copy_chis is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11636 consolidate_hashtags_from_manual_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11633 rle_unique_nms_with_priority is not consistent : 3 used against 1 in the step definition ! WARNING : number of outputs for step 11633 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11640 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11634 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11637 blur_detection have less inputs used (0) than in the step definition (1) : maybe we manage optionnal inputs ! Step 11638 brightness have less inputs used (0) than in the step definition (1) : maybe we manage optionnal inputs ! Step 11639 send_mail_cod have less inputs used (4) than in the step definition (5) : maybe we manage optionnal inputs ! Step 11641 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11635 have datatype=11 whereas input 0 of step 11633 have datatype=2 WARNING : type of output 1 of step 11636 doesn't seem to be define in the database( WARNING : type of input 3 of step 11634 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11636 doesn't seem to be define in the database( WARNING : type of input 1 of step 11633 doesn't seem to be define in the database( WARNING : output 0 of step 11640 have datatype=10 whereas input 3 of step 11639 have datatype=6 WARNING : type of input 1 of step 11640 doesn't seem to be define in the database( WARNING : output 1 of step 11633 have datatype=7 whereas input 1 of step 11640 have datatype=None WARNING : type of output 1 of step 11640 doesn't seem to be define in the database( WARNING : type of input 4 of step 11634 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11635 doesn't seem to be define in the database( WARNING : type of input 1 of step 11636 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : id de la photo (peut être local ou global) was removed should we ? donnée sous forme de nombre was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? donnée sous forme de texte was removed should we ? [ (photo_id, photo_id_loc, hashtag_type, x0, x1, y0, y1, score), ...] was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? id de la photo (peut être local ou global) was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? donnée sous forme de texte was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? donnée sous forme de texte was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? load thcls load pdts Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec no input labels no input values updating current state to 1 list_input_json: {} Current got : datou_id : 4254, datou_cur_ids : ['3763082'] with mtr_portfolio_ids : ['25127822'] and first list_photo_ids : [] new path : /proc/2973802/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11635 copy_chis is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11636 consolidate_hashtags_from_manual_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11633 rle_unique_nms_with_priority is not consistent : 3 used against 1 in the step definition ! WARNING : number of outputs for step 11633 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11640 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11634 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11637 blur_detection have less inputs used (0) than in the step definition (1) : maybe we manage optionnal inputs ! Step 11638 brightness have less inputs used (0) than in the step definition (1) : maybe we manage optionnal inputs ! Step 11639 send_mail_cod have less inputs used (4) than in the step definition (5) : maybe we manage optionnal inputs ! Step 11641 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11635 have datatype=11 whereas input 0 of step 11633 have datatype=2 WARNING : type of output 1 of step 11636 doesn't seem to be define in the database( WARNING : type of input 3 of step 11634 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11636 doesn't seem to be define in the database( WARNING : type of input 1 of step 11633 doesn't seem to be define in the database( WARNING : output 0 of step 11640 have datatype=10 whereas input 3 of step 11639 have datatype=6 WARNING : type of input 1 of step 11640 doesn't seem to be define in the database( WARNING : output 1 of step 11633 have datatype=7 whereas input 1 of step 11640 have datatype=None WARNING : type of output 1 of step 11640 doesn't seem to be define in the database( WARNING : type of input 4 of step 11634 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11635 doesn't seem to be define in the database( WARNING : type of input 1 of step 11636 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : copy_chis, consolidate_hashtags_from_manual_portfolio, rle_unique_nms_with_priority, ventilate_hashtags_in_portfolio, final, blur_detection, brightness, send_mail_cod, split_time_score over limit max, limiting to limit_max 100 list_input_json : {} origin We have 1 , BFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFBFwe have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 18 ; length of list_pids : 18 ; length of list_args : 18 time to download the photos : 6.48130989074707 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 9 step1:copy_chis Tue Sep 23 11:18:33 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Begin step datou_step_copy_crop batch 1 Loaded 19 chid ids of type : 4200 batch 1 Loaded 188 chid ids of type : 4200 Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : copy_chis we use saveGeneral [1372886396, 1372886393, 1372886392, 1372886391, 1372886390, 1372886382, 1372886332, 1372886330, 1372886329, 1372886224, 1372886150, 1372886148, 1372883541, 1372883511, 1372883509, 1372883406, 1372883392, 1372883390] Looping around the photos to save general results len do output : 18 /1372883390Didn't retrieve data . /1372883392Didn't retrieve data . /1372883406Didn't retrieve data . /1372883509Didn't retrieve data . /1372883511Didn't retrieve data . /1372883541Didn't retrieve data . /1372886148Didn't retrieve data . /1372886150Didn't retrieve data . /1372886224Didn't retrieve data . /1372886329Didn't retrieve data . /1372886330Didn't retrieve data . /1372886332Didn't retrieve data . /1372886382Didn't retrieve data . /1372886390Didn't retrieve data . /1372886391Didn't retrieve data . /1372886392Didn't retrieve data . /1372886393Didn't retrieve data . /1372886396Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886396', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886393', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886392', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886391', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886390', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886382', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886332', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886330', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886329', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886224', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886150', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886148', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372883541', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372883511', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372883509', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372883406', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372883392', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372883390', None, None, None, None, None, '3763082') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 36 time used for this insertion : 0.01999378204345703 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.11835145950317383 time spend to save output : 0.02060866355895996 total time spend for step 1 : 0.1389601230621338 step2:consolidate_hashtags_from_manual_portfolio Tue Sep 23 11:18:33 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step consolidate_hashtags_from_manual_portfolio Iterating over portfolio : 25127822 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=25127822 AND mptpi.`type`=4200 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('barquette_opaque','carton','ela','environnement','etiquette','film_plastique','kraft','metal','papier','pehd','pet_clair','pet_opaque','textiles_sanitaires','pet_fonce','mal_croppe','flou','background')) To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=25127822 AND mptpi.`type`=4200 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('barquette_opaque','carton','ela','environnement','etiquette','film_plastique','kraft','metal','papier','pehd','pet_clair','pet_opaque','textiles_sanitaires','pet_fonce','mal_croppe','flou','background')) To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=25127822 AND mptpi.`type`=4200 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('barquette_opaque','carton','ela','environnement','etiquette','film_plastique','kraft','metal','papier','pehd','pet_clair','pet_opaque','textiles_sanitaires','pet_fonce','mal_croppe','flou','background')) To do TODO : # On doit donc construire les chi a partir des informations dans les photos filles query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=25132917 AND mpp.hide_status=0 ORDER BY ph.size desc TODO : # On doit donc construire les chi a partir des informations dans les photos filles query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=25132918 AND mpp.hide_status=0 ORDER BY ph.size desc batch 1 Loaded 106 chid ids of type : 4200 begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : TODO : # On doit donc construire les chi a partir des informations dans les photos filles query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=25132919 AND mpp.hide_status=0 ORDER BY ph.size desc batch 1 Loaded 109 chid ids of type : 4200 begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : TODO : # On doit donc construire les chi a partir des informations dans les photos filles query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=25132920 AND mpp.hide_status=0 ORDER BY ph.size desc TODO : # On doit donc construire les chi a partir des informations dans les photos filles query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=25132921 AND mpp.hide_status=0 ORDER BY ph.size desc batch 1 Loaded 87 chid ids of type : 4200 begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : TODO : # On doit donc construire les chi a partir des informations dans les photos filles query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=25132922 AND mpp.hide_status=0 ORDER BY ph.size desc batch 1 Loaded 22 chid ids of type : 4200 begin to find the sub_photo_id : begin to find the sub_photo_id : TODO : # On doit donc construire les chi a partir des informations dans les photos filles query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=25132923 AND mpp.hide_status=0 ORDER BY ph.size desc TODO : # On doit donc construire les chi a partir des informations dans les photos filles query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=25132925 AND mpp.hide_status=0 ORDER BY ph.size desc batch 1 Loaded 87 chid ids of type : 4200 begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : TODO : # On doit donc construire les chi a partir des informations dans les photos filles query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=25132926 AND mpp.hide_status=0 ORDER BY ph.size desc TODO : # On doit donc construire les chi a partir des informations dans les photos filles query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=25132927 AND mpp.hide_status=0 ORDER BY ph.size desc batch 1 Loaded 84 chid ids of type : 4200 begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : TODO : # On doit donc construire les chi a partir des informations dans les photos filles query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=25132928 AND mpp.hide_status=0 ORDER BY ph.size desc batch 1 Loaded 188 chid ids of type : 4200 begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : TODO : # On doit donc construire les chi a partir des informations dans les photos filles query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=25132929 AND mpp.hide_status=0 ORDER BY ph.size desc TODO : # On doit donc construire les chi a partir des informations dans les photos filles query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=25132930 AND mpp.hide_status=0 ORDER BY ph.size desc batch 1 Loaded 23 chid ids of type : 4200 begin to find the sub_photo_id : begin to find the sub_photo_id : TODO : # On doit donc construire les chi a partir des informations dans les photos filles query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=25132931 AND mpp.hide_status=0 ORDER BY ph.size desc TODO : # On doit donc construire les chi a partir des informations dans les photos filles query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=25132932 AND mpp.hide_status=0 ORDER BY ph.size desc TODO : # On doit donc construire les chi a partir des informations dans les photos filles query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=25132933 AND mpp.hide_status=0 ORDER BY ph.size desc batch 1 Loaded 136 chid ids of type : 4200 begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : begin to find the sub_photo_id : TODO : # On doit donc construire les chi a partir des informations dans les photos filles query : SELECT ph.photo_id FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=27128155 AND mpp.hide_status=0 ORDER BY ph.size desc To test ! Use context local managing function ! Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : consolidate_hashtags_from_manual_portfolio we use saveGeneral [1372886396, 1372886393, 1372886392, 1372886391, 1372886390, 1372886382, 1372886332, 1372886330, 1372886329, 1372886224, 1372886150, 1372886148, 1372883541, 1372883511, 1372883509, 1372883406, 1372883392, 1372883390] Looping around the photos to save general results len do output : 6 /Tassign value string error string indices must be integers invalid literal for int() with base 10: 'T' begin to insert list_values into mtr_datou_result : length of list_values in save_final : 18 time used for this insertion : 0.01822662353515625 save_final save missing photos in datou_result : time spend for datou_step_exec : 2.8866772651672363 time spend to save output : 0.018428325653076172 total time spend for step 2 : 2.9051055908203125 step3:rle_unique_nms_with_priority Tue Sep 23 11:18:36 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array We expect there is only one output and this part is used while all output are not tuple or array VR 22-3-18 : For now we do not clean correctly the datou structure Begin step rle-unique-nms batch 1 Loaded 156 chid ids of type : 4200 seulement à utiliser dans la step consolidation batch 1 Loaded 3 chid ids of type : 4200 Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! save time : 0.023842334747314453 seulement à utiliser dans la step consolidation batch 1 Loaded 12 chid ids of type : 4200 Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! save time : 0.025102853775024414 seulement à utiliser dans la step consolidation batch 1 Loaded 11 chid ids of type : 4200 Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! save time : 0.02703261375427246 seulement à utiliser dans la step consolidation batch 1 Loaded 17 chid ids of type : 4200 Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! save time : 0.027602195739746094 seulement à utiliser dans la step consolidation batch 1 Loaded 12 chid ids of type : 4200 Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! save time : 0.026808977127075195 seulement à utiliser dans la step consolidation batch 1 Loaded 11 chid ids of type : 4200 Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! save time : 0.025913238525390625 seulement à utiliser dans la step consolidation batch 1 Loaded 11 chid ids of type : 4200 Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! save time : 0.026897192001342773 seulement à utiliser dans la step consolidation batch 1 Loaded 6 chid ids of type : 4200 Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! save time : 0.02510237693786621 seulement à utiliser dans la step consolidation batch 1 Loaded 11 chid ids of type : 4200 Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! save time : 0.027901887893676758 seulement à utiliser dans la step consolidation batch 1 Loaded 11 chid ids of type : 4200 Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! save time : 0.025989294052124023 seulement à utiliser dans la step consolidation batch 1 Loaded 12 chid ids of type : 4200 Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! save time : 0.026953935623168945 seulement à utiliser dans la step consolidation batch 1 Loaded 10 chid ids of type : 4200 Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! save time : 0.027075767517089844 seulement à utiliser dans la step consolidation batch 1 Loaded 10 chid ids of type : 4200 Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! save time : 0.027344703674316406 seulement à utiliser dans la step consolidation batch 1 Loaded 12 chid ids of type : 4200 Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! save time : 0.027443885803222656 seulement à utiliser dans la step consolidation batch 1 Loaded 9 chid ids of type : 4200 Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! save time : 0.026974201202392578 seulement à utiliser dans la step consolidation batch 1 Loaded 13 chid ids of type : 4200 Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! save time : 0.026870012283325195 seulement à utiliser dans la step consolidation batch 1 Loaded 12 chid ids of type : 4200 Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! save time : 0.027869462966918945 seulement à utiliser dans la step consolidation batch 1 Loaded 5 chid ids of type : 4200 Number RLEs to save : 0 TO DO : save crop sub photo not yet done ! save time : 0.02483677864074707 map_output_result : {1372886396: (0.0, 'Should be the crop_list due to order', 0), 1372886393: (0.0, 'Should be the crop_list due to order', 0), 1372886392: (0.0, 'Should be the crop_list due to order', 0), 1372886391: (0.0, 'Should be the crop_list due to order', 0), 1372886390: (0.0, 'Should be the crop_list due to order', 0), 1372886382: (0.0, 'Should be the crop_list due to order', 0), 1372886332: (0.0, 'Should be the crop_list due to order', 0), 1372886330: (0.0, 'Should be the crop_list due to order', 0), 1372886329: (0.0, 'Should be the crop_list due to order', 0), 1372886224: (0.0, 'Should be the crop_list due to order', 0), 1372886150: (0.0, 'Should be the crop_list due to order', 0), 1372886148: (0.0, 'Should be the crop_list due to order', 0), 1372883541: (0.0, 'Should be the crop_list due to order', 0), 1372883511: (0.0, 'Should be the crop_list due to order', 0), 1372883509: (0.0, 'Should be the crop_list due to order', 0), 1372883406: (0.0, 'Should be the crop_list due to order', 0), 1372883392: (0.0, 'Should be the crop_list due to order', 0), 1372883390: (0.0, 'Should be the crop_list due to order', 0)} End step rle-unique-nms Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : rle_unique_nms_with_priority we use saveGeneral [1372886396, 1372886393, 1372886392, 1372886391, 1372886390, 1372886382, 1372886332, 1372886330, 1372886329, 1372886224, 1372886150, 1372886148, 1372883541, 1372883511, 1372883509, 1372883406, 1372883392, 1372883390] Looping around the photos to save general results len do output : 18 /1372886396.Didn't retrieve data . /1372886393.Didn't retrieve data . /1372886392.Didn't retrieve data . /1372886391.Didn't retrieve data . /1372886390.Didn't retrieve data . /1372886382.Didn't retrieve data . /1372886332.Didn't retrieve data . /1372886330.Didn't retrieve data . /1372886329.Didn't retrieve data . /1372886224.Didn't retrieve data . /1372886150.Didn't retrieve data . /1372886148.Didn't retrieve data . /1372883541.Didn't retrieve data . /1372883511.Didn't retrieve data . /1372883509.Didn't retrieve data . /1372883406.Didn't retrieve data . /1372883392.Didn't retrieve data . /1372883390.Didn't retrieve data . before output type Used above Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886396', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886393', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886392', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886391', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886390', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886382', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886332', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886330', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886329', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886224', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886150', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886148', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372883541', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372883511', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372883509', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372883406', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372883392', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372883390', None, None, None, None, None, '3763082') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 54 time used for this insertion : 0.014226436614990234 save_final save missing photos in datou_result : time spend for datou_step_exec : 2.892789602279663 time spend to save output : 0.014889955520629883 total time spend for step 3 : 2.907679557800293 step4:ventilate_hashtags_in_portfolio Tue Sep 23 11:18:39 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure beginning of datou step ventilate_hashtags_in_portfolio : To implement ! Iterating over portfolio : 25127822 get user id for portfolio 25127822 SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=25127822 AND mptpi.`type`=4200 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('barquette_opaque','carton','ela','environnement','etiquette','film_plastique','kraft','metal','papier','pehd','pet_clair','pet_opaque','textiles_sanitaires','pet_fonce','mal_croppe','flou')) AND mptpi.`min_score`=0.5 To do To do ! Use context local managing function ! SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=25127822 AND mptpi.`type`=4200 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('barquette_opaque','carton','ela','environnement','etiquette','film_plastique','kraft','metal','papier','pehd','pet_clair','pet_opaque','textiles_sanitaires','pet_fonce','mal_croppe','flou')) AND mptpi.`min_score`=0.5 To do lien utilise dans velours : https://marlene.fotonower.com/velours/25132917,25132918,25132919,25132920,25132921,25132922,25132923,25132925,25132926,25132927,25132928,25132929,25132930,25132931,25132932,25132933?tags=textiles_sanitaires,pet_opaque,environnement,kraft,film_plastique,papier,pehd,pet_clair,ela,barquette_opaque,mal_croppe,metal,etiquette,flou,pet_fonce,carton Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : ventilate_hashtags_in_portfolio we use saveGeneral [1372886396, 1372886393, 1372886392, 1372886391, 1372886390, 1372886382, 1372886332, 1372886330, 1372886329, 1372886224, 1372886150, 1372886148, 1372883541, 1372883511, 1372883509, 1372883406, 1372883392, 1372883390] Looping around the photos to save general results len do output : 1 /25127822. before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886396', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886393', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886392', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886391', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886390', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886382', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886332', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886330', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886329', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886224', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886150', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886148', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372883541', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372883511', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372883509', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372883406', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372883392', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372883390', None, None, None, None, None, '3763082') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 19 time used for this insertion : 0.02465653419494629 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.3479440212249756 time spend to save output : 0.024966716766357422 total time spend for step 4 : 0.372910737991333 step5:final Tue Sep 23 11:18:39 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 2 VR 22-3-18 : For now we do not clean correctly the datou structure Beginning of datou step final ! Inside saveOutput : final : False verbose : 0 original output for save of step final : {1372886396: ('0.9820645135352716',), 1372886393: ('0.9820645135352716',), 1372886392: ('0.9820645135352716',), 1372886391: ('0.9820645135352716',), 1372886390: ('0.9820645135352716',), 1372886382: ('0.9820645135352716',), 1372886332: ('0.9820645135352716',), 1372886330: ('0.9820645135352716',), 1372886329: ('0.9820645135352716',), 1372886224: ('0.9820645135352716',), 1372886150: ('0.9820645135352716',), 1372886148: ('0.9820645135352716',), 1372883541: ('0.9820645135352716',), 1372883511: ('0.9820645135352716',), 1372883509: ('0.9820645135352716',), 1372883406: ('0.9820645135352716',), 1372883392: ('0.9820645135352716',), 1372883390: ('0.9820645135352716',)} new output for save of step final : {1372886396: ('0.9820645135352716',), 1372886393: ('0.9820645135352716',), 1372886392: ('0.9820645135352716',), 1372886391: ('0.9820645135352716',), 1372886390: ('0.9820645135352716',), 1372886382: ('0.9820645135352716',), 1372886332: ('0.9820645135352716',), 1372886330: ('0.9820645135352716',), 1372886329: ('0.9820645135352716',), 1372886224: ('0.9820645135352716',), 1372886150: ('0.9820645135352716',), 1372886148: ('0.9820645135352716',), 1372883541: ('0.9820645135352716',), 1372883511: ('0.9820645135352716',), 1372883509: ('0.9820645135352716',), 1372883406: ('0.9820645135352716',), 1372883392: ('0.9820645135352716',), 1372883390: ('0.9820645135352716',)} [1372886396, 1372886393, 1372886392, 1372886391, 1372886390, 1372886382, 1372886332, 1372886330, 1372886329, 1372886224, 1372886150, 1372886148, 1372883541, 1372883511, 1372883509, 1372883406, 1372883392, 1372883390] Looping around the photos to save general results len do output : 18 /1372886396.Didn't retrieve data . /1372886393.Didn't retrieve data . /1372886392.Didn't retrieve data . /1372886391.Didn't retrieve data . /1372886390.Didn't retrieve data . /1372886382.Didn't retrieve data . /1372886332.Didn't retrieve data . /1372886330.Didn't retrieve data . /1372886329.Didn't retrieve data . /1372886224.Didn't retrieve data . /1372886150.Didn't retrieve data . /1372886148.Didn't retrieve data . /1372883541.Didn't retrieve data . /1372883511.Didn't retrieve data . /1372883509.Didn't retrieve data . /1372883406.Didn't retrieve data . /1372883392.Didn't retrieve data . /1372883390.Didn't retrieve data . before output type Used above Used above Managing all output in save final without adding information in the mtr_datou_result ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886396', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886393', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886392', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886391', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886390', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886382', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886332', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886330', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886329', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886224', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886150', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886148', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372883541', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372883511', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372883509', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372883406', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372883392', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372883390', None, None, None, None, None, '3763082') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 54 time used for this insertion : 0.014998435974121094 save_final save missing photos in datou_result : time spend for datou_step_exec : 0.10424017906188965 time spend to save output : 0.015800952911376953 total time spend for step 5 : 0.1200411319732666 step6:blur_detection Tue Sep 23 11:18:39 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step blur_detection toutes les photos sont déjà traitées, on saute les calculs Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 18 time used for this insertion : 0.00983285903930664 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 18 time used for this insertion : 0.01578974723815918 save missing photos in datou_result : time spend for datou_step_exec : 0.030612945556640625 time spend to save output : 0.030328989028930664 total time spend for step 6 : 0.06094193458557129 step7:brightness Tue Sep 23 11:18:39 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure inside step calcul brightness toutes les photos sont déjà traitées, on saute les calculs Inside saveOutput : final : False verbose : 0 begin to insert list_values into class_photo_scores : length of list_valuse in save_photo_hashtag_id_thcl_score : 18 time used for this insertion : 0.010418891906738281 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 18 time used for this insertion : 0.01014089584350586 save missing photos in datou_result : time spend for datou_step_exec : 0.02737903594970703 time spend to save output : 0.025760650634765625 total time spend for step 7 : 0.053139686584472656 step8:send_mail_cod Tue Sep 23 11:18:39 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed complete output_args for input 0 complete output_args for input 1 complete output_args for input 2 Inconsistent number of input and output, step which parrallelize and manage error in input by avoiding sending an output for this data can't be used in tree dependencies of input and output complete output_args for input 3 We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! VR 22-3-18 : For now we do not clean correctly the datou structure dans la step send mail cod work_area: /home/admin/workarea/git/Velours/python in order to get the selector url, please entre the license of selector results_COD_P25127822_23-09-2025_11_18_39.pdf 25132917 imagette251329171758619119 25132918 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette251329181758619119 25132920 imagette251329201758619121 25132921 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette251329211758619121 25132922 change filename to text .change filename to text .change filename to text .imagette251329221758619122 25132923 imagette251329231758619122 25132925 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette251329251758619122 25132926 imagette251329261758619124 25132927 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette251329271758619124 25132929 imagette251329291758619127 25132930 change filename to text .change filename to text .imagette251329301758619127 25132933 change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .change filename to text .imagette251329331758619127 SELECT h.hashtag,pcr.value FROM MTRUser.portfolio_carac_ratio pcr, MTRBack.hashtags h where pcr.portfolio_id=25127822 and hashtag_type = 4200 and pcr.hashtag_id = h.hashtag_id; velour_link : https://marlene.fotonower.com/velours/25132917,25132918,25132919,25132920,25132921,25132922,25132923,25132925,25132926,25132927,25132928,25132929,25132930,25132931,25132932,25132933?tags=textiles_sanitaires,pet_opaque,environnement,kraft,film_plastique,papier,pehd,pet_clair,ela,barquette_opaque,mal_croppe,metal,etiquette,flou,pet_fonce,carton your option no_mail is active, we will not send the real mail to your client args[1372886396] : ((1372886396, -0.7126995266886128, 492688767), (1372886396, 0.8191160524280322, 2107752395), '0.9820645135352716') We are sending mail with results at kexin@fotonower.com args[1372886393] : ((1372886393, -0.6077304102558146, 492688767), (1372886393, -0.4459476476925409, 496442774), '0.9820645135352716') We are sending mail with results at kexin@fotonower.com args[1372886392] : ((1372886392, -3.654454367888017, 492609224), (1372886392, -0.1241712907920593, 496442774), '0.9820645135352716') We are sending mail with results at kexin@fotonower.com args[1372886391] : ((1372886391, -1.7892236493888574, 492688767), (1372886391, -0.38156077414766376, 496442774), '0.9820645135352716') We are sending mail with results at kexin@fotonower.com args[1372886390] : ((1372886390, -0.11594010668605602, 492688767), (1372886390, -0.2359785261922584, 496442774), '0.9820645135352716') We are sending mail with results at kexin@fotonower.com args[1372886382] : ((1372886382, -0.48381747431951566, 492688767), (1372886382, -0.18545270506138592, 496442774), '0.9820645135352716') We are sending mail with results at kexin@fotonower.com args[1372886332] : ((1372886332, -4.3769293187356215, 492609224), (1372886332, -0.2655593996444795, 496442774), '0.9820645135352716') We are sending mail with results at kexin@fotonower.com args[1372886330] : ((1372886330, -0.5574943066196671, 492688767), (1372886330, -0.20230880008069593, 496442774), '0.9820645135352716') We are sending mail with results at kexin@fotonower.com args[1372886329] : ((1372886329, 1.8623658946598793, 492688767), (1372886329, -0.24412625432543975, 496442774), '0.9820645135352716') We are sending mail with results at kexin@fotonower.com args[1372886224] : ((1372886224, -4.390953403319224, 492609224), (1372886224, -0.08866528962240725, 496442774), '0.9820645135352716') We are sending mail with results at kexin@fotonower.com args[1372886150] : ((1372886150, 1.2588043590489681, 492688767), (1372886150, -0.26635373109193616, 496442774), '0.9820645135352716') We are sending mail with results at kexin@fotonower.com args[1372886148] : ((1372886148, 0.5052974818146324, 492688767), (1372886148, -0.24795636356549858, 496442774), '0.9820645135352716') We are sending mail with results at kexin@fotonower.com args[1372883541] : ((1372883541, -5.102082911630643, 492609224), (1372883541, -0.10954590428193783, 496442774), '0.9820645135352716') We are sending mail with results at kexin@fotonower.com args[1372883511] : ((1372883511, -2.3128419697817137, 492609224), (1372883511, -0.13438383647842875, 496442774), '0.9820645135352716') We are sending mail with results at kexin@fotonower.com args[1372883509] : ((1372883509, -1.267067183648938, 492688767), (1372883509, -0.23046474613808476, 496442774), '0.9820645135352716') We are sending mail with results at kexin@fotonower.com args[1372883406] : ((1372883406, -1.1317171727300566, 492688767), (1372883406, -0.0868744598445639, 496442774), '0.9820645135352716') We are sending mail with results at kexin@fotonower.com args[1372883392] : ((1372883392, -2.004726431855926, 492609224), (1372883392, -0.04345881291398973, 2107752395), '0.9820645135352716') We are sending mail with results at kexin@fotonower.com args[1372883390] : ((1372883390, -0.8126504632263407, 492688767), (1372883390, 0.869107049070189, 2107752395), '0.9820645135352716') We are sending mail with results at kexin@fotonower.com refus_total : 0.9820645135352716 2022-04-13 10:29:59 0 SELECT ph.photo_id,ph.url,ph.username,ph.uploaded_at,ph.text FROM MTRBack.photos ph, MTRUser.mtr_portfolio_photos mpp WHERE ph.photo_id=mpp.mtr_photo_id AND mpp.mtr_portfolio_id=25127822 AND mpp.hide_status=0 ORDER BY mpp.order LIMIT 0, 1000 start upload file to ovh https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_COD_P25127822_23-09-2025_11_18_39.pdf results_COD_P25127822_23-09-2025_11_18_39.pdf uploaded to url https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_COD_P25127822_23-09-2025_11_18_39.pdf start insert file to database insert into MTRUser.mtr_files (mtd_id,mtr_portfolio_id,text,url,format,tags,file_size,value) values ('4254','25127822','results_COD_P25127822_23-09-2025_11_18_39.pdf','https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_COD_P25127822_23-09-2025_11_18_39.pdf','pdf','','1.71','0.9820645135352716') Inside saveOutput : final : False verbose : 0 saveOutput not yet implemented for datou_step.type : send_mail_cod we use saveGeneral [1372886396, 1372886393, 1372886392, 1372886391, 1372886390, 1372886382, 1372886332, 1372886330, 1372886329, 1372886224, 1372886150, 1372886148, 1372883541, 1372883511, 1372883509, 1372883406, 1372883392, 1372883390] Looping around the photos to save general results len do output : 0 before output type Used above Managing all output in save final without adding information in the mtr_datou_result ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886396', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886393', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886392', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886391', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886390', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886382', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886332', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886330', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886329', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886224', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886150', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886148', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372883541', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372883511', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372883509', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372883406', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372883392', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372883390', None, None, None, None, None, '3763082') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 18 time used for this insertion : 0.020783662796020508 save_final save missing photos in datou_result : time spend for datou_step_exec : 14.418033838272095 time spend to save output : 0.021012067794799805 total time spend for step 8 : 14.439045906066895 step9:split_time_score Tue Sep 23 11:18:54 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Currently we do not manage missing dependencies information, that could maybe be correctly interpreted with default behavior Some of the step done at execution of the step could be done before when the tree of execution is build and the dependencies of different step analysed We should have FATAL ERROR but same_nb_input_output==True : this should be an optionnal input ! complete output_args for input 1 VR 22-3-18 : For now we do not clean correctly the datou structure begin split time score TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 861, 'mtr_user_id': 31, 'name': 'Rungis_class_dechets_1212', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'Rungis_Aluminium,Rungis_Carton,Rungis_Papier,Rungis_Plastique_clair,Rungis_Plastique_dur,Rungis_Plastique_fonce,Rungis_Tapis_vide,Rungis_Tetrapak', 'svm_portfolios_learning': '1160730,571842,571844,571839,571933,571840,571841,572307', 'photo_hashtag_type': 999, 'photo_desc_type': 3963, 'type_classification': 'caffe', 'hashtag_id_list': '2107751280,2107750907,2107750908,2107750909,2107750910,2107750911,2107750912,2107750913'}] thcls : [{'id': 758, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 856, 'photo_desc_type': 3853, 'type_classification': 'caffe', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('03', 18),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {} 10072025 25127822 Nombre de photos uploadées : 18 / 23040 (0%) 10072025 25127822 Nombre de photos taguées (types de déchets): 0 / 18 (0%) 10072025 25127822 Nombre de photos taguées (volume) : 0 / 18 (0%) elapsed_time : load_data_split_time_score 2.1457672119140625e-06 elapsed_time : order_list_meta_photo_and_scores 5.0067901611328125e-06 ?????????????????? elapsed_time : fill_and_build_computed_from_old_data 0.0008287429809570312 Catched exception ! Connect or reconnect ! Catched exception ! Connect or reconnect ! elapsed_time : insert_dashboard_record_day_entry 0.20804429054260254 We will return after consolidate but for now we need the day, how to get it, for now depending on the previous heavy steps find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25110342_20-07-2025_08_59_13.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 25110342 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25110343_20-07-2025_09_01_19.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 25110343 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25110344_20-07-2025_08_52_31.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 25110344 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25110345_20-07-2025_09_23_18.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 25110345 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25110346_20-07-2025_08_56_01.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 25110346 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25110348_20-07-2025_09_11_53.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 25110348 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25110350_20-07-2025_08_54_20.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 25110350 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25127804_21-07-2025_10_51_40.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 25127804 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25127808_21-07-2025_11_12_19.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 25127808 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25127812_21-07-2025_10_59_25.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 25127812 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25127816_21-07-2025_11_13_03.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 25127816 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25127819_21-07-2025_10_55_09.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 25127819 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25127827_21-07-2025_10_50_47.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 25127827 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25127834_21-07-2025_10_57_26.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 25127834 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25127837_21-07-2025_10_54_18.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 25127837 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25127841_21-07-2025_11_05_44.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 25127841 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25127845_21-07-2025_11_13_41.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 25127845 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25127853_21-07-2025_11_01_21.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 25127853 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25127875_21-07-2025_10_54_29.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 25127875 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25127879_21-07-2025_11_03_19.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 25127879 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25127880_21-07-2025_11_23_43.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 25127880 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25127881_21-07-2025_10_46_45.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 25127881 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25127884_21-07-2025_10_57_48.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 25127884 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25127885_21-07-2025_10_41_45.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 25127885 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25127886_21-07-2025_10_44_07.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 25127886 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25127887_21-07-2025_15_34_44.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 25127887 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25127899_21-07-2025_10_55_58.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 25127899 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25110353_20-07-2025_09_11_31.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 25110353 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25110355_20-07-2025_08_56_03.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 25110355 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25110357_20-07-2025_08_59_46.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 25110357 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25110358_20-07-2025_08_52_28.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 25110358 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25110359_20-07-2025_08_48_17.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 25110359 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25110361_20-07-2025_08_52_30.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 25110361 order by id desc limit 1 find url: https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/results_Auto_P25110362_20-07-2025_08_58_26.pdf select completion_json, dashboard_run_id from MTRPhoto.dashboard_results where mtr_portfolio_id = 25110362 order by id desc limit 1 NUMBER BATCH : 0 # DISPLAY ALL COLLECTED DATA : {'10072025': {'nb_upload': 18, 'nb_taggue_class': 0, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1372886396, 1372886393, 1372886392, 1372886391, 1372886390, 1372886382, 1372886332, 1372886330, 1372886329, 1372886224, 1372886150, 1372886148, 1372883541, 1372883511, 1372883509, 1372883406, 1372883392, 1372883390] Looping around the photos to save general results len do output : 1 /25127822Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886396', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886393', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886392', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886391', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886390', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886382', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886332', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886330', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886329', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886224', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886150', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372886148', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372883541', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372883511', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372883509', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372883406', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372883392', None, None, None, None, None, '3763082') ('4254', None, None, None, None, None, None, None, '3763082') ('4254', '25127822', '1372883390', None, None, None, None, None, '3763082') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 19 time used for this insertion : 0.018388032913208008 save_final save missing photos in datou_result : time spend for datou_step_exec : 16.754510641098022 time spend to save output : 0.018639326095581055 total time spend for step 9 : 16.773149967193604 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 18 set_done_treatment 5.22user 1.82system 0:46.27elapsed 15%CPU (0avgtext+0avgdata 221536maxresident)k 952inputs+96056outputs (10major+190427minor)pagefaults 0swaps