python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 8 -a ' -a 4311 ' -s datou_current_4220 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 4127984 load datou : 4311 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11939 matching_dashboard is not consistent : 1 used against 0 in the step definition ! Step 11940 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 0 of step 11939 doesn't seem to be define in the database( WARNING : type of input 2 of step 11940 doesn't seem to be define in the database( DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : None was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? load thcls load pdts Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec no input labels no input values updating current state to 1 we have a portfolio with more photos than limit : 2820>1000 please execute split_portfolio.py -i 21625899 -l 1000 size over we load limit photo not treated list_input_json: {} Current got : datou_id : 4311, datou_cur_ids : ['2685090'] with mtr_portfolio_ids : ['21625899'] and first list_photo_ids : [] new path : /proc/4127984/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11939 matching_dashboard is not consistent : 1 used against 0 in the step definition ! list printed: [[242], [721], [1123], [1409, 1410], [1422], [1429, 1430], [2448, 2449], [2452, 2453], [2456, 2457, 2458, 2459, 2460], [2517], [2606], [2609], [2611, 2612, 2613, 2614, 2615, 2616, 2617, 2618, 2619], [2621], [2626, 2627], [2633], [2635], [2638], [2689, 2690, 2691], [2706]] forced_hashtag: truck force hashtag to truck elapsed_time : SPLIT_TRUCK 0.10293245315551758 ***** END SPLIT TRUCK ***** NUMBER BATCH : 20 list_ponderation used : [0.001, 0.001, 0.001, 0.001, 0.001] , list_hashtag_class_create_as_list : ['truck'] ERROR missing amount info ERROR missing amount info result_one_balle_Type_truck:{'day': '24032025', 'map_nb_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'map_time_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'duration': 0, 'nb_balles_papier': 0, 'begin_time_port': 'image'} Production hashtag (incorrect ponderation at 20-10-18) : 0 ERROR missing amount info ERROR missing amount info result_one_balle_Type_truck:{'day': '24032025', 'map_nb_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'map_time_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'duration': 0, 'nb_balles_papier': 0, 'begin_time_port': 'image'} Production hashtag (incorrect ponderation at 20-10-18) : 0 ERROR missing amount info ERROR missing amount info result_one_balle_Type_truck:{'day': '24032025', 'map_nb_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'map_time_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'duration': 0, 'nb_balles_papier': 0, 'begin_time_port': 'image'} Production hashtag (incorrect ponderation at 20-10-18) : 0 ERROR missing amount info ERROR missing amount info result_one_balle_Type_truck:{'day': '24032025', 'map_nb_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'map_time_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'duration': 0, 'nb_balles_papier': 0, 'begin_time_port': 'image'} Production hashtag (incorrect ponderation at 20-10-18) : 0 ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info result_one_balle_Type_truck:{'day': '24032025', 'map_nb_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'map_time_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'duration': 20.00098991394043, 'nb_balles_papier': 0, 'begin_time_port': 'image_24032025_07_36_26_9432.jpg'} Production hashtag (incorrect ponderation at 20-10-18) : 0 ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info result_one_balle_Type_truck:{'day': '24032025', 'map_nb_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'map_time_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'duration': 80.00002408027649, 'nb_balles_papier': 0, 'begin_time_port': 'image_24032025_08_10_45_10412.jpg'} Production hashtag (incorrect ponderation at 20-10-18) : 0 ERROR missing amount info ERROR missing amount info result_one_balle_Type_truck:{'day': '24032025', 'map_nb_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'map_time_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'duration': 0, 'nb_balles_papier': 0, 'begin_time_port': 'image'} Production hashtag (incorrect ponderation at 20-10-18) : 0 ERROR missing amount info ERROR missing amount info ERROR missing amount info result_one_balle_Type_truck:{'day': '24032025', 'map_nb_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'map_time_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'duration': 20.00109100341797, 'nb_balles_papier': 0, 'begin_time_port': 'image_24032025_08_24_05_9333.jpg'} Production hashtag (incorrect ponderation at 20-10-18) : 0 We have rejected 12 photos because of the batch_size condition ! NUMBER BATCH list_of_portfolios_to_create : 8 list_same_port_ids : [21633379] find same portfolio which already exist 21633379 , we will use it list_same_port_ids : [21633405] find same portfolio which already exist 21633405 , we will use it list_same_port_ids : [21640765] find same portfolio which already exist 21640765 , we will use it list_same_port_ids : [21640766] find same portfolio which already exist 21640766 , we will use it list_same_port_ids : [21640767] find same portfolio which already exist 21640767 , we will use it list_same_port_ids : [] https://marlene.fotonower.com/api/v1/secured/portfolio/new?name=truck_diff_batch__24032025_08_10_45_010412&access_token=f1621a5ac35fdbb2264e149659fde12f list_same_port_ids : [] https://marlene.fotonower.com/api/v1/secured/portfolio/new?name=truck_diff_batch__24032025_08_13_15_009365&access_token=f1621a5ac35fdbb2264e149659fde12f list_same_port_ids : [] https://marlene.fotonower.com/api/v1/secured/portfolio/new?name=truck_diff_batch__24032025_08_24_05_009333&access_token=f1621a5ac35fdbb2264e149659fde12f todo SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21633379 AND mptpi.`type`=4403 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('plate_crops')) To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21633405 AND mptpi.`type`=4403 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('plate_crops')) To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21640765 AND mptpi.`type`=4403 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('plate_crops')) To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21640766 AND mptpi.`type`=4403 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('plate_crops')) To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21640767 AND mptpi.`type`=4403 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('plate_crops')) To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21642401 AND mptpi.`type`=4403 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('plate_crops')) To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21642401 AND mptpi.`type`=4403 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('plate_crops')) To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21642402 AND mptpi.`type`=4403 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('plate_crops')) To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21642402 AND mptpi.`type`=4403 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('plate_crops')) To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21642403 AND mptpi.`type`=4403 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('plate_crops')) To do To do SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21642403 AND mptpi.`type`=4403 AND mptpi.`hashtag_id` in (select hashtag_id FROM MTRBack.hashtags where hashtag in ('plate_crops')) To do Qualite : 0.1487912535620003 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21633379 AND mptpi.`type`=4230 To do Qualite : 0.19256649634315598 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21633405 AND mptpi.`type`=4230 To do Qualite : 0.14357440855290463 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21640765 AND mptpi.`type`=4230 To do Qualite : 0.1431161030530753 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21640766 AND mptpi.`type`=4230 To do Qualite : 0.12619834243545033 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21640767 AND mptpi.`type`=4230 To do # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21642401 AND mptpi.`type`=4230 To do # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21642402 AND mptpi.`type`=4230 To do # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=21642403 AND mptpi.`type`=4230 To do elapsed_time : count_nb_balles_and_create_portfolio 6.737776517868042 do something to do to do to do to do to do to do to do to do TODO : Insert select and so on # DISPLAY ALL COLLECTED DATA : {'24032025': {'nb_upload': 2820, 'nb_taggue_class': 2712, 'nb_taggue_densite': 0}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1346765150, 1346765091, 1346765082, 1346765074, 1346765067, 1346765061, 1346764592, 1346764571, 1346764539, 1346764502, 1346764457, 1346764412, 1346764244, 1346764221, 1346764195, 1346764165, 1346764139, 1346764133, 1346764049, 1346764046, 1346763997, 1346763991, 1346763987, 1346763983, 1346763921, 1346763919, 1346763899, 1346763895, 1346763890, 1346763887, 1346763846, 1346763841, 1346763836, 1346763798, 1346763764, 1346763723, 1346763702, 1346763699, 1346763695, 1346763687, 1346763673, 1346763659, 1346763551, 1346763533, 1346763514, 1346763503, 1346763497, 1346763483, 1346763226, 1346763224, 1346763220, 1346763218, 1346763215, 1346763212, 1346763154, 1346763149, 1346763137, 1346763134, 1346763131, 1346763128, 1346763087, 1346763086, 1346763081, 1346763076, 1346763071, 1346763065, 1346762999, 1346762995, 1346762992, 1346762965, 1346762960, 1346762958, 1346762881, 1346762878, 1346762875, 1346762871, 1346762868, 1346762865, 1346762797, 1346762794, 1346762789, 1346762784, 1346762780, 1346762776, 1346762743, 1346762728, 1346762724, 1346762719, 1346762715, 1346762711, 1346762657, 1346762654, 1346762651, 1346762646, 1346762644, 1346762641, 1346762589, 1346762588, 1346762586, 1346762579] Looping around the photos to save general results len do output : 1 /21625899Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346765150', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346765091', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346765082', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346765074', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346765067', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346765061', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346764592', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346764571', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346764539', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346764502', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346764457', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346764412', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346764244', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346764221', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346764195', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346764165', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346764139', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346764133', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346764049', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346764046', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346763997', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346763991', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346763987', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346763983', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346763921', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346763919', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346763899', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346763895', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346763890', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346763887', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346763846', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346763841', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346763836', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346763798', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346763764', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346763723', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346763702', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346763699', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346763695', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346763687', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346763673', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346763659', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346763551', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346763533', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346763514', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346763503', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346763497', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346763483', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346763226', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346763224', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346763220', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346763218', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346763215', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346763212', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346763154', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346763149', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346763137', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346763134', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346763131', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346763128', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346763087', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346763086', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346763081', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346763076', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346763071', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346763065', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346762999', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346762995', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346762992', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346762965', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346762960', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346762958', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346762881', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346762878', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346762875', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346762871', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346762868', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346762865', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346762797', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346762794', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346762789', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346762784', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346762780', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346762776', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346762743', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346762728', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346762724', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346762719', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346762715', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346762711', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346762657', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346762654', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346762651', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346762646', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346762644', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346762641', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346762589', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346762588', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346762586', None, None, None, None, None, '2685090') ('4311', None, None, None, None, None, None, None, '2685090') ('4311', '21625899', '1346762579', None, None, None, None, None, '2685090') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 101 time used for this insertion : 0.02729630470275879 save_final save missing photos in datou_result : time spend for datou_step_exec : 7.267274379730225 time spend to save output : 0.028265953063964844 total time spend for step 2 : 7.2955403327941895 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 1 set_done_treatment 1.63user 1.22system 0:10.87elapsed 26%CPU (0avgtext+0avgdata 111036maxresident)k 528inputs+160outputs (18major+51762minor)pagefaults 0swaps