WARNING : number of inputs for step 13615 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 13615 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 13621 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 13621 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 13617 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 13617 ventilate_hashtags_in_portfolio have less outputs used (0) than in the step definition (1) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 13611 doesn't seem to be define in the database( WARNING : type of input 2 of step 13613 doesn't seem to be define in the database( WARNING : output 1 of step 13611 have datatype=2 whereas input 1 of step 13614 have datatype=7 WARNING : type of output 2 of step 13614 doesn't seem to be define in the database( WARNING : type of input 1 of step 13615 doesn't seem to be define in the database( WARNING : type of output 3 of step 13614 doesn't seem to be define in the database( WARNING : type of input 1 of step 13617 doesn't seem to be define in the database( WARNING : type of input 2 of step 13621 doesn't seem to be define in the database( WARNING : output 1 of step 13615 have datatype=7 whereas input 2 of step 13621 have datatype=None WARNING : type of output 3 of step 13621 doesn't seem to be define in the database( WARNING : type of input 2 of step 13617 doesn't seem to be define in the database( WARNING : output 0 of step 13614 have datatype=1 whereas input 0 of step 13615 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22109190 AND mptpi.`type`=3726 To do error result not float, to be checked, testing the next We don't need to reset the datou_current for this portfolio, que ce soit dans le cas ou il est fini (et found_correct devrait etre vrai, ou dans le cas ou il n'est pas fini, mais du coup on le retrouvera et mettra simplement à jour) # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 13611 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 13613 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 13613 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 13614 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 13614 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 13615 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 13615 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 13621 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 13621 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 13617 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 13617 ventilate_hashtags_in_portfolio have less outputs used (0) than in the step definition (1) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 13611 doesn't seem to be define in the database( WARNING : type of input 2 of step 13613 doesn't seem to be define in the database( WARNING : output 1 of step 13611 have datatype=2 whereas input 1 of step 13614 have datatype=7 WARNING : type of output 2 of step 13614 doesn't seem to be define in the database( WARNING : type of input 1 of step 13615 doesn't seem to be define in the database( WARNING : type of output 3 of step 13614 doesn't seem to be define in the database( WARNING : type of input 1 of step 13617 doesn't seem to be define in the database( WARNING : type of input 2 of step 13621 doesn't seem to be define in the database( WARNING : output 1 of step 13615 have datatype=7 whereas input 2 of step 13621 have datatype=None WARNING : type of output 3 of step 13621 doesn't seem to be define in the database( WARNING : type of input 2 of step 13617 doesn't seem to be define in the database( WARNING : output 0 of step 13614 have datatype=1 whereas input 0 of step 13615 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22109267 AND mptpi.`type`=3726 To do error result not float, to be checked, testing the next We don't need to reset the datou_current for this portfolio, que ce soit dans le cas ou il est fini (et found_correct devrait etre vrai, ou dans le cas ou il n'est pas fini, mais du coup on le retrouvera et mettra simplement à jour) # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 13611 mask_detect is not consistent : 3 used against 2 in the step definition ! Step 13613 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Step 13613 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 13614 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 13614 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 13615 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 13615 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 13621 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 13621 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 13617 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 13617 ventilate_hashtags_in_portfolio have less outputs used (0) than in the step definition (1) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 13611 doesn't seem to be define in the database( WARNING : type of input 2 of step 13613 doesn't seem to be define in the database( WARNING : output 1 of step 13611 have datatype=2 whereas input 1 of step 13614 have datatype=7 WARNING : type of output 2 of step 13614 doesn't seem to be define in the database( WARNING : type of input 1 of step 13615 doesn't seem to be define in the database( WARNING : type of output 3 of step 13614 doesn't seem to be define in the database( WARNING : type of input 1 of step 13617 doesn't seem to be define in the database( WARNING : type of input 2 of step 13621 doesn't seem to be define in the database( WARNING : output 1 of step 13615 have datatype=7 whereas input 2 of step 13621 have datatype=None WARNING : type of output 3 of step 13621 doesn't seem to be define in the database( WARNING : type of input 2 of step 13617 doesn't seem to be define in the database( WARNING : output 0 of step 13614 have datatype=1 whereas input 0 of step 13615 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=22109413 AND mptpi.`type`=3726 To do elapsed_time : count_nb_balles_and_create_portfolio 4.293439149856567 # DISPLAY ALL COLLECTED DATA : {'16042025': {'nb_upload': 100, 'nb_taggue_class': 3351, 'nb_taggue_densite': 3351}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [] Looping around the photos to save general results len do output : 1 /22055303Didn't retrieve data . before output type Here is an output not treated by saveGeneral : error in checking portfolio photo link list index out of range [('4883', None, '22055303', 'None', None, None, None, None, '2727250')] [] begin to insert list_values into mtr_datou_result : length of list_values in save_final : 1 time used for this insertion : 0.009685993194580078 save_final save missing photos in datou_result : time spend for datou_step_exec : 4.71312689781189 time spend to save output : 0.010021209716796875 total time spend for step 1 : 4.7231481075286865 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 1 set_done_treatment 1.21user 0.52system 0:06.31elapsed 27%CPU (0avgtext+0avgdata 102676maxresident)k 0inputs+152outputs (2major+49579minor)pagefaults 0swaps