python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 4189' -s datou_current_4189 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 274224 load datou : 4189 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? load thcls load pdts Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Date 2026-04-18 08:35:26.455503 Current got : datou_id : 4189, datou_cur_ids : ['4375606'] with mtr_portfolio_ids : ['30984947'] and first list_photo_ids : [] new path : /proc/274224/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! List Step Type Loaded in datou : split_time_score over limit max, limiting to limit_max 100 list_input_json : [] origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! we have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB time to download the photos : 0.08782076835632324 About to test input to load Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 1 step1:split_time_score Sat Apr 18 08:35:26 2026 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec begin split time score 2022-04-13 10:29:59 0 TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 3379, 'mtr_user_id': 31, 'name': 'learn_classif_flux_maj_generique_effnet_v2_s_02062022', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'aluminium,ela,film_pedb,flux_dev,jrm,pcm,pcnc,pehd_pp,pet_clair,refus,tapis_vide', 'svm_portfolios_learning': '5515864,5515840,5515844,5515850,6244400,6237996,6237998,5515847,5515841,5515868,5515866', 'photo_hashtag_type': 4374, 'photo_desc_type': 5680, 'type_classification': 'tf_classification2', 'hashtag_id_list': '493546845,492741797,2107760237,2107760238,495916461,560181804,1284539308,2107760239,2107755846,538914404,2107748999'}] thcls : [{'id': 3513, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2_tf', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 4557, 'photo_desc_type': 5767, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('10', 359), ('14', 366)) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {1: 622} L15042026 30984947 Nombre de photos uploadées : 725 / 23040 (3%) 15042026 30984947 Nombre de photos taguées (types de déchets): 622 / 725 (85%) 15042026 30984947 Nombre de photos taguées (volume) : 622 / 725 (85%) elapsed_time : load_data_split_time_score 3.0994415283203125e-06 elapsed_time : order_list_meta_photo_and_scores 0.00045943260192871094 LLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLL??????????????????????????????????????????????????????????????????????????????????????????????????????? elapsed_time : fill_and_build_computed_from_old_data 0.030569076538085938 LCatched exception ! Connect or reconnect ! Catched exception ! Connect or reconnect ! elapsed_time : insert_dashboard_record_day_entry 0.24116110801696777 LLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLL Creating list_photo_by_hashtags Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None elapsed_time : list_photo_by_hashtags 0.07308721542358398 Creating list_photo_total elapsed_time : select_descriptors 51.076717138290405 15042026 30984947 Nombre de photos avec descriptors (type 5680) : 549 / 652 (84%) Missing descriptors for photos 0 and 1415836208 L0:00:00|ON:Missing descriptors for photos 1415836208 and 1415836209 LMissing descriptors for photos 1415836209 and 1415836210 LMissing descriptors for photos 1415836210 and 1415836211 LMissing descriptors for photos 1415836211 and 1415836212 LMissing descriptors for photos 1415836212 and 1415836213 LMissing descriptors for photos 1415836213 and 1415836214 LMissing descriptors for photos 1415836214 and 1415836215 LMissing descriptors for photos 1415836215 and 1415836217 LMissing descriptors for photos 1415836217 and 1415836218 LMissing descriptors for photos 1415836218 and 1415836219 LMissing descriptors for photos 1415836219 and 1415836220 LMissing descriptors for photos 1415836220 and 1415836221 LMissing descriptors for photos 1415836221 and 1415836222 LMissing descriptors for photos 1415836222 and 1415836223 LMissing descriptors for photos 1415836223 and 1415836224 LMissing descriptors for photos 1415836224 and 1415836226 LERROR : Hum hum, what can we do for different size of descriptors (ignore the difference ) : 0 vs 1280 photo_id : 1415836227 photo_id_prec : 1415836226 LLLLLL0:13:59.188520|OFF:0:00:00|ON:ERROR : Hum hum, what can we do for different size of descriptors (ignore the difference ) : 1280 vs 0 photo_id : 1415836637 photo_id_prec : 1415836635 LERROR : Hum hum, what can we do for different size of descriptors (ignore the difference ) : 0 vs 1280 photo_id : 1415836639 photo_id_prec : 1415836637 ERROR : Hum hum, what can we do for different size of descriptors (ignore the difference ) : 1280 vs 0 photo_id : 1415837508 photo_id_prec : 1415837507 LMissing descriptors for photos 1415837508 and 1415837509 LMissing descriptors for photos 1415837509 and 1415837512 LMissing descriptors for photos 1415837512 and 1415837513 LMissing descriptors for photos 1415837513 and 1415837514 LMissing descriptors for photos 1415837514 and 1415837515 LMissing descriptors for photos 1415837515 and 1415837521 LMissing descriptors for photos 1415837521 and 1415837523 LMissing descriptors for photos 1415837523 and 1415837525 LMissing descriptors for photos 1415837525 and 1415837526 LMissing descriptors for photos 1415837526 and 1415837527 LMissing descriptors for photos 1415837527 and 1415837528 LMissing descriptors for photos 1415837528 and 1415837832 LMissing descriptors for photos 1415837832 and 1415837998 LMissing descriptors for photos 1415837998 and 1415838115 LMissing descriptors for photos 1415838115 and 1415838116 LMissing descriptors for photos 1415838116 and 1415838117 LMissing descriptors for photos 1415838117 and 1415838118 LMissing descriptors for photos 1415838118 and 1415838285 LMissing descriptors for photos 1415838285 and 1415838332 LMissing descriptors for photos 1415838332 and 1415838365 LMissing descriptors for photos 1415838365 and 1415838370 LMissing descriptors for photos 1415838370 and 1415838372 LMissing descriptors for photos 1415838372 and 1415838373 LMissing descriptors for photos 1415838373 and 1415838374 LMissing descriptors for photos 1415838374 and 1415838375 LMissing descriptors for photos 1415838375 and 1415838376 LMissing descriptors for photos 1415838376 and 1415838377 LMissing descriptors for photos 1415838377 and 1415838378 LMissing descriptors for photos 1415838378 and 1415838379 LMissing descriptors for photos 1415838379 and 1415838381 LMissing descriptors for photos 1415838381 and 1415838382 LMissing descriptors for photos 1415838382 and 1415838383 LMissing descriptors for photos 1415838383 and 1415838385 LMissing descriptors for photos 1415838385 and 1415838387 LMissing descriptors for photos 1415838387 and 1415838388 LERROR : Hum hum, what can we do for different size of descriptors (ignore the difference ) : 0 vs 1280 photo_id : 1415838389 photo_id_prec : 1415838388 LLMissing descriptors for photos 1415838395 and 1415838397 LMissing descriptors for photos 1415838397 and 1415838399 LMissing descriptors for photos 1415838399 and 1415838400 LMissing descriptors for photos 1415838400 and 1415838401 LMissing descriptors for photos 1415838401 and 1415838402 LMissing descriptors for photos 1415838402 and 1415838404 LMissing descriptors for photos 1415838404 and 1415838405 LMissing descriptors for photos 1415838405 and 1415838407 LMissing descriptors for photos 1415838407 and 1415838408 LMissing descriptors for photos 1415838408 and 1415838410 LMissing descriptors for photos 1415838410 and 1415838412 LMissing descriptors for photos 1415838412 and 1415838413 LMissing descriptors for photos 1415838413 and 1415838418 LMissing descriptors for photos 1415838418 and 1415838420 LMissing descriptors for photos 1415838420 and 1415838422 LMissing descriptors for photos 1415838422 and 1415838423 LMissing descriptors for photos 1415838423 and 1415838424 LMissing descriptors for photos 1415838424 and 1415838426 LMissing descriptors for photos 1415838426 and 1415838428 LMissing descriptors for photos 1415838428 and 1415838429 LMissing descriptors for photos 1415838429 and 1415838430 LMissing descriptors for photos 1415838430 and 1415838431 LMissing descriptors for photos 1415838431 and 1415838432 LMissing descriptors for photos 1415838432 and 1415838433 LMissing descriptors for photos 1415838433 and 1415838436 LMissing descriptors for photos 1415838436 and 1415838438 LMissing descriptors for photos 1415838438 and 1415838439 LMissing descriptors for photos 1415838439 and 1415838440 LMissing descriptors for photos 1415838440 and 1415838441 LMissing descriptors for photos 1415838441 and 1415838442 LMissing descriptors for photos 1415838442 and 1415838447 LMissing descriptors for photos 1415838447 and 1415838449 LMissing descriptors for photos 1415838449 and 1415838451 LMissing descriptors for photos 1415838451 and 1415838452 LMissing descriptors for photos 1415838452 and 1415838455 LMissing descriptors for photos 1415838455 and 1415838456 LMissing descriptors for photos 1415838456 and 1415838461 LMissing descriptors for photos 1415838461 and 1415838462 LMissing descriptors for photos 1415838462 and 1415838464 LMissing descriptors for photos 1415838464 and 1415838465 LMissing descriptors for photos 1415838465 and 1415838467 LMissing descriptors for photos 1415838467 and 1415838470 LMissing descriptors for photos 1415838470 and 1415838476 LMissing descriptors for photos 1415838476 and 1415838477 LMissing descriptors for photos 1415838477 and 1415838478 LMissing descriptors for photos 1415838478 and 1415838481 LMissing descriptors for photos 1415838481 and 1415838482 LMissing descriptors for photos 1415838482 and 1415838483 LERROR : Hum hum, what can we do for different size of descriptors (ignore the difference ) : 0 vs 1280 photo_id : 1415838484 photo_id_prec : 1415838483 LMissing descriptors for photos 1415838584 and 1415838586 LMissing descriptors for photos 1415838586 and 1415838587 LLMissing descriptors for photos 1415838591 and 1415838592 LMissing descriptors for photos 1415838592 and 1415838593 LMissing descriptors for photos 1415838593 and 1415838595 LMissing descriptors for photos 1415838595 and 1415838596 LLMissing descriptors for photos 1415838598 and 1415838599 LMissing descriptors for photos 1415838599 and 1415838600 LMissing descriptors for photos 1415838600 and 1415838604 LMissing descriptors for photos 1415838604 and 1415838606 LLLMissing descriptors for photos 1415846284 and 1415846285 LMissing descriptors for photos 1415846285 and 1415846286 LMissing descriptors for photos 1415846286 and 1415846287 LMissing descriptors for photos 1415846287 and 1415846288 LMissing descriptors for photos 1415846288 and 1415846289 LMissing descriptors for photos 1415846289 and 1415846290 LMissing descriptors for photos 1415846290 and 1415846294 LMissing descriptors for photos 1415846294 and 1415846295 LMissing descriptors for photos 1415846295 and 1415846296 LMissing descriptors for photos 1415846296 and 1415846304 LMissing descriptors for photos 1415846304 and 1415846305 LMissing descriptors for photos 1415846305 and 1415846312 LMissing descriptors for photos 1415846312 and 1415846349 LMissing descriptors for photos 1415846349 and 1415846356 LMissing descriptors for photos 1415846356 and 1415846357 LMissing descriptors for photos 1415846357 and 1415846359 LMissing descriptors for photos 1415846359 and 1415846361 LMissing descriptors for photos 1415846361 and 1415846362 LMissing descriptors for photos 1415846362 and 1415846363 LMissing descriptors for photos 1415846363 and 1415846472 LMissing descriptors for photos 1415846472 and 1415846588 LMissing descriptors for photos 1415846588 and 1415846589 LMissing descriptors for photos 1415846589 and 1415846590 LLMissing descriptors for photos 1415846683 and 1415846684 LMissing descriptors for photos 1415846684 and 1415846685 LMissing descriptors for photos 1415846685 and 1415846686 LMissing descriptors for photos 1415846686 and 1415846687 LMissing descriptors for photos 1415846687 and 1415846688 LMissing descriptors for photos 1415846688 and 1415846690 LMissing descriptors for photos 1415846690 and 1415846691 LMissing descriptors for photos 1415846691 and 1415846694 LMissing descriptors for photos 1415846694 and 1415846696 LLLLL4:19:00.100952|OFF:0:00:59.886996|ON:LLMissing descriptors for photos 1415850700 and 1415852559 LMissing descriptors for photos 1415852559 and 1415852578 LMissing descriptors for photos 1415852578 and 1415852560 LMissing descriptors for photos 1415852560 and 1415852561 LMissing descriptors for photos 1415852561 and 1415852562 LMissing descriptors for photos 1415852562 and 1415852563 LMissing descriptors for photos 1415852563 and 1415852564 LMissing descriptors for photos 1415852564 and 1415852580 LMissing descriptors for photos 1415852580 and 1415852581 LMissing descriptors for photos 1415852581 and 1415852582 LMissing descriptors for photos 1415852582 and 1415852583 LMissing descriptors for photos 1415852583 and 1415852584 LLLLERROR : Hum hum, what can we do for different size of descriptors (ignore the difference ) : 1280 vs 0 photo_id : 1415941607 photo_id_prec : 1415859466 L 15042026 Removing 2 photos because of the 'same image' condition Total on : 16379.289472 list_time_on Total off : 59.886996 list_time_off dist_desc begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 725 time used for this insertion : 0.33515238761901855 photos_removed : len 2 elapsed_time : remove_photo_duplicate 0.6190652847290039 To do, maybe not at the correct place ! .L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L................................L.......................L..L..L..........L................................................................L....................................L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L...L...L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L......................................................L.L.L..L.L.L.L.L..L.L.L.L.L................................................L...L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L..L.L.L.L.L.L.L.L.L.L................................................................L...L.....................................................................L.......................................L..L.L.L.L.L.L.L.L.L.L.L.L.L...................................................................L..L.....................L..................Lforce hashtag to JRM elapsed_time : CREATE_PORT_BATCH_BY_HOUR 0.02495288848876953 NUMBER BATCH : 8 list_ponderation used : [1e-05, 1e-05, 1e-05, 1e-05, 1e-05] , list_hashtag_class_create_as_list : ['jrm'] LLLLLLLLLLLLLLLLLLERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info Lresult_one_balle_Type_JRM:{'day': '15042026', 'map_nb_amount': {0: 9, 1: 54, 2: 2, 3: 3, 4: 0}, 'map_time_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'duration': 839.1885199546814, 'nb_balles_papier': 0.0006800000000000011, 'begin_time_port': 'image_15042026_10_00_03_074445_0000.jpg'} Production hashtag (incorrect ponderation at 20-10-18) : 0.0006800000000000011 LERROR missing amount info result_one_balle_Type_JRM:{'day': '15042026', 'map_nb_amount': {0: 15, 1: 72, 2: 1, 3: 1, 4: 0}, 'map_time_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'duration': 839.8197529315948, 'nb_balles_papier': 0.0008900000000000017, 'begin_time_port': 'image_15042026_10_15_03_117196m0.jpg 1e-05 for time 1, id_amount 2 this amount prod time diff : 1e-05'} Production hashtag (incorrect ponderation at 20-10-18) : 0.0008900000000000017 LLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info Lresult_one_balle_Type_JRM:{'day': '15042026', 'map_nb_amount': {0: 0, 1: 9, 2: 0, 3: 0, 4: 0}, 'map_time_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'duration': 839.4770319461823, 'nb_balles_papier': 9e-05, 'begin_time_port': 'image_15042026_10_30_02_793621m0.jpg 1e-05 for time 1, id_amount 2 this amount prod time diff : 1e-05'} Production hashtag (incorrect ponderation at 20-10-18) : 9e-05 LLLLLLERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info Lresult_one_balle_Type_JRM:{'day': '15042026', 'map_nb_amount': {0: 12, 1: 57, 2: 2, 3: 0, 4: 0}, 'map_time_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'duration': 839.559247970581, 'nb_balles_papier': 0.0007100000000000012, 'begin_time_port': 'image_15042026_10_45_03_200895_0000.jpg'} Production hashtag (incorrect ponderation at 20-10-18) : 0.0007100000000000012 result_one_balle_Type_JRM:{'day': '15042026', 'map_nb_amount': {0: 6, 1: 49, 2: 0, 3: 0, 4: 0}, 'map_time_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'duration': 839.166111946106, 'nb_balles_papier': 0.0005500000000000008, 'begin_time_port': 'image_15042026_14_00_03_313270m0.jpg 1e-05 for time 1, id_amount 2 this amount prod time diff : 1e-05'} Production hashtag (incorrect ponderation at 20-10-18) : 0.0005500000000000008 result_one_balle_Type_JRM:{'day': '15042026', 'map_nb_amount': {0: 9, 1: 77, 2: 1, 3: 1, 4: 0}, 'map_time_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'duration': 839.9332909584045, 'nb_balles_papier': 0.0008800000000000017, 'begin_time_port': 'image_15042026_14_15_03_086889m0.jpg 1e-05 for time 1, id_amount 2 this amount prod time diff : 1e-05'} Production hashtag (incorrect ponderation at 20-10-18) : 0.0008800000000000017 result_one_balle_Type_JRM:{'day': '15042026', 'map_nb_amount': {0: 12, 1: 68, 2: 0, 3: 2, 4: 0}, 'map_time_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'duration': 839.4338779449463, 'nb_balles_papier': 0.0008200000000000015, 'begin_time_port': 'image_15042026_14_30_02_894161m0.jpg 1e-05 for time 1, id_amount 1 this amount prod time diff : 1e-05'} Production hashtag (incorrect ponderation at 20-10-18) : 0.0008200000000000015 LERROR missing amount info Lresult_one_balle_Type_JRM:{'day': '15042026', 'map_nb_amount': {0: 3, 1: 82, 2: 0, 3: 0, 4: 0}, 'map_time_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'duration': 840.733883857727, 'nb_balles_papier': 0.0008500000000000016, 'begin_time_port': 'image_15042026_14_45_02_218368m0.jpg 1e-05 for time 1, id_amount 2 this amount prod time diff : 1e-05'} Production hashtag (incorrect ponderation at 20-10-18) : 0.0008500000000000016 We have rejected 0 photos because of the batch_size condition ! NUMBER BATCH list_of_portfolios_to_create : 8 list_same_port_ids : [30985725] find same portfolio which already exist 30985725 , we will use it list_same_port_ids : [30985727] find same portfolio which already exist 30985727 , we will use it list_same_port_ids : [30985728] find same portfolio which already exist 30985728 , we will use it list_same_port_ids : [30986179] find same portfolio which already exist 30986179 , we will use it list_same_port_ids : [30986180] find same portfolio which already exist 30986180 , we will use it list_same_port_ids : [30986181] find same portfolio which already exist 30986181 , we will use it list_same_port_ids : [30986515] find same portfolio which already exist 30986515 , we will use it list_same_port_ids : [] Qualite : 0.22317763069519345 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=30985725 AND mptpi.`type`=3726 To do Qualite : 0.23631924276981323 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=30985727 AND mptpi.`type`=3726 To do Qualite : 0.2349286368296265 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=30985728 AND mptpi.`type`=3726 To do Qualite : 0.22095602432694822 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=30986179 AND mptpi.`type`=3726 To do Qualite : 0.22448279248594155 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=30986180 AND mptpi.`type`=3726 To do Qualite : 0.16147758894219166 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=30986181 AND mptpi.`type`=3726 To do Qualite : 0.2281555766420951 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=30986515 AND mptpi.`type`=3726 To do # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=30993631 AND mptpi.`type`=3726 To do elapsed_time : count_nb_balles_and_create_portfolio 6.163213491439819 # DISPLAY ALL COLLECTED DATA : {'15042026': {'nb_upload': 725, 'nb_taggue_class': 622, 'nb_taggue_densite': 622, 'nb_descriptors': 549}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1415941607, 1415859466, 1415859465, 1415859464, 1415859463, 1415859462, 1415858561, 1415858560, 1415858559, 1415858558, 1415858557, 1415858556, 1415858544, 1415858542, 1415858541, 1415858540, 1415858539, 1415858538, 1415857863, 1415857862, 1415857861, 1415857860, 1415857859, 1415857857, 1415856729, 1415856728, 1415856727, 1415856724, 1415856723, 1415856721, 1415855382, 1415855381, 1415855380, 1415855379, 1415855378, 1415855377, 1415855263, 1415855262, 1415855261, 1415855260, 1415855259, 1415855258, 1415855244, 1415855243, 1415855242, 1415855240, 1415855238, 1415855237, 1415855216, 1415855215, 1415855214, 1415855213, 1415855211, 1415855210, 1415855200, 1415855199, 1415855198, 1415855196, 1415855194, 1415855193, 1415855183, 1415855182, 1415855181, 1415855180, 1415855179, 1415855177, 1415855150, 1415855149, 1415855148, 1415855147, 1415855146, 1415855145, 1415854470, 1415854469, 1415854468, 1415854467, 1415854466, 1415854465, 1415854429, 1415854428, 1415854426, 1415854425, 1415854424, 1415854423, 1415854027, 1415854026, 1415854025, 1415854024, 1415854023, 1415854021, 1415853144, 1415853143, 1415853142, 1415853141, 1415853139, 1415853137, 1415853093, 1415853092, 1415853091, 1415853090] Looping around the photos to save general results len do output : 1 /30984947Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415941607', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415859466', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415859465', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415859464', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415859463', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415859462', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415858561', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415858560', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415858559', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415858558', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415858557', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415858556', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415858544', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415858542', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415858541', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415858540', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415858539', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415858538', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415857863', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415857862', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415857861', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415857860', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415857859', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415857857', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415856729', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415856728', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415856727', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415856724', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415856723', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415856721', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415855382', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415855381', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415855380', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415855379', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415855378', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415855377', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415855263', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415855262', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415855261', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415855260', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415855259', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415855258', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415855244', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415855243', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415855242', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415855240', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415855238', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415855237', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415855216', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415855215', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415855214', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415855213', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415855211', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415855210', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415855200', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415855199', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415855198', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415855196', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415855194', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415855193', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415855183', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415855182', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415855181', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415855180', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415855179', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415855177', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415855150', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415855149', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415855148', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415855147', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415855146', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415855145', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415854470', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415854469', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415854468', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415854467', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415854466', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415854465', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415854429', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415854428', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415854426', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415854425', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415854424', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415854423', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415854027', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415854026', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415854025', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415854024', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415854023', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415854021', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415853144', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415853143', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415853142', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415853141', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415853139', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415853137', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415853093', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415853092', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415853091', None, None, None, None, None, '4375606') ('4189', None, None, None, None, None, None, None, '4375606') ('4189', '30984947', '1415853090', None, None, None, None, None, '4375606') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 101 time used for this insertion : 0.06407761573791504 save_final save missing photos in datou_result : time spend for datou_step_exec : 58.485450744628906 time spend to save output : 0.06490898132324219 total time spend for step 1 : 58.55035972595215 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 1 set_done_treatment 2.38user 0.60system 1:00.65elapsed 4%CPU (0avgtext+0avgdata 115584maxresident)k 264inputs+4576outputs (6major+54196minor)pagefaults 0swaps