python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 4189' -s datou_current_4189 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 881398 load datou : 4189 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? load thcls load pdts Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec no input labels no input values updating current state to 1 list_input_json: {} Current got : datou_id : 4189, datou_cur_ids : ['2587955'] with mtr_portfolio_ids : ['20550732'] and first list_photo_ids : [] new path : /proc/881398/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! List Step Type Loaded in datou : split_time_score over limit max, limiting to limit_max 100 list_input_json : {} origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! we have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB time to download the photos : 0.03349804878234863 About to test input to load Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 1 step1:split_time_score Fri Feb 14 15:00:27 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec begin split time score 2022-04-13 10:29:59 0 TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 3379, 'mtr_user_id': 31, 'name': 'learn_classif_flux_maj_generique_effnet_v2_s_02062022', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'aluminium,ela,film_pedb,flux_dev,jrm,pcm,pcnc,pehd_pp,pet_clair,refus,tapis_vide', 'svm_portfolios_learning': '5515864,5515840,5515844,5515850,6244400,6237996,6237998,5515847,5515841,5515868,5515866', 'photo_hashtag_type': 4374, 'photo_desc_type': 5680, 'type_classification': 'tf_classification2', 'hashtag_id_list': '493546845,492741797,2107760237,2107760238,495916461,560181804,1284539308,2107760239,2107755846,538914404,2107748999'}] thcls : [{'id': 3513, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2_tf', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 4557, 'photo_desc_type': 5767, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('10', 357), ('14', 335)) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {1: 476} 14022025 20550732 Nombre de photos uploadées : 692 / 23040 (3%) 14022025 20550732 Nombre de photos taguées (types de déchets): 476 / 692 (68%) 14022025 20550732 Nombre de photos taguées (volume) : 476 / 692 (68%) elapsed_time : load_data_split_time_score 6.198883056640625e-06 elapsed_time : order_list_meta_photo_and_scores 0.00022792816162109375 LLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLL???????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????? elapsed_time : fill_and_build_computed_from_old_data 0.06928396224975586 elapsed_time : insert_dashboard_record_day_entry 0.02216506004333496 LLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLL Creating list_photo_by_hashtags Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None elapsed_time : list_photo_by_hashtags 0.04816007614135742 Creating list_photo_total elapsed_time : select_descriptors 26.572169065475464 14022025 20550732 Nombre de photos avec descriptors (type 5680) : 418 / 634 (65%) ERROR : Hum hum, what can we do for different size of descriptors (ignore the difference ) : 0 vs 1280 photo_id : 1337584894 photo_id_prec : 0 0:00:00|ON:LL0:03:59.799653|OFF:LL0:00:00|ON:LL0:05:00.558790|OFF:LLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLL0:07:00.098537|ON:LL0:00:59.716614|OFF:0:00:59.766646|ON:LL0:09:00.336045|OFF:0:00:00|ON:LL0:04:59.329527|OFF:LL0:01:00.766494|ON:LL0:00:59.687602|OFF:LL0:00:59.689809|ON:LLL0:10:00.120784|OFF:LL0:00:59.687747|ON:LMissing descriptors for photos 1337594816 and 1337594859 LMissing descriptors for photos 1337594859 and 1337594862 LLLL0:07:00.500979|OFF:0:00:59.627051|ON:LLLLL3:07:00.708250|OFF:L0:00:00|ON:Missing descriptors for photos 1337622195 and 1337622196 LLLL0:00:59.877050|OFF:L0:00:00|ON:Missing descriptors for photos 1337622297 and 1337622339 LMissing descriptors for photos 1337622339 and 1337622402 LMissing descriptors for photos 1337622402 and 1337622423 LLLL0:00:59.855992|OFF:LLLLL0:00:59.890153|ON:LLL0:00:00|OFF:LLL0:00:59.823968|ON:LL0:00:00|OFF:L0:00:00|ON:LLL0:00:59.903233|OFF:LLLLLLL0:00:00|ON:Missing descriptors for photos 1337623920 and 1337624042 LMissing descriptors for photos 1337624042 and 1337624066 LLL0:00:59.857410|OFF:LLLLLLLLL0:00:59.876114|ON:LMissing descriptors for photos 1337624223 and 1337624539 LLL0:01:00.941060|OFF:LLL0:00:00|ON:LL0:00:00|OFF:L0:00:59.817970|ON:Missing descriptors for photos 1337624767 and 1337624799 LLL0:00:00|OFF:LLL0:00:00|ON:LMissing descriptors for photos 1337624881 and 1337624884 LMissing descriptors for photos 1337624884 and 1337624885 LLL0:00:59.891103|OFF:L0:00:59.893136|ON:Missing descriptors for photos 1337624915 and 1337624919 LMissing descriptors for photos 1337624919 and 1337624923 LMissing descriptors for photos 1337624923 and 1337624925 LMissing descriptors for photos 1337624925 and 1337624928 LLL0:00:59.817399|OFF:L0:00:00|ON:Missing descriptors for photos 1337625146 and 1337625150 LLL0:00:00|OFF:LLL0:00:59.818436|ON:LL0:00:00|OFF:L0:00:00|ON:Missing descriptors for photos 1337625235 and 1337625239 LMissing descriptors for photos 1337625239 and 1337625261 LLL0:00:59.856526|OFF:LLL0:00:00|ON:Missing descriptors for photos 1337625289 and 1337625294 LLL0:01:00.824552|OFF:L0:00:00|ON:LMissing descriptors for photos 1337625727 and 1337625737 LLMissing descriptors for photos 1337625872 and 1337625873 LLMissing descriptors for photos 1337625878 and 1337625880 LLL0:01:59.776187|OFF:LLL0:00:00|ON:Missing descriptors for photos 1337625914 and 1337625918 LMissing descriptors for photos 1337625918 and 1337625922 LLL0:00:59.863458|OFF:LLLLL0:00:00|ON:ERROR : Hum hum, what can we do for different size of descriptors (ignore the difference ) : 1280 vs 0 photo_id : 1337626650 photo_id_prec : 1337626629 LMissing descriptors for photos 1337626650 and 1337626821 LMissing descriptors for photos 1337626821 and 1337626826 LMissing descriptors for photos 1337626826 and 1337626873 LMissing descriptors for photos 1337626873 and 1337626892 LMissing descriptors for photos 1337626892 and 1337626917 LMissing descriptors for photos 1337626917 and 1337626943 LMissing descriptors for photos 1337626943 and 1337626950 LMissing descriptors for photos 1337626950 and 1337626957 LMissing descriptors for photos 1337626957 and 1337626993 LMissing descriptors for photos 1337626993 and 1337627000 LMissing descriptors for photos 1337627000 and 1337627012 LMissing descriptors for photos 1337627012 and 1337627041 LMissing descriptors for photos 1337627041 and 1337627061 LMissing descriptors for photos 1337627061 and 1337627065 LMissing descriptors for photos 1337627065 and 1337627069 LMissing descriptors for photos 1337627069 and 1337627113 LMissing descriptors for photos 1337627113 and 1337627118 LMissing descriptors for photos 1337627118 and 1337627137 LMissing descriptors for photos 1337627137 and 1337627141 LMissing descriptors for photos 1337627141 and 1337627146 LMissing descriptors for photos 1337627146 and 1337627152 LMissing descriptors for photos 1337627152 and 1337627156 LMissing descriptors for photos 1337627156 and 1337627162 LMissing descriptors for photos 1337627162 and 1337627175 LMissing descriptors for photos 1337627175 and 1337627178 LMissing descriptors for photos 1337627178 and 1337627181 LMissing descriptors for photos 1337627181 and 1337627185 LMissing descriptors for photos 1337627185 and 1337627204 LMissing descriptors for photos 1337627204 and 1337627274 LMissing descriptors for photos 1337627274 and 1337627361 LMissing descriptors for photos 1337627361 and 1337627365 LMissing descriptors for photos 1337627365 and 1337627371 LMissing descriptors for photos 1337627371 and 1337627376 LMissing descriptors for photos 1337627376 and 1337627382 LMissing descriptors for photos 1337627382 and 1337627389 LMissing descriptors for photos 1337627389 and 1337627490 LMissing descriptors for photos 1337627490 and 1337627613 LMissing descriptors for photos 1337627613 and 1337627738 LMissing descriptors for photos 1337627738 and 1337627875 LMissing descriptors for photos 1337627875 and 1337627999 LMissing descriptors for photos 1337627999 and 1337628126 LMissing descriptors for photos 1337628126 and 1337628445 LMissing descriptors for photos 1337628445 and 1337628449 LMissing descriptors for photos 1337628449 and 1337628451 LMissing descriptors for photos 1337628451 and 1337628453 LMissing descriptors for photos 1337628453 and 1337628507 LMissing descriptors for photos 1337628507 and 1337628512 LMissing descriptors for photos 1337628512 and 1337628930 LMissing descriptors for photos 1337628930 and 1337628938 LMissing descriptors for photos 1337628938 and 1337628981 LMissing descriptors for photos 1337628981 and 1337629004 LMissing descriptors for photos 1337629004 and 1337629010 LMissing descriptors for photos 1337629010 and 1337629014 LMissing descriptors for photos 1337629014 and 1337629026 LMissing descriptors for photos 1337629026 and 1337629031 LMissing descriptors for photos 1337629031 and 1337629037 LMissing descriptors for photos 1337629037 and 1337629094 LMissing descriptors for photos 1337629094 and 1337629096 LMissing descriptors for photos 1337629096 and 1337629099 LMissing descriptors for photos 1337629099 and 1337629114 LMissing descriptors for photos 1337629114 and 1337629119 LMissing descriptors for photos 1337629119 and 1337629124 LMissing descriptors for photos 1337629124 and 1337629129 LMissing descriptors for photos 1337629129 and 1337629204 LMissing descriptors for photos 1337629204 and 1337629205 LMissing descriptors for photos 1337629205 and 1337629520 LMissing descriptors for photos 1337629520 and 1337629573 LMissing descriptors for photos 1337629573 and 1337629600 LMissing descriptors for photos 1337629600 and 1337629607 LMissing descriptors for photos 1337629607 and 1337629618 LMissing descriptors for photos 1337629618 and 1337629620 LMissing descriptors for photos 1337629620 and 1337629725 LMissing descriptors for photos 1337629725 and 1337629727 LMissing descriptors for photos 1337629727 and 1337629730 LMissing descriptors for photos 1337629730 and 1337629734 LMissing descriptors for photos 1337629734 and 1337629736 LMissing descriptors for photos 1337629736 and 1337629739 LMissing descriptors for photos 1337629739 and 1337629755 LMissing descriptors for photos 1337629755 and 1337629757 LMissing descriptors for photos 1337629757 and 1337629763 LMissing descriptors for photos 1337629763 and 1337629769 LMissing descriptors for photos 1337629769 and 1337629774 LMissing descriptors for photos 1337629774 and 1337629780 LMissing descriptors for photos 1337629780 and 1337630406 LMissing descriptors for photos 1337630406 and 1337630557 LMissing descriptors for photos 1337630557 and 1337630717 LMissing descriptors for photos 1337630717 and 1337630886 LMissing descriptors for photos 1337630886 and 1337630909 LMissing descriptors for photos 1337630909 and 1337630919 LMissing descriptors for photos 1337630919 and 1337630931 LMissing descriptors for photos 1337630931 and 1337630953 LMissing descriptors for photos 1337630953 and 1337631083 LMissing descriptors for photos 1337631083 and 1337631185 LMissing descriptors for photos 1337631185 and 1337631198 LMissing descriptors for photos 1337631198 and 1337631201 LMissing descriptors for photos 1337631201 and 1337631294 LMissing descriptors for photos 1337631294 and 1337631295 LMissing descriptors for photos 1337631295 and 1337631297 LMissing descriptors for photos 1337631297 and 1337631301 LMissing descriptors for photos 1337631301 and 1337631311 LMissing descriptors for photos 1337631311 and 1337631316 LMissing descriptors for photos 1337631316 and 1337631626 LMissing descriptors for photos 1337631626 and 1337631627 LMissing descriptors for photos 1337631627 and 1337631629 LMissing descriptors for photos 1337631629 and 1337631634 LMissing descriptors for photos 1337631634 and 1337631637 LMissing descriptors for photos 1337631637 and 1337631641 LMissing descriptors for photos 1337631641 and 1337631667 LMissing descriptors for photos 1337631667 and 1337631670 LMissing descriptors for photos 1337631670 and 1337631673 LMissing descriptors for photos 1337631673 and 1337631679 LMissing descriptors for photos 1337631679 and 1337631698 LMissing descriptors for photos 1337631698 and 1337631746 LMissing descriptors for photos 1337631746 and 1337631893 LMissing descriptors for photos 1337631893 and 1337631898 LMissing descriptors for photos 1337631898 and 1337631902 LMissing descriptors for photos 1337631902 and 1337631907 LMissing descriptors for photos 1337631907 and 1337631910 LMissing descriptors for photos 1337631910 and 1337631913 LMissing descriptors for photos 1337631913 and 1337631926 LMissing descriptors for photos 1337631926 and 1337631931 LMissing descriptors for photos 1337631931 and 1337631935 LMissing descriptors for photos 1337631935 and 1337631938 LMissing descriptors for photos 1337631938 and 1337631943 LMissing descriptors for photos 1337631943 and 1337631950 LMissing descriptors for photos 1337631950 and 1337631974 LMissing descriptors for photos 1337631974 and 1337631979 LMissing descriptors for photos 1337631979 and 1337631984 LMissing descriptors for photos 1337631984 and 1337631988 LMissing descriptors for photos 1337631988 and 1337631998 LMissing descriptors for photos 1337631998 and 1337632001 LMissing descriptors for photos 1337632001 and 1337632012 LMissing descriptors for photos 1337632012 and 1337632016 LMissing descriptors for photos 1337632016 and 1337632021 LMissing descriptors for photos 1337632021 and 1337632027 LMissing descriptors for photos 1337632027 and 1337632034 LMissing descriptors for photos 1337632034 and 1337632039 LMissing descriptors for photos 1337632039 and 1337632081 LMissing descriptors for photos 1337632081 and 1337632084 LMissing descriptors for photos 1337632084 and 1337632087 LMissing descriptors for photos 1337632087 and 1337632090 LMissing descriptors for photos 1337632090 and 1337632170 LMissing descriptors for photos 1337632170 and 1337632196 LMissing descriptors for photos 1337632196 and 1337632268 LMissing descriptors for photos 1337632268 and 1337632273 LMissing descriptors for photos 1337632273 and 1337632277 LMissing descriptors for photos 1337632277 and 1337632280 LMissing descriptors for photos 1337632280 and 1337632285 LMissing descriptors for photos 1337632285 and 1337632291 LMissing descriptors for photos 1337632291 and 1337632301 LMissing descriptors for photos 1337632301 and 1337632305 LMissing descriptors for photos 1337632305 and 1337632307 LMissing descriptors for photos 1337632307 and 1337632310 LMissing descriptors for photos 1337632310 and 1337632314 LMissing descriptors for photos 1337632314 and 1337632320 LMissing descriptors for photos 1337632320 and 1337632331 LMissing descriptors for photos 1337632331 and 1337632340 LMissing descriptors for photos 1337632340 and 1337632343 LMissing descriptors for photos 1337632343 and 1337632347 LMissing descriptors for photos 1337632347 and 1337632358 LMissing descriptors for photos 1337632358 and 1337632369 LMissing descriptors for photos 1337632369 and 1337632397 LMissing descriptors for photos 1337632397 and 1337632402 LMissing descriptors for photos 1337632402 and 1337632405 LMissing descriptors for photos 1337632405 and 1337632409 LMissing descriptors for photos 1337632409 and 1337632413 LMissing descriptors for photos 1337632413 and 1337632418 LMissing descriptors for photos 1337632418 and 1337632431 LMissing descriptors for photos 1337632431 and 1337632434 LMissing descriptors for photos 1337632434 and 1337632446 LMissing descriptors for photos 1337632446 and 1337632450 LMissing descriptors for photos 1337632450 and 1337632453 LMissing descriptors for photos 1337632453 and 1337632456 LMissing descriptors for photos 1337632456 and 1337632474 LMissing descriptors for photos 1337632474 and 1337632477 LMissing descriptors for photos 1337632477 and 1337632482 LMissing descriptors for photos 1337632482 and 1337632485 LMissing descriptors for photos 1337632485 and 1337632489 LMissing descriptors for photos 1337632489 and 1337632627 LMissing descriptors for photos 1337632627 and 1337632663 LMissing descriptors for photos 1337632663 and 1337632668 LMissing descriptors for photos 1337632668 and 1337632672 LMissing descriptors for photos 1337632672 and 1337632676 LMissing descriptors for photos 1337632676 and 1337632678 LMissing descriptors for photos 1337632678 and 1337632684 LMissing descriptors for photos 1337632684 and 1337632728 LMissing descriptors for photos 1337632728 and 1337632733 LMissing descriptors for photos 1337632733 and 1337632735 LMissing descriptors for photos 1337632735 and 1337632738 LMissing descriptors for photos 1337632738 and 1337632742 LMissing descriptors for photos 1337632742 and 1337632744 LMissing descriptors for photos 1337632744 and 1337632775 LMissing descriptors for photos 1337632775 and 1337632780 LMissing descriptors for photos 1337632780 and 1337632783 LMissing descriptors for photos 1337632783 and 1337632788 LMissing descriptors for photos 1337632788 and 1337632914 LMissing descriptors for photos 1337632914 and 1337633179 LMissing descriptors for photos 1337633179 and 1337634276 LMissing descriptors for photos 1337634276 and 1337634341 LMissing descriptors for photos 1337634341 and 1337634414 LMissing descriptors for photos 1337634414 and 1337634541 LMissing descriptors for photos 1337634541 and 1337634665 LMissing descriptors for photos 1337634665 and 1337634790 LMissing descriptors for photos 1337634790 and 1337634881 LMissing descriptors for photos 1337634881 and 1337634997 LMissing descriptors for photos 1337634997 and 1337635235 LMissing descriptors for photos 1337635235 and 1337635405 LMissing descriptors for photos 1337635405 and 1337635715 LMissing descriptors for photos 1337635715 and 1337635847 LMissing descriptors for photos 1337635847 and 1337636182 LMissing descriptors for photos 1337636182 and 1337636237 LMissing descriptors for photos 1337636237 and 1337636241 LMissing descriptors for photos 1337636241 and 1337636305 LMissing descriptors for photos 1337636305 and 1337636495 LMissing descriptors for photos 1337636495 and 1337636639 L 14022025 Removing 88 photos because of the 'same image' condition Total on : 14461.222213999996 list_time_on Total off : 1078.756061 list_time_off dist_desc begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 692 time used for this insertion : 0.10706305503845215 photos_removed : len 88 elapsed_time : remove_photo_duplicate 0.18761754035949707 To do, maybe not at the correct place ! .........................L.L.............................L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L...........L.........................................................L................................L.L............L.L.............................L...................................L.L..................................L.L.L..L.........L.....................................L..L..L..L.L.L..L..L.L.L.L.L..L..L.L.L.L..L..L.L.L..L.L..L..L.L.L.L.L.L.L..L.L.L.L.L.L..L.L..L.L.L..L.L.L..L.L.L..L.L.L..L.L.L.L.L.L..L.L.L..L.L.L..L.L.L.L..L.L.L.L..L.L..L.L..L.L..L.L..L.L.L.L.L..L.L.L.L..L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.Lforce hashtag to JRM elapsed_time : CREATE_PORT_BATCH_BY_HOUR 0.023900270462036133 NUMBER BATCH : 8 list_ponderation used : [1e-05, 1e-05, 1e-05, 1e-05, 1e-05] , list_hashtag_class_create_as_list : ['jrm'] result_one_balle_Type_JRM:{'day': '14022025', 'map_nb_amount': {0: 4, 1: 47, 2: 0, 3: 1, 4: 0}, 'map_time_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'duration': 540.3584430217743, 'nb_balles_papier': 0.0005200000000000007, 'begin_time_port': 'image_14022025_10_00_02_767938m0.jpg 1e-05 for time 1, id_amount 2 this amount prod time diff : 1e-05'} Production hashtag (incorrect ponderation at 20-10-18) : 0.0005200000000000007 result_one_balle_Type_JRM:{'day': '14022025', 'map_nb_amount': {0: 10, 1: 70, 2: 0, 3: 0, 4: 0}, 'map_time_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'duration': 779.1860852241516, 'nb_balles_papier': 0.0008100000000000015, 'begin_time_port': 'image_14022025_10_16_03_224918m0.jpg 1e-05 for time 1, id_amount 2 this amount prod time diff : 1e-05'} Production hashtag (incorrect ponderation at 20-10-18) : 0.0008100000000000015 result_one_balle_Type_JRM:{'day': '14022025', 'map_nb_amount': {0: 10, 1: 71, 2: 0, 3: 3, 4: 0}, 'map_time_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'duration': 839.9503998756409, 'nb_balles_papier': 0.0008500000000000016, 'begin_time_port': 'image_14022025_10_30_03_045253m0.jpg 1e-05 for time 1, id_amount 2 this amount prod time diff : 1e-05'} Production hashtag (incorrect ponderation at 20-10-18) : 0.0008500000000000016 result_one_balle_Type_JRM:{'day': '14022025', 'map_nb_amount': {0: 6, 1: 71, 2: 3, 3: 1, 4: 0}, 'map_time_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'duration': 840.1056749820709, 'nb_balles_papier': 0.0008100000000000015, 'begin_time_port': 'image_14022025_10_45_02_638439m0.jpg 1e-05 for time 1, id_amount 2 this amount prod time diff : 1e-05'} Production hashtag (incorrect ponderation at 20-10-18) : 0.0008100000000000015 result_one_balle_Type_JRM:{'day': '14022025', 'map_nb_amount': {0: 0, 1: 22, 2: 0, 3: 0, 4: 0}, 'map_time_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'duration': 840.1906208992004, 'nb_balles_papier': 0.00023, 'begin_time_port': 'image_14022025_14_00_02_234869m0.jpg 1e-05 for time 1, id_amount 2 this amount prod time diff : 1e-05'} Production hashtag (incorrect ponderation at 20-10-18) : 0.00023 LLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info ERROR missing amount info Lresult_one_balle_Type_JRM:{'day': '14022025', 'map_nb_amount': {0: 0, 1: 8, 2: 0, 3: 0, 4: 0}, 'map_time_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'duration': 840.3741688728333, 'nb_balles_papier': 8e-05, 'begin_time_port': 'image_14022025_14_15_02_282016m0.jpg 1e-05 for time 1, id_amount 2 this amount prod time diff : 1e-05'} Production hashtag (incorrect ponderation at 20-10-18) : 8e-05 LLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLWe filter photos on hashtag condition ! LLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLWe filter photos on hashtag condition ! We have rejected 0 photos because of the batch_size condition ! NUMBER BATCH list_of_portfolios_to_create : 6 list_same_port_ids : [20551920] find same portfolio which already exist 20551920 , we will use it list_same_port_ids : [20553017] find same portfolio which already exist 20553017 , we will use it list_same_port_ids : [20554393] find same portfolio which already exist 20554393 , we will use it list_same_port_ids : [20558650] find same portfolio which already exist 20558650 , we will use it list_same_port_ids : [20558651] find same portfolio which already exist 20558651 , we will use it list_same_port_ids : [] https://marlene.fotonower.com/api/v1/secured/portfolio/new?name=JRM_diff_batch__14022025_14_15_02_282016&access_token=b05576c56a0e42ad0cb9b46155f68f82 Qualite : 0.12688593194964148 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20551920 AND mptpi.`type`=3726 To do Qualite : 0.11267466792228975 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20553017 AND mptpi.`type`=3726 To do Qualite : 0.0966749533638519 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20554393 AND mptpi.`type`=3726 To do # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20558650 AND mptpi.`type`=3726 To do # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20558651 AND mptpi.`type`=3726 To do # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=20560036 AND mptpi.`type`=3726 To do elapsed_time : count_nb_balles_and_create_portfolio 3.182309865951538 # DISPLAY ALL COLLECTED DATA : {'14022025': {'nb_upload': 692, 'nb_taggue_class': 476, 'nb_taggue_densite': 476, 'nb_descriptors': 418}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1337636639, 1337636495, 1337636305, 1337636241, 1337636237, 1337636182, 1337635847, 1337635715, 1337635405, 1337635235, 1337634997, 1337634881, 1337634790, 1337634665, 1337634541, 1337634414, 1337634341, 1337634276, 1337633179, 1337632914, 1337632788, 1337632783, 1337632780, 1337632775, 1337632744, 1337632742, 1337632738, 1337632735, 1337632733, 1337632728, 1337632684, 1337632678, 1337632676, 1337632672, 1337632668, 1337632663, 1337632627, 1337632489, 1337632485, 1337632482, 1337632477, 1337632474, 1337632456, 1337632453, 1337632450, 1337632446, 1337632434, 1337632431, 1337632418, 1337632413, 1337632409, 1337632405, 1337632402, 1337632397, 1337632369, 1337632358, 1337632347, 1337632343, 1337632340, 1337632331, 1337632320, 1337632314, 1337632310, 1337632307, 1337632305, 1337632301, 1337632291, 1337632285, 1337632280, 1337632277, 1337632273, 1337632268, 1337632196, 1337632170, 1337632090, 1337632087, 1337632084, 1337632081, 1337632039, 1337632034, 1337632027, 1337632021, 1337632016, 1337632012, 1337632001, 1337631998, 1337631988, 1337631984, 1337631979, 1337631974, 1337631950, 1337631943, 1337631938, 1337631935, 1337631931, 1337631926, 1337631913, 1337631910, 1337631907, 1337631902] Looping around the photos to save general results len do output : 1 /20550732Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337636639', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337636495', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337636305', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337636241', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337636237', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337636182', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337635847', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337635715', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337635405', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337635235', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337634997', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337634881', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337634790', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337634665', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337634541', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337634414', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337634341', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337634276', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337633179', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632914', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632788', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632783', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632780', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632775', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632744', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632742', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632738', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632735', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632733', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632728', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632684', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632678', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632676', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632672', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632668', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632663', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632627', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632489', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632485', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632482', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632477', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632474', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632456', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632453', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632450', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632446', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632434', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632431', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632418', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632413', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632409', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632405', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632402', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632397', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632369', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632358', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632347', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632343', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632340', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632331', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632320', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632314', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632310', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632307', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632305', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632301', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632291', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632285', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632280', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632277', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632273', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632268', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632196', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632170', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632090', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632087', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632084', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632081', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632039', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632034', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632027', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632021', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632016', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632012', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337632001', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337631998', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337631988', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337631984', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337631979', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337631974', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337631950', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337631943', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337631938', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337631935', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337631931', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337631926', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337631913', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337631910', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337631907', None, None, None, None, None, '2587955') ('4189', None, None, None, None, None, None, None, '2587955') ('4189', '20550732', '1337631902', None, None, None, None, None, '2587955') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 101 time used for this insertion : 0.030210256576538086 save_final save missing photos in datou_result : time spend for datou_step_exec : 30.793257474899292 time spend to save output : 0.03113722801208496 total time spend for step 1 : 30.824394702911377 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 1 set_done_treatment 2.83user 0.76system 0:33.95elapsed 10%CPU (0avgtext+0avgdata 105480maxresident)k 0inputs+3520outputs (0major+49994minor)pagefaults 0swaps