python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 4189' -s datou_current_4189 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 2690151 load datou : 4189 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? load thcls load pdts Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec updating current state to 1 list_input_json: [] Date 2026-05-09 08:35:27.190631 Current got : datou_id : 4189, datou_cur_ids : ['4379329'] with mtr_portfolio_ids : ['31042617'] and first list_photo_ids : [] new path : /proc/2690151/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! List Step Type Loaded in datou : split_time_score over limit max, limiting to limit_max 100 list_input_json : [] origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! we have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB time to download the photos : 0.011986494064331055 About to test input to load Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 1 step1:split_time_score Sat May 9 08:35:27 2026 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec begin split time score 2022-04-13 10:29:59 0 TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 3379, 'mtr_user_id': 31, 'name': 'learn_classif_flux_maj_generique_effnet_v2_s_02062022', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'aluminium,ela,film_pedb,flux_dev,jrm,pcm,pcnc,pehd_pp,pet_clair,refus,tapis_vide', 'svm_portfolios_learning': '5515864,5515840,5515844,5515850,6244400,6237996,6237998,5515847,5515841,5515868,5515866', 'photo_hashtag_type': 4374, 'photo_desc_type': 5680, 'type_classification': 'tf_classification2', 'hashtag_id_list': '493546845,492741797,2107760237,2107760238,495916461,560181804,1284539308,2107760239,2107755846,538914404,2107748999'}] thcls : [{'id': 3513, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2_tf', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 4557, 'photo_desc_type': 5767, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('10', 385), ('14', 275)) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {1: 604} 29042026 31042617 Nombre de photos uploadées : 660 / 23040 (2%) 29042026 31042617 Nombre de photos taguées (types de déchets): 604 / 660 (91%) 29042026 31042617 Nombre de photos taguées (volume) : 604 / 660 (91%) elapsed_time : load_data_split_time_score 2.86102294921875e-06 elapsed_time : order_list_meta_photo_and_scores 0.0002696514129638672 LLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLL???????????????????????????????????????????????????????? elapsed_time : fill_and_build_computed_from_old_data 0.027431249618530273 Catched exception ! Connect or reconnect ! Catched exception ! Connect or reconnect ! elapsed_time : insert_dashboard_record_day_entry 0.4274179935455322 LLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLL Creating list_photo_by_hashtags Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None elapsed_time : list_photo_by_hashtags 0.04967308044433594 Creating list_photo_total elapsed_time : select_descriptors 28.153757095336914 29042026 31042617 Nombre de photos avec descriptors (type 5680) : 320 / 376 (85%) ERROR : Hum hum, what can we do for different size of descriptors (ignore the difference ) : 0 vs 1280 photo_id : 1416670592 photo_id_prec : 0 0:00:00|ON:LMissing descriptors for photos 1416671111 and 1416671138 LMissing descriptors for photos 1416671138 and 1416671141 LMissing descriptors for photos 1416671141 and 1416671142 LMissing descriptors for photos 1416671142 and 1416671143 LLMissing descriptors for photos 1416671145 and 1416671157 LMissing descriptors for photos 1416671157 and 1416671158 LMissing descriptors for photos 1416671158 and 1416671159 LMissing descriptors for photos 1416671159 and 1416671160 LMissing descriptors for photos 1416671160 and 1416671161 LMissing descriptors for photos 1416671161 and 1416671162 LMissing descriptors for photos 1416671162 and 1416671200 LMissing descriptors for photos 1416671200 and 1416671201 LMissing descriptors for photos 1416671201 and 1416671202 LMissing descriptors for photos 1416671202 and 1416671203 LMissing descriptors for photos 1416671203 and 1416671204 LMissing descriptors for photos 1416671204 and 1416671205 LMissing descriptors for photos 1416671205 and 1416671314 LMissing descriptors for photos 1416671314 and 1416671315 LMissing descriptors for photos 1416671315 and 1416671317 LMissing descriptors for photos 1416671317 and 1416671319 LMissing descriptors for photos 1416671319 and 1416671320 LMissing descriptors for photos 1416671320 and 1416671321 LMissing descriptors for photos 1416671321 and 1416671329 LMissing descriptors for photos 1416671329 and 1416671346 LMissing descriptors for photos 1416671346 and 1416671330 LMissing descriptors for photos 1416671330 and 1416671332 LMissing descriptors for photos 1416671332 and 1416671335 LMissing descriptors for photos 1416671335 and 1416671347 LMissing descriptors for photos 1416671347 and 1416671348 LMissing descriptors for photos 1416671348 and 1416671350 LMissing descriptors for photos 1416671350 and 1416671351 LMissing descriptors for photos 1416671351 and 1416671352 LMissing descriptors for photos 1416671352 and 1416671403 LMissing descriptors for photos 1416671403 and 1416671404 LMissing descriptors for photos 1416671404 and 1416671405 LMissing descriptors for photos 1416671405 and 1416671406 LMissing descriptors for photos 1416671406 and 1416671407 LMissing descriptors for photos 1416671407 and 1416671408 LMissing descriptors for photos 1416671408 and 1416673673 LMissing descriptors for photos 1416673673 and 1416673674 LMissing descriptors for photos 1416673674 and 1416673676 LMissing descriptors for photos 1416673676 and 1416673677 LMissing descriptors for photos 1416673677 and 1416673678 LLMissing descriptors for photos 1416679263 and 1416679264 LLMissing descriptors for photos 1416679266 and 1416679267 LMissing descriptors for photos 1416679267 and 1416679268 LMissing descriptors for photos 1416679268 and 1416679297 LLL0:35:59.945330|OFF:0:00:00|ON:LL0:04:59.526207|OFF:0:00:00|ON:LMissing descriptors for photos 1416679711 and 1416679712 LLLLLMissing descriptors for photos 1416684572 and 1416684573 LMissing descriptors for photos 1416684573 and 1416684574 LMissing descriptors for photos 1416684574 and 1416684575 LMissing descriptors for photos 1416684575 and 1416685737 LMissing descriptors for photos 1416685737 and 1416685738 LMissing descriptors for photos 1416685738 and 1416685739 LMissing descriptors for photos 1416685739 and 1416685740 LMissing descriptors for photos 1416685740 and 1416685741 LMissing descriptors for photos 1416685741 and 1416685743 LMissing descriptors for photos 1416685743 and 1416685750 LMissing descriptors for photos 1416685750 and 1416685751 LLLLLLMissing descriptors for photos 1416686381 and 1416686428 LLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLMissing descriptors for photos 1416693344 and 1416693390 LMissing descriptors for photos 1416693390 and 1416693391 LMissing descriptors for photos 1416693391 and 1416693392 LMissing descriptors for photos 1416693392 and 1416693394 LMissing descriptors for photos 1416693394 and 1416693396 LMissing descriptors for photos 1416693396 and 1416693398 LMissing descriptors for photos 1416693398 and 1416694891 LMissing descriptors for photos 1416694891 and 1416694894 LMissing descriptors for photos 1416694894 and 1416694896 LMissing descriptors for photos 1416694896 and 1416694898 LMissing descriptors for photos 1416694898 and 1416694900 LMissing descriptors for photos 1416694900 and 1416694901 LLLMissing descriptors for photos 1416696066 and 1416696067 LMissing descriptors for photos 1416696067 and 1416696068 LMissing descriptors for photos 1416696068 and 1416696069 LMissing descriptors for photos 1416696069 and 1416696070 LMissing descriptors for photos 1416696070 and 1416696118 LMissing descriptors for photos 1416696118 and 1416696119 LMissing descriptors for photos 1416696119 and 1416696120 LMissing descriptors for photos 1416696120 and 1416696121 LMissing descriptors for photos 1416696121 and 1416696123 LMissing descriptors for photos 1416696123 and 1416696128 LMissing descriptors for photos 1416696128 and 1416697648 LMissing descriptors for photos 1416697648 and 1416697650 LMissing descriptors for photos 1416697650 and 1416697653 LMissing descriptors for photos 1416697653 and 1416697655 LMissing descriptors for photos 1416697655 and 1416697657 LMissing descriptors for photos 1416697657 and 1416697658 LMissing descriptors for photos 1416697658 and 1416697724 LMissing descriptors for photos 1416697724 and 1416697725 LMissing descriptors for photos 1416697725 and 1416697727 LMissing descriptors for photos 1416697727 and 1416697771 LMissing descriptors for photos 1416697771 and 1416697772 LMissing descriptors for photos 1416697772 and 1416697775 LMissing descriptors for photos 1416697775 and 1416697777 LMissing descriptors for photos 1416697777 and 1416697779 LMissing descriptors for photos 1416697779 and 1416697782 LMissing descriptors for photos 1416697782 and 1416697827 LMissing descriptors for photos 1416697827 and 1416697829 LMissing descriptors for photos 1416697829 and 1416697832 LMissing descriptors for photos 1416697832 and 1416697833 LMissing descriptors for photos 1416697833 and 1416697834 LLLMissing descriptors for photos 1416697881 and 1416697882 LMissing descriptors for photos 1416697882 and 1416697883 LMissing descriptors for photos 1416697883 and 1416697884 LMissing descriptors for photos 1416697884 and 1416697885 LMissing descriptors for photos 1416697885 and 1416697888 LMissing descriptors for photos 1416697888 and 1416697929 LMissing descriptors for photos 1416697929 and 1416697931 LMissing descriptors for photos 1416697931 and 1416697933 LMissing descriptors for photos 1416697933 and 1416697935 LMissing descriptors for photos 1416697935 and 1416697938 LMissing descriptors for photos 1416697938 and 1416697940 LMissing descriptors for photos 1416697940 and 1416697988 LMissing descriptors for photos 1416697988 and 1416697990 LMissing descriptors for photos 1416697990 and 1416697993 LMissing descriptors for photos 1416697993 and 1416697994 LMissing descriptors for photos 1416697994 and 1416697995 LLLMissing descriptors for photos 1416698090 and 1416698091 LMissing descriptors for photos 1416698091 and 1416698092 LMissing descriptors for photos 1416698092 and 1416698093 LMissing descriptors for photos 1416698093 and 1416698095 LMissing descriptors for photos 1416698095 and 1416698097 LMissing descriptors for photos 1416698097 and 1416698140 LMissing descriptors for photos 1416698140 and 1416698153 LMissing descriptors for photos 1416698153 and 1416698155 LMissing descriptors for photos 1416698155 and 1416698158 LMissing descriptors for photos 1416698158 and 1416698161 LMissing descriptors for photos 1416698161 and 1416698172 LMissing descriptors for photos 1416698172 and 1416698286 LMissing descriptors for photos 1416698286 and 1416698289 LMissing descriptors for photos 1416698289 and 1416698291 LMissing descriptors for photos 1416698291 and 1416698292 LMissing descriptors for photos 1416698292 and 1416698293 LMissing descriptors for photos 1416698293 and 1416698294 LMissing descriptors for photos 1416698294 and 1416698397 LMissing descriptors for photos 1416698397 and 1416698399 LMissing descriptors for photos 1416698399 and 1416698401 LMissing descriptors for photos 1416698401 and 1416698403 LMissing descriptors for photos 1416698403 and 1416698406 LMissing descriptors for photos 1416698406 and 1416698408 LMissing descriptors for photos 1416698408 and 1416698453 LMissing descriptors for photos 1416698453 and 1416698456 LMissing descriptors for photos 1416698456 and 1416698458 LMissing descriptors for photos 1416698458 and 1416698459 LMissing descriptors for photos 1416698459 and 1416698460 LMissing descriptors for photos 1416698460 and 1416698461 LMissing descriptors for photos 1416698461 and 1416698516 LMissing descriptors for photos 1416698516 and 1416698517 LMissing descriptors for photos 1416698517 and 1416698518 LMissing descriptors for photos 1416698518 and 1416698519 LMissing descriptors for photos 1416698519 and 1416698520 LMissing descriptors for photos 1416698520 and 1416698523 LMissing descriptors for photos 1416698523 and 1416698564 LMissing descriptors for photos 1416698564 and 1416698566 LMissing descriptors for photos 1416698566 and 1416698568 LMissing descriptors for photos 1416698568 and 1416698571 LMissing descriptors for photos 1416698571 and 1416698573 LMissing descriptors for photos 1416698573 and 1416698575 LMissing descriptors for photos 1416698575 and 1416698623 LMissing descriptors for photos 1416698623 and 1416698626 LMissing descriptors for photos 1416698626 and 1416698628 LMissing descriptors for photos 1416698628 and 1416698629 LMissing descriptors for photos 1416698629 and 1416698630 LMissing descriptors for photos 1416698630 and 1416698631 LMissing descriptors for photos 1416698631 and 1416698679 LMissing descriptors for photos 1416698679 and 1416698680 LMissing descriptors for photos 1416698680 and 1416698681 LMissing descriptors for photos 1416698681 and 1416698682 LMissing descriptors for photos 1416698682 and 1416698683 LMissing descriptors for photos 1416698683 and 1416698686 LMissing descriptors for photos 1416698686 and 1416698784 LMissing descriptors for photos 1416698784 and 1416698786 LMissing descriptors for photos 1416698786 and 1416698787 LMissing descriptors for photos 1416698787 and 1416698788 LMissing descriptors for photos 1416698788 and 1416698789 LMissing descriptors for photos 1416698789 and 1416698790 LMissing descriptors for photos 1416698790 and 1416698853 LMissing descriptors for photos 1416698853 and 1416698855 LMissing descriptors for photos 1416698855 and 1416698857 LMissing descriptors for photos 1416698857 and 1416698859 LMissing descriptors for photos 1416698859 and 1416698862 LMissing descriptors for photos 1416698862 and 1416698864 LMissing descriptors for photos 1416698864 and 1416698899 LMissing descriptors for photos 1416698899 and 1416698900 LMissing descriptors for photos 1416698900 and 1416698901 LMissing descriptors for photos 1416698901 and 1416698902 LMissing descriptors for photos 1416698902 and 1416698903 LMissing descriptors for photos 1416698903 and 1416698904 LMissing descriptors for photos 1416698904 and 1416698907 LMissing descriptors for photos 1416698907 and 1416698908 LMissing descriptors for photos 1416698908 and 1416698909 LMissing descriptors for photos 1416698909 and 1416698910 LMissing descriptors for photos 1416698910 and 1416698911 LMissing descriptors for photos 1416698911 and 1416698912 LMissing descriptors for photos 1416698912 and 1416698913 LMissing descriptors for photos 1416698913 and 1416698914 LMissing descriptors for photos 1416698914 and 1416698915 LMissing descriptors for photos 1416698915 and 1416698916 LMissing descriptors for photos 1416698916 and 1416698917 LMissing descriptors for photos 1416698917 and 1416698918 LMissing descriptors for photos 1416698918 and 1416698922 LMissing descriptors for photos 1416698922 and 1416698923 LMissing descriptors for photos 1416698923 and 1416698924 LMissing descriptors for photos 1416698924 and 1416698925 LMissing descriptors for photos 1416698925 and 1416698926 LMissing descriptors for photos 1416698926 and 1416698927 LMissing descriptors for photos 1416698927 and 1416698928 LMissing descriptors for photos 1416698928 and 1416698929 LMissing descriptors for photos 1416698929 and 1416698930 LMissing descriptors for photos 1416698930 and 1416698931 LMissing descriptors for photos 1416698931 and 1416698932 LMissing descriptors for photos 1416698932 and 1416698933 LMissing descriptors for photos 1416698933 and 1416698936 LMissing descriptors for photos 1416698936 and 1416698937 LMissing descriptors for photos 1416698937 and 1416698938 LMissing descriptors for photos 1416698938 and 1416698939 LMissing descriptors for photos 1416698939 and 1416698940 LMissing descriptors for photos 1416698940 and 1416698941 LMissing descriptors for photos 1416698941 and 1416698942 LMissing descriptors for photos 1416698942 and 1416698943 LMissing descriptors for photos 1416698943 and 1416698944 LMissing descriptors for photos 1416698944 and 1416698945 LMissing descriptors for photos 1416698945 and 1416698946 LMissing descriptors for photos 1416698946 and 1416698947 LMissing descriptors for photos 1416698947 and 1416698997 LMissing descriptors for photos 1416698997 and 1416698999 LMissing descriptors for photos 1416698999 and 1416699001 LMissing descriptors for photos 1416699001 and 1416699004 LMissing descriptors for photos 1416699004 and 1416699006 LMissing descriptors for photos 1416699006 and 1416699008 LMissing descriptors for photos 1416699008 and 1416699059 LMissing descriptors for photos 1416699059 and 1416699061 LMissing descriptors for photos 1416699061 and 1416699063 LMissing descriptors for photos 1416699063 and 1416699064 LMissing descriptors for photos 1416699064 and 1416699065 LMissing descriptors for photos 1416699065 and 1416699066 LMissing descriptors for photos 1416699066 and 1416699121 LMissing descriptors for photos 1416699121 and 1416699122 LMissing descriptors for photos 1416699122 and 1416699123 LMissing descriptors for photos 1416699123 and 1416699125 LMissing descriptors for photos 1416699125 and 1416699127 LMissing descriptors for photos 1416699127 and 1416699130 LMissing descriptors for photos 1416699130 and 1416699181 LMissing descriptors for photos 1416699181 and 1416699183 LMissing descriptors for photos 1416699183 and 1416699185 LMissing descriptors for photos 1416699185 and 1416699187 LMissing descriptors for photos 1416699187 and 1416699189 LMissing descriptors for photos 1416699189 and 1416699190 LMissing descriptors for photos 1416699190 and 1416699276 LMissing descriptors for photos 1416699276 and 1416699278 LMissing descriptors for photos 1416699278 and 1416699280 LMissing descriptors for photos 1416699280 and 1416699282 LMissing descriptors for photos 1416699282 and 1416699285 LMissing descriptors for photos 1416699285 and 1416699287 LMissing descriptors for photos 1416699287 and 1416699336 LMissing descriptors for photos 1416699336 and 1416699339 LMissing descriptors for photos 1416699339 and 1416699342 LMissing descriptors for photos 1416699342 and 1416699345 LMissing descriptors for photos 1416699345 and 1416699346 LMissing descriptors for photos 1416699346 and 1416699347 LMissing descriptors for photos 1416699347 and 1416699390 LMissing descriptors for photos 1416699390 and 1416699391 LMissing descriptors for photos 1416699391 and 1416699392 LMissing descriptors for photos 1416699392 and 1416699393 LMissing descriptors for photos 1416699393 and 1416699394 LMissing descriptors for photos 1416699394 and 1416699397 LMissing descriptors for photos 1416699397 and 1416699458 LMissing descriptors for photos 1416699458 and 1416699460 LMissing descriptors for photos 1416699460 and 1416699462 LMissing descriptors for photos 1416699462 and 1416699464 LMissing descriptors for photos 1416699464 and 1416699467 LMissing descriptors for photos 1416699467 and 1416699469 LMissing descriptors for photos 1416699469 and 1416699566 LMissing descriptors for photos 1416699566 and 1416699567 LMissing descriptors for photos 1416699567 and 1416699568 LMissing descriptors for photos 1416699568 and 1416699570 LMissing descriptors for photos 1416699570 and 1416699572 LMissing descriptors for photos 1416699572 and 1416699575 LMissing descriptors for photos 1416699575 and 1416699616 LMissing descriptors for photos 1416699616 and 1416699618 LMissing descriptors for photos 1416699618 and 1416699620 LMissing descriptors for photos 1416699620 and 1416699622 LMissing descriptors for photos 1416699622 and 1416699624 LMissing descriptors for photos 1416699624 and 1416699625 LMissing descriptors for photos 1416699625 and 1416699670 LMissing descriptors for photos 1416699670 and 1416699671 LMissing descriptors for photos 1416699671 and 1416699672 LMissing descriptors for photos 1416699672 and 1416699673 LMissing descriptors for photos 1416699673 and 1416699674 LMissing descriptors for photos 1416699674 and 1416699675 LMissing descriptors for photos 1416699675 and 1416699733 LMissing descriptors for photos 1416699733 and 1416699734 LMissing descriptors for photos 1416699734 and 1416699736 LMissing descriptors for photos 1416699736 and 1416699738 LMissing descriptors for photos 1416699738 and 1416699740 LMissing descriptors for photos 1416699740 and 1416699742 LMissing descriptors for photos 1416699742 and 1416699790 LMissing descriptors for photos 1416699790 and 1416699792 LMissing descriptors for photos 1416699792 and 1416699795 LMissing descriptors for photos 1416699795 and 1416699797 LMissing descriptors for photos 1416699797 and 1416699798 LMissing descriptors for photos 1416699798 and 1416699799 LMissing descriptors for photos 1416699799 and 1416699842 LMissing descriptors for photos 1416699842 and 1416699843 LMissing descriptors for photos 1416699843 and 1416699844 LMissing descriptors for photos 1416699844 and 1416699845 LMissing descriptors for photos 1416699845 and 1416699846 LMissing descriptors for photos 1416699846 and 1416699847 LMissing descriptors for photos 1416699847 and 1416699860 LMissing descriptors for photos 1416699860 and 1416699905 LMissing descriptors for photos 1416699905 and 1416699906 LMissing descriptors for photos 1416699906 and 1416699908 LMissing descriptors for photos 1416699908 and 1416699911 LMissing descriptors for photos 1416699911 and 1416699913 LMissing descriptors for photos 1416699913 and 1416699915 LMissing descriptors for photos 1416699915 and 1416699961 LMissing descriptors for photos 1416699961 and 1416699964 LMissing descriptors for photos 1416699964 and 1416699969 LMissing descriptors for photos 1416699969 and 1416699971 LMissing descriptors for photos 1416699971 and 1416699973 LMissing descriptors for photos 1416699973 and 1416699974 LMissing descriptors for photos 1416699974 and 1416700089 LMissing descriptors for photos 1416700089 and 1416700091 LMissing descriptors for photos 1416700091 and 1416700094 LMissing descriptors for photos 1416700094 and 1416700099 LMissing descriptors for photos 1416700099 and 1416700102 LMissing descriptors for photos 1416700102 and 1416700104 L 29042026 Removing 2 photos because of the 'same image' condition Total on : 2459.471537 list_time_on Total off : 0.0 list_time_off Warning in study_and_display_distrib_list : min=max : 0.0 0.0 dist_desc begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 660 time used for this insertion : 0.4838290214538574 photos_removed : len 2 elapsed_time : remove_photo_duplicate 0.6151244640350342 To do, maybe not at the correct place ! .................................................................L.L.L.L.L..L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L................................................................................................................L.L..L.L.L.L...L............................L...............................L.L..L............................L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.Lforce hashtag to JRM elapsed_time : CREATE_PORT_BATCH_BY_HOUR 0.019850492477416992 NUMBER BATCH : 7 list_ponderation used : [1e-05, 1e-05, 1e-05, 1e-05, 1e-05] , list_hashtag_class_create_as_list : ['jrm'] result_one_balle_Type_JRM:{'day': '29042026', 'map_nb_amount': {0: 14, 1: 50, 2: 1, 3: 0, 4: 0}, 'map_time_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'duration': 660.1896131038666, 'nb_balles_papier': 0.0006500000000000011, 'begin_time_port': 'image_29042026_10_00_02_733064m0.jpg 1e-05 for time 1, id_amount 2 this amount prod time diff : 1e-05'} Production hashtag (incorrect ponderation at 20-10-18) : 0.0006500000000000011 result_one_balle_Type_JRM:{'day': '29042026', 'map_nb_amount': {0: 5, 1: 71, 2: 0, 3: 0, 4: 0}, 'map_time_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'duration': 719.7992050647736, 'nb_balles_papier': 0.0007600000000000013, 'begin_time_port': 'image_29042026_10_17_02_477705m0.jpg 1e-05 for time 1, id_amount 2 this amount prod time diff : 1e-05'} Production hashtag (incorrect ponderation at 20-10-18) : 0.0007600000000000013 result_one_balle_Type_JRM:{'day': '29042026', 'map_nb_amount': {0: 8, 1: 77, 2: 1, 3: 2, 4: 0}, 'map_time_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'duration': 839.2681760787964, 'nb_balles_papier': 0.0008800000000000017, 'begin_time_port': 'image_29042026_10_30_03_092998m0.jpg 1e-05 for time 1, id_amount 1 this amount prod time diff : 1e-05'} Production hashtag (incorrect ponderation at 20-10-18) : 0.0008800000000000017 LLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLresult_one_balle_Type_JRM:{'day': '29042026', 'map_nb_amount': {0: 6, 1: 79, 2: 1, 3: 0, 4: 0}, 'map_time_amount': {0: 0, 1: 0, 2: 0, 3: 0, 4: 0}, 'duration': 839.4268260002136, 'nb_balles_papier': 0.0008600000000000016, 'begin_time_port': 'image_29042026_10_45_03_067139m0.jpg 1e-05 for time 1, id_amount 2 this amount prod time diff : 1e-05'} Production hashtag (incorrect ponderation at 20-10-18) : 0.0008600000000000016 LLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLWe filter photos on hashtag condition ! We have rejected 5 photos because of the batch_size condition ! NUMBER BATCH list_of_portfolios_to_create : 4 list_same_port_ids : [31042787] find same portfolio which already exist 31042787 , we will use it list_same_port_ids : [31043430] find same portfolio which already exist 31043430 , we will use it list_same_port_ids : [31043627] find same portfolio which already exist 31043627 , we will use it list_same_port_ids : [] Qualite : 0.1458439020222561 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=31042787 AND mptpi.`type`=3726 To do Qualite : 0.16727131941966242 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=31043430 AND mptpi.`type`=3726 To do # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=31043627 AND mptpi.`type`=3726 To do # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 12489 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 12499 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12500 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 12492 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 12492 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! WARNING : number of outputs for step 12493 merge_mask_thcl_custom is not consistent : 4 used against 2 in the step definition ! WARNING : number of inputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12494 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 12502 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 12502 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 12496 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 12495 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 12495 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 2 of step 12492 doesn't seem to be define in the database( WARNING : output 1 of step 12489 have datatype=2 whereas input 1 of step 12493 have datatype=7 WARNING : type of output 2 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12494 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 3 of step 12493 doesn't seem to be define in the database( WARNING : type of input 1 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12496 doesn't seem to be define in the database( WARNING : type of input 3 of step 12495 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12500 doesn't seem to be define in the database( WARNING : type of output 2 of step 12489 doesn't seem to be define in the database( WARNING : type of input 1 of step 12499 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 12496 have datatype=10 whereas input 3 of step 12498 have datatype=6 WARNING : type of input 5 of step 12498 doesn't seem to be define in the database( WARNING : output 0 of step 12501 have datatype=11 whereas input 5 of step 12498 have datatype=None WARNING : output 0 of step 12496 have datatype=10 whereas input 0 of step 12501 have datatype=18 WARNING : type of input 2 of step 12502 doesn't seem to be define in the database( WARNING : output 1 of step 12494 have datatype=7 whereas input 2 of step 12502 have datatype=None WARNING : type of output 3 of step 12502 doesn't seem to be define in the database( WARNING : type of input 2 of step 12496 doesn't seem to be define in the database( WARNING : type of output 1 of step 12499 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : type of output 1 of step 12500 doesn't seem to be define in the database( WARNING : type of input 3 of step 12492 doesn't seem to be define in the database( WARNING : output 0 of step 12493 have datatype=1 whereas input 0 of step 12494 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=31044504 AND mptpi.`type`=3726 To do elapsed_time : count_nb_balles_and_create_portfolio 54.94614124298096 # DISPLAY ALL COLLECTED DATA : {'29042026': {'nb_upload': 660, 'nb_taggue_class': 604, 'nb_taggue_densite': 604, 'nb_descriptors': 320}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1416700104, 1416700102, 1416700099, 1416700094, 1416700091, 1416700089, 1416699974, 1416699973, 1416699971, 1416699969, 1416699964, 1416699961, 1416699915, 1416699913, 1416699911, 1416699908, 1416699906, 1416699905, 1416699860, 1416699847, 1416699846, 1416699845, 1416699844, 1416699843, 1416699842, 1416699799, 1416699798, 1416699797, 1416699795, 1416699792, 1416699790, 1416699742, 1416699740, 1416699738, 1416699736, 1416699734, 1416699733, 1416699675, 1416699674, 1416699673, 1416699672, 1416699671, 1416699670, 1416699625, 1416699624, 1416699622, 1416699620, 1416699618, 1416699616, 1416699575, 1416699572, 1416699570, 1416699568, 1416699567, 1416699566, 1416699469, 1416699467, 1416699464, 1416699462, 1416699460, 1416699458, 1416699397, 1416699394, 1416699393, 1416699392, 1416699391, 1416699390, 1416699347, 1416699346, 1416699345, 1416699342, 1416699339, 1416699336, 1416699287, 1416699285, 1416699282, 1416699280, 1416699278, 1416699276, 1416699190, 1416699189, 1416699187, 1416699185, 1416699183, 1416699181, 1416699130, 1416699127, 1416699125, 1416699123, 1416699122, 1416699121, 1416699066, 1416699065, 1416699064, 1416699063, 1416699061, 1416699059, 1416699008, 1416699006, 1416699004] Looping around the photos to save general results len do output : 1 /31042617Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416700104', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416700102', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416700099', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416700094', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416700091', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416700089', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699974', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699973', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699971', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699969', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699964', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699961', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699915', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699913', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699911', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699908', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699906', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699905', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699860', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699847', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699846', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699845', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699844', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699843', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699842', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699799', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699798', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699797', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699795', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699792', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699790', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699742', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699740', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699738', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699736', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699734', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699733', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699675', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699674', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699673', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699672', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699671', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699670', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699625', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699624', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699622', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699620', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699618', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699616', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699575', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699572', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699570', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699568', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699567', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699566', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699469', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699467', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699464', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699462', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699460', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699458', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699397', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699394', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699393', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699392', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699391', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699390', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699347', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699346', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699345', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699342', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699339', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699336', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699287', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699285', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699282', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699280', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699278', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699276', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699190', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699189', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699187', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699185', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699183', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699181', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699130', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699127', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699125', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699123', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699122', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699121', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699066', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699065', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699064', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699063', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699061', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699059', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699008', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699006', None, None, None, None, None, '4379329') ('4189', None, None, None, None, None, None, None, '4379329') ('4189', '31042617', '1416699004', None, None, None, None, None, '4379329') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 101 time used for this insertion : 0.05089926719665527 save_final save missing photos in datou_result : time spend for datou_step_exec : 84.82503056526184 time spend to save output : 0.05167961120605469 total time spend for step 1 : 84.8767101764679 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 1 set_done_treatment 1.83user 0.62system 1:26.35elapsed 2%CPU (0avgtext+0avgdata 115496maxresident)k 704inputs+2712outputs (10major+54204minor)pagefaults 0swaps