python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 8 -a ' -a 4323 -l 20 --fifo' -s datou_current_sts -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 702197 load datou : 4323 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? load thcls load pdts Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec no input labels no input values updating current state to 1 we have a portfolio with more photos than limit : 281>20 please execute split_portfolio.py -i 23100212 -l 20 size over we load limit photo not treated list_input_json: {} Current got : datou_id : 4323, datou_cur_ids : ['2890706'] with mtr_portfolio_ids : ['23100212'] and first list_photo_ids : [] new path : /proc/702197/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! List Step Type Loaded in datou : split_time_score list_input_json : {} origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! we have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB time to download the photos : 0.020674943923950195 About to test input to load Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 1 step1:split_time_score Mon May 19 07:44:06 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec begin split time score 2022-04-13 10:29:59 0 TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 3847, 'mtr_user_id': 31, 'name': 'learn_MM_generique_050224', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'aluminium,ela,emr,film_pedb,flux_dev,jrm,pcm,pcnc,pehd_pp,pet_clair,refus,tapis_vide', 'svm_portfolios_learning': '13096157,13096155,13096163,13096159,13301956,13095886,13096162,13096160,13358264,13096158,5515868,13276803', 'photo_hashtag_type': 4932, 'photo_desc_type': 6032, 'type_classification': 'tf_classification2', 'hashtag_id_list': '493546845,492741797,616987804,2107760237,2107760238,495916461,560181804,1284539308,2107760239,2107755846,538914404,2107748999'}] thcls : [{'id': 3513, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2_tf', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 4557, 'photo_desc_type': 5767, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('00', 79), ('01', 19), ('02', 5), ('03', 3), ('04', 3), ('05', 109), ('06', 63)) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {1: 260} 19052025 23100212 Nombre de photos uploadées : 281 / 23040 (1%) 19052025 23100212 Nombre de photos taguées (types de déchets): 260 / 281 (92%) 19052025 23100212 Nombre de photos taguées (volume) : 260 / 281 (92%) elapsed_time : load_data_split_time_score 3.0994415283203125e-06 elapsed_time : order_list_meta_photo_and_scores 0.00013256072998046875 ????????????????????? elapsed_time : fill_and_build_computed_from_old_data 0.011683225631713867 elapsed_time : insert_dashboard_record_day_entry 0.035037994384765625 Creating list_photo_total elapsed_time : select_descriptors 2.1795313358306885 19052025 23100212 Nombre de photos avec descriptors (type 6032) : 39 / 60 (65%) Missing descriptors for photos 0 and 1359269030 0:00:00|ON:Missing descriptors for photos 1359269030 and 1359263496 Missing descriptors for photos 1359263496 and 1359264943 Missing descriptors for photos 1359264943 and 1359265715 Missing descriptors for photos 1359265715 and 1359268273 Missing descriptors for photos 1359268273 and 1359268355 Missing descriptors for photos 1359268355 and 1359268476 Missing descriptors for photos 1359268476 and 1359268688 Missing descriptors for photos 1359268688 and 1359268845 Missing descriptors for photos 1359268845 and 1359269028 Missing descriptors for photos 1359269028 and 1359263466 Missing descriptors for photos 1359263466 and 1359263492 Missing descriptors for photos 1359263492 and 1359263523 Missing descriptors for photos 1359263523 and 1359264940 Missing descriptors for photos 1359264940 and 1359265713 Missing descriptors for photos 1359265713 and 1359267996 Missing descriptors for photos 1359267996 and 1359263519 Missing descriptors for photos 1359263519 and 1359263541 Missing descriptors for photos 1359263541 and 1359263673 Missing descriptors for photos 1359263673 and 1359264934 Missing descriptors for photos 1359264934 and 1359265002 Missing descriptors for photos 1359265002 and 1359265683 Missing descriptors for photos 1359265683 and 1359265711 Missing descriptors for photos 1359265711 and 1359267994 Missing descriptors for photos 1359267994 and 1359263538 Missing descriptors for photos 1359263538 and 1359263599 Missing descriptors for photos 1359263599 and 1359263629 Missing descriptors for photos 1359263629 and 1359263669 Missing descriptors for photos 1359263669 and 1359264776 Missing descriptors for photos 1359264776 and 1359263624 Missing descriptors for photos 1359263624 and 1359263665 Missing descriptors for photos 1359263665 and 1359263713 Missing descriptors for photos 1359263713 and 1359263847 Missing descriptors for photos 1359263847 and 1359264298 Missing descriptors for photos 1359264298 and 1359264723 Missing descriptors for photos 1359264723 and 1359263645 Missing descriptors for photos 1359263645 and 1359263663 Missing descriptors for photos 1359263663 and 1359263710 Missing descriptors for photos 1359263710 and 1359263812 Missing descriptors for photos 1359263812 and 1359263827 Missing descriptors for photos 1359263827 and 1359263844 Missing descriptors for photos 1359263844 and 1359264191 Missing descriptors for photos 1359264191 and 1359264618 Missing descriptors for photos 1359264618 and 1359263809 Missing descriptors for photos 1359263809 and 1359263839 Missing descriptors for photos 1359263839 and 1359264076 Missing descriptors for photos 1359264076 and 1359264613 Missing descriptors for photos 1359264613 and 1359263782 Missing descriptors for photos 1359263782 and 1359263808 Missing descriptors for photos 1359263808 and 1359263835 Missing descriptors for photos 1359263835 and 1359263966 Missing descriptors for photos 1359263966 and 1359264608 Missing descriptors for photos 1359264608 and 1359263882 Missing descriptors for photos 1359263882 and 1359264604 Missing descriptors for photos 1359264604 and 1359264901 Missing descriptors for photos 1359264901 and 1359264930 Missing descriptors for photos 1359264930 and 1359264999 Missing descriptors for photos 1359264999 and 1359265020 Missing descriptors for photos 1359265020 and 1359265681 Missing descriptors for photos 1359265681 and 1359265710 Missing descriptors for photos 1359265710 and 1359265789 Missing descriptors for photos 1359265789 and 1359267226 Missing descriptors for photos 1359267226 and 1359267284 Missing descriptors for photos 1359267284 and 1359267992 Missing descriptors for photos 1359267992 and 1359265788 Missing descriptors for photos 1359265788 and 1359265808 Missing descriptors for photos 1359265808 and 1359267132 Missing descriptors for photos 1359267132 and 1359267223 Missing descriptors for photos 1359267223 and 1359267282 Missing descriptors for photos 1359267282 and 1359267974 Missing descriptors for photos 1359267974 and 1359267983 Missing descriptors for photos 1359267983 and 1359267131 Missing descriptors for photos 1359267131 and 1359267187 Missing descriptors for photos 1359267187 and 1359267222 Missing descriptors for photos 1359267222 and 1359267281 Missing descriptors for photos 1359267281 and 1359267905 Missing descriptors for photos 1359267905 and 1359267982 Missing descriptors for photos 1359267982 and 1359267302 Missing descriptors for photos 1359267302 and 1359267975 Missing descriptors for photos 1359267975 and 1359268130 Missing descriptors for photos 1359268130 and 1359268252 Missing descriptors for photos 1359268252 and 1359268271 Missing descriptors for photos 1359268271 and 1359268291 Missing descriptors for photos 1359268291 and 1359268327 Missing descriptors for photos 1359268327 and 1359268342 Missing descriptors for photos 1359268342 and 1359268354 Missing descriptors for photos 1359268354 and 1359268407 Missing descriptors for photos 1359268407 and 1359268475 Missing descriptors for photos 1359268475 and 1359268628 Missing descriptors for photos 1359268628 and 1359268660 Missing descriptors for photos 1359268660 and 1359268674 Missing descriptors for photos 1359268674 and 1359268765 Missing descriptors for photos 1359268765 and 1359268802 Missing descriptors for photos 1359268802 and 1359268844 Missing descriptors for photos 1359268844 and 1359268893 Missing descriptors for photos 1359268893 and 1359269004 Missing descriptors for photos 1359269004 and 1359269027 Missing descriptors for photos 1359269027 and 1359269270 Missing descriptors for photos 1359269270 and 1359269371 Missing descriptors for photos 1359269371 and 1359269770 Missing descriptors for photos 1359269770 and 1359270325 Missing descriptors for photos 1359270325 and 1359270464 Missing descriptors for photos 1359270464 and 1359270621 Missing descriptors for photos 1359270621 and 1359272565 Missing descriptors for photos 1359272565 and 1359272924 Missing descriptors for photos 1359272924 and 1359273121 Missing descriptors for photos 1359273121 and 1359273209 Missing descriptors for photos 1359273209 and 1359273506 Missing descriptors for photos 1359273506 and 1359273679 Missing descriptors for photos 1359273679 and 1359276029 Missing descriptors for photos 1359276029 and 1359276088 Missing descriptors for photos 1359276088 and 1359276086 Missing descriptors for photos 1359276086 and 1359276134 Missing descriptors for photos 1359276134 and 1359276083 Missing descriptors for photos 1359276083 and 1359276130 Missing descriptors for photos 1359276130 and 1359276162 Missing descriptors for photos 1359276162 and 1359276158 Missing descriptors for photos 1359276158 and 1359276154 Missing descriptors for photos 1359276154 and 1359276192 Missing descriptors for photos 1359276192 and 1359276188 Missing descriptors for photos 1359276188 and 1359276183 Missing descriptors for photos 1359276183 and 1359276227 Missing descriptors for photos 1359276227 and 1359276222 Missing descriptors for photos 1359276222 and 1359276218 5:09:08.999837|OFF:0:00:11.000406|ON:0:00:00|OFF:0:00:09.999820|ON:0:00:00|OFF:0:00:09.999746|ON:Missing descriptors for photos 1359276300 and 1359276298 Missing descriptors for photos 1359276298 and 1359276293 Missing descriptors for photos 1359276293 and 1359276291 Missing descriptors for photos 1359276291 and 1359276372 Missing descriptors for photos 1359276372 and 1359276420 Missing descriptors for photos 1359276420 and 1359276446 Missing descriptors for photos 1359276446 and 1359276537 Missing descriptors for photos 1359276537 and 1359276613 Missing descriptors for photos 1359276613 and 1359276674 Missing descriptors for photos 1359276674 and 1359276873 Missing descriptors for photos 1359276873 and 1359276871 Missing descriptors for photos 1359276871 and 1359276869 Missing descriptors for photos 1359276869 and 1359276864 Missing descriptors for photos 1359276864 and 1359276914 Missing descriptors for photos 1359276914 and 1359276911 Missing descriptors for photos 1359276911 and 1359276902 Missing descriptors for photos 1359276902 and 1359276893 Missing descriptors for photos 1359276893 and 1359276889 Missing descriptors for photos 1359276889 and 1359276885 Missing descriptors for photos 1359277004 and 1359277202 Missing descriptors for photos 1359277202 and 1359277120 Missing descriptors for photos 1359277120 and 1359277197 Missing descriptors for photos 1359277197 and 1359277042 Missing descriptors for photos 1359277042 and 1359277116 Missing descriptors for photos 1359277116 and 1359277168 Missing descriptors for photos 1359277168 and 1359277193 Missing descriptors for photos 1359277193 and 1359277039 Missing descriptors for photos 1359277039 and 1359277113 Missing descriptors for photos 1359277113 and 1359277165 Missing descriptors for photos 1359277165 and 1359277189 Missing descriptors for photos 1359277189 and 1359277036 Missing descriptors for photos 1359277036 and 1359277110 Missing descriptors for photos 1359277110 and 1359277162 Missing descriptors for photos 1359277162 and 1359277187 Missing descriptors for photos 1359277187 and 1359277108 Missing descriptors for photos 1359277108 and 1359277159 Missing descriptors for photos 1359277159 and 1359277184 Missing descriptors for photos 1359277184 and 1359277259 Missing descriptors for photos 1359277321 and 1359277319 Missing descriptors for photos 1359277319 and 1359277435 Missing descriptors for photos 1359277435 and 1359277381 Missing descriptors for photos 1359277381 and 1359277378 Missing descriptors for photos 1359277378 and 1359277375 Missing descriptors for photos 1359277375 and 1359277373 Missing descriptors for photos 1359277373 and 1359277370 0:36:20.000129|OFF:0:00:09.999907|ON:Missing descriptors for photos 1359277883 and 1359277881 Missing descriptors for photos 1359277881 and 1359277877 Missing descriptors for photos 1359277877 and 1359277874 Missing descriptors for photos 1359277874 and 1359277870 Missing descriptors for photos 1359277870 and 1359277897 Missing descriptors for photos 1359277897 and 1359278025 Missing descriptors for photos 1359278025 and 1359279798 Missing descriptors for photos 1359279798 and 1359278279 Missing descriptors for photos 1359278279 and 1359278342 Missing descriptors for photos 1359278342 and 1359279795 Missing descriptors for photos 1359279795 and 1359278093 Missing descriptors for photos 1359278093 and 1359278172 Missing descriptors for photos 1359278172 and 1359278231 Missing descriptors for photos 1359278231 and 1359278276 Missing descriptors for photos 1359278276 and 1359278336 Missing descriptors for photos 1359278336 and 1359278635 Missing descriptors for photos 1359278635 and 1359278855 Missing descriptors for photos 1359278855 and 1359279206 Missing descriptors for photos 1359279206 and 1359279721 Missing descriptors for photos 1359279721 and 1359279792 Missing descriptors for photos 1359279792 and 1359280135 Missing descriptors for photos 1359280135 and 1359280617 Missing descriptors for photos 1359280617 and 1359280886 Missing descriptors for photos 1359280886 and 1359282053 Missing descriptors for photos 1359282053 and 1359278169 Missing descriptors for photos 1359278169 and 1359278229 Missing descriptors for photos 1359278229 and 1359278274 Missing descriptors for photos 1359278274 and 1359278333 Missing descriptors for photos 1359278333 and 1359278364 Missing descriptors for photos 1359278364 and 1359278630 Missing descriptors for photos 1359278630 and 1359278826 Missing descriptors for photos 1359278826 and 1359279190 Missing descriptors for photos 1359279190 and 1359279249 Missing descriptors for photos 1359279249 and 1359279716 Missing descriptors for photos 1359279716 and 1359279789 Missing descriptors for photos 1359279789 and 1359279850 Missing descriptors for photos 1359279850 and 1359280133 Missing descriptors for photos 1359280133 and 1359280614 Missing descriptors for photos 1359280614 and 1359280885 Missing descriptors for photos 1359280885 and 1359281986 Missing descriptors for photos 1359281986 and 1359282050 Missing descriptors for photos 1359282050 and 1359278328 Missing descriptors for photos 1359278328 and 1359278362 Missing descriptors for photos 1359278362 and 1359278534 Missing descriptors for photos 1359278534 and 1359278591 Missing descriptors for photos 1359278591 and 1359278624 Missing descriptors for photos 1359278624 and 1359278824 Missing descriptors for photos 1359278824 and 1359278985 Missing descriptors for photos 1359278985 and 1359279159 Missing descriptors for photos 1359279159 and 1359279246 Missing descriptors for photos 1359279246 and 1359279297 Missing descriptors for photos 1359279297 and 1359279353 Missing descriptors for photos 1359279353 and 1359279548 Missing descriptors for photos 1359279548 and 1359279604 Missing descriptors for photos 1359279604 and 1359279639 Missing descriptors for photos 1359279639 and 1359279712 Missing descriptors for photos 1359279712 and 1359279785 Missing descriptors for photos 1359279785 and 1359279846 Missing descriptors for photos 1359279846 and 1359279871 Missing descriptors for photos 1359279871 and 1359280034 Missing descriptors for photos 1359280034 and 1359280131 Missing descriptors for photos 1359280131 and 1359280609 Missing descriptors for photos 1359280609 and 1359280813 Missing descriptors for photos 1359280813 and 1359280880 Missing descriptors for photos 1359280880 and 1359281330 Missing descriptors for photos 1359281330 and 1359281383 Missing descriptors for photos 1359281383 and 1359281980 Missing descriptors for photos 1359281980 and 1359282046 Missing descriptors for photos 1359282046 and 1359282292 19052025 Removing 12 photos because of the 'same image' condition Total on : 20728.999966 list_time_on Total off : 40.999879 list_time_off dist_desc begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 281 time used for this insertion : 0.048123836517333984 photos_removed : len 12 elapsed_time : remove_photo_duplicate 0.10813260078430176 Creating list_photo_total XXXXXXXXXXXXWarning list_scores null Warning list_scores null Warning list_scores null Warning list_scores null Warning list_scores null Warning list_scores null Warning list_scores null Warning list_scores null Warning list_scores null Warning list_scores null Warning list_scores null Warning list_scores null Warning list_scores null Warning list_scores null Warning list_scores null Warning list_scores null Warning list_scores null Warning list_scores null Warning list_scores null Warning list_scores null Warning list_scores null time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average time_diff is bigger than the limit of interval, we ignore the result of this image in moving average ERROR ! ERROR ! ERROR ! ERROR ! ERROR ! ERROR ! ERROR ! ERROR ! ERROR ! ERROR ! ERROR ! ERROR ! ERROR ! ERROR ! ERROR ! ERROR ! ERROR ! ERROR ! ERROR ! ERROR ! ERROR ! elapsed_time : count_sum_diff_and_build_graph 0.016796112060546875 Total photos : 281 .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .Change port : 0 hashtag : 616987804 photo_id =1359276215 : emr .....can't find max_score_info ..can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info ..can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info ...can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .Change port : 9 hashtag : 2107760238 photo_id =1359276982 : flux_dev ....can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .Change port : 4 hashtag : 616987804 photo_id =1359277303 : emr .....can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .Change port : 5 hashtag : 1284539308 photo_id =1359277560 : pcnc ...can't find max_score_info .......can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info Total photos : 281 Number of lists : 5 counter photos in port : 27 hashtag : aluminium(493546845) : 0 photos in 0 portfolios ! hashtag : ela(492741797) : 0 photos in 0 portfolios ! hashtag : emr(616987804) : 14 photos in 2 portfolios ! hashtag : film_pedb(2107760237) : 0 photos in 0 portfolios ! hashtag : flux_dev(2107760238) : 4 photos in 1 portfolios ! hashtag : jrm(495916461) : 0 photos in 0 portfolios ! hashtag : pcm(560181804) : 0 photos in 0 portfolios ! hashtag : pcnc(1284539308) : 9 photos in 1 portfolios ! hashtag : pehd_pp(2107760239) : 0 photos in 0 portfolios ! hashtag : pet_clair(2107755846) : 0 photos in 0 portfolios ! hashtag : refus(538914404) : 0 photos in 0 portfolios ! hashtag : tapis_vide(2107748999) : 0 photos in 1 portfolios ! elapsed_time : group_photo_by_moyenne_exp 0.009128093719482422 elapsed_time : compute_and_correct_tag_with_moyenne_mobile 3.5762786865234375e-06 today str has not a value , we define it as the date of the first image todaystr_first : 19052025 attention , prev_timestamp is 0 , we do nothing Unexpected type of data : ({'photo_id': 1359263496, 'url': 'https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/2025/5/19/08b2b10e5e931dd0b64b01918e334f84.jpg', 'username': None, 'uploaded_at': 1747605947, 'text': 'image_19052025_00_01_35_10190.jpg', 'time': 1747612895.01019}, 'different') *oooooooUnexpected type of data : ({'photo_id': 1359263466, 'url': 'https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/2025/5/19/a8c9d8c654b7dbb59bdf9dad7b4d580c.jpg', 'username': None, 'uploaded_at': 1747605887, 'text': 'image_19052025_00_01_55_10141.jpg', 'time': 1747612915.010141}, 'different') oooo*ooUnexpected type of data : ({'photo_id': 1359264934, 'url': 'https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/2025/5/19/03d3d370947617aa7bf47f2359e0762d.jpg', 'username': None, 'uploaded_at': 1747607266, 'text': 'image_19052025_00_02_35_10178.jpg', 'time': 1747612955.010178}, 'different') oooo*oooo*ooooo*oUnexpected type of data : ({'photo_id': 1359263710, 'url': 'https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/2025/5/19/6cfef293d142bf6e1bd4b625b64e0399.jpg', 'username': None, 'uploaded_at': 1747606547, 'text': 'image_19052025_00_09_15_10083.jpg', 'time': 1747613355.010083}, 'different') oooooUnexpected type of data : ({'photo_id': 1359263809, 'url': 'https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/2025/5/19/c4fe322657af9518d09333053d6a50b0.jpg', 'username': None, 'uploaded_at': 1747606847, 'text': 'image_19052025_00_10_15_10193.jpg', 'time': 1747613415.010193}, 'different') *oo*oooUnexpected type of data : ({'photo_id': 1359264608, 'url': 'https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/2025/5/19/8097dde494011e547c823f93eacaae08.jpg', 'username': None, 'uploaded_at': 1747607082, 'text': 'image_19052025_00_15_05_10137.jpg', 'time': 1747613705.010137}, 'different') *o*ooooooooo*oooooo*ooooo*o*ooooooooooooooooo***************oo***ooooooooooo* BIG TIME 229.99912691116333 (131.00000619888306, 2, 0, 0, 0.79193085, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1.8588998770952225, 0.0144, 0.0, 1.9, 0.13999896812438964, -0.21666500012079876) on 9 1359276372 2025-05-19 05:15:04.009127 id_data : 148 ooooo* BIG TIME 550.0009541511536 (131.00000619888306, 2, 0, 0, 0.79193085, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1.8588998770952225, 0.0144, 0.0, 1.9, 0.13999896812438964, -0.21666500012079876) on 9 1359276873 2025-05-19 05:24:14.010081 id_data : 154 * BIG TIME 170.00025415420532 (131.00000619888306, 2, 0, 0, 0.79193085, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1.8588998770952225, 0.0144, 0.0, 1.9, 0.13999896812438964, -0.21666500012079876) on 9 1359276982 2025-05-19 05:28:34.010225 id_data : 164 *ooooooooooo* BIG TIME 179.0001780986786 (190.99901795387268, 5, 0, 0, 0, 0, 0, 0.39203092, 0, 0, 0, 0, 0, 0, 1.972999994802475, 0.0168, 0.0, 1.9, 0.00999986219406128, -0.0) on 12 1359277108 2025-05-19 05:33:24.010108 id_data : 184 oo* BIG TIME 379.99984192848206 (190.99901795387268, 5, 0, 0, 0, 0, 0, 0.39203092, 0, 0, 0, 0, 0, 0, 1.972999994802475, 0.0168, 0.0, 1.9, 0.00999986219406128, -0.0) on 12 1359277259 2025-05-19 05:39:44.009950 id_data : 187 ******** BIG TIME 198.99995803833008 (581.9991312026978, 7, 0, 0, 0, 0, 0, 0, 0, 0.7966068, 0, 0, 0, 0, 2.08189986948967, 0.0211, 0.0009999999046325684, 2.9, 0.22999891281127932, -0.19999812324841817) on 26 1359278025 2025-05-19 05:56:34.010205 id_data : 217 * BIG TIME 279.9987828731537 (581.9991312026978, 7, 0, 0, 0, 0, 0, 0, 0, 0.7966068, 0, 0, 0, 0, 2.08189986948967, 0.0211, 0.0009999999046325684, 2.9, 0.22999891281127932, -0.19999812324841817) on 26 1359279798 2025-05-19 06:01:14.008988 id_data : 218 ooooooooooooUnexpected type of data : ({'photo_id': 1359280617, 'url': 'https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/2025/5/19/bf88cb79f8ee73e46572693ef51ef83c.jpg', 'username': None, 'uploaded_at': 1747631028, 'text': 'image_19052025_06_01_35_10227.jpg', 'time': 1747634495.010227}, 'different') Unexpected type of data : ({'photo_id': 1359280886, 'url': 'https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/2025/5/19/610d6d86305c8be52f9744f15499aa85.jpg', 'username': None, 'uploaded_at': 1747631387, 'text': 'image_19052025_06_01_35_10227.jpg', 'time': 1747634495.010227}, 'different') Unexpected type of data : ({'photo_id': 1359282053, 'url': 'https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/2025/5/19/ec37bc859dfa2b064509d69114f78bfc.jpg', 'username': None, 'uploaded_at': 1747632767, 'text': 'image_19052025_06_01_35_10227.jpg', 'time': 1747634495.010227}, 'different') ooooooooooooUnexpected type of data : ({'photo_id': 1359280614, 'url': 'https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/2025/5/19/f91902f57ecf933376a5a0c6689cfa19.jpg', 'username': None, 'uploaded_at': 1747631028, 'text': 'image_19052025_06_01_55_10232.jpg', 'time': 1747634515.010232}, 'different') Unexpected type of data : ({'photo_id': 1359280885, 'url': 'https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/2025/5/19/2b4d96f84714ebf2a84596390fd24764.jpg', 'username': None, 'uploaded_at': 1747631387, 'text': 'image_19052025_06_01_55_10232.jpg', 'time': 1747634515.010232}, 'different') Unexpected type of data : ({'photo_id': 1359281986, 'url': 'https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/2025/5/19/efba3ea9c53a0bbe6f78bc396dad1f3f.jpg', 'username': None, 'uploaded_at': 1747632648, 'text': 'image_19052025_06_01_55_10232.jpg', 'time': 1747634515.010232}, 'different') Unexpected type of data : ({'photo_id': 1359282050, 'url': 'https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/2025/5/19/6cbdf5f10cefca2075b4bf4a48c4f60e.jpg', 'username': None, 'uploaded_at': 1747632767, 'text': 'image_19052025_06_01_55_10232.jpg', 'time': 1747634515.010232}, 'different') oooooooooooooooooooUnexpected type of data : ({'photo_id': 1359280609, 'url': 'https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/2025/5/19/e07e6a4bc46d1e127810c1c987591a55.jpg', 'username': None, 'uploaded_at': 1747631028, 'text': 'image_19052025_06_02_04_9217.jpg', 'time': 1747634524.009217}, 'different') Unexpected type of data : ({'photo_id': 1359280813, 'url': 'https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/2025/5/19/33f122144db420e2291e7f77f26cf9ed.jpg', 'username': None, 'uploaded_at': 1747631267, 'text': 'image_19052025_06_02_04_9217.jpg', 'time': 1747634524.009217}, 'different') Unexpected type of data : ({'photo_id': 1359280880, 'url': 'https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/2025/5/19/b350bd045bfdb5446b05a60153e61596.jpg', 'username': None, 'uploaded_at': 1747631387, 'text': 'image_19052025_06_02_04_9217.jpg', 'time': 1747634524.009217}, 'different') Unexpected type of data : ({'photo_id': 1359281330, 'url': 'https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/2025/5/19/92c55f1f4f5938148a38ed616d12c783.jpg', 'username': None, 'uploaded_at': 1747631987, 'text': 'image_19052025_06_02_04_9217.jpg', 'time': 1747634524.009217}, 'different') Unexpected type of data : ({'photo_id': 1359281383, 'url': 'https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/2025/5/19/4a1b031de08746a858ac545433fee737.jpg', 'username': None, 'uploaded_at': 1747632107, 'text': 'image_19052025_06_02_04_9217.jpg', 'time': 1747634524.009217}, 'different') Unexpected type of data : ({'photo_id': 1359281980, 'url': 'https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/2025/5/19/3f4a539891421e1250c34f8198e33dab.jpg', 'username': None, 'uploaded_at': 1747632648, 'text': 'image_19052025_06_02_04_9217.jpg', 'time': 1747634524.009217}, 'different') Unexpected type of data : ({'photo_id': 1359282046, 'url': 'https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/2025/5/19/5f0ac7ed4cb9e39165c842e79415e953.jpg', 'username': None, 'uploaded_at': 1747632767, 'text': 'image_19052025_06_02_04_9217.jpg', 'time': 1747634524.009217}, 'different') Unexpected type of data : ({'photo_id': 1359282292, 'url': 'https://storage.sbg.cloud.ovh.net/v1/AUTH_3b171620e76e4af496c5fd050759c9f0/media.fotonower.com/2025/5/19/338483a1e4613a8c6c3ef4b74e18dc22.jpg', 'username': None, 'uploaded_at': 1747633188, 'text': 'image_19052025_06_02_04_9217.jpg', 'time': 1747634524.009217}, 'different') Count Time bigger than 30s : 46 #Number Photos for regression : {'19052025': {493546845: {2107751013: 0, 2107751014: 0, 2107751015: 0, 2107751016: 0, 2107751017: 0}, 492741797: {2107751013: 0, 2107751014: 0, 2107751015: 0, 2107751016: 0, 2107751017: 0}, 616987804: {2107751013: 0, 2107751014: 261.0000813007355, 2107751015: 28.998981952667236, 2107751016: 10.000072002410889, 2107751017: 0}, 2107760237: {2107751013: 0, 2107751014: 0, 2107751015: 0, 2107751016: 0, 2107751017: 0}, 2107760238: {2107751013: 0, 2107751014: 0, 2107751015: 51.00004982948303, 2107751016: 0, 2107751017: 0}, 495916461: {2107751013: 0, 2107751014: 0, 2107751015: 9.99986219406128, 2107751016: 0, 2107751017: 0}, 560181804: {2107751013: 0, 2107751014: 0, 2107751015: 0, 2107751016: 0, 2107751017: 0}, 1284539308: {2107751013: 0, 2107751014: 51.000118017196655, 2107751015: 130.00007009506226, 2107751016: 48.998724699020386, 2107751017: 0}, 2107760239: {2107751013: 0, 2107751014: 0, 2107751015: 0, 2107751016: 0, 2107751017: 0}, 2107755846: {2107751013: 0, 2107751014: 0, 2107751015: 0, 2107751016: 0, 2107751017: 0}, 538914404: {2107751013: 0, 2107751014: 0, 2107751015: 0, 2107751016: 0, 2107751017: 0}, 2107748999: {2107751013: 0, 2107751014: 0, 2107751015: 1270.0008668899536, 2107751016: 0, 2107751017: 0}}} 19052025|aluminium, 05102018_papier_non_papier_dense:0 19052025|aluminium, 05102018_papier_non_papier_peu_dense:0 19052025|aluminium, 05102018_papier_non_papier_presque_vide:0 19052025|aluminium, 05102018_papier_non_papier_tres_dense:0 19052025|aluminium, 05102018_papier_non_papier_tres_peu_dense:0 19052025|ela, 05102018_papier_non_papier_dense:0 19052025|ela, 05102018_papier_non_papier_peu_dense:0 19052025|ela, 05102018_papier_non_papier_presque_vide:0 19052025|ela, 05102018_papier_non_papier_tres_dense:0 19052025|ela, 05102018_papier_non_papier_tres_peu_dense:0 19052025|emr, 05102018_papier_non_papier_dense:0 19052025|emr, 05102018_papier_non_papier_peu_dense:261.0000813007355 19052025|emr, 05102018_papier_non_papier_presque_vide:28.998981952667236 19052025|emr, 05102018_papier_non_papier_tres_dense:10.000072002410889 19052025|emr, 05102018_papier_non_papier_tres_peu_dense:0 19052025|film_pedb, 05102018_papier_non_papier_dense:0 19052025|film_pedb, 05102018_papier_non_papier_peu_dense:0 19052025|film_pedb, 05102018_papier_non_papier_presque_vide:0 19052025|film_pedb, 05102018_papier_non_papier_tres_dense:0 19052025|film_pedb, 05102018_papier_non_papier_tres_peu_dense:0 19052025|flux_dev, 05102018_papier_non_papier_dense:0 19052025|flux_dev, 05102018_papier_non_papier_peu_dense:0 19052025|flux_dev, 05102018_papier_non_papier_presque_vide:51.00004982948303 19052025|flux_dev, 05102018_papier_non_papier_tres_dense:0 19052025|flux_dev, 05102018_papier_non_papier_tres_peu_dense:0 19052025|jrm, 05102018_papier_non_papier_dense:0 19052025|jrm, 05102018_papier_non_papier_peu_dense:0 19052025|jrm, 05102018_papier_non_papier_presque_vide:9.99986219406128 19052025|jrm, 05102018_papier_non_papier_tres_dense:0 19052025|jrm, 05102018_papier_non_papier_tres_peu_dense:0 19052025|pcm, 05102018_papier_non_papier_dense:0 19052025|pcm, 05102018_papier_non_papier_peu_dense:0 19052025|pcm, 05102018_papier_non_papier_presque_vide:0 19052025|pcm, 05102018_papier_non_papier_tres_dense:0 19052025|pcm, 05102018_papier_non_papier_tres_peu_dense:0 19052025|pcnc, 05102018_papier_non_papier_dense:0 19052025|pcnc, 05102018_papier_non_papier_peu_dense:51.000118017196655 19052025|pcnc, 05102018_papier_non_papier_presque_vide:130.00007009506226 19052025|pcnc, 05102018_papier_non_papier_tres_dense:48.998724699020386 19052025|pcnc, 05102018_papier_non_papier_tres_peu_dense:0 19052025|pehd_pp, 05102018_papier_non_papier_dense:0 19052025|pehd_pp, 05102018_papier_non_papier_peu_dense:0 19052025|pehd_pp, 05102018_papier_non_papier_presque_vide:0 19052025|pehd_pp, 05102018_papier_non_papier_tres_dense:0 19052025|pehd_pp, 05102018_papier_non_papier_tres_peu_dense:0 19052025|pet_clair, 05102018_papier_non_papier_dense:0 19052025|pet_clair, 05102018_papier_non_papier_peu_dense:0 19052025|pet_clair, 05102018_papier_non_papier_presque_vide:0 19052025|pet_clair, 05102018_papier_non_papier_tres_dense:0 19052025|pet_clair, 05102018_papier_non_papier_tres_peu_dense:0 19052025|refus, 05102018_papier_non_papier_dense:0 19052025|refus, 05102018_papier_non_papier_peu_dense:0 19052025|refus, 05102018_papier_non_papier_presque_vide:0 19052025|refus, 05102018_papier_non_papier_tres_dense:0 19052025|refus, 05102018_papier_non_papier_tres_peu_dense:0 19052025|tapis_vide, 05102018_papier_non_papier_dense:0 19052025|tapis_vide, 05102018_papier_non_papier_peu_dense:0 19052025|tapis_vide, 05102018_papier_non_papier_presque_vide:1270.0008668899536 19052025|tapis_vide, 05102018_papier_non_papier_tres_dense:0 19052025|tapis_vide, 05102018_papier_non_papier_tres_peu_dense:0 #Number Photos for regression amount gros magasin papier (time_diff then nb_photo) : We have not displayed the number of photos removed for one material since Rungis_Papier wasn't in the thcl used ! 19052025_time_diff_distrib Number amount portfolio for this type of dechet : aluminium 0 Number amount portfolio for this type of dechet : ela 0 Number amount portfolio for this type of dechet : emr 14 https://marlene.fotonower.com/api/v1/secured/portfolio/new?name=19052025_emr_05102018_papier_non_papier_peu_dense&access_token=3e00c8f3e5af383c48e3c68940ba6fe7 Created to study and clean : 23103336 with name like 19052025_emr_05102018_papier_non_papier_peu_dense https://marlene.fotonower.com/api/v1/secured/portfolio/new?name=19052025_emr_05102018_papier_non_papier_presque_vide&access_token=3e00c8f3e5af383c48e3c68940ba6fe7 Created to study and clean : 23103337 with name like 19052025_emr_05102018_papier_non_papier_presque_vide https://marlene.fotonower.com/api/v1/secured/portfolio/new?name=19052025_emr_05102018_papier_non_papier_tres_dense&access_token=3e00c8f3e5af383c48e3c68940ba6fe7 Created to study and clean : 23103338 with name like 19052025_emr_05102018_papier_non_papier_tres_dense Number amount portfolio for this type of dechet : film_pedb 0 Number amount portfolio for this type of dechet : flux_dev 2 Number amount portfolio for this type of dechet : jrm 1 Number amount portfolio for this type of dechet : pcm 0 Number amount portfolio for this type of dechet : pcnc 9 https://marlene.fotonower.com/api/v1/secured/portfolio/new?name=19052025_pcnc_05102018_papier_non_papier_peu_dense&access_token=3e00c8f3e5af383c48e3c68940ba6fe7 Created to study and clean : 23103339 with name like 19052025_pcnc_05102018_papier_non_papier_peu_dense https://marlene.fotonower.com/api/v1/secured/portfolio/new?name=19052025_pcnc_05102018_papier_non_papier_presque_vide&access_token=3e00c8f3e5af383c48e3c68940ba6fe7 Created to study and clean : 23103340 with name like 19052025_pcnc_05102018_papier_non_papier_presque_vide https://marlene.fotonower.com/api/v1/secured/portfolio/new?name=19052025_pcnc_05102018_papier_non_papier_tres_dense&access_token=3e00c8f3e5af383c48e3c68940ba6fe7 Created to study and clean : 23103341 with name like 19052025_pcnc_05102018_papier_non_papier_tres_dense Number amount portfolio for this type of dechet : pehd_pp 0 Number amount portfolio for this type of dechet : pet_clair 0 Number amount portfolio for this type of dechet : refus 0 Number amount portfolio for this type of dechet : tapis_vide 192 https://marlene.fotonower.com/api/v1/secured/portfolio/new?name=19052025_tapis_vide_05102018_papier_non_papier_presque_vide&access_token=3e00c8f3e5af383c48e3c68940ba6fe7 Created to study and clean : 23103342 with name like 19052025_tapis_vide_05102018_papier_non_papier_presque_vide NUMBER BATCH : 5 list_ponderation used : [0.001, 0.001, 0.001, 0.001, 0.001, 0.001, 0.001, 0.001, 0.001, 0.001, 0.001] , list_hashtag_class_create_as_list : ['pcnc', 'gm', 'emr', 'ela', 'pet_clair', 'film_pedb', 'pehd_pp', 'flux_dev_rigide', 'aluminium', 'tapis_vide', 'refus', 'gros_cartons', 'gm', 'flux_dev_rigide', 'gros_cartons'] We filter photos on hashtag condition ! We filter photos on hashtag condition ! result_one_balle_Type_emr:{'day': '19052025', 'map_nb_amount': {0: 6, 1: 3, 2: 0, 3: 0, 4: 0}, 'map_time_amount': {0: 110.99998617172241, 1: 28.998981952667236, 2: 0, 3: 0, 4: 0}, 'duration': 129.99900007247925, 'nb_balles_papier': 0.13999896812438964, 'begin_time_port': 'image_19052025_05_08_24_009960m0.jpg 0.009999968051910401 for time 9.9999680519104, id_amount 2 this amount prod time diff : 0.009999968051910401'} Production hashtag (incorrect ponderation at 20-10-18) : 0.13999896812438964 We filter photos on hashtag condition ! hashtag non trouvé ! We filter photos on hashtag condition ! hashtag non trouvé ! hashtag non trouvé ! We filter photos on hashtag condition ! We filter photos on hashtag condition ! We filter photos on hashtag condition ! We filter photos on hashtag condition ! We filter photos on hashtag condition ! We filter photos on hashtag condition ! We filter photos on hashtag condition ! We filter photos on hashtag condition ! We filter photos on hashtag condition ! We filter photos on hashtag condition ! hashtag non trouvé ! We filter photos on hashtag condition ! hashtag non trouvé ! hashtag non trouvé ! We filter photos on hashtag condition ! We filter photos on hashtag condition ! We filter photos on hashtag condition ! We filter photos on hashtag condition ! result_one_balle_Type_flux_dev_rigide:{'day': '19052025', 'map_nb_amount': {0: 0, 1: 4, 2: 0, 3: 0, 4: 0}, 'map_time_amount': {0: 0, 1: 60.99991202354431, 2: 0, 3: 0, 4: 0}, 'duration': 60.99991202354431, 'nb_balles_papier': 0.06199991202354431, 'begin_time_port': 'image_19052025_05_28_34_010225m0.jpg 0.001 for time 1, id_amount 2 this amount prod time diff : 0.001'} Production hashtag (incorrect ponderation at 20-10-18) : 0.06199991202354431 We filter photos on hashtag condition ! We filter photos on hashtag condition ! We filter photos on hashtag condition ! result_one_balle_Type_emr:{'day': '19052025', 'map_nb_amount': {0: 4, 1: 0, 2: 1, 3: 0, 4: 0}, 'map_time_amount': {0: 150.00009512901306, 1: 0, 2: 10.000072002410889, 3: 0, 4: 0}, 'duration': 38.99996614456177, 'nb_balles_papier': 0.16000016713142398, 'begin_time_port': 'image_19052025_05_41_45_010151m0.jpg 0.12100020098686219 for time 121.00020098686218, id_amount 1 this amount prod time diff : 0.12100020098686219'} Production hashtag (incorrect ponderation at 20-10-18) : 0.16000016713142398 We filter photos on hashtag condition ! hashtag non trouvé ! We filter photos on hashtag condition ! hashtag non trouvé ! hashtag non trouvé ! We filter photos on hashtag condition ! We filter photos on hashtag condition ! We filter photos on hashtag condition ! We filter photos on hashtag condition ! We filter photos on hashtag condition ! We filter photos on hashtag condition ! We filter photos on hashtag condition ! We filter photos on hashtag condition ! We filter photos on hashtag condition ! We filter photos on hashtag condition ! hashtag non trouvé ! We filter photos on hashtag condition ! hashtag non trouvé ! hashtag non trouvé ! We filter photos on hashtag condition ! We filter photos on hashtag condition ! We filter photos on hashtag condition ! We filter photos on hashtag condition ! We filter photos on hashtag condition ! result_one_balle_Type_gros_cartons:{'day': '19052025', 'map_nb_amount': {0: 2, 1: 2, 2: 5, 3: 0, 4: 0}, 'map_time_amount': {0: 51.000118017196655, 1: 130.00007009506226, 2: 48.998724699020386, 3: 0, 4: 0}, 'duration': 119.9988739490509, 'nb_balles_papier': 0.22999891281127932, 'begin_time_port': 'image_19052025_05_45_44_010010m0.jpg 0.12000003790855408 for time 120.00003790855408, id_amount 2 this amount prod time diff : 0.12000003790855408'} Production hashtag (incorrect ponderation at 20-10-18) : 0.22999891281127932 We have rejected 0 photos because of the batch_size condition ! NUMBER BATCH list_of_portfolios_to_create : 4 list_same_port_ids : [23103035] find same portfolio which already exist 23103035 , we will use it list_same_port_ids : [23103036] find same portfolio which already exist 23103036 , we will use it list_same_port_ids : [23103037] find same portfolio which already exist 23103037 , we will use it list_same_port_ids : [23103039] find same portfolio which already exist 23103039 , we will use it # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23103035 AND mptpi.`type`=4207 To do # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 13185 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 13193 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 13194 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 13189 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 13189 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 13191 argmax have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 13192 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 13186 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 13186 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 13197 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 13197 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 13188 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 13187 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 13187 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 13196 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 13188 doesn't seem to be define in the database( WARNING : type of input 3 of step 13187 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 13185 doesn't seem to be define in the database( WARNING : type of input 2 of step 13189 doesn't seem to be define in the database( WARNING : output 0 of step 13185 have datatype=16 whereas input 0 of step 13192 have datatype=1 WARNING : output 1 of step 13185 have datatype=2 whereas input 1 of step 13192 have datatype=7 WARNING : output 0 of step 13191 have datatype=6 whereas input 2 of step 13192 have datatype=5 WARNING : type of output 2 of step 13192 doesn't seem to be define in the database( WARNING : type of input 1 of step 13186 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 13188 have datatype=10 whereas input 3 of step 13195 have datatype=6 WARNING : type of input 2 of step 13197 doesn't seem to be define in the database( WARNING : output 1 of step 13186 have datatype=7 whereas input 2 of step 13197 have datatype=None WARNING : type of output 3 of step 13197 doesn't seem to be define in the database( WARNING : type of input 1 of step 13188 doesn't seem to be define in the database( WARNING : output 0 of step 13188 have datatype=10 whereas input 0 of step 13198 have datatype=18 WARNING : type of input 5 of step 13195 doesn't seem to be define in the database( WARNING : output 0 of step 13198 have datatype=11 whereas input 5 of step 13195 have datatype=None WARNING : type of output 1 of step 13193 doesn't seem to be define in the database( WARNING : type of input 3 of step 13189 doesn't seem to be define in the database( WARNING : type of output 1 of step 13194 doesn't seem to be define in the database( WARNING : type of input 3 of step 13189 doesn't seem to be define in the database( WARNING : output 0 of step 13192 have datatype=1 whereas input 0 of step 13186 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23103036 AND mptpi.`type`=4200 To do # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23103037 AND mptpi.`type`=4207 To do # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11500 mask_detect is not consistent : 3 used against 2 in the step definition ! WARNING : number of outputs for step 11508 brightness is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11509 blur_detection is not consistent : 2 used against 1 in the step definition ! WARNING : number of inputs for step 11504 crop_condition is not consistent : 3 used against 2 in the step definition ! Step 11504 crop_condition have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 11507 merge_mask_thcl_custom have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11507 merge_mask_thcl_custom is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11501 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11576 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11576 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of outputs for step 11503 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11502 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11502 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11511 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 1 of step 11503 doesn't seem to be define in the database( WARNING : type of input 3 of step 11502 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of output 2 of step 11500 doesn't seem to be define in the database( WARNING : type of input 2 of step 11504 doesn't seem to be define in the database( WARNING : output 1 of step 11500 have datatype=2 whereas input 1 of step 11507 have datatype=7 WARNING : type of output 2 of step 11507 doesn't seem to be define in the database( WARNING : type of input 1 of step 11501 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11503 have datatype=10 whereas input 3 of step 11510 have datatype=6 WARNING : type of input 2 of step 11576 doesn't seem to be define in the database( WARNING : output 1 of step 11501 have datatype=7 whereas input 2 of step 11576 have datatype=None WARNING : type of output 3 of step 11576 doesn't seem to be define in the database( WARNING : type of input 1 of step 11503 doesn't seem to be define in the database( WARNING : output 0 of step 11503 have datatype=10 whereas input 0 of step 11582 have datatype=18 WARNING : type of input 5 of step 11510 doesn't seem to be define in the database( WARNING : output 0 of step 11582 have datatype=11 whereas input 5 of step 11510 have datatype=None WARNING : type of output 1 of step 11508 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : type of output 1 of step 11509 doesn't seem to be define in the database( WARNING : type of input 3 of step 11504 doesn't seem to be define in the database( WARNING : output 0 of step 11507 have datatype=1 whereas input 0 of step 11501 have datatype=2 DataTypes for each output/input checked ! TODO Duplicate data, are they consistent 3 ? Duplicate data, are they consistent 4 ? SELECT mptpi.id, mptpi.mtr_portfolio_id_1, mptpi.mtr_portfolio_id_2, mptpi.type, mptpi.hashtag_id, mptpi.min_score, mptpi.mtr_user_id, mptpi.created_at, mptpi.updated_at, mptpi.last_updated_at_desc, mptpi.last_updated_at_asc, h.hashtag FROM MTRPhoto.mtr_port_to_port_ids mptpi, MTRBack.hashtags h WHERE h.hashtag_id=mptpi.hashtag_id AND mptpi.`mtr_portfolio_id_1`=23103039 AND mptpi.`type`=4207 To do elapsed_time : count_nb_balles_and_create_portfolio 29.330612182617188 # DISPLAY ALL COLLECTED DATA : {'19052025': {'nb_upload': 281, 'nb_taggue_class': 260, 'nb_taggue_densite': 260, 'nb_descriptors': 39, 'number_port': 5, 'count_photo_in_port': 27, 'nb_port_per_class': {'aluminium': {'nb_photos': 0, 'nb_portfolios': 0}, 'ela': {'nb_photos': 0, 'nb_portfolios': 0}, 'emr': {'nb_photos': 14, 'nb_portfolios': 2}, 'film_pedb': {'nb_photos': 0, 'nb_portfolios': 0}, 'flux_dev': {'nb_photos': 4, 'nb_portfolios': 1}, 'jrm': {'nb_photos': 0, 'nb_portfolios': 0}, 'pcm': {'nb_photos': 0, 'nb_portfolios': 0}, 'pcnc': {'nb_photos': 9, 'nb_portfolios': 1}, 'pehd_pp': {'nb_photos': 0, 'nb_portfolios': 0}, 'pet_clair': {'nb_photos': 0, 'nb_portfolios': 0}, 'refus': {'nb_photos': 0, 'nb_portfolios': 0}, 'tapis_vide': {'nb_photos': 0, 'nb_portfolios': 1}}}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1359282292, 1359282053, 1359282050, 1359282046, 1359281986, 1359281980, 1359281383, 1359281330, 1359280886, 1359280885, 1359280880, 1359280813, 1359280617, 1359280614, 1359280609, 1359280135, 1359280133, 1359280131, 1359280034, 1359279871] Looping around the photos to save general results len do output : 1 /23100212Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4323', None, None, None, None, None, None, None, '2890706') ('4323', '23100212', '1359282292', None, None, None, None, None, '2890706') ('4323', None, None, None, None, None, None, None, '2890706') ('4323', '23100212', '1359282053', None, None, None, None, None, '2890706') ('4323', None, None, None, None, None, None, None, '2890706') ('4323', '23100212', '1359282050', None, None, None, None, None, '2890706') ('4323', None, None, None, None, None, None, None, '2890706') ('4323', '23100212', '1359282046', None, None, None, None, None, '2890706') ('4323', None, None, None, None, None, None, None, '2890706') ('4323', '23100212', '1359281986', None, None, None, None, None, '2890706') ('4323', None, None, None, None, None, None, None, '2890706') ('4323', '23100212', '1359281980', None, None, None, None, None, '2890706') ('4323', None, None, None, None, None, None, None, '2890706') ('4323', '23100212', '1359281383', None, None, None, None, None, '2890706') ('4323', None, None, None, None, None, None, None, '2890706') ('4323', '23100212', '1359281330', None, None, None, None, None, '2890706') ('4323', None, None, None, None, None, None, None, '2890706') ('4323', '23100212', '1359280886', None, None, None, None, None, '2890706') ('4323', None, None, None, None, None, None, None, '2890706') ('4323', '23100212', '1359280885', None, None, None, None, None, '2890706') ('4323', None, None, None, None, None, None, None, '2890706') ('4323', '23100212', '1359280880', None, None, None, None, None, '2890706') ('4323', None, None, None, None, None, None, None, '2890706') ('4323', '23100212', '1359280813', None, None, None, None, None, '2890706') ('4323', None, None, None, None, None, None, None, '2890706') ('4323', '23100212', '1359280617', None, None, None, None, None, '2890706') ('4323', None, None, None, None, None, None, None, '2890706') ('4323', '23100212', '1359280614', None, None, None, None, None, '2890706') ('4323', None, None, None, None, None, None, None, '2890706') ('4323', '23100212', '1359280609', None, None, None, None, None, '2890706') ('4323', None, None, None, None, None, None, None, '2890706') ('4323', '23100212', '1359280135', None, None, None, None, None, '2890706') ('4323', None, None, None, None, None, None, None, '2890706') ('4323', '23100212', '1359280133', None, None, None, None, None, '2890706') ('4323', None, None, None, None, None, None, None, '2890706') ('4323', '23100212', '1359280131', None, None, None, None, None, '2890706') ('4323', None, None, None, None, None, None, None, '2890706') ('4323', '23100212', '1359280034', None, None, None, None, None, '2890706') ('4323', None, None, None, None, None, None, None, '2890706') ('4323', '23100212', '1359279871', None, None, None, None, None, '2890706') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 21 time used for this insertion : 0.06662154197692871 save_final save missing photos in datou_result : time spend for datou_step_exec : 31.797712564468384 time spend to save output : 0.06698870658874512 total time spend for step 1 : 31.86470127105713 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 1 set_done_treatment 1.48user 1.32system 0:33.83elapsed 8%CPU (0avgtext+0avgdata 104140maxresident)k 272inputs+512outputs (12major+50260minor)pagefaults 0swaps