python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 8 -a ' -a 4323 -l 20 --fifo' -s datou_current_sts -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 1122875 load datou : 4323 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? load thcls load pdts Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec no input labels no input values updating current state to 1 we have a portfolio with more photos than limit : 316>20 please execute split_portfolio.py -i 21878891 -l 20 size over we load limit photo not treated list_input_json: {} Current got : datou_id : 4323, datou_cur_ids : ['2707418'] with mtr_portfolio_ids : ['21878891'] and first list_photo_ids : [] new path : /proc/1122875/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! List Step Type Loaded in datou : split_time_score list_input_json : {} origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! we have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB time to download the photos : 0.04959416389465332 About to test input to load Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 1 step1:split_time_score Mon Mar 31 02:02:06 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec begin split time score 2022-04-13 10:29:59 0 TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 3847, 'mtr_user_id': 31, 'name': 'learn_MM_generique_050224', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'aluminium,ela,emr,film_pedb,flux_dev,jrm,pcm,pcnc,pehd_pp,pet_clair,refus,tapis_vide', 'svm_portfolios_learning': '13096157,13096155,13096163,13096159,13301956,13095886,13096162,13096160,13358264,13096158,5515868,13276803', 'photo_hashtag_type': 4932, 'photo_desc_type': 6032, 'type_classification': 'tf_classification2', 'hashtag_id_list': '493546845,492741797,616987804,2107760237,2107760238,495916461,560181804,1284539308,2107760239,2107755846,538914404,2107748999'}] thcls : [{'id': 3513, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2_tf', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 4557, 'photo_desc_type': 5767, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('00', 11), ('01', 12), ('03', 12), ('04', 12), ('05', 12), ('06', 12), ('07', 12), ('08', 12), ('09', 13), ('10', 12), ('11', 12), ('12', 14), ('13', 12), ('14', 13), ('15', 16), ('16', 29), ('17', 27), ('18', 13), ('19', 12), ('20', 12), ('21', 12), ('22', 12), ('23', 12)) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {1: 316} 30032025 21878891 Nombre de photos uploadées : 316 / 23040 (1%) 30032025 21878891 Nombre de photos taguées (types de déchets): 316 / 316 (100%) 30032025 21878891 Nombre de photos taguées (volume) : 316 / 316 (100%) elapsed_time : load_data_split_time_score 1.6689300537109375e-06 elapsed_time : order_list_meta_photo_and_scores 0.0001423358917236328 elapsed_time : fill_and_build_computed_from_old_data 0.014420747756958008 elapsed_time : insert_dashboard_record_day_entry 0.026134490966796875 Creating list_photo_total elapsed_time : select_descriptors 0.01137232780456543 30032025 21878891 Nombre de photos avec descriptors (type 6032) : 0 / 0 (0%) Missing descriptors for photos 0 and 1348680875 0:00:00|ON:Missing descriptors for photos 1348680875 and 1348685439 Missing descriptors for photos 1348685439 and 1348685782 Missing descriptors for photos 1348685782 and 1348688700 Missing descriptors for photos 1348688700 and 1348693033 Missing descriptors for photos 1348693033 and 1348696493 Missing descriptors for photos 1348696493 and 1348697939 Missing descriptors for photos 1348697939 and 1348701511 Missing descriptors for photos 1348701511 and 1348704914 Missing descriptors for photos 1348704914 and 1348706533 Missing descriptors for photos 1348706533 and 1348707062 Missing descriptors for photos 1348707062 and 1348707193 Missing descriptors for photos 1348707193 and 1348709349 Missing descriptors for photos 1348709349 and 1348713463 Missing descriptors for photos 1348713463 and 1348716327 Missing descriptors for photos 1348716327 and 1348717883 Missing descriptors for photos 1348717883 and 1348718200 Missing descriptors for photos 1348718200 and 1348718803 Missing descriptors for photos 1348718803 and 1348718929 Missing descriptors for photos 1348718929 and 1348719024 Missing descriptors for photos 1348719024 and 1348719125 Missing descriptors for photos 1348719125 and 1348719222 Missing descriptors for photos 1348719222 and 1348719305 Missing descriptors for photos 1348719305 and 1348720824 Missing descriptors for photos 1348720824 and 1348723174 Missing descriptors for photos 1348723174 and 1348725134 Missing descriptors for photos 1348725134 and 1348727867 Missing descriptors for photos 1348727867 and 1348729453 Missing descriptors for photos 1348729453 and 1348730604 Missing descriptors for photos 1348730604 and 1348730707 Missing descriptors for photos 1348730707 and 1348730792 Missing descriptors for photos 1348730792 and 1348730968 Missing descriptors for photos 1348730968 and 1348732148 Missing descriptors for photos 1348732148 and 1348732895 Missing descriptors for photos 1348732895 and 1348733527 Missing descriptors for photos 1348733527 and 1348736174 Missing descriptors for photos 1348736174 and 1348737120 Missing descriptors for photos 1348737120 and 1348737260 Missing descriptors for photos 1348737260 and 1348737387 Missing descriptors for photos 1348737387 and 1348737510 Missing descriptors for photos 1348737510 and 1348737652 Missing descriptors for photos 1348737652 and 1348737776 Missing descriptors for photos 1348737776 and 1348739047 Missing descriptors for photos 1348739047 and 1348739850 Missing descriptors for photos 1348739850 and 1348740526 Missing descriptors for photos 1348740526 and 1348740644 Missing descriptors for photos 1348740644 and 1348740770 Missing descriptors for photos 1348740770 and 1348742810 Missing descriptors for photos 1348742810 and 1348743469 Missing descriptors for photos 1348743469 and 1348743604 Missing descriptors for photos 1348743604 and 1348743746 Missing descriptors for photos 1348743746 and 1348744590 Missing descriptors for photos 1348744590 and 1348745605 Missing descriptors for photos 1348745605 and 1348746567 Missing descriptors for photos 1348746567 and 1348746840 Missing descriptors for photos 1348746840 and 1348746970 Missing descriptors for photos 1348746970 and 1348747114 Missing descriptors for photos 1348747114 and 1348747220 Missing descriptors for photos 1348747220 and 1348747341 Missing descriptors for photos 1348747341 and 1348749529 Missing descriptors for photos 1348749529 and 1348751713 Missing descriptors for photos 1348751713 and 1348751883 Missing descriptors for photos 1348751883 and 1348752009 Missing descriptors for photos 1348752009 and 1348752166 Missing descriptors for photos 1348752166 and 1348752276 Missing descriptors for photos 1348752276 and 1348752414 Missing descriptors for photos 1348752414 and 1348752559 Missing descriptors for photos 1348752559 and 1348752688 Missing descriptors for photos 1348752688 and 1348752819 Missing descriptors for photos 1348752819 and 1348752960 Missing descriptors for photos 1348752960 and 1348753087 Missing descriptors for photos 1348753087 and 1348754809 Missing descriptors for photos 1348754809 and 1348755838 Missing descriptors for photos 1348755838 and 1348755940 Missing descriptors for photos 1348755940 and 1348756064 Missing descriptors for photos 1348756064 and 1348756234 Missing descriptors for photos 1348756234 and 1348756346 Missing descriptors for photos 1348756346 and 1348756479 Missing descriptors for photos 1348756479 and 1348756578 Missing descriptors for photos 1348756578 and 1348756722 Missing descriptors for photos 1348756722 and 1348756827 Missing descriptors for photos 1348756827 and 1348756943 Missing descriptors for photos 1348756943 and 1348757061 Missing descriptors for photos 1348757061 and 1348758883 Missing descriptors for photos 1348758883 and 1348760131 Missing descriptors for photos 1348760131 and 1348760259 Missing descriptors for photos 1348760259 and 1348760404 Missing descriptors for photos 1348760404 and 1348760532 Missing descriptors for photos 1348760532 and 1348760666 Missing descriptors for photos 1348760666 and 1348760806 Missing descriptors for photos 1348760806 and 1348760945 Missing descriptors for photos 1348760945 and 1348761087 Missing descriptors for photos 1348761087 and 1348762124 Missing descriptors for photos 1348762124 and 1348764872 Missing descriptors for photos 1348764872 and 1348772011 Missing descriptors for photos 1348772011 and 1348777914 Missing descriptors for photos 1348777914 and 1348782655 Missing descriptors for photos 1348782655 and 1348786766 Missing descriptors for photos 1348786766 and 1348791764 Missing descriptors for photos 1348791764 and 1348795702 Missing descriptors for photos 1348795702 and 1348800664 Missing descriptors for photos 1348800664 and 1348804679 Missing descriptors for photos 1348804679 and 1348809251 Missing descriptors for photos 1348809251 and 1348814608 Missing descriptors for photos 1348814608 and 1348818331 Missing descriptors for photos 1348818331 and 1348836111 Missing descriptors for photos 1348836111 and 1348820418 Missing descriptors for photos 1348820418 and 1348823242 Missing descriptors for photos 1348823242 and 1348824697 Missing descriptors for photos 1348824697 and 1348826825 Missing descriptors for photos 1348826825 and 1348827184 Missing descriptors for photos 1348827184 and 1348827358 Missing descriptors for photos 1348827358 and 1348827532 Missing descriptors for photos 1348827532 and 1348827693 Missing descriptors for photos 1348827693 and 1348827866 Missing descriptors for photos 1348827866 and 1348828048 Missing descriptors for photos 1348828048 and 1348828251 Missing descriptors for photos 1348828251 and 1348828408 Missing descriptors for photos 1348828408 and 1348828565 Missing descriptors for photos 1348828565 and 1348828683 Missing descriptors for photos 1348828683 and 1348830626 Missing descriptors for photos 1348830626 and 1348831003 Missing descriptors for photos 1348831003 and 1348831187 Missing descriptors for photos 1348831187 and 1348831350 Missing descriptors for photos 1348831350 and 1348831464 Missing descriptors for photos 1348831464 and 1348831657 Missing descriptors for photos 1348831657 and 1348831820 Missing descriptors for photos 1348831820 and 1348832729 Missing descriptors for photos 1348832729 and 1348833537 Missing descriptors for photos 1348833537 and 1348834047 Missing descriptors for photos 1348834047 and 1348834173 Missing descriptors for photos 1348834173 and 1348834300 Missing descriptors for photos 1348834300 and 1348835957 Missing descriptors for photos 1348835957 and 1348836107 Missing descriptors for photos 1348836107 and 1348838272 Missing descriptors for photos 1348838272 and 1348836902 Missing descriptors for photos 1348836902 and 1348837060 Missing descriptors for photos 1348837060 and 1348837225 Missing descriptors for photos 1348837225 and 1348837395 Missing descriptors for photos 1348837395 and 1348837579 Missing descriptors for photos 1348837579 and 1348837737 Missing descriptors for photos 1348837737 and 1348837911 Missing descriptors for photos 1348837911 and 1348838084 Missing descriptors for photos 1348838084 and 1348838244 Missing descriptors for photos 1348838244 and 1348846941 Missing descriptors for photos 1348846941 and 1348838391 Missing descriptors for photos 1348838391 and 1348839133 Missing descriptors for photos 1348839133 and 1348840292 Missing descriptors for photos 1348840292 and 1348842238 Missing descriptors for photos 1348842238 and 1348842389 Missing descriptors for photos 1348842389 and 1348842555 Missing descriptors for photos 1348842555 and 1348842698 Missing descriptors for photos 1348842698 and 1348842875 Missing descriptors for photos 1348842875 and 1348843016 Missing descriptors for photos 1348843016 and 1348843160 Missing descriptors for photos 1348843160 and 1348843275 Missing descriptors for photos 1348843275 and 1348843421 Missing descriptors for photos 1348843421 and 1348843542 Missing descriptors for photos 1348843542 and 1348845408 Missing descriptors for photos 1348845408 and 1348845756 Missing descriptors for photos 1348845756 and 1348845949 Missing descriptors for photos 1348845949 and 1348846110 Missing descriptors for photos 1348846110 and 1348846243 Missing descriptors for photos 1348846243 and 1348846420 Missing descriptors for photos 1348846420 and 1348846581 Missing descriptors for photos 1348846581 and 1348846698 Missing descriptors for photos 1348846698 and 1348846831 Missing descriptors for photos 1348846831 and 1348851331 Missing descriptors for photos 1348851331 and 1348846980 Missing descriptors for photos 1348846980 and 1348847089 Missing descriptors for photos 1348847089 and 1348847216 Missing descriptors for photos 1348847216 and 1348848452 Missing descriptors for photos 1348848452 and 1348849744 Missing descriptors for photos 1348849744 and 1348850007 Missing descriptors for photos 1348850007 and 1348850157 Missing descriptors for photos 1348850157 and 1348850315 Missing descriptors for photos 1348850315 and 1348850464 Missing descriptors for photos 1348850464 and 1348850628 Missing descriptors for photos 1348850628 and 1348850783 Missing descriptors for photos 1348850783 and 1348850989 Missing descriptors for photos 1348850989 and 1348851180 Missing descriptors for photos 1348851180 and 1348851378 Missing descriptors for photos 1348851378 and 1348851327 Missing descriptors for photos 1348851327 and 1348852822 Missing descriptors for photos 1348852822 and 1348851461 Missing descriptors for photos 1348851461 and 1348852782 Missing descriptors for photos 1348852782 and 1348852881 Missing descriptors for photos 1348852881 and 1348853343 Missing descriptors for photos 1348853343 and 1348852866 Missing descriptors for photos 1348852866 and 1348853024 Missing descriptors for photos 1348853024 and 1348853187 Missing descriptors for photos 1348853187 and 1348853382 Missing descriptors for photos 1348853382 and 1348853338 Missing descriptors for photos 1348853338 and 1348853490 Missing descriptors for photos 1348853490 and 1348853488 Missing descriptors for photos 1348853488 and 1348853485 Missing descriptors for photos 1348853485 and 1348853501 Missing descriptors for photos 1348853501 and 1348853748 Missing descriptors for photos 1348853748 and 1348853482 Missing descriptors for photos 1348853482 and 1348853630 Missing descriptors for photos 1348853630 and 1348853778 Missing descriptors for photos 1348853778 and 1348853799 Missing descriptors for photos 1348853799 and 1348853776 Missing descriptors for photos 1348853776 and 1348853838 Missing descriptors for photos 1348853838 and 1348854082 Missing descriptors for photos 1348854082 and 1348853879 Missing descriptors for photos 1348853879 and 1348854009 Missing descriptors for photos 1348854009 and 1348854079 Missing descriptors for photos 1348854079 and 1348854119 Missing descriptors for photos 1348854119 and 1348854117 Missing descriptors for photos 1348854117 and 1348854115 Missing descriptors for photos 1348854115 and 1348854271 Missing descriptors for photos 1348854271 and 1348854248 Missing descriptors for photos 1348854248 and 1348855007 Missing descriptors for photos 1348855007 and 1348854369 Missing descriptors for photos 1348854369 and 1348855446 Missing descriptors for photos 1348855446 and 1348855833 Missing descriptors for photos 1348855833 and 1348855416 Missing descriptors for photos 1348855416 and 1348856039 Missing descriptors for photos 1348856039 and 1348856286 Missing descriptors for photos 1348856286 and 1348856313 Missing descriptors for photos 1348856313 and 1348856637 Missing descriptors for photos 1348856637 and 1348856340 Missing descriptors for photos 1348856340 and 1348856464 Missing descriptors for photos 1348856464 and 1348856580 Missing descriptors for photos 1348856580 and 1348856634 Missing descriptors for photos 1348856634 and 1348856671 Missing descriptors for photos 1348856671 and 1348856701 Missing descriptors for photos 1348856701 and 1348856806 Missing descriptors for photos 1348856806 and 1348856699 Missing descriptors for photos 1348856699 and 1348856831 Missing descriptors for photos 1348856831 and 1348856922 Missing descriptors for photos 1348856922 and 1348856873 Missing descriptors for photos 1348856873 and 1348856979 Missing descriptors for photos 1348856979 and 1348857075 Missing descriptors for photos 1348857075 and 1348857019 Missing descriptors for photos 1348857019 and 1348857110 Missing descriptors for photos 1348857110 and 1348860105 Missing descriptors for photos 1348860105 and 1348857133 Missing descriptors for photos 1348857133 and 1348857248 Missing descriptors for photos 1348857248 and 1348857381 Missing descriptors for photos 1348857381 and 1348857497 Missing descriptors for photos 1348857497 and 1348857605 Missing descriptors for photos 1348857605 and 1348859411 Missing descriptors for photos 1348859411 and 1348859652 Missing descriptors for photos 1348859652 and 1348859806 Missing descriptors for photos 1348859806 and 1348859943 Missing descriptors for photos 1348859943 and 1348863291 Missing descriptors for photos 1348863291 and 1348860102 Missing descriptors for photos 1348860102 and 1348860247 Missing descriptors for photos 1348860247 and 1348860381 Missing descriptors for photos 1348860381 and 1348860510 Missing descriptors for photos 1348860510 and 1348860634 Missing descriptors for photos 1348860634 and 1348860766 Missing descriptors for photos 1348860766 and 1348860897 Missing descriptors for photos 1348860897 and 1348861020 Missing descriptors for photos 1348861020 and 1348861775 Missing descriptors for photos 1348861775 and 1348862640 Missing descriptors for photos 1348862640 and 1348862777 Missing descriptors for photos 1348862777 and 1348862901 Missing descriptors for photos 1348862901 and 1348863006 Missing descriptors for photos 1348863006 and 1348863098 Missing descriptors for photos 1348863098 and 1348863189 Missing descriptors for photos 1348863189 and 1348863289 Missing descriptors for photos 1348863289 and 1348863392 Missing descriptors for photos 1348863392 and 1348863496 Missing descriptors for photos 1348863496 and 1348863609 Missing descriptors for photos 1348863609 and 1348863705 Missing descriptors for photos 1348863705 and 1348864989 Missing descriptors for photos 1348864989 and 1348865409 Missing descriptors for photos 1348865409 and 1348865596 Missing descriptors for photos 1348865596 and 1348865757 Missing descriptors for photos 1348865757 and 1348865894 Missing descriptors for photos 1348865894 and 1348866024 Missing descriptors for photos 1348866024 and 1348866183 Missing descriptors for photos 1348866183 and 1348866323 Missing descriptors for photos 1348866323 and 1348866439 Missing descriptors for photos 1348866439 and 1348866579 Missing descriptors for photos 1348866579 and 1348866704 Missing descriptors for photos 1348866704 and 1348866835 Missing descriptors for photos 1348866835 and 1348867853 Missing descriptors for photos 1348867853 and 1348869134 Missing descriptors for photos 1348869134 and 1348869281 Missing descriptors for photos 1348869281 and 1348869414 Missing descriptors for photos 1348869414 and 1348869549 Missing descriptors for photos 1348869549 and 1348869681 Missing descriptors for photos 1348869681 and 1348869818 Missing descriptors for photos 1348869818 and 1348869927 Missing descriptors for photos 1348869927 and 1348870102 Missing descriptors for photos 1348870102 and 1348870275 Missing descriptors for photos 1348870275 and 1348870402 Missing descriptors for photos 1348870402 and 1348870518 Missing descriptors for photos 1348870518 and 1348871904 Missing descriptors for photos 1348871904 and 1348875865 Missing descriptors for photos 1348875865 and 1348880199 Missing descriptors for photos 1348880199 and 1348886413 Missing descriptors for photos 1348886413 and 1348892733 Missing descriptors for photos 1348892733 and 1348898941 Missing descriptors for photos 1348898941 and 1348901582 Missing descriptors for photos 1348901582 and 1348901683 Missing descriptors for photos 1348901683 and 1348901805 Missing descriptors for photos 1348901805 and 1348901910 Missing descriptors for photos 1348901910 and 1348902029 Missing descriptors for photos 1348902029 and 1348902131 Missing descriptors for photos 1348902131 and 1348903838 Missing descriptors for photos 1348903838 and 1348904793 Missing descriptors for photos 1348904793 and 1348904930 Missing descriptors for photos 1348904930 and 1348905077 Missing descriptors for photos 1348905077 and 1348905196 Missing descriptors for photos 1348905196 and 1348905293 Missing descriptors for photos 1348905293 and 1348905412 Missing descriptors for photos 1348905412 and 1348905524 Missing descriptors for photos 1348905524 and 1348905638 Missing descriptors for photos 1348905638 and 1348905756 Missing descriptors for photos 1348905756 and 1348905861 Missing descriptors for photos 1348905861 and 1348905972 30032025 Removing 0 photos because of the 'same image' condition Total on : 0 Total off : 0.0 list_time_off Warning in study_and_display_distrib_list : min=max : 0.0 0.0 dist_desc Warning in study_and_display_distrib_list : min=max : -1 -1 begin to insert list_values into photo_hahstag_ids : length of list_valuse in save_photo_hashtag_id_type : 316 time used for this insertion : 0.07006144523620605 photos_removed : len 0 elapsed_time : remove_photo_duplicate 0.10198807716369629 Creating list_photo_total elapsed_time : count_sum_diff_and_build_graph 0.0026092529296875 Total photos : 316 .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info .can't find max_score_info Total photos : 316 Number of lists : 0 counter photos in port : 0 hashtag : aluminium(493546845) : 0 photos in 0 portfolios ! hashtag : ela(492741797) : 0 photos in 0 portfolios ! hashtag : emr(616987804) : 0 photos in 0 portfolios ! hashtag : film_pedb(2107760237) : 0 photos in 0 portfolios ! hashtag : flux_dev(2107760238) : 0 photos in 0 portfolios ! hashtag : jrm(495916461) : 0 photos in 0 portfolios ! hashtag : pcm(560181804) : 0 photos in 0 portfolios ! hashtag : pcnc(1284539308) : 0 photos in 0 portfolios ! hashtag : pehd_pp(2107760239) : 0 photos in 0 portfolios ! hashtag : pet_clair(2107755846) : 0 photos in 0 portfolios ! hashtag : refus(538914404) : 0 photos in 0 portfolios ! hashtag : tapis_vide(2107748999) : 0 photos in 0 portfolios ! elapsed_time : group_photo_by_moyenne_exp 0.013107776641845703 elapsed_time : compute_and_correct_tag_with_moyenne_mobile 2.384185791015625e-06 today str has not a value , we define it as the date of the first image todaystr_first : 30032025 attention , prev_timestamp is 0 , we do nothing **************************************************************************************************************************************************************************************************************************************************************************************************************Count Time bigger than 30s : 302 #Number Photos for regression : {'30032025': {493546845: {2107751013: 0, 2107751014: 0, 2107751015: 0, 2107751016: 0, 2107751017: 0}, 492741797: {2107751013: 0, 2107751014: 0, 2107751015: 0, 2107751016: 0, 2107751017: 0}, 616987804: {2107751013: 0, 2107751014: 0, 2107751015: 0, 2107751016: 0, 2107751017: 0}, 2107760237: {2107751013: 0, 2107751014: 0, 2107751015: 0, 2107751016: 0, 2107751017: 0}, 2107760238: {2107751013: 0, 2107751014: 0, 2107751015: 0, 2107751016: 0, 2107751017: 0}, 495916461: {2107751013: 0, 2107751014: 0, 2107751015: 0, 2107751016: 0, 2107751017: 0}, 560181804: {2107751013: 0, 2107751014: 0, 2107751015: 0, 2107751016: 0, 2107751017: 0}, 1284539308: {2107751013: 0, 2107751014: 0, 2107751015: 0, 2107751016: 0, 2107751017: 0}, 2107760239: {2107751013: 0, 2107751014: 0, 2107751015: 0, 2107751016: 0, 2107751017: 0}, 2107755846: {2107751013: 0, 2107751014: 0, 2107751015: 0, 2107751016: 0, 2107751017: 0}, 538914404: {2107751013: 0, 2107751014: 0, 2107751015: 0, 2107751016: 0, 2107751017: 0}, 2107748999: {2107751013: 0, 2107751014: 0, 2107751015: 2704.9959626197815, 2107751016: 0, 2107751017: 0}}} 30032025|aluminium, 05102018_papier_non_papier_dense:0 30032025|aluminium, 05102018_papier_non_papier_peu_dense:0 30032025|aluminium, 05102018_papier_non_papier_presque_vide:0 30032025|aluminium, 05102018_papier_non_papier_tres_dense:0 30032025|aluminium, 05102018_papier_non_papier_tres_peu_dense:0 30032025|ela, 05102018_papier_non_papier_dense:0 30032025|ela, 05102018_papier_non_papier_peu_dense:0 30032025|ela, 05102018_papier_non_papier_presque_vide:0 30032025|ela, 05102018_papier_non_papier_tres_dense:0 30032025|ela, 05102018_papier_non_papier_tres_peu_dense:0 30032025|emr, 05102018_papier_non_papier_dense:0 30032025|emr, 05102018_papier_non_papier_peu_dense:0 30032025|emr, 05102018_papier_non_papier_presque_vide:0 30032025|emr, 05102018_papier_non_papier_tres_dense:0 30032025|emr, 05102018_papier_non_papier_tres_peu_dense:0 30032025|film_pedb, 05102018_papier_non_papier_dense:0 30032025|film_pedb, 05102018_papier_non_papier_peu_dense:0 30032025|film_pedb, 05102018_papier_non_papier_presque_vide:0 30032025|film_pedb, 05102018_papier_non_papier_tres_dense:0 30032025|film_pedb, 05102018_papier_non_papier_tres_peu_dense:0 30032025|flux_dev, 05102018_papier_non_papier_dense:0 30032025|flux_dev, 05102018_papier_non_papier_peu_dense:0 30032025|flux_dev, 05102018_papier_non_papier_presque_vide:0 30032025|flux_dev, 05102018_papier_non_papier_tres_dense:0 30032025|flux_dev, 05102018_papier_non_papier_tres_peu_dense:0 30032025|jrm, 05102018_papier_non_papier_dense:0 30032025|jrm, 05102018_papier_non_papier_peu_dense:0 30032025|jrm, 05102018_papier_non_papier_presque_vide:0 30032025|jrm, 05102018_papier_non_papier_tres_dense:0 30032025|jrm, 05102018_papier_non_papier_tres_peu_dense:0 30032025|pcm, 05102018_papier_non_papier_dense:0 30032025|pcm, 05102018_papier_non_papier_peu_dense:0 30032025|pcm, 05102018_papier_non_papier_presque_vide:0 30032025|pcm, 05102018_papier_non_papier_tres_dense:0 30032025|pcm, 05102018_papier_non_papier_tres_peu_dense:0 30032025|pcnc, 05102018_papier_non_papier_dense:0 30032025|pcnc, 05102018_papier_non_papier_peu_dense:0 30032025|pcnc, 05102018_papier_non_papier_presque_vide:0 30032025|pcnc, 05102018_papier_non_papier_tres_dense:0 30032025|pcnc, 05102018_papier_non_papier_tres_peu_dense:0 30032025|pehd_pp, 05102018_papier_non_papier_dense:0 30032025|pehd_pp, 05102018_papier_non_papier_peu_dense:0 30032025|pehd_pp, 05102018_papier_non_papier_presque_vide:0 30032025|pehd_pp, 05102018_papier_non_papier_tres_dense:0 30032025|pehd_pp, 05102018_papier_non_papier_tres_peu_dense:0 30032025|pet_clair, 05102018_papier_non_papier_dense:0 30032025|pet_clair, 05102018_papier_non_papier_peu_dense:0 30032025|pet_clair, 05102018_papier_non_papier_presque_vide:0 30032025|pet_clair, 05102018_papier_non_papier_tres_dense:0 30032025|pet_clair, 05102018_papier_non_papier_tres_peu_dense:0 30032025|refus, 05102018_papier_non_papier_dense:0 30032025|refus, 05102018_papier_non_papier_peu_dense:0 30032025|refus, 05102018_papier_non_papier_presque_vide:0 30032025|refus, 05102018_papier_non_papier_tres_dense:0 30032025|refus, 05102018_papier_non_papier_tres_peu_dense:0 30032025|tapis_vide, 05102018_papier_non_papier_dense:0 30032025|tapis_vide, 05102018_papier_non_papier_peu_dense:0 30032025|tapis_vide, 05102018_papier_non_papier_presque_vide:2704.9959626197815 30032025|tapis_vide, 05102018_papier_non_papier_tres_dense:0 30032025|tapis_vide, 05102018_papier_non_papier_tres_peu_dense:0 #Number Photos for regression amount gros magasin papier (time_diff then nb_photo) : We have not displayed the number of photos removed for one material since Rungis_Papier wasn't in the thcl used ! 30032025_time_diff_distrib Number amount portfolio for this type of dechet : aluminium 0 Number amount portfolio for this type of dechet : ela 0 Number amount portfolio for this type of dechet : emr 0 Number amount portfolio for this type of dechet : film_pedb 0 Number amount portfolio for this type of dechet : flux_dev 0 Number amount portfolio for this type of dechet : jrm 0 Number amount portfolio for this type of dechet : pcm 0 Number amount portfolio for this type of dechet : pcnc 0 Number amount portfolio for this type of dechet : pehd_pp 0 Number amount portfolio for this type of dechet : pet_clair 0 Number amount portfolio for this type of dechet : refus 0 Number amount portfolio for this type of dechet : tapis_vide 46 https://marlene.fotonower.com/api/v1/secured/portfolio/new?name=30032025_tapis_vide_05102018_papier_non_papier_presque_vide&access_token=0fc1cdda0f63f39f777d9cb33b1aa204 Created to study and clean : 21893283 with name like 30032025_tapis_vide_05102018_papier_non_papier_presque_vide NUMBER BATCH : 0 list_ponderation used : [0.001, 0.001, 0.001, 0.001, 0.001, 0.001, 0.001, 0.001, 0.001, 0.001, 0.001] , list_hashtag_class_create_as_list : ['pcnc', 'gm', 'emr', 'ela', 'pet_clair', 'film_pedb', 'pehd_pp', 'flux_dev_rigide', 'aluminium', 'tapis_vide', 'refus', 'gros_cartons', 'gm', 'flux_dev_rigide', 'gros_cartons'] We have rejected 0 photos because of the batch_size condition ! NUMBER BATCH list_of_portfolios_to_create : 0 elapsed_time : count_nb_balles_and_create_portfolio 1.6469292640686035 # DISPLAY ALL COLLECTED DATA : {'30032025': {'nb_upload': 316, 'nb_taggue_class': 316, 'nb_taggue_densite': 316, 'nb_descriptors': 0, 'number_port': 0, 'count_photo_in_port': 0, 'nb_port_per_class': {'aluminium': {'nb_photos': 0, 'nb_portfolios': 0}, 'ela': {'nb_photos': 0, 'nb_portfolios': 0}, 'emr': {'nb_photos': 0, 'nb_portfolios': 0}, 'film_pedb': {'nb_photos': 0, 'nb_portfolios': 0}, 'flux_dev': {'nb_photos': 0, 'nb_portfolios': 0}, 'jrm': {'nb_photos': 0, 'nb_portfolios': 0}, 'pcm': {'nb_photos': 0, 'nb_portfolios': 0}, 'pcnc': {'nb_photos': 0, 'nb_portfolios': 0}, 'pehd_pp': {'nb_photos': 0, 'nb_portfolios': 0}, 'pet_clair': {'nb_photos': 0, 'nb_portfolios': 0}, 'refus': {'nb_photos': 0, 'nb_portfolios': 0}, 'tapis_vide': {'nb_photos': 0, 'nb_portfolios': 0}}}} Inside saveOutput : final : True verbose : 0 saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral [1348905972, 1348905861, 1348905756, 1348905638, 1348905524, 1348905412, 1348905293, 1348905196, 1348905077, 1348904930, 1348904793, 1348903838, 1348902131, 1348902029, 1348901910, 1348901805, 1348901683, 1348901582, 1348898941, 1348892733] Looping around the photos to save general results len do output : 1 /21878891Didn't retrieve data . before output type Here is an output not treated by saveGeneral : Managing all output in save final without adding information in the mtr_datou_result ('4323', None, None, None, None, None, None, None, '2707418') ('4323', '21878891', '1348905972', None, None, None, None, None, '2707418') ('4323', None, None, None, None, None, None, None, '2707418') ('4323', '21878891', '1348905861', None, None, None, None, None, '2707418') ('4323', None, None, None, None, None, None, None, '2707418') ('4323', '21878891', '1348905756', None, None, None, None, None, '2707418') ('4323', None, None, None, None, None, None, None, '2707418') ('4323', '21878891', '1348905638', None, None, None, None, None, '2707418') ('4323', None, None, None, None, None, None, None, '2707418') ('4323', '21878891', '1348905524', None, None, None, None, None, '2707418') ('4323', None, None, None, None, None, None, None, '2707418') ('4323', '21878891', '1348905412', None, None, None, None, None, '2707418') ('4323', None, None, None, None, None, None, None, '2707418') ('4323', '21878891', '1348905293', None, None, None, None, None, '2707418') ('4323', None, None, None, None, None, None, None, '2707418') ('4323', '21878891', '1348905196', None, None, None, None, None, '2707418') ('4323', None, None, None, None, None, None, None, '2707418') ('4323', '21878891', '1348905077', None, None, None, None, None, '2707418') ('4323', None, None, None, None, None, None, None, '2707418') ('4323', '21878891', '1348904930', None, None, None, None, None, '2707418') ('4323', None, None, None, None, None, None, None, '2707418') ('4323', '21878891', '1348904793', None, None, None, None, None, '2707418') ('4323', None, None, None, None, None, None, None, '2707418') ('4323', '21878891', '1348903838', None, None, None, None, None, '2707418') ('4323', None, None, None, None, None, None, None, '2707418') ('4323', '21878891', '1348902131', None, None, None, None, None, '2707418') ('4323', None, None, None, None, None, None, None, '2707418') ('4323', '21878891', '1348902029', None, None, None, None, None, '2707418') ('4323', None, None, None, None, None, None, None, '2707418') ('4323', '21878891', '1348901910', None, None, None, None, None, '2707418') ('4323', None, None, None, None, None, None, None, '2707418') ('4323', '21878891', '1348901805', None, None, None, None, None, '2707418') ('4323', None, None, None, None, None, None, None, '2707418') ('4323', '21878891', '1348901683', None, None, None, None, None, '2707418') ('4323', None, None, None, None, None, None, None, '2707418') ('4323', '21878891', '1348901582', None, None, None, None, None, '2707418') ('4323', None, None, None, None, None, None, None, '2707418') ('4323', '21878891', '1348898941', None, None, None, None, None, '2707418') ('4323', None, None, None, None, None, None, None, '2707418') ('4323', '21878891', '1348892733', None, None, None, None, None, '2707418') begin to insert list_values into mtr_datou_result : length of list_values in save_final : 21 time used for this insertion : 0.1879105567932129 save_final save missing photos in datou_result : time spend for datou_step_exec : 1.9500057697296143 time spend to save output : 0.18823957443237305 total time spend for step 1 : 2.1382453441619873 caffe_path_current : About to save ! 2 After save, about to update current ! ret : 2 len(input) + len(total_photo_id_missing) : 1 set_done_treatment 1.09user 0.76system 0:03.64elapsed 51%CPU (0avgtext+0avgdata 102880maxresident)k 0inputs+88outputs (4major+49172minor)pagefaults 0swaps