python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 20 -a ' -a 4189' -s datou_current_4189 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 1797127 load datou : 4189 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? load thcls load pdts Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec no input labels no input values updating current state to 1 list_input_json: {} Date 2026-03-04 10:30:29.752706 Current got : datou_id : 4189, datou_cur_ids : ['4356661'] with mtr_portfolio_ids : ['30620530'] and first list_photo_ids : [] new path : /proc/1797127/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! List Step Type Loaded in datou : split_time_score over limit max, limiting to limit_max 100 list_input_json : {} origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! we have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB time to download the photos : 0.020575761795043945 About to test input to load Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 1 step1:split_time_score Wed Mar 4 10:30:29 2026 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec begin split time score 2022-04-13 10:29:59 0 TODO : Insert select and so on Begin split_port_in_batch_balle thcls : [{'id': 3379, 'mtr_user_id': 31, 'name': 'learn_classif_flux_maj_generique_effnet_v2_s_02062022', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': 'aluminium,ela,film_pedb,flux_dev,jrm,pcm,pcnc,pehd_pp,pet_clair,refus,tapis_vide', 'svm_portfolios_learning': '5515864,5515840,5515844,5515850,6244400,6237996,6237998,5515847,5515841,5515868,5515866', 'photo_hashtag_type': 4374, 'photo_desc_type': 5680, 'type_classification': 'tf_classification2', 'hashtag_id_list': '493546845,492741797,2107760237,2107760238,495916461,560181804,1284539308,2107760239,2107755846,538914404,2107748999'}] thcls : [{'id': 3513, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2_tf', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 4557, 'photo_desc_type': 5767, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] (('10', 143),) ERROR counted https://github.com/fotonower/Velours/issues/663#issuecomment-421136223 {1: 137} L04032026 30620530 Nombre de photos uploadées : 143 / 23040 (0%) 04032026 30620530 Nombre de photos taguées (types de déchets): 137 / 143 (95%) 04032026 30620530 Nombre de photos taguées (volume) : 137 / 143 (95%) elapsed_time : load_data_split_time_score 1.430511474609375e-06 elapsed_time : order_list_meta_photo_and_scores 6.0558319091796875e-05 LLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLL?????? elapsed_time : fill_and_build_computed_from_old_data 0.009386539459228516 LCatched exception ! Connect or reconnect ! Catched exception ! Connect or reconnect ! elapsed_time : insert_dashboard_record_day_entry 0.23038268089294434 LLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLLL Creating list_photo_by_hashtags Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None Hashtag is None elapsed_time : list_photo_by_hashtags 0.012233257293701172 Creating list_photo_total Object GET failed: https://storage.sbg.cloud.ovh.net/v1/AUTH_b878d1ec389e4877b1a6590d8f2dcede/desc.fotonower.com/5680.1410837506.desc 404 Not Found [first 60 chars of response] b'
The resource could not be found.<' Object GET failed: https://storage.sbg.cloud.ovh.net/v1/AUTH_b878d1ec389e4877b1a6590d8f2dcede/desc.fotonower.com/5680.1410839348.desc 404 Not Found [first 60 chars of response] b'
The resource could not be found.<' Object GET failed: https://storage.sbg.cloud.ovh.net/v1/AUTH_b878d1ec389e4877b1a6590d8f2dcede/desc.fotonower.com/5680.1410839899.desc 404 Not Found [first 60 chars of response] b'
The resource could not be found.<'
Unable to find some descriptors in ovh, trying to find them in s3
Number missing : 3
Warning this was a bad idea s3 !
elapsed_time : select_descriptors 0.8710768222808838
Missing descriptors : 3
04032026 30620530 Nombre de photos avec descriptors (type 5680) : 0 / 9 (0%)
Missing descriptors for photos 0 and 1410836147
L0:00:00|ON:Missing descriptors for photos 1410836147 and 1410836148
LMissing descriptors for photos 1410836148 and 1410836149
LMissing descriptors for photos 1410836149 and 1410836150
LMissing descriptors for photos 1410836150 and 1410836152
LMissing descriptors for photos 1410836152 and 1410836159
LMissing descriptors for photos 1410836159 and 1410836160
LMissing descriptors for photos 1410836160 and 1410836167
LMissing descriptors for photos 1410836167 and 1410836226
LMissing descriptors for photos 1410836226 and 1410836280
LMissing descriptors for photos 1410836280 and 1410836379
LMissing descriptors for photos 1410836379 and 1410836779
LMissing descriptors for photos 1410836779 and 1410836813
LMissing descriptors for photos 1410836813 and 1410836913
LMissing descriptors for photos 1410836913 and 1410837012
LMissing descriptors for photos 1410837012 and 1410837061
LMissing descriptors for photos 1410837061 and 1410837115
LMissing descriptors for photos 1410837115 and 1410837506
LMissing descriptors for photos 1410837506 and 1410837510
LMissing descriptors for photos 1410837510 and 1410837511
LMissing descriptors for photos 1410837511 and 1410837533
LMissing descriptors for photos 1410837533 and 1410837584
LMissing descriptors for photos 1410837584 and 1410837599
LMissing descriptors for photos 1410837599 and 1410837708
LMissing descriptors for photos 1410837708 and 1410837709
LMissing descriptors for photos 1410837709 and 1410837710
LMissing descriptors for photos 1410837710 and 1410837711
LMissing descriptors for photos 1410837711 and 1410837712
LMissing descriptors for photos 1410837712 and 1410837713
LMissing descriptors for photos 1410837713 and 1410837787
LMissing descriptors for photos 1410837787 and 1410837841
LMissing descriptors for photos 1410837841 and 1410837895
LMissing descriptors for photos 1410837895 and 1410837951
LMissing descriptors for photos 1410837951 and 1410838036
LMissing descriptors for photos 1410838036 and 1410838136
LMissing descriptors for photos 1410838136 and 1410838437
LMissing descriptors for photos 1410838437 and 1410838438
LMissing descriptors for photos 1410838438 and 1410838439
LMissing descriptors for photos 1410838439 and 1410838440
LMissing descriptors for photos 1410838440 and 1410838441
LMissing descriptors for photos 1410838441 and 1410838442
LMissing descriptors for photos 1410838442 and 1410838503
LMissing descriptors for photos 1410838503 and 1410838558
LMissing descriptors for photos 1410838558 and 1410838614
LMissing descriptors for photos 1410838614 and 1410838672
LMissing descriptors for photos 1410838672 and 1410838726
LMissing descriptors for photos 1410838726 and 1410838781
LMissing descriptors for photos 1410838781 and 1410838878
LMissing descriptors for photos 1410838878 and 1410838913
LMissing descriptors for photos 1410838913 and 1410838929
LMissing descriptors for photos 1410838929 and 1410838931
LMissing descriptors for photos 1410838931 and 1410838933
LMissing descriptors for photos 1410838933 and 1410838935
LMissing descriptors for photos 1410838935 and 1410839226
LMissing descriptors for photos 1410839226 and 1410839235
LMissing descriptors for photos 1410839235 and 1410839237
LMissing descriptors for photos 1410839237 and 1410839239
LMissing descriptors for photos 1410839239 and 1410839241
LMissing descriptors for photos 1410839241 and 1410839242
LMissing descriptors for photos 1410839242 and 1410839244
LMissing descriptors for photos 1410839244 and 1410839246
LMissing descriptors for photos 1410839246 and 1410839247
LMissing descriptors for photos 1410839247 and 1410839248
LMissing descriptors for photos 1410839248 and 1410839250
LMissing descriptors for photos 1410839250 and 1410839266
LMissing descriptors for photos 1410839266 and 1410839317
LMissing descriptors for photos 1410839317 and 1410839319
LMissing descriptors for photos 1410839319 and 1410839320
LMissing descriptors for photos 1410839320 and 1410839336
LMissing descriptors for photos 1410839336 and 1410839337
LMissing descriptors for photos 1410839337 and 1410839338
LMissing descriptors for photos 1410839338 and 1410839343
LMissing descriptors for photos 1410839343 and 1410839344
LMissing descriptors for photos 1410839344 and 1410839345
LMissing descriptors for photos 1410839345 and 1410839346
LMissing descriptors for photos 1410839346 and 1410839347
LMissing descriptors for photos 1410839347 and 1410839348
LMissing descriptors for photos 1410839348 and 1410839362
LMissing descriptors for photos 1410839362 and 1410839369
LMissing descriptors for photos 1410839369 and 1410839384
LMissing descriptors for photos 1410839384 and 1410839387
LMissing descriptors for photos 1410839387 and 1410839388
LMissing descriptors for photos 1410839388 and 1410839389
LMissing descriptors for photos 1410839389 and 1410839412
LMissing descriptors for photos 1410839412 and 1410839466
LMissing descriptors for photos 1410839466 and 1410839528
LMissing descriptors for photos 1410839528 and 1410839638
LMissing descriptors for photos 1410839638 and 1410839694
LMissing descriptors for photos 1410839694 and 1410839749
LMissing descriptors for photos 1410839749 and 1410839794
LMissing descriptors for photos 1410839794 and 1410839837
LMissing descriptors for photos 1410839837 and 1410839849
LMissing descriptors for photos 1410839849 and 1410839854
LMissing descriptors for photos 1410839854 and 1410839856
LMissing descriptors for photos 1410839856 and 1410839860
LMissing descriptors for photos 1410839860 and 1410839862
LMissing descriptors for photos 1410839862 and 1410839864
LMissing descriptors for photos 1410839864 and 1410839867
LMissing descriptors for photos 1410839867 and 1410839870
LMissing descriptors for photos 1410839870 and 1410839872
LMissing descriptors for photos 1410839872 and 1410839873
LMissing descriptors for photos 1410839873 and 1410839884
LMissing descriptors for photos 1410839884 and 1410839887
LMissing descriptors for photos 1410839887 and 1410839888
LMissing descriptors for photos 1410839888 and 1410839889
LMissing descriptors for photos 1410839889 and 1410839890
LMissing descriptors for photos 1410839890 and 1410839893
LMissing descriptors for photos 1410839893 and 1410839898
LMissing descriptors for photos 1410839898 and 1410839899
LMissing descriptors for photos 1410839899 and 1410839900
LMissing descriptors for photos 1410839900 and 1410839902
LMissing descriptors for photos 1410839902 and 1410839904
LMissing descriptors for photos 1410839904 and 1410839910
LMissing descriptors for photos 1410839910 and 1410839915
LMissing descriptors for photos 1410839915 and 1410839917
LMissing descriptors for photos 1410839917 and 1410839920
LMissing descriptors for photos 1410839920 and 1410839922
LMissing descriptors for photos 1410839922 and 1410839923
LMissing descriptors for photos 1410839923 and 1410839924
LMissing descriptors for photos 1410839924 and 1410839926
LMissing descriptors for photos 1410839926 and 1410839927
LMissing descriptors for photos 1410839927 and 1410839928
LMissing descriptors for photos 1410839928 and 1410839929
LMissing descriptors for photos 1410839929 and 1410839930
LMissing descriptors for photos 1410839930 and 1410839931
LMissing descriptors for photos 1410839931 and 1410839933
LMissing descriptors for photos 1410839933 and 1410839934
LMissing descriptors for photos 1410839934 and 1410839935
LMissing descriptors for photos 1410839935 and 1410839936
LMissing descriptors for photos 1410839936 and 1410839937
LMissing descriptors for photos 1410839937 and 1410839938
LMissing descriptors for photos 1410839938 and 1410839940
LMissing descriptors for photos 1410839940 and 1410839941
LMissing descriptors for photos 1410839941 and 1410839942
LMissing descriptors for photos 1410839942 and 1410839943
LMissing descriptors for photos 1410839943 and 1410839944
LMissing descriptors for photos 1410839944 and 1410839945
LMissing descriptors for photos 1410839945 and 1410839947
LMissing descriptors for photos 1410839947 and 1410839948
LMissing descriptors for photos 1410839948 and 1410839951
LMissing descriptors for photos 1410839951 and 1410839953
LMissing descriptors for photos 1410839953 and 1410839954
LMissing descriptors for photos 1410839954 and 1410839955
L
04032026 Removing 0 photos because of the 'same image' condition
Total on : 0
Total off : 0.0
list_time_off
Warning in study_and_display_distrib_list : min=max : 0.0 0.0
dist_desc
Warning in study_and_display_distrib_list : min=max : -1 -1
begin to insert list_values into photo_hahstag_ids :
length of list_valuse in save_photo_hashtag_id_type : 143
time used for this insertion : 0.0315699577331543
photos_removed : len 0
elapsed_time : remove_photo_duplicate 0.06664681434631348
To do, maybe not at the correct place !
.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.L.Lforce hashtag to JRM
elapsed_time : CREATE_PORT_BATCH_BY_HOUR 0.009865522384643555
NUMBER BATCH : 2
list_ponderation used : [1e-05, 1e-05, 1e-05, 1e-05, 1e-05] , list_hashtag_class_create_as_list : ['jrm']
We have rejected 9 photos because of the batch_size condition !
NUMBER BATCH list_of_portfolios_to_create : 0
elapsed_time : count_nb_balles_and_create_portfolio 0.022099971771240234
# DISPLAY ALL COLLECTED DATA :
{'04032026': {'nb_upload': 143, 'nb_taggue_class': 137, 'nb_taggue_densite': 137, 'nb_descriptors': 0}}
Inside saveOutput : final : True verbose : 0
saveOutput not yet implemented for datou_step.type : split_time_score we use saveGeneral
[1410839955, 1410839954, 1410839953, 1410839951, 1410839948, 1410839947, 1410839945, 1410839944, 1410839943, 1410839942, 1410839941, 1410839940, 1410839938, 1410839937, 1410839936, 1410839935, 1410839934, 1410839933, 1410839931, 1410839930, 1410839929, 1410839928, 1410839927, 1410839926, 1410839924, 1410839923, 1410839922, 1410839920, 1410839917, 1410839915, 1410839910, 1410839904, 1410839902, 1410839900, 1410839899, 1410839898, 1410839893, 1410839890, 1410839889, 1410839888, 1410839887, 1410839884, 1410839873, 1410839872, 1410839870, 1410839867, 1410839864, 1410839862, 1410839860, 1410839856, 1410839854, 1410839849, 1410839837, 1410839794, 1410839749, 1410839694, 1410839638, 1410839528, 1410839466, 1410839412, 1410839389, 1410839388, 1410839387, 1410839384, 1410839369, 1410839362, 1410839348, 1410839347, 1410839346, 1410839345, 1410839344, 1410839343, 1410839338, 1410839337, 1410839336, 1410839320, 1410839319, 1410839317, 1410839266, 1410839250, 1410839248, 1410839247, 1410839246, 1410839244, 1410839242, 1410839241, 1410839239, 1410839237, 1410839235, 1410839226, 1410838935, 1410838933, 1410838931, 1410838929, 1410838913, 1410838878, 1410838781, 1410838726, 1410838672, 1410838614]
Looping around the photos to save general results
len do output : 1
/30620530Didn't retrieve data
.
before output type
Here is an output not treated by saveGeneral :