python /home/admin/mtr/script_for_cron.py -j datou_current3 -m 8 -a ' --fifo -a 533 -l 100 ' -s datou_current_533 -M 0 -S 0 -U 95,95,120 import MySQLdb succeeded Import error (python version) ['/Users/moilerat/Documents/Fotonower/install/caffe/distribute/python', '/home/admin/workarea/git/Velours/python/prod', '/home/admin/workarea/install/caffe_cuda8_python3/python', '/home/admin/workarea/install/darknet', '/home/admin/workarea/git/Velours/python', '/home/admin/workarea/install/caffe_frcnn_python3/py-faster-rcnn/caffe-fast-rcnn/python', '/home/admin/mtr/.credentials', '/home/admin/workarea/install/caffe/python', '/home/admin/workarea/install/caffe_frcnn/py-faster-rcnn/tools', '/home/admin/workarea/git/fotonowerpip', '/home/admin/workarea/install/segment-anything', '/home/admin/workarea/git/pyfvs', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/home/admin/.local/lib/python3.8/site-packages', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages'] process id : 3149432 load datou : 533 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! DataTypes for each output/input checked ! Unexpected type for variable list_input_json ERROR or WARNING : can't parse json string Expecting value: line 1 column 1 (char 0) Tried to parse : chemin de la photo was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? [ (photo_id, hashtag_id_0, score_0), (photo_id, hashtag_id_1, score_1), ...] was removed should we ? (photo_id, hashtag_id, score_max) was removed should we ? chemin de la photo was removed should we ? load thcls load pdts Running datou job : batch_current TODO datou_current to load to do maybe to take outside batchDatouExec no input labels no input values updating current state to 1 we have a portfolio with more photos than limit : 19272>100 please execute split_portfolio.py -i 23282116 -l 100 size over we load limit photo not treated list_input_json: {} Current got : datou_id : 533, datou_cur_ids : ['2939103'] with mtr_portfolio_ids : ['23282116'] and first list_photo_ids : [] new path : /proc/3149432/ Inside batchDatouExec : verbose : 0 # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! DataTypes for each output/input checked ! List Step Type Loaded in datou : tfhub_classification2, argmax list_input_json : {} origin We have 1 , WARNING: data may be incomplete, need to offset and complete ! BHTTP Error 404: Not Found can't download the photo : 1383487920 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487920_5c5e874c72253ee56e113524cd36b38b.jpg FBFBFBFBFBHTTP Error 404: Not Found can't download the photo : 1383487903 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487903_df683300d9fe99b6c73b366f732664e8.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487900 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487900_2540ca5f543407563f09729ab243aaf7.jpg FBFBFBHTTP Error 404: Not Found can't download the photo : 1383487893 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487893_f477deffdd25a7ba59184f9739279c1c.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487891 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487891_16b8a4ed8f297970e2c7a99587e4522b.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487887 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487887_66491560984ef31d3df65e7bd07061e4.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487865 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487865_d256e068710b98c2b3971b6a2a4ace0e.jpg FBFBHTTP Error 404: Not Found can't download the photo : 1383487857 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487857_bbf573abd114bf084182362484b8014b.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487855 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487855_d01d033c61923cd407656ed450d0cf94.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487853 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487853_a25dc64d11cf2136bb87dd3f09f3be74.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487850 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487850_cd1728ed581390bea0fee9bdc69fefef.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487848 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487848_2050c7925c5765d4db2d6c980d282cee.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487847 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487847_044a3e7967447c2a999ae08e04fc92b6.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487845 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487845_a236a6aad5fce40c481c2309961ca7f0.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487843 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487843_80507fab5d965d57f7f2e4cc80e5850f.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487841 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487841_fa9d98f56e443321ea2ee48c22b5446d.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487839 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487839_e59ab06e6447a48552908ba3cda52a16.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487813 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487813_101b23c3d60216bc6376c0cfb2540d8e.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487810 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487810_d561db64f2bca8d3ef14457120d0a6a1.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487807 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487807_6b822ee428d7502768bd570da058d4d1.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487805 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487805_96cc44270ffa96d0f198a2faec1392e7.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487803 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487803_ee699d40401ab52e0674cf6697e9e7ec.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487801 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487801_47e7d11d8980c219648966ecb3f74681.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487799 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487799_327ad0ee708d37aa22377577b2e8fe4b.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487797 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487797_579a81f5dd124a403e9f15bd08fa1078.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487795 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487795_acadbc1b89151a0b1425a36cdddd4630.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487793 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487793_d3ad6f80cfddd82bac4d566e6dbe2182.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487791 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487791_df303e80fad31b750a4560c84833727b.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487789 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487789_dd4bd2166f936f11b16421c014f2d128.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487770 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487770_0797c4e31a97754d9ed87883a1dba14c.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487767 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487767_19edac059beff51ae4c5c9c36c2fc605.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487764 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487764_d9ff9c0b22e17ffc2572fb435739f12c.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487761 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487761_f50a21ee1bf6acf1f9c637a0dbc5f12d.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487759 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487759_5b898c0d962a38b174b5fa1e497d9a82.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487757 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487757_3a9f8a31a9d1ac5e5f6629381508b366.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487753 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487753_aaa33277da43e4f944472aa0a4a92d72.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487751 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487751_0ba6500b7da8a377eb0724ee81daea7a.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487749 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487749_958b980fa6b006d4abc90a0f3e0c9525.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487747 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487747_d36d2d224ee8b2aabce39a589721b6fd.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487745 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487745_4cf9446157474d4d6c623d156b20b0ec.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487743 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487743_c3840574ec9ce8167119aa2265a70ae7.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487590 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487590_344feb163f1367b1bacb7b9241107042.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487588 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487588_b17e5c2481b101da76ba4ac7aacac834.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487586 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487586_3d019112a74853e379a1758afb7edc55.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487584 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487584_4cf2c92a04d57c61c3648c2483440bda.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487582 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487582_3ce865e84dab087665b73277e23bc9fd.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487579 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487579_6e45b685d7eaa63b5887302b366b8a73.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487554 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487554_2e58cef94ff7af8f9f193fc3763427e0.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487550 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487550_24342b11772d710badc8247431899676.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487547 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487547_0c6e0111508ebf6fa68feb8830445a67.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487544 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487544_cbac7fbd5b0715c55253b734c91e1a2d.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487542 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487542_d5c09a8202deb315156288b273d2962c.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487539 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487539_d300291dd176c4d93f6d7b4d49bb41ad.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487536 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487536_2b6276e1905742e5d319a5e680077e31.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487534 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487534_3ff5d1d0e57dfa66994c92d492467b77.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487532 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487532_06a137f1410b81339ae96e136ee19fbb.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487530 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487530_e815185ac5e736fc34a223c239f2fc29.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487528 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487528_384fd013f57d26c4eeebf056e90aa139.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487526 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487526_1a4ccc4be44275c8e606c224e9bad37e.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487502 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487502_35bf02c1ce9469b3f7cb41d642bfc4a2.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487499 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487499_b3e243e6c74a6a3d45d2f864f59f86a3.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487496 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487496_e97b3d6720643041ff35bb3a957f9927.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487493 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487493_aadc25fd4556d231f97414013eff1a9b.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487491 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487491_16276e322c39ae1855040cf77769d156.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487488 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487488_54a9d0952d7db4084c5e9f2cce4bd094.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487487 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487487_5db053249fc47db97fcd4fd52b52e89f.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487485 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487485_a886774edaa92544dbf648ffd3995abc.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487483 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487483_83bec87c59e6415c7f5e3103632e55f3.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487481 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487481_a0266ccb24275bb593a177a353c37d65.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487479 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487479_fcfc077e3a29f48332a5dea99c106c18.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487477 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487477_02cee694818b3dfe0c8f8c91f79e80bc.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487449 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487449_b29eac06521ef5ec079ae997fcca2ef6.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487446 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487446_a295c6162d2b5380f6fd8b26e55a8eab.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487443 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487443_0a5abb454ba536e9bb384fd277e39ac9.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487440 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487440_1fd27c174fd5cc95079694bb4c10a5f7.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487438 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487438_d1228aa797cffab97317df50b70527d7.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487435 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487435_93a7e4d4d941b693b05b1ce87d19813e.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487427 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487427_6b9cba443e7816a64e8976d048cb0ad8.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487425 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487425_b12e4ff393d90089e674f817c0c34392.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487423 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487423_6ff515fd617ad421dda0684019baf4ab.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487421 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487421_b653fcc1348925ac7726433bc85ae9df.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487419 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487419_715e8d8114affaa3956f46304cd9911c.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487417 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487417_07b3c93cb65319eb330340977150b2de.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487394 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487394_0b56b9f093c0154e1df2e79acfeefa60.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487391 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487391_730cfa30d4e44ff43b470d19ba807add.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487388 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487388_8e18fcace99cc3594ae377e2b95f2b89.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487385 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487385_c18c863e69829a2a620382b4b7225ee2.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487383 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487383_109fe274c718868fc642145ef8096c51.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487380 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487380_9d1fb4ab36164bcaf35cb6bcc34f16d5.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487377 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487377_d8b635ac366768a4a0a6eb59e3597452.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487375 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487375_e49f1719e494c2af899e231fbe5ea3f2.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487373 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487373_08801817f726c2a72eb34882a89283d8.jpg FBHTTP Error 404: Not Found can't download the photo : 1383487371 complete_path : /home/admin/workarea/git/Velours/python/temp/1757686768_3149432_1383487371_a045682f3a94c7fe2e499d79646741c3.jpg Fwe have missing 93 photos in the step downloads : photo missing : [1383487920, 1383487903, 1383487900, 1383487893, 1383487891, 1383487887, 1383487865, 1383487857, 1383487855, 1383487853, 1383487850, 1383487848, 1383487847, 1383487845, 1383487843, 1383487841, 1383487839, 1383487813, 1383487810, 1383487807, 1383487805, 1383487803, 1383487801, 1383487799, 1383487797, 1383487795, 1383487793, 1383487791, 1383487789, 1383487770, 1383487767, 1383487764, 1383487761, 1383487759, 1383487757, 1383487753, 1383487751, 1383487749, 1383487747, 1383487745, 1383487743, 1383487590, 1383487588, 1383487586, 1383487584, 1383487582, 1383487579, 1383487554, 1383487550, 1383487547, 1383487544, 1383487542, 1383487539, 1383487536, 1383487534, 1383487532, 1383487530, 1383487528, 1383487526, 1383487502, 1383487499, 1383487496, 1383487493, 1383487491, 1383487488, 1383487487, 1383487485, 1383487483, 1383487481, 1383487479, 1383487477, 1383487449, 1383487446, 1383487443, 1383487440, 1383487438, 1383487435, 1383487427, 1383487425, 1383487423, 1383487421, 1383487419, 1383487417, 1383487394, 1383487391, 1383487388, 1383487385, 1383487383, 1383487380, 1383487377, 1383487375, 1383487373, 1383487371] try to delete the photos missing in DB Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487920 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487903 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487900 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487893 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487891 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487887 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487865 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487857 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487855 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487853 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487850 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487848 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487847 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487845 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487843 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487841 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487839 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487813 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487810 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487807 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487805 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487803 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487801 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487799 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487797 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487795 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487793 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487791 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487789 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487770 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487767 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487764 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487761 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487759 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487757 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487753 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487751 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487749 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487747 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487745 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487743 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487590 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487588 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487586 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487584 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487582 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487579 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487554 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487550 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487547 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487544 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487542 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487539 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487536 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487534 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487532 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487530 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487528 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487526 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487502 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487499 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487496 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487493 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487491 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487488 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487487 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487485 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487483 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487481 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487479 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487477 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487449 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487446 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487443 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487440 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487438 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487435 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487427 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487425 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487423 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487421 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487419 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487417 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487394 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487391 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487388 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487385 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487383 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487380 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487377 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487375 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487373 Url missing : :https://www.fotonower.com/api/v1/photo/onerrorloadautomation/1383487371 length of list_filenames : 7 ; length of list_pids : 7 ; length of list_args : 7 time to download the photos : 46.52355122566223 About to test input to load we should then remove the video here, and this would fix the bug of datou_current ! Calling datou_exec Inside datou_exec : verbose : 0 number of steps : 2 step1:tfhub_classification2 Fri Sep 12 16:20:14 2025 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou_step TFHub with tf2 ! nombre de thcls : 3 we are using the classfication for multi_thcl [3513, 3379, 3847] begin to check gpu status inside check gpu memory 2025-09-12 16:20:17.527078: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2025-09-12 16:20:17.574652: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-12 16:20:17.574979: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-12 16:20:17.577120: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-12 16:20:17.579230: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-12 16:20:17.579543: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-12 16:20:17.581879: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-12 16:20:17.582937: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-12 16:20:17.587726: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-12 16:20:17.589392: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-12 16:20:17.589897: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2025-09-12 16:20:17.624574: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3492910000 Hz 2025-09-12 16:20:17.626222: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7f2640000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2025-09-12 16:20:17.626272: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2025-09-12 16:20:17.754831: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x3b59e060 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2025-09-12 16:20:17.754890: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2025-09-12 16:20:17.756182: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2025-09-12 16:20:17.756290: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-12 16:20:17.756320: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2025-09-12 16:20:17.756366: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2025-09-12 16:20:17.756397: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2025-09-12 16:20:17.756424: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2025-09-12 16:20:17.756472: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2025-09-12 16:20:17.756501: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2025-09-12 16:20:17.758662: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2025-09-12 16:20:17.758733: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2025-09-12 16:20:17.759994: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2025-09-12 16:20:17.760019: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2025-09-12 16:20:17.760034: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2025-09-12 16:20:17.762493: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 3096 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) l 3637 free memory gpu now : 10591 max_wait_temp : 1 max_wait : 5 1 Physical GPUs, 1 Logical GPUs tagging for thcl : 3513 To do loadFromThcl(), then load ParamDescType : thcl3513 thcls : [{'id': 3513, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2_tf', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 4557, 'photo_desc_type': 5767, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'}] thcl {'id': 3513, 'mtr_user_id': 31, 'name': 'Rungis_amount_dechets_fall_2018_v2_tf', 'pb_hashtag_id': 0, 'live': b'\x00', 'list_hashtags': '05102018_Papier_non_papier_dense,05102018_Papier_non_papier_peu_dense,05102018_Papier_non_papier_presque_vide,05102018_Papier_non_papier_tres_dense,05102018_Papier_non_papier_tres_peu_dense', 'svm_portfolios_learning': '1108385,1108386,1108388,1108384,1108387', 'photo_hashtag_type': 4557, 'photo_desc_type': 5767, 'type_classification': 'tf_classification2', 'hashtag_id_list': '2107751013,2107751014,2107751015,2107751016,2107751017'} Update svm_hashtag_type_desc : 5767 FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (5767, 'Rungis_amount_dechets_fall_2018_v2_tf', 2048, 2048, 'Rungis_amount_dechets_fall_2018_v2_tf', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 3, datetime.datetime(2023, 3, 16, 15, 52, 10), datetime.datetime(2023, 3, 16, 15, 52, 10)) model_name : Rungis_amount_dechets_fall_2018_v2_tf model_param file didn't exist ERROR in datou_step_exec, will save and exit ! Unauthorized. Check username/id, password, tenant name/id and user/tenant domain name/id. File "/home/admin/workarea/git/Velours/python/mtr/datou/datou_lib.py", line 2329, in datou_exec output = datou_step_exec(sNext, args, cache, context, map_info, verbose, mtr_user_id) File "/home/admin/workarea/git/Velours/python/mtr/datou/datou_lib.py", line 2523, in datou_step_exec return lib_process.datou_step_tfhub2(param, json_param, args, cache, context, map_info, verbose) File "/home/admin/workarea/git/Velours/python/mtr/datou/lib_step_exec/lib_step_process.py", line 3130, in datou_step_tfhub2 CDM = CacheModelData(0, os.environ["CAFFE_MODELS"], storage_server=storage_server) File "/home/admin/workarea/git/Velours/python/mtr/database_queries/CacheModelData_queries.py", line 58, in __init__ s3_desc.connect() File "/home/admin/workarea/git/Velours/python/mtr/utils/cdn/swift_upload_manager.py", line 41, in connect resp_headers, containers = self._conn.get_account() File "/usr/lib/python3/dist-packages/swiftclient/client.py", line 1804, in get_account return self._retry(None, get_account, marker=marker, limit=limit, File "/usr/lib/python3/dist-packages/swiftclient/client.py", line 1738, in _retry self.url, self.token = self.get_auth() File "/usr/lib/python3/dist-packages/swiftclient/client.py", line 1682, in get_auth self.url, self.token = get_auth(self.authurl, self.user, self.key, File "/usr/lib/python3/dist-packages/swiftclient/client.py", line 729, in get_auth storage_url, token = get_auth_keystone(auth_url, user, File "/usr/lib/python3/dist-packages/swiftclient/client.py", line 644, in get_auth_keystone raise ClientException(msg) [1383487914, 1383487911, 1383487908, 1383487905, 1383487898, 1383487895, 1383487860] begin to insert list_values into mtr_datou_result : length of list_values in save_final : 7 time used for this insertion : 0.01765608787536621 save_final ERROR in last step tfhub_classification2, Unauthorized. Check username/id, password, tenant name/id and user/tenant domain name/id. time spend for datou_step_exec : 3.535341739654541 time spend to save output : 0.01990652084350586 total time spend for step 0 : 3.555248260498047 caffe_path_current : About to save ! 2 After save, about to update current ! update_current_state 3.83user 2.92system 0:53.91elapsed 12%CPU (0avgtext+0avgdata 557156maxresident)k 2072inputs+6688outputs (54major+149586minor)pagefaults 0swaps