python /home/admin/mtr/script_for_cron.py -j default -m 20 -a 'pytest /home/admin/workarea/git/Velours/python/tests/test_migrate ' -s test_pytest -M 0 -S 0 -U 97,80,97 ============================= test session starts ============================== platform linux -- Python 3.8.10, pytest-7.4.3, pluggy-1.3.0 rootdir: /home/admin plugins: hydra-core-1.3.2, html-4.1.0, cov-4.1.0, anyio-3.7.1, metadata-3.0.0, xonsh-0.9.13 collected 18 items workarea/git/Velours/python/tests/test_migrate/pytest_cache_photo_data_test.py . [ 5%] . [ 11%] workarea/git/Velours/python/tests/test_migrate/pytest_cod_main_test.py . [ 16%] . [ 22%] workarea/git/Velours/python/tests/test_migrate/pytest_datou_local_test.py F [ 27%] . [ 33%] workarea/git/Velours/python/tests/test_migrate/pytest_datou_long_test.py F [ 38%] FF [ 50%] workarea/git/Velours/python/tests/test_migrate/pytest_datou_varroa_test.py F [ 55%] =================================== FAILURES =================================== ______________________________ test_local_export _______________________________ context = def test_local_export(context): """Test datou local export""" list_input_json = "" local_data = [os.path.join( os.environ["GITVELOURS"], "python/tests/data/datou_local/ref/face_test_ssd.jpg" )] contextDataFile_out = os.path.join( os.environ["GITVELOURS"], "python/tests/data/datou_local/out/datou_2200.json" ) contextDataFile_ref = os.path.join( os.environ["GITVELOURS"], "python/tests/data/datou_local/ref/datou_2200.json" ) limit = 0 offset = 0 rand = 0 datou_id = 2200 save = 0 ret, map_info = datouLocal( context, datou_id, local_data, list_input_json, limit, offset, save, verbose=False, contextDataFile=contextDataFile_out, export_or_import=1 ) # Load reference JSON with open(contextDataFile_ref, "r") as f: data = "".join(f.readlines()) import ast json_non_reg = ast.literal_eval(data) # Load output JSON with open(contextDataFile_out, "r") as g: data = "".join(g.readlines()) import ast check_json = ast.literal_eval(data) # Check JSON match > assert ordered(json_non_reg) == ordered(check_json) E AssertionError: assert [('datous', [...v2'), ...])])] == [('datous', [...v2'), ...])])] E At index 0 diff: ('datous', [(2200, [('depends_step', []), ('estimated_time', 0), ('id', 2200), ('limit_max', 100), ('list_steps', [[('id', 5351), ('is_datou_depend', True), ('is_local', True), ('is_photo_id_local', False), ('json_param', []), ('json_param_as_string', '{}'), ('list_data', []), ('nb_input', 1), ('nb_output', 2), ('param', '1978'), ('prod', False), ('type', 'ssd')]]), ('name', 'test_ssd_face_1704_v2'), ('nb_steps', 1)])]) != ('datous', [(2200, [('depends_step', []), ('estimated_time', 0), ('id', 2200), ('limit_max', 100), ('list_steps', [[('id', 5351), ('... E E ...Full output truncated (2 lines hidden), use '-vv' to show workarea/git/Velours/python/tests/test_migrate/pytest_datou_local_test.py:58: AssertionError ----------------------------- Captured stdout call ----------------------------- Inside batchDatouExec : verbose : False # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! FOUND : 1 Here is data_from_sql_as_vec to set the ParamDescriptorType : (4617, 'test_ssd_face_1704_v2', 16384, 25088, 'test_ssd_face_1704_v2', 'pool5', 10.0, None, None, 256, None, 0, None, 8, None, None, -1000.0, 1, datetime.datetime(2020, 4, 17, 12, 52, 31), datetime.datetime(2020, 4, 17, 12, 52, 31)) model_param file didn't exist model_name : test_ssd_face_1704_v2 model_type : caffe_faster_rcnn list file need : ['caffemodel', 'test.prototxt'] file exist in s3 : [] file manque in s3 : ['caffemodel', 'test.prototxt'] local folder : /data/models_weight/test_ssd_face_1704_v2 chemin de la photo was removed should we ? [ (photo_id_loc, hashtag_id, hashtag_type, x0, x1, y0, y1, score, None), ...] was removed should we ? chemin de la photo was removed should we ? Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! DataTypes for each output/input checked ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) DONE and to test : checkNoCycle ! We are managing only one step so we do not consider checkConsistencyNbInputNbOutput ! We are managing only one step so we do not consider checkConsistencyTypeOutputInput ! TODO load 1978 ##List Step Type Loaded in datou : ssd list_input_json : time to download the photos : 1.0728836059570312e-05 Inside datou_exec : verbose : False we use local cache db, so we are in local job, but when commit will be implemented for local cache db, we could again use save number of steps : 1 step1:ssd Wed Jan 7 13:18:05 2026 VR 17-11-17 : now, only for linear exec dependencies tree, some output goes to fill the input of the next VR 22-3-18 : now we test the dependencies tree, but keep two separate code for datou_prepare_output_input until the code is correctly tested, clean and works in both case VR 22-3-18 : but we use the first code for the first step id = -1, build in the code of datou_exec VR 22-3-18 : we should manage here the case when we are at the first step instead of building this step before datou_exec Beginning of datou step detection ssd ! To loadFromThcl() model_param file didn't exist model_name : test_ssd_face_1704_v2 model_type : tf_detection list file need : ['checkpoint', 'frozen_inference_graph.pb', 'model.ckpt.data-00000-of-00001', 'model.ckpt.index', 'model.ckpt.meta'] file exist in s3 : ['checkpoint', 'frozen_inference_graph.pb', 'model.ckpt.data-00000-of-00001', 'model.ckpt.index', 'model.ckpt.meta'] file manque in s3 : [] local folder : /data/models_weight/test_ssd_face_1704_v2/test /data/models_weight/test_ssd_face_1704_v2/test/checkpoint size_local : 77 size in s3 : 77 create time local : 2021-10-29 19:35:14 create time in s3 : 2021-08-06 21:09:52 checkpoint already exist and didn't need to update /data/models_weight/test_ssd_face_1704_v2/test/frozen_inference_graph.pb size_local : 16381495 size in s3 : 16381495 create time local : 2021-10-29 19:35:15 create time in s3 : 2021-08-06 21:09:53 frozen_inference_graph.pb already exist and didn't need to update /data/models_weight/test_ssd_face_1704_v2/test/model.ckpt.data-00000-of-00001 size_local : 15791780 size in s3 : 15791780 create time local : 2021-10-29 19:35:15 create time in s3 : 2021-08-06 21:09:51 model.ckpt.data-00000-of-00001 already exist and didn't need to update /data/models_weight/test_ssd_face_1704_v2/test/model.ckpt.index size_local : 15041 size in s3 : 15041 create time local : 2021-10-29 19:35:15 create time in s3 : 2021-08-06 21:09:53 model.ckpt.index already exist and didn't need to update /data/models_weight/test_ssd_face_1704_v2/test/model.ckpt.meta size_local : 1460828 size in s3 : 1460828 create time local : 2021-10-29 19:35:15 create time in s3 : 2021-08-06 21:09:51 model.ckpt.meta already exist and didn't need to update About to compute detect_ssd : len(args) : 1 Inside frcnn step exec : nb paths : 1 the count of objects is: 1 [([662, 1547, 574, 1634], 0.99965024, 1.0)] len de result ssd : 1 time spend for datou_step_exec : 7.519404649734497 time spend to save output : 3.0040740966796875e-05 total time spend for step 1 : 7.519434690475464 {0: [[(0, 492588132, 2447, 662, 1547, 574, 1634, 0.99965024, None)], '/home/admin/workarea/git/Velours/python/tests/data/datou_local/ref/face_test_ssd.jpg']} ----------------------------- Captured stderr call ----------------------------- 2026-01-07 13:18:09.153349: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1 2026-01-07 13:18:09.184884: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2026-01-07 13:18:09.185529: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2026-01-07 13:18:09.189353: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2026-01-07 13:18:09.191865: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2026-01-07 13:18:09.192933: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2026-01-07 13:18:09.196050: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2026-01-07 13:18:09.197413: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2026-01-07 13:18:09.202900: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2026-01-07 13:18:09.204686: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2026-01-07 13:18:09.205128: I tensorflow/core/platform/cpu_feature_guard.cc:143] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA 2026-01-07 13:18:09.230512: I tensorflow/core/platform/profile_utils/cpu_utils.cc:102] CPU Frequency: 3493010000 Hz 2026-01-07 13:18:09.231952: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x7fdebc000b60 initialized for platform Host (this does not guarantee that XLA will be used). Devices: 2026-01-07 13:18:09.232003: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): Host, Default Version 2026-01-07 13:18:09.439790: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x26c45820 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices: 2026-01-07 13:18:09.439842: I tensorflow/compiler/xla/service/service.cc:176] StreamExecutor device (0): NVIDIA GeForce RTX 2080 Ti, Compute Capability 7.5 2026-01-07 13:18:09.441199: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1561] Found device 0 with properties: pciBusID: 0000:41:00.0 name: NVIDIA GeForce RTX 2080 Ti computeCapability: 7.5 coreClock: 1.545GHz coreCount: 68 deviceMemorySize: 10.76GiB deviceMemoryBandwidth: 573.69GiB/s 2026-01-07 13:18:09.441276: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2026-01-07 13:18:09.441307: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 2026-01-07 13:18:09.441334: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10 2026-01-07 13:18:09.441361: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10 2026-01-07 13:18:09.441387: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10 2026-01-07 13:18:09.441413: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10 2026-01-07 13:18:09.441439: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2026-01-07 13:18:09.443560: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1703] Adding visible gpu devices: 0 2026-01-07 13:18:09.443628: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 2026-01-07 13:18:09.444808: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102] Device interconnect StreamExecutor with strength 1 edge matrix: 2026-01-07 13:18:09.444851: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1108] 0 2026-01-07 13:18:09.444868: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1121] 0: N 2026-01-07 13:18:09.447099: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1247] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 10034 MB memory) -> physical GPU (device: 0, name: NVIDIA GeForce RTX 2080 Ti, pci bus id: 0000:41:00.0, compute capability: 7.5) 2026-01-07 13:18:11.390570: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7 2026-01-07 13:18:12.580403: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10 ______________________________ test_datou_varroa _______________________________ context = def test_datou_varroa(context): """Test varroa datou""" datou_id = 39 list_photo_ids = ["947246065"] mtr_porfolio_ids = [] limit = 0 offset = 0 rand = 0 list_input_json = "168, 0,0,victor@fotonower.com" ret = batchDatouExec( context, datou_id, mtr_porfolio_ids, list_photo_ids, list_input_json, limit, offset, rand, 2, False ) ret_non_reg = {947246065: [[ 'u948231305', 'u948231304', 'u948231307', 'u948231306', 'u948231301', 'u948231300', 'u948231303', 'u948231302', 'u948231288', 'u948231289', 'u948231284', 'u948231285', 'u948231286', 'u948231287', 'u948231280', 'u948231281', 'u948231282', 'u948231283', 'u948231293', 'u948231292', 'u948231299', 'u948231298', 'u948231279', 'u948231278', 'u948231291', 'u948231290', 'u948231297', 'u948231296', 'u948231295', 'u948231294' ], [ 'temp/1532085081_21437_947246065_6969857b9a2e18758c1684c02f5983fc_resize_7_bib_crop_15726715_0.jpg', 'temp/1532085081_21437_947246065_6969857b9a2e18758c1684c02f5983fc_resize_bib_crop_15722621_0.jpg', # ... (truncated for brevity) ]]} > assert ret is not None E assert None is not None workarea/git/Velours/python/tests/test_migrate/pytest_datou_long_test.py:34: AssertionError ----------------------------- Captured stdout call ----------------------------- Inside batchDatouExec : verbose : False # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! Step 102 frcnn have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of outputs for step 103 glue_tag is not consistent : 3 used against 1 in the step definition ! Step 461 tile have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 462 crop have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 543 argmax have less outputs used (1) than in the step definition (2) : some outputs may be not used ! WARNING : number of inputs for step 383 count is not consistent : 5 used against 3 in the step definition ! Step 383 count have less outputs used (0) than in the step definition (1) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : output 1 of step 108 have datatype=1 whereas input 0 of step 383 have datatype=2 WARNING : type of output 1 of step 103 doesn't seem to be define in the database( WARNING : output 1 of step 103 have datatype=None whereas input 0 of step 462 have datatype=9 WARNING : type of input 4 of step 383 doesn't seem to be define in the database( WARNING : output 0 of step 462 have datatype=1 whereas input 4 of step 383 have datatype=None We ignore checkConsistencyTypeOutputInput for datou_step group_by ! WARNING : type of output 2 of step 103 doesn't seem to be define in the database( WARNING : type of input 3 of step 461 doesn't seem to be define in the database( WARNING : type of input 3 of step 383 doesn't seem to be define in the database( WARNING : output 0 of step 543 have datatype=6 whereas input 3 of step 383 have datatype=None DataTypes for each output/input checked ! List Step Type Loaded in datou : init_tile_detect_glue, tile, frcnn, glue_tag, tile, crop, thcl, argmax, count, group_by list_input_json : 168, 0,0,victor@fotonower.com origin maybe url didn't existe for photo_id 947246065 , please check it all photo missing in this thread we have missing 1 photos in the step downloads : photo missing : [947246065] try to delete the photos missing in DB HTTP Error 500: Internal Server Error length of list_filenames : 0 ; length of list_pids : 0 ; length of list_args : 0 time to download the photos : 34.60846257209778 About to test input to load empty input or other problem ret : 0 no datou_current to update __________________________ test_pipeline_carte_grise ___________________________ context = def test_pipeline_carte_grise(context): """Test pipeline carte grise""" datou_id = 188 list_photo_ids = ["929082694", "930325495"] mtr_porfolio_ids = [] limit = 0 offset = 0 rand = 0 list_input_json = [] ret = batchDatouExec( context, datou_id, mtr_porfolio_ids, list_photo_ids, list_input_json, limit, offset, rand, 2, False ) ret_non_reg = {929082694: [], 930325495: []} > assert len(ret) == len(ret_non_reg) E TypeError: object of type 'NoneType' has no len() workarea/git/Velours/python/tests/test_migrate/pytest_datou_long_test.py:58: TypeError ----------------------------- Captured stdout call ----------------------------- Inside batchDatouExec : verbose : False # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! Step 243 rotate have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 207 argmax have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 424 crop have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 209 crop have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 210 tesseract have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 444 group_by have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Step 443 group_by have less outputs used (2) than in the step definition (3) : some outputs may be not used ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step group_by ! We ignore checkConsistencyTypeOutputInput for datou_step group_by ! We ignore checkConsistencyTypeOutputInput for datou_step group_by ! We ignore checkConsistencyTypeOutputInput for datou_step group_by ! We ignore checkConsistencyTypeOutputInput for datou_step group_by ! DataTypes for each output/input checked ! List Step Type Loaded in datou : thcl, argmax, rotate, thcl, argmax, frcnn, crop, frcnn, crop, image_processing, tesseract, group_by, group_by, final list_input_json : [] origin we have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 0 ; length of list_pids : 0 ; length of list_args : 0 time to download the photos : 0.015757083892822266 About to test input to load empty input or other problem ret : 0 no datou_current to update _____________________________ test_pipeline_papier _____________________________ context = def test_pipeline_papier(context): """Test pipeline papier""" datou_id = 370 list_photo_ids = ["947557587", "947559224"] mtr_portfolio_ids = [] limit = 0 offset = 0 rand = 0 list_input_json = [] ret = batchDatouExec( context, datou_id, mtr_portfolio_ids, list_photo_ids, list_input_json, limit, offset, rand, 2, False ) # Basic validation that ret is not None > assert ret is not None E assert None is not None workarea/git/Velours/python/tests/test_migrate/pytest_datou_long_test.py:79: AssertionError ----------------------------- Captured stdout call ----------------------------- Inside batchDatouExec : verbose : False # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! DataTypes for each output/input checked ! List Step Type Loaded in datou : mask_detect, thcl, argmax, mask_detect, final list_input_json : [] origin we have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 0 ; length of list_pids : 0 ; length of list_args : 0 time to download the photos : 0.018402814865112305 About to test input to load empty input or other problem ret : 0 no datou_current to update _____________________________ test_datou_varroa38 ______________________________ context = def test_datou_varroa38(context): """Test varroa datou 38""" datou_id = 38 list_photo_ids = ["983612311"] mtr_porfolio_ids = [] limit = 0 offset = 0 rand = 0 list_input_json = ",0,-1,aucun,0,0,125" ret = batchDatouExec( context, datou_id, mtr_porfolio_ids, list_photo_ids, list_input_json, limit, offset, rand, 2, False ) photo_id = 983612311 ret_non_reg = {photo_id: {'varroa': 4}} > assert len(ret) == len(ret_non_reg) E TypeError: object of type 'NoneType' has no len() workarea/git/Velours/python/tests/test_migrate/pytest_datou_varroa_test.py:25: TypeError ----------------------------- Captured stdout call ----------------------------- Inside batchDatouExec : verbose : False # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! Step 3473 tile have less inputs used (1) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 3471 detect_points is not consistent : 2 used against 1 in the step definition ! Step 3475 tile have less inputs used (1) than in the step definition (3) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 3477 argmax is not consistent : 3 used against 2 in the step definition ! WARNING : number of inputs for step 3474 count_final is not consistent : 4 used against 2 in the step definition ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 1 of step 3471 doesn't seem to be define in the database( WARNING : type of input 2 of step 3474 doesn't seem to be define in the database( WARNING : type of output 2 of step 3477 doesn't seem to be define in the database( WARNING : type of input 3 of step 3474 doesn't seem to be define in the database( DataTypes for each output/input checked ! List Step Type Loaded in datou : init_detect_points, tile, detect_points, tile, thcl, argmax, count_final list_input_json : ,0,-1,aucun,0,0,125 origin we have missing 0 photos in the step downloads : photo missing : [] try to delete the photos missing in DB length of list_filenames : 0 ; length of list_pids : 0 ; length of list_args : 0 time to download the photos : 0.01768040657043457 About to test input to load empty input or other problem ret : 0 no datou_current to update =============================== warnings summary =============================== ../../usr/lib/python3/dist-packages/babel/localedata.py:17 /usr/lib/python3/dist-packages/babel/localedata.py:17: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.10 it will stop working from collections import MutableMapping workarea/git/Velours/python/tests/test_migrate/pytest_datou_local_test.py::test_local_export workarea/git/Velours/python/tests/test_migrate/pytest_datou_local_test.py::test_local_export workarea/git/Velours/python/tests/test_migrate/pytest_datou_local_test.py::test_local_export workarea/git/Velours/python/tests/test_migrate/pytest_datou_local_test.py::test_local_export workarea/git/Velours/python/tests/test_migrate/pytest_datou_varroa_test.py::test_datou_varroa_local workarea/git/Velours/python/tests/test_migrate/pytest_datou_varroa_test.py::test_datou_varroa_local /usr/lib/python3/dist-packages/keystoneclient/v3/client.py:218: DeprecationWarning: Using the 'tenant_name' argument is deprecated in version '1.7.0' and will be removed in version '2.0.0', please use the 'project_name' argument instead super(Client, self).__init__(**kwargs) workarea/git/Velours/python/tests/test_migrate/pytest_datou_local_test.py::test_local_export workarea/git/Velours/python/tests/test_migrate/pytest_datou_local_test.py::test_local_export workarea/git/Velours/python/tests/test_migrate/pytest_datou_local_test.py::test_local_export workarea/git/Velours/python/tests/test_migrate/pytest_datou_local_test.py::test_local_export workarea/git/Velours/python/tests/test_migrate/pytest_datou_varroa_test.py::test_datou_varroa_local workarea/git/Velours/python/tests/test_migrate/pytest_datou_varroa_test.py::test_datou_varroa_local /usr/lib/python3/dist-packages/debtcollector/renames.py:43: DeprecationWarning: Using the 'tenant_id' argument is deprecated in version '1.7.0' and will be removed in version '2.0.0', please use the 'project_id' argument instead return wrapped(*args, **kwargs) workarea/git/Velours/python/tests/test_migrate/pytest_datou_local_test.py::test_local_export workarea/git/Velours/python/tests/test_migrate/pytest_datou_local_test.py::test_local_export workarea/git/Velours/python/tests/test_migrate/pytest_datou_local_test.py::test_local_export workarea/git/Velours/python/tests/test_migrate/pytest_datou_local_test.py::test_local_export workarea/git/Velours/python/tests/test_migrate/pytest_datou_varroa_test.py::test_datou_varroa_local workarea/git/Velours/python/tests/test_migrate/pytest_datou_varroa_test.py::test_datou_varroa_local /usr/lib/python3/dist-packages/keystoneclient/httpclient.py:363: DeprecationWarning: Constructing an HTTPClient instance without using a session is deprecated as of the 1.7.0 release and may be removed in the 2.0.0 release. warnings.warn( workarea/git/Velours/python/tests/test_migrate/pytest_datou_local_test.py::test_local_export workarea/git/Velours/python/tests/test_migrate/pytest_datou_local_test.py::test_local_export workarea/git/Velours/python/tests/test_migrate/pytest_datou_local_test.py::test_local_export workarea/git/Velours/python/tests/test_migrate/pytest_datou_local_test.py::test_local_export workarea/git/Velours/python/tests/test_migrate/pytest_datou_varroa_test.py::test_datou_varroa_local workarea/git/Velours/python/tests/test_migrate/pytest_datou_varroa_test.py::test_datou_varroa_local /usr/lib/python3/dist-packages/keystoneclient/session.py:139: DeprecationWarning: keystoneclient.session.Session is deprecated as of the 2.1.0 release in favor of keystoneauth1.session.Session. It will be removed in future releases. warnings.warn( workarea/git/Velours/python/tests/test_migrate/pytest_datou_local_test.py::test_local_export workarea/git/Velours/python/tests/test_migrate/pytest_datou_local_test.py::test_local_export workarea/git/Velours/python/tests/test_migrate/pytest_datou_local_test.py::test_local_export workarea/git/Velours/python/tests/test_migrate/pytest_datou_local_test.py::test_local_export workarea/git/Velours/python/tests/test_migrate/pytest_datou_varroa_test.py::test_datou_varroa_local workarea/git/Velours/python/tests/test_migrate/pytest_datou_varroa_test.py::test_datou_varroa_local /usr/lib/python3/dist-packages/keystoneclient/v3/client.py:221: DeprecationWarning: Constructing an instance of the keystoneclient.v3.client.Client class without a session is deprecated as of the 1.7.0 release and may be removed in the 2.0.0 release. warnings.warn( workarea/git/Velours/python/tests/test_migrate/pytest_datou_local_test.py::test_local_export workarea/git/Velours/python/tests/test_migrate/pytest_datou_local_test.py::test_local_export workarea/git/Velours/python/tests/test_migrate/pytest_datou_local_test.py::test_local_export workarea/git/Velours/python/tests/test_migrate/pytest_datou_local_test.py::test_local_export workarea/git/Velours/python/tests/test_migrate/pytest_datou_varroa_test.py::test_datou_varroa_local workarea/git/Velours/python/tests/test_migrate/pytest_datou_varroa_test.py::test_datou_varroa_local /usr/lib/python3/dist-packages/keystoneclient/auth/identity/base.py:51: DeprecationWarning: keystoneclient auth plugins are deprecated as of the 2.1.0 release in favor of keystoneauth1 plugins. They will be removed in future releases. warnings.warn( workarea/git/Velours/python/tests/test_migrate/pytest_datou_local_test.py::test_local_export /usr/local/lib/python3.8/dist-packages/tensorflow/python/pywrap_tensorflow_internal.py:15: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses import imp workarea/git/Velours/python/tests/test_migrate/pytest_datou_local_test.py::test_local_export /usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/dtypes.py:513: DeprecationWarning: `np.object` is a deprecated alias for the builtin `object`. To silence this warning, use `object` by itself. Doing this will not modify any behavior and is safe. Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations np.object, workarea/git/Velours/python/tests/test_migrate/pytest_datou_local_test.py::test_local_export /usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/dtypes.py:521: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. To silence this warning, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here. Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations np.bool, workarea/git/Velours/python/tests/test_migrate/pytest_datou_local_test.py::test_local_export /usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/dtypes.py:555: DeprecationWarning: `np.object` is a deprecated alias for the builtin `object`. To silence this warning, use `object` by itself. Doing this will not modify any behavior and is safe. Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations np.object, workarea/git/Velours/python/tests/test_migrate/pytest_datou_local_test.py::test_local_export /usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/dtypes.py:565: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. To silence this warning, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here. Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations np.bool, workarea/git/Velours/python/tests/test_migrate/pytest_datou_local_test.py::test_local_export :219: RuntimeWarning: numpy.ndarray size changed, may indicate binary incompatibility. Expected 88, got 80 workarea/git/Velours/python/tests/test_migrate/pytest_datou_local_test.py::test_local_export /usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/tensor_util.py:106: DeprecationWarning: `np.object` is a deprecated alias for the builtin `object`. To silence this warning, use `object` by itself. Doing this will not modify any behavior and is safe. Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations np.object: workarea/git/Velours/python/tests/test_migrate/pytest_datou_local_test.py::test_local_export /usr/local/lib/python3.8/dist-packages/tensorflow/python/framework/tensor_util.py:108: DeprecationWarning: `np.bool` is a deprecated alias for the builtin `bool`. To silence this warning, use `bool` by itself. Doing this will not modify any behavior and is safe. If you specifically wanted the numpy scalar type, use `np.bool_` here. Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations np.bool: workarea/git/Velours/python/tests/test_migrate/pytest_datou_local_test.py::test_local_export :219: RuntimeWarning: numpy.ndarray size changed, may indicate binary incompatibility. Expected 80 from C header, got 88 from PyObject workarea/git/Velours/python/tests/test_migrate/pytest_datou_local_test.py::test_local_export /usr/local/lib/python3.8/dist-packages/h5py/__init__.py:46: DeprecationWarning: `np.typeDict` is a deprecated alias for `np.sctypeDict`. from ._conv import register_converters as _register_converters workarea/git/Velours/python/tests/test_migrate/pytest_datou_local_test.py::test_local_export /usr/local/lib/python3.8/dist-packages/scipy/fft/__init__.py:97: DeprecationWarning: The module numpy.dual is deprecated. Instead of using dual, use the functions directly from numpy or scipy. from numpy.dual import register_func workarea/git/Velours/python/tests/test_migrate/pytest_datou_local_test.py: 15 warnings /usr/local/lib/python3.8/dist-packages/scipy/sparse/sputils.py:17: DeprecationWarning: `np.typeDict` is a deprecated alias for `np.sctypeDict`. supported_dtypes = [np.typeDict[x] for x in supported_dtypes] workarea/git/Velours/python/tests/test_migrate/pytest_datou_varroa_test.py::test_datou_varroa_local workarea/git/Velours/python/tests/test_migrate/pytest_datou_varroa_test.py::test_datou_varroa_local /usr/local/lib/python3.8/dist-packages/scipy/special/orthogonal.py:81: DeprecationWarning: `np.int` is a deprecated alias for the builtin `int`. To silence this warning, use `int` by itself. Doing this will not modify any behavior and is safe. When replacing `np.int`, you may wish to use e.g. `np.int64` or `np.int32` to specify the precision. If you wish to review your current use, check the release note link for additional information. Deprecated in NumPy 1.20; for more details and guidance: https://numpy.org/devdocs/release/1.20.0-notes.html#deprecations from numpy import (exp, inf, pi, sqrt, floor, sin, cos, around, int, workarea/git/Velours/python/tests/test_migrate/pytest_datou_varroa_test.py::test_datou_varroa_local workarea/git/Velours/python/tests/test_migrate/pytest_datou_varroa_test.py::test_datou_varroa_local workarea/git/Velours/python/tests/test_migrate/pytest_datou_varroa_test.py::test_datou_varroa_local workarea/git/Velours/python/tests/test_migrate/pytest_datou_varroa_test.py::test_datou_varroa_local /usr/local/lib/python3.8/dist-packages/skimage/util/dtype.py:226: DeprecationWarning: Converting `np.inexact` or `np.floating` to a dtype is deprecated. The current result is `float64` which is not strictly correct. dtypeobj_out = np.dtype(dtype) -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html =========================== short test summary info ============================ FAILED workarea/git/Velours/python/tests/test_migrate/pytest_datou_local_test.py::test_local_export FAILED workarea/git/Velours/python/tests/test_migrate/pytest_datou_long_test.py::test_datou_varroa FAILED workarea/git/Velours/python/tests/test_migrate/pytest_datou_long_test.py::test_pipeline_carte_grise FAILED workarea/git/Velours/python/tests/test_migrate/pytest_datou_long_test.py::test_pipeline_papier FAILED workarea/git/Velours/python/tests/test_migrate/pytest_datou_varroa_test.py::test_datou_varroa38 !!!!!!!!!!!!!!!!!!!!!!!!!!!!!! KeyboardInterrupt !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! /home/admin/workarea/git/Velours/python/mtr/datou/lib_step_exec/lib_step_util.py:201: KeyboardInterrupt (to show a full traceback on KeyboardInterrupt use --full-trace) ============= 5 failed, 5 passed, 69 warnings in 758.73s (0:12:38) ============= Command exited with non-zero status 2 23.30user 20.55system 12:42.15elapsed 5%CPU (0avgtext+0avgdata 7469380maxresident)k 615528inputs+776outputs (3162major+3233336minor)pagefaults 0swaps