python /home/admin/mtr/script_for_cron.py -j default -m 20 -a 'python3 ~/workarea/git/Velours/python/prod/memo/memo.py --job=sla_per_day --raspi=20084 --dashboard_places=dechet_Valcor_v2 --last_days 7 -u_s' -s sla_RUBBIA -M 0 -S 0 -U 90,80,95 import MySQLdb succeeded Import error (python version) Need env var : LOGRASPI Raspi : 20084, Date : _cr_cnf_ : /home/admin/mtr/.credentials/credentials.py _cr_cnf_ : /home/admin/mtr/.credentials/credentials.py warning , we can't find thcl infos in json_data warning , we can't find pdt infos in json_data VR TODO TOCHECK : due to this bug it shouldn't have being able to work, or maybe it was due to a change directory not done ! VR TODO TOCHECK : due to this bug it shouldn't have being able to work, or maybe it was due to a change directory not done ! # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11939 matching_dashboard is not consistent : 1 used against 0 in the step definition ! Step 11940 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 0 of step 11939 doesn't seem to be define in the database( WARNING : type of input 2 of step 11940 doesn't seem to be define in the database( DataTypes for each output/input checked ! We expected an STS datou with a camera_place_name, so there is a PB, datou_id: 4311 ! None was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? {'id': '4311', 'name': 'sts_dechet_valcor_matching_v2_2022Q3', 'limit_max': 100, 'nb_steps': 2, 'estimated_time': 8640, 'list_steps': [{'id': 11939, 'type': 'matching_dashboard', 'param': '0', 'json_param_as_string': "{'camera_matching': 'plaque_Valcor', 'max_time_diff': 120, 'matching_method': '1_vs_n', 'input_default_values': []}", 'json_param': {'camera_matching': 'plaque_Valcor', 'max_time_diff': 120, 'matching_method': '1_vs_n', 'input_default_values': []}, 'nb_input': 1, 'nb_output': 0, 'prod': False, 'is_local': False, 'is_datou_depend': False, 'is_photo_id_local': False, 'list_data': [{'step_id': 11939, 'datou_id': 4311, 'type_id': 18, 'name': 'portfolio_id', 'output_or_input': False, 'data_order_id': 0, 'type_python': 'int'}]}, {'id': 11940, 'type': 'split_time_score', 'param': '0', 'json_param_as_string': "{'camera_place_name': 'dechet_Valcor_v2', 'ETA': 8640, 'param_moyenne_mobile': 5, 'thcl': 3442, 'list_task_complete': [''], 'list_tasks': 'LOAD,ORDER,CONVENTION,SPLIT_TRUCK,merge_port_according_to_plate_info,remove_no_plate_batch,CREATE_PORT_LAUNCH_CARAC_FILL_DASHBOARD', 'forced_hashtag': 'truck', 'list_hashtag_class_create': 'truck', 'ponderation': '0.001,0.001,0.001,0.001,0.001', 'host': 'www.fotonower.com', 'datou_proportion_papier': {'test': 370, 'truck': 4234}, 'datou_matching_id': 4311, 'cod': 1, 'min_number_photo_per_batch': 2, 'datou_truck_for_info': 4234}", 'json_param': {'camera_place_name': 'dechet_Valcor_v2', 'ETA': 8640, 'param_moyenne_mobile': 5, 'thcl': 3442, 'list_task_complete': [''], 'list_tasks': 'LOAD,ORDER,CONVENTION,SPLIT_TRUCK,merge_port_according_to_plate_info,remove_no_plate_batch,CREATE_PORT_LAUNCH_CARAC_FILL_DASHBOARD', 'forced_hashtag': 'truck', 'list_hashtag_class_create': 'truck', 'ponderation': '0.001,0.001,0.001,0.001,0.001', 'host': 'www.fotonower.com', 'datou_proportion_papier': {'test': 370, 'truck': 4234}, 'datou_matching_id': 4311, 'cod': 1, 'min_number_photo_per_batch': 2, 'datou_truck_for_info': 4234}, 'nb_input': 2, 'nb_output': 1, 'prod': False, 'is_local': False, 'is_datou_depend': False, 'is_photo_id_local': False, 'list_data': [{'step_id': 11940, 'datou_id': 4311, 'type_id': 18, 'name': 'portfolio_id', 'output_or_input': False, 'data_order_id': 0, 'type_python': 'int'}, {'step_id': 11940, 'datou_id': 4311, 'type_id': 10, 'name': 'data_string', 'output_or_input': False, 'data_order_id': 1, 'type_python': 'string'}, {'step_id': 11940, 'datou_id': 4311, 'type_id': 22, 'name': 'portfolios_ids_list', 'output_or_input': True, 'data_order_id': 0, 'type_python': 'liste_de_int'}]}], 'depends_step': [{'output': 11939, 'output_id': 0, 'input': 11940, 'input_id': 2}]} check if it is the case where we have multiple sts in one datou no, it is not this case # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! DataTypes for each output/input checked ! Unexpected error treated as WARNING for now expected given param in final step {'min_score': 0.7, 'name_pipeline': 'proportion_papier_tapis', 'hashtag_type': 705, 'hashtag_proportion': 492668766, 'hashtag_parmi': 538914404, 'hashtag_to_consider': 'papier_only', 'hashtag_background': 493012381} Found hashtag_parmi in final step, will be used and useful if different from matiere_majoritaire in classifier ventilate_hashtags_in_portfolio step not found ! # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! Unexpected error treated as WARNING for now expected given param in final step {'name_pipeline': 'aggregate_carac_ratio', 'hashtag_type': 4230, 'hashtag_proportion': 'papier,carton,metal,pet_clair,autre,pehd,pet_fonce', 'hashtag_parmi': 'refus', 'hashtag_weights': {'autre': 3.0, 'barquette_avec_film': 3.0, 'carton': 3.0, 'metal': 3.0, 'papier': 3.0, 'pehd': 3.0, 'pet_fonce': 3.0, 'pet_clair': 1.0, 'flou': 1.0, 'mal_croppe': 1.0}, 'option_sccu': 1} Found hashtag_parmi in final step, will be used and useful if different from matiere_majoritaire in classifier TODO : Insert select and so on # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11939 matching_dashboard is not consistent : 1 used against 0 in the step definition ! Step 11940 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! WARNING : type of output 0 of step 11939 doesn't seem to be define in the database( WARNING : type of input 2 of step 11940 doesn't seem to be define in the database( DataTypes for each output/input checked ! We expected an STS datou with a camera_place_name, so there is a PB, datou_id: 4311 ! None was removed should we ? None was removed should we ? donnée sous forme de texte was removed should we ? [ptf_id0,ptf_id1...] was removed should we ? {'id': 4311, 'name': 'sts_dechet_valcor_matching_v2_2022Q3', 'limit_max': 100, 'nb_steps': 2, 'estimated_time': 8640, 'list_steps': [{'id': 11939, 'type': 'matching_dashboard', 'param': '0', 'json_param_as_string': "{'camera_matching': 'plaque_Valcor', 'max_time_diff': 120, 'matching_method': '1_vs_n', 'input_default_values': []}", 'json_param': {'camera_matching': 'plaque_Valcor', 'max_time_diff': 120, 'matching_method': '1_vs_n', 'input_default_values': []}, 'nb_input': 1, 'nb_output': 0, 'prod': False, 'is_local': False, 'is_datou_depend': False, 'is_photo_id_local': False, 'list_data': [{'step_id': 11939, 'datou_id': 4311, 'type_id': 18, 'name': 'portfolio_id', 'output_or_input': False, 'data_order_id': 0, 'type_python': 'int'}]}, {'id': 11940, 'type': 'split_time_score', 'param': '0', 'json_param_as_string': "{'camera_place_name': 'dechet_Valcor_v2', 'ETA': 8640, 'param_moyenne_mobile': 5, 'thcl': 3442, 'list_task_complete': [''], 'list_tasks': 'LOAD,ORDER,CONVENTION,SPLIT_TRUCK,merge_port_according_to_plate_info,remove_no_plate_batch,CREATE_PORT_LAUNCH_CARAC_FILL_DASHBOARD', 'forced_hashtag': 'truck', 'list_hashtag_class_create': 'truck', 'ponderation': '0.001,0.001,0.001,0.001,0.001', 'host': 'www.fotonower.com', 'datou_proportion_papier': {'test': 370, 'truck': 4234}, 'datou_matching_id': 4311, 'cod': 1, 'min_number_photo_per_batch': 2, 'datou_truck_for_info': 4234}", 'json_param': {'camera_place_name': 'dechet_Valcor_v2', 'ETA': 8640, 'param_moyenne_mobile': 5, 'thcl': 3442, 'list_task_complete': [''], 'list_tasks': 'LOAD,ORDER,CONVENTION,SPLIT_TRUCK,merge_port_according_to_plate_info,remove_no_plate_batch,CREATE_PORT_LAUNCH_CARAC_FILL_DASHBOARD', 'forced_hashtag': 'truck', 'list_hashtag_class_create': 'truck', 'ponderation': '0.001,0.001,0.001,0.001,0.001', 'host': 'www.fotonower.com', 'datou_proportion_papier': {'test': 370, 'truck': 4234}, 'datou_matching_id': 4311, 'cod': 1, 'min_number_photo_per_batch': 2, 'datou_truck_for_info': 4234}, 'nb_input': 2, 'nb_output': 1, 'prod': False, 'is_local': False, 'is_datou_depend': False, 'is_photo_id_local': False, 'list_data': [{'step_id': 11940, 'datou_id': 4311, 'type_id': 18, 'name': 'portfolio_id', 'output_or_input': False, 'data_order_id': 0, 'type_python': 'int'}, {'step_id': 11940, 'datou_id': 4311, 'type_id': 10, 'name': 'data_string', 'output_or_input': False, 'data_order_id': 1, 'type_python': 'string'}, {'step_id': 11940, 'datou_id': 4311, 'type_id': 22, 'name': 'portfolios_ids_list', 'output_or_input': True, 'data_order_id': 0, 'type_python': 'liste_de_int'}]}], 'depends_step': [{'output': 11939, 'output_id': 0, 'input': 11940, 'input_id': 2}]} check if it is the case where we have multiple sts in one datou no, it is not this case # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! We ignore checkConsistencyTypeOutputInput for datou_step final ! DataTypes for each output/input checked ! Unexpected error treated as WARNING for now expected given param in final step {'min_score': 0.7, 'name_pipeline': 'proportion_papier_tapis', 'hashtag_type': 705, 'hashtag_proportion': 492668766, 'hashtag_parmi': 538914404, 'hashtag_to_consider': 'papier_only', 'hashtag_background': 493012381} Found hashtag_parmi in final step, will be used and useful if different from matiere_majoritaire in classifier ventilate_hashtags_in_portfolio step not found ! # VR 17-11-17 : to create in DB ! Here we check the datou graph and we reorder steps ! Tree builded and cycle checked, now we need to re-order the steps ! We have currenlty an error because there is no dependence between the last step for the case tile - detect - glue We can either keep the depence of, it is better to keep an order compatible with the id of steps if we do not have sons, so a lexical order : (number_son, step_id) All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! All sons are already in current list ! DONE and to test : checkNoCycle ! Here we check the consistency of inputs/outputs number between the given ones and the db ! eke 1-6-18 : checkConsistencyNbInputNbOutput should be processed after step reordering ! WARNING : number of outputs for step 11415 rle_unique_nms_with_priority is not consistent : 2 used against 1 in the step definition ! Step 11419 crop_condition have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! WARNING : number of outputs for step 11419 crop_condition is not consistent : 4 used against 3 in the step definition ! WARNING : number of inputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! WARNING : number of outputs for step 11416 ventilate_hashtags_in_portfolio is not consistent : 2 used against 1 in the step definition ! Step 11417 final have less inputs used (2) than in the step definition (3) : maybe we manage optionnal inputs ! Step 11417 final have less outputs used (1) than in the step definition (2) : some outputs may be not used ! Step 11422 split_time_score have less inputs used (1) than in the step definition (2) : maybe we manage optionnal inputs ! Number of inputs / outputs for each step checked ! Here we check the consistency of outputs/inputs types during steps connections eke 1-6-18 : checkConsistencyTypeOutputInput should be processed after checkConsistencyNbInputNbOutput ! We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 2 of step 11418 have datatype=6 We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : type of input 2 of step 11419 doesn't seem to be define in the database( WARNING : output 1 of step 11415 have datatype=7 whereas input 2 of step 11419 have datatype=None WARNING : type of output 3 of step 11419 doesn't seem to be define in the database( WARNING : type of input 1 of step 11416 doesn't seem to be define in the database( WARNING : type of output 1 of step 11416 doesn't seem to be define in the database( WARNING : type of input 3 of step 11417 doesn't seem to be define in the database( We ignore checkConsistencyTypeOutputInput for datou_step final ! WARNING : output 0 of step 11416 have datatype=10 whereas input 0 of step 11420 have datatype=18 WARNING : type of input 5 of step 11418 doesn't seem to be define in the database( WARNING : output 0 of step 11420 have datatype=11 whereas input 5 of step 11418 have datatype=None WARNING : type of input 2 of step 11416 doesn't seem to be define in the database( WARNING : output 0 of step 11421 have datatype=5 whereas input 2 of step 11416 have datatype=None WARNING : output 0 of step 11418 have datatype=10 whereas input 0 of step 11422 have datatype=18 DataTypes for each output/input checked ! Unexpected error treated as WARNING for now expected given param in final step {'name_pipeline': 'aggregate_carac_ratio', 'hashtag_type': 4230, 'hashtag_proportion': 'papier,carton,metal,pet_clair,autre,pehd,pet_fonce', 'hashtag_parmi': 'refus', 'hashtag_weights': {'autre': 3.0, 'barquette_avec_film': 3.0, 'carton': 3.0, 'metal': 3.0, 'papier': 3.0, 'pehd': 3.0, 'pet_fonce': 3.0, 'pet_clair': 1.0, 'flou': 1.0, 'mal_croppe': 1.0}, 'option_sccu': 1} Found hashtag_parmi in final step, will be used and useful if different from matiere_majoritaire in classifier Fonction SLA_RUBBIA TODO : Insert select and so on ***** analysis of device with port 20084 for dashboard dechet_Valcor_v2 ***** **** analysis of day 2025/02/20/ **** VR TODO TOCHECK : due to this bug it shouldn't have being able to work, or maybe it was due to a change directory not done ! VR TODO TOCHECK : due to this bug it shouldn't have being able to work, or maybe it was due to a change directory not done ! VR TODO TOCHECK : due to this bug it shouldn't have being able to work, or maybe it was due to a change directory not done ! VR TODO TOCHECK : due to this bug it shouldn't have being able to work, or maybe it was due to a change directory not done ! 0:00:00 apple pause between two photos 10 Find filename in sqlite. Info from logs total number of images : 8574 coverage for 10 seconds : 23:16:32 (96.98%) coverage for 20 seconds : 23:53:04 (99.52%) max time between two photos : 0:01:19 results of pre-diagnostic : light ok 8574, not duplicated 6241, two criteria ok 6241, nb forced upload 0 end of day status of photos as found in sqllite duplicated : 2329 nothing_done : 13 ok_for_neural_classification : 1930 uploaded : 4302 Info from dashboard total number of uploaded images : 6337 number of useful images : 6241 number of finished photos : 15 (0.24% of two criteria ok) number of finished photos for older algorithm : 0 (0.00% of two criteria ok) number of started photos : 54 (0.87% of two criteria ok) number of finished batches : 5 (45.45% of total) number of finished batches for older algorithm : 0 (0.00% of total) number of started batches : 6 (54.55% of total) last update : 2025-02-22 11:32:35 **** analysis of day 2025/02/21/ **** VR TODO TOCHECK : due to this bug it shouldn't have being able to work, or maybe it was due to a change directory not done ! VR TODO TOCHECK : due to this bug it shouldn't have being able to work, or maybe it was due to a change directory not done ! VR TODO TOCHECK : due to this bug it shouldn't have being able to work, or maybe it was due to a change directory not done ! VR TODO TOCHECK : due to this bug it shouldn't have being able to work, or maybe it was due to a change directory not done ! 0:00:00 apple pause between two photos 10 Find filename in sqlite. Info from logs total number of images : 8452 coverage for 10 seconds : 22:59:35 (95.81%) coverage for 20 seconds : 23:33:03 (98.13%) max time between two photos : 0:13:30 results of pre-diagnostic : light ok 8469, not duplicated 5635, two criteria ok 5635, nb forced upload 0 end of day status of photos as found in sqllite duplicated : 2828 nothing_done : 7 ok_for_neural_classification : 10 uploaded : 5607 Info from dashboard total number of uploaded images : 5703 number of useful images : 5635 number of finished photos : 269 (4.77% of two criteria ok) number of finished photos for older algorithm : 0 (0.00% of two criteria ok) number of started photos : 0 (0.00% of two criteria ok) number of finished batches : 44 (100.00% of total) number of finished batches for older algorithm : 0 (0.00% of total) number of started batches : 0 (0.00% of total) last update : 2025-02-23 08:36:51 **** analysis of day 2025/02/22/ **** VR TODO TOCHECK : due to this bug it shouldn't have being able to work, or maybe it was due to a change directory not done ! VR TODO TOCHECK : due to this bug it shouldn't have being able to work, or maybe it was due to a change directory not done ! VR TODO TOCHECK : due to this bug it shouldn't have being able to work, or maybe it was due to a change directory not done ! VR TODO TOCHECK : due to this bug it shouldn't have being able to work, or maybe it was due to a change directory not done ! 0:00:00 apple pause between two photos 10 Find filename in sqlite. Info from logs total number of images : 8526 coverage for 10 seconds : 23:14:43 (96.86%) coverage for 20 seconds : 23:45:09 (98.97%) max time between two photos : 0:01:21 results of pre-diagnostic : light ok 8538, not duplicated 5780, two criteria ok 5780, nb forced upload 0 end of day status of photos as found in sqllite duplicated : 2757 nothing_done : 1 uploaded : 5768 Info from dashboard total number of uploaded images : 5877 number of useful images : 5780 number of finished photos : 87 (1.51% of two criteria ok) number of finished photos for older algorithm : 0 (0.00% of two criteria ok) number of started photos : 0 (0.00% of two criteria ok) number of finished batches : 18 (100.00% of total) number of finished batches for older algorithm : 0 (0.00% of total) number of started batches : 0 (0.00% of total) last update : 2025-02-23 08:45:48 **** analysis of day 2025/02/23/ **** VR TODO TOCHECK : due to this bug it shouldn't have being able to work, or maybe it was due to a change directory not done ! VR TODO TOCHECK : due to this bug it shouldn't have being able to work, or maybe it was due to a change directory not done ! VR TODO TOCHECK : due to this bug it shouldn't have being able to work, or maybe it was due to a change directory not done ! VR TODO TOCHECK : due to this bug it shouldn't have being able to work, or maybe it was due to a change directory not done ! 0:00:00 apple pause between two photos 10 Find filename in sqlite. Info from logs total number of images : 8513 coverage for 10 seconds : 23:11:51 (96.66%) coverage for 20 seconds : 23:42:56 (98.82%) max time between two photos : 0:01:20 results of pre-diagnostic : light ok 8525, not duplicated 5820, two criteria ok 5820, nb forced upload 0 end of day status of photos as found in sqllite duplicated : 2697 nothing_done : 1 ok_for_neural_classification : 5 uploaded : 5810 Info from dashboard total number of uploaded images : 5900 number of useful images : 5820 number of finished photos : 5 (0.09% of two criteria ok) number of finished photos for older algorithm : 0 (0.00% of two criteria ok) number of started photos : 0 (0.00% of two criteria ok) number of finished batches : 2 (100.00% of total) number of finished batches for older algorithm : 0 (0.00% of total) number of started batches : 0 (0.00% of total) last update : 2025-02-25 08:36:50 **** analysis of day 2025/02/24/ **** VR TODO TOCHECK : due to this bug it shouldn't have being able to work, or maybe it was due to a change directory not done ! VR TODO TOCHECK : due to this bug it shouldn't have being able to work, or maybe it was due to a change directory not done ! VR TODO TOCHECK : due to this bug it shouldn't have being able to work, or maybe it was due to a change directory not done ! VR TODO TOCHECK : due to this bug it shouldn't have being able to work, or maybe it was due to a change directory not done ! 0:00:00 apple pause between two photos 10 Find filename in sqlite. Info from logs total number of images : 8526 coverage for 10 seconds : 23:00:54 (95.90%) coverage for 20 seconds : 23:45:10 (98.97%) max time between two photos : 0:01:20 results of pre-diagnostic : light ok 8530, not duplicated 6815, two criteria ok 6808, nb forced upload 0 end of day status of photos as found in sqllite dark : 7 duplicated : 1720 nothing_done : 1 ok_for_neural_classification : 6 uploaded : 6792 Info from dashboard total number of uploaded images : 6911 number of useful images : 6808 number of finished photos : 542 (7.96% of two criteria ok) number of finished photos for older algorithm : 0 (0.00% of two criteria ok) number of started photos : 0 (0.00% of two criteria ok) number of finished batches : 76 (100.00% of total) number of finished batches for older algorithm : 0 (0.00% of total) number of started batches : 0 (0.00% of total) last update : 2025-02-26 08:36:52 **** analysis of day 2025/02/25/ **** VR TODO TOCHECK : due to this bug it shouldn't have being able to work, or maybe it was due to a change directory not done ! VR TODO TOCHECK : due to this bug it shouldn't have being able to work, or maybe it was due to a change directory not done ! VR TODO TOCHECK : due to this bug it shouldn't have being able to work, or maybe it was due to a change directory not done ! VR TODO TOCHECK : due to this bug it shouldn't have being able to work, or maybe it was due to a change directory not done ! 0:00:00 apple pause between two photos 10 Find filename in sqlite. Info from logs total number of images : 8520 coverage for 10 seconds : 23:07:36 (96.36%) coverage for 20 seconds : 23:44:12 (98.90%) max time between two photos : 0:01:21 results of pre-diagnostic : light ok 8532, not duplicated 6738, two criteria ok 6738, nb forced upload 0 end of day status of photos as found in sqllite duplicated : 1790 nothing_done : 1 ok_for_neural_classification : 12 uploaded : 6717 Info from dashboard total number of uploaded images : 6861 number of useful images : 6738 number of finished photos : 294 (4.36% of two criteria ok) number of finished photos for older algorithm : 0 (0.00% of two criteria ok) number of started photos : 0 (0.00% of two criteria ok) number of finished batches : 60 (100.00% of total) number of finished batches for older algorithm : 0 (0.00% of total) number of started batches : 0 (0.00% of total) last update : 2025-02-26 06:01:10 **** analysis of day 2025/02/26/ **** VR TODO TOCHECK : due to this bug it shouldn't have being able to work, or maybe it was due to a change directory not done ! VR TODO TOCHECK : due to this bug it shouldn't have being able to work, or maybe it was due to a change directory not done ! VR TODO TOCHECK : due to this bug it shouldn't have being able to work, or maybe it was due to a change directory not done ! VR TODO TOCHECK : due to this bug it shouldn't have being able to work, or maybe it was due to a change directory not done ! 0:00:00 apple pause between two photos 10 Find filename in sqlite. Info from logs total number of images : 8520 coverage for 10 seconds : 23:01:52 (95.96%) coverage for 20 seconds : 23:44:13 (98.91%) max time between two photos : 0:01:20 results of pre-diagnostic : light ok 8526, not duplicated 6581, two criteria ok 6581, nb forced upload 0 end of day status of photos as found in sqllite duplicated : 1945 nothing_done : 1 ok_for_neural_classification : 3 uploaded : 6571 Info from dashboard total number of uploaded images : 6676 number of useful images : 6581 number of finished photos : 477 (7.25% of two criteria ok) number of finished photos for older algorithm : 0 (0.00% of two criteria ok) number of started photos : 52 (0.79% of two criteria ok) number of finished batches : 80 (94.12% of total) number of finished batches for older algorithm : 0 (0.00% of total) number of started batches : 5 (5.88% of total) last update : 2025-02-27 06:03:53 **** analysis for all days **** Info from logs total number of images : 59631 coverage for 10 seconds : 6 days, 17:53:03 (96.36%) coverage for 20 seconds : 6 days, 22:07:47 (98.89%) max time between two photos : 0:21:31 results of pre-diagnostic : light ok 59694, not duplicated 43610, two criteria ok 43603, nb forced upload 0 end of day status of photos as found in sqllite duplicated : 16066 nothing_done : 25 ok_for_neural_classification : 1966 uploaded : 41567 dark : 7 Info from dashboard total number of uploaded images : 44265 number of useful images : 43603 number of finished photos : 1689 (3.87% of two criteria ok) number of finished photos for older algorithm : 0 (0.00% of two criteria ok) number of started photos : 106 (0.24% of two criteria ok) number of finished batches : 285 (96.28% of total) number of finished batches for older algorithm : 0 (0.00% of total) number of started batches : 11 (3.72% of total) last update : 2025-02-27 06:03:53 You can launch datou with following url to update dashboard https://www.fotonower.com/api/v1/datou/current/set?mtr_portfolio_id=20675559,20822434&mtr_datou_id=4311 cvs resume : date,nb_photos,% time,nb ok,uploaded,to upload, % uploaded, nb useful, % completed photos, last_update, remark 2025/02/20,8574,96.98%,6241,4302,1930,69.03%,69,21.74%,2025/02/22, 2025/02/21,8452,95.81%,5635,5607,10,99.82%,269,100.00%,2025/02/23, 2025/02/22,8526,96.86%,5780,5768,0,100.00%,87,100.00%,2025/02/23, 2025/02/23,8513,96.66%,5820,5810,5,99.91%,5,100.00%,2025/02/25, 2025/02/24,8526,95.90%,6808,6792,6,99.91%,542,100.00%,2025/02/26, 2025/02/25,8520,96.36%,6738,6717,12,99.82%,294,100.00%,2025/02/26, 2025/02/26,8520,95.96%,6581,6571,3,99.95%,529,90.17%,2025/02/27, coverage for this period for every 10 second 2025/02/20/ 0.9698260396532368 2025/02/21/ 0.9580550700818297 2025/02/22/ 0.9685644509774418 2025/02/23/ 0.9665736871954537 2025/02/24/ 0.9589694325165801 2025/02/25/ 0.9636222641465758 2025/02/26/ 0.9636073830219596 mean value for this period : 0.9641740467990111 coverage for this period for every 20 second 2025/02/20/ 0.9951967036655517 2025/02/21/ 0.9812960798157386 2025/02/22/ 0.9896989548490145 2025/02/23/ 0.988159585180384 2025/02/24/ 0.9897105290570493 2025/02/25/ 0.98903922499103 2025/02/26/ 0.9888788395368333 mean value for this period : 0.9888542738708003 3.38user 1.56system 0:37.71elapsed 13%CPU (0avgtext+0avgdata 196408maxresident)k 349792inputs+104outputs (82major+124398minor)pagefaults 0swaps