March 2024 - Hpcoatmeal - Comm.Collab.Science.Gc.Ca

by LeBel, Antoine (SSC/SPC)

Unclassified | Non classifié Does that work now? Antoine LeBel Chef d'équipe | Team Lead Optimisation ordinateur haute performance (HPCO) High Performance Computing Optimization (HPCO) Services partagés Canada | Shared Services Canada antoine.lebel(a)ssc-spc.gc.ca<mailto:antoine.lebel@ssc-spc.gc.ca> Gouvernement du Canada | Government of Canada [cid:image001.png@01DA76B3.F47CBCD0]

1 year, 1 month

1
0
0 / 0

FW: Connection DB

by LeBel, Antoine (SSC/SPC)

Unclassified | Non classifié Salutations, Est-ce que vous savez la réponse? From: Kadirzada, Kabir (SSC/SPC) <Kabir.Kadirzada(a)ssc-spc.gc.ca> Sent: Wednesday, March 13, 2024 12:14 PM To: LeBel, Antoine (SSC/SPC) <Antoine.Lebel(a)ssc-spc.gc.ca> Subject: RE: Connection DB Unclassified | Non classifié Bonjour Antoine, Avez-vous changé le compte d’utilisateur reader_all? Je ne peux plus me connecter. Merci, [cid:image001.png@01DA7539.804E6D20] Kabir Kadirzada Chef d’Équipe, Évolution du Service CHP Gestion du calcul de haute performance (CHP) intégré Services Partagés Canada, Gouvernement du Canada Kabir.kadirzada(a)canada.ca<mailto:Kabir.kadirzada@canada.ca> Tél.: 514-258-1917 Team Lead, HPC Service Evolution Integrated High Performance Computing (HPC) Management Shared Services Canada, Government of Canada Kabir.kadirzada(a)canada.ca<mailto:Kabir.kadirzada@canada.ca> / Tél.: 514-258-1917 From: LeBel, Antoine (SSC/SPC) <Antoine.Lebel(a)ssc-spc.gc.ca<mailto:Antoine.Lebel@ssc-spc.gc.ca>> Sent: Monday, February 26, 2024 11:46 AM To: Kadirzada, Kabir (SSC/SPC) <Kabir.Kadirzada(a)ssc-spc.gc.ca<mailto:Kabir.Kadirzada@ssc-spc.gc.ca>> Subject: RE: Connection DB Unclassified | Non classifié Bonjour Kabir, est-ce que tu as une des crédentials pour aller sur le VPN de EC? From: Kadirzada, Kabir (SSC/SPC) <Kabir.Kadirzada(a)ssc-spc.gc.ca<mailto:Kabir.Kadirzada@ssc-spc.gc.ca>> Sent: Monday, February 26, 2024 10:49 AM To: LeBel, Antoine (SSC/SPC) <Antoine.Lebel(a)ssc-spc.gc.ca<mailto:Antoine.Lebel@ssc-spc.gc.ca>> Subject: Connection DB Unclassified | Non classifié Bonjour Antoine, Je suis les instructions que vous avez fourni et je ne suis pas capable de me connecter au BD. Je me connecte avec VPN SSC sur Windows, vous dites qu’il faut être sur le réseau de science, comment je peux me connecter avec Windows et VPN? Merci Kabir Kadirzada Chef d’Équipe, Surveillance du Service CHP Gestion du calcul de haute performance (CHP) intégré Services Partagés Canada, Gouvernement du Canada Kabir.kadirzada(a)canada.ca<mailto:Kabir.kadirzada@canada.ca> Tél.: 514-258-1917<tel:514-258-1917> Team Lead, HPC Service Oversight Integrated High Performance Computing (HPC) Management Shared Services Canada, Government of Canada Kabir.kadirzada(a)canada.ca<mailto:Kabir.kadirzada@canada.ca> / Tél.: 514-258-1917<tel:514-258-1917>

1 year, 1 month

1
0
0 / 0

FW: 23-12 RMAAS GPSC core-hour & storage month-end status update

by LeBel, Antoine (SSC/SPC)

Unclassified | Non classifié From: Weaver, Ross <Ross.Weaver(a)dfo-mpo.gc.ca> Sent: Tuesday, March 12, 2024 11:03 AM To: Mowbray, Dale (SSC/SPC) <Dale.Mowbray(a)ssc-spc.gc.ca>; LeBel, Antoine (SSC/SPC) <Antoine.Lebel(a)ssc-spc.gc.ca>; Tran, Ryan (SSC/SPC) <Ryan.Tran(a)ssc-spc.gc.ca> Cc: Ma, Robert (he, him / il, lui) (DFO/MPO) <Robert.Ma(a)dfo-mpo.gc.ca> Subject: FW: 23-12 RMAAS GPSC core-hour & storage month-end status update Here is a copy of the last report I sent out concerning core hour and storage usage for DFO COMDA and DPNM allocations on GPSC. Like I said these are not perfect, and require a fair amount of hands on work to create them. If I could generate similar information and a similar report using a system from HPCO in the near term I would consider that a significant success for reporting. Ross Weaver Science Advisor, Ocean and Climate Change Science Fisheries and Oceans Canada, Government of Canada Ross.Weaver(a)dfo-mpo.gc.ca<mailto:Ross.Weaver@dfo-mpo.gc.ca> / Tel. : 343.576.2076 Conseiller scientifique, Sciences océaniques et des changements climatiques Pêches et Océans Canada, Gouvernement du Canada Ross.Weaver(a)dfo-mpo.gc.ca<mailto:Ross.Weaver@dfo-mpo.gc.ca> / Tél. : 343.576.2076 From: Weaver, Ross Sent: Tuesday, February 20, 2024 10:21 AM Subject: 23-12 RMAAS GPSC core-hour & storage month-end status update Here’s the GPSC core-hour month-end status and storage update. Please note that Core Hours is for all usage with the project tag, Normal and Low QoS. For COMDA we have separated out the other projects (PFM,AFM, Aquaculture) that share the same Normal QoS Queue with COMDA. [cid:image001.png@01DA63E6.82B32F60] Please note 85% of a partition is an industry guideline of how full a partition can safely get. 15% of a storage partition in traditional storage structures should be kept empty as swing space to allow the system space to create temporary files and operational space for things such as moving files or copying files or system storage activities. When there is not enough swing space the storage ought to be actively managed to avoid problems with the system. Rate of data growth and how often files are created and deleted as well as how large files can grow temporarily during actively modelling are factors in determining the risk when a partition begins to fill up. Top 5 Storage Users Name Space Used (TB) Total Space Used (TB) User Percentage of Total Minghong Zhang 121.82 733.97 17% Yongxing Ma 106.22 733.97 14% Colin Hughes 74.84 733.97 10% Guoqi Han 58.37 733.97 8% SeDOO 53.7 733.97 7% Reporting for 2023-12 COMDA allocation is 458936.4 core-hours/month 99.8% COMDA project usage - lower than last month 10 COMDA project users 83.3% of total usage from 3 users Name COMDA project C-H usage Zhenxia Long 203522.0 Guoqi Han 101384.0 Catarina Wor 76401.2 Sarah MacDermid 58589.0 William Perrie 6658.1 Olivier Riche 5725.4 Jacqueline Dumas 3763.8 Dan Greenberg 1590.7 Yongxing Ma 356.3 Adam Drozdowski 0.0 Total Usage 457990.7 Total Allocation 458936.4 Total % 99.79% [cid:image002.png@01DA63E6.82B32F60] [cid:image003.png@01DA63E6.82B32F60] Please note 85% of a partition is an industry guideline of how full a partition can safely get. 15% of a storage partition in traditional storage structures should be kept empty as swing space to allow the system space to create temporary files and operational space for things such as moving files or copying files or system storage activities. When there is not enough swing space the storage ought to be actively managed to avoid problems with the system. Rate of data growth and how often files are created and deleted as well as how large files can grow temporarily during actively modelling are factors in determining the risk when a partition begins to fill up. Name Space Used (TB) Total Space Used (TB) User Percentage of Total Port Leads Group 188.24 829.2 23% Michael Dunphy 107.83 829.2 13% Stephanne Taylor 85.83 829.2 10% SeDOO group 72.36 829.2 9% Hauke Blanken 67.3 829.2 8% Reporting for 2023-12 OPP allocation is 1350500.0 core-hours/month 47% OPP project usage - lower than last month 14 OPP project users 97% of total usage from 3 users Name OPP project C-H usage Stephanne Taylor 389232.12 Maxim Krassovski 155636.039 Rachel Horwitz 68359.0492 Andy Lin 8415.52972 Michael Dunphy 6073.83111 Michael Dunphy 3121.53306 Simon St-OngeDrouin 1038.74528 Simon St-OngeDrouin 695.825833 Hauke Blanken 572.125833 Rachel Horwitz 64.273333 Nancy Soontiens 45.675 Maxim Krassovski 15.595 Corinne Bourgault-Brunelle 0.8425 Corinne Bourgault-Brunelle 0.205278 Total Usage 633271.4 Total Allocation 1350500 Total % 47% [cid:image004.png@01DA63E6.82B32F60] Ross Weaver Science Advisor, Ocean and Climate Change Science Fisheries and Oceans Canada, Government of Canada Ross.Weaver(a)dfo-mpo.gc.ca<mailto:Ross.Weaver@dfo-mpo.gc.ca> / Tel. : 343.576.2076 Conseiller scientifique, Sciences océaniques et des changements climatiques Pêches et Océans Canada, Gouvernement du Canada Ross.Weaver(a)dfo-mpo.gc.ca<mailto:Ross.Weaver@dfo-mpo.gc.ca> / Tél. : 343.576.2076

1 year, 1 month

1
0
0 / 0

Re: HPC stats tool request (Elastic Search)

by Genswein, Bernie (SSC/SPC)

Unclassified | Non classifié Ok, sounds good – thanks Antoine. Bernie From: LeBel, Antoine (SSC/SPC) <Antoine.Lebel(a)ssc-spc.gc.ca> Sent: Wednesday, March 6, 2024 5:20 AM To: Genswein, Bernie (SSC/SPC) <Bernie.Genswein(a)ssc-spc.gc.ca> Cc: hpcoatmeal(a)comm.collab.science.gc.ca Subject: RE: HPC stats tool request (Elastic Search) Unclassified | Non classifié Hi Bernie, We’ll try to make use of our newly implemented database to generate these on PowerBI. Thanks From: Genswein, Bernie (SSC/SPC) <Bernie.Genswein(a)ssc-spc.gc.ca<mailto:Bernie.Genswein@ssc-spc.gc.ca>> Sent: Tuesday, March 5, 2024 3:38 PM To: LeBel, Antoine (SSC/SPC) <Antoine.Lebel(a)ssc-spc.gc.ca<mailto:Antoine.Lebel@ssc-spc.gc.ca>> Subject: FW: HPC stats tool request (Elastic Search) Unclassified | Non classifié Hi Antoine, Below is an email requesting additional features for hpc-stats. It’s not something that’s urgent, just wanted to make you aware of the request. I think the 3rd one, the discrepancy, is not an issue for you to worry about right now. I think it’s probably a user thing….. Thanks. Bernie From: Clarke, Kurtis (AAFC/AAC) <kurtis.clarke(a)agr.gc.ca<mailto:kurtis.clarke@agr.gc.ca>> Sent: Wednesday, February 21, 2024 1:30 PM To: Genswein, Bernie (SSC/SPC) <Bernie.Genswein(a)ssc-spc.gc.ca<mailto:Bernie.Genswein@ssc-spc.gc.ca>> Cc: Bilkhu, Satpal (AAFC/AAC) <satpal.bilkhu(a)AGR.GC.CA<mailto:satpal.bilkhu@AGR.GC.CA>>; Kess, Tony <Tony.Kess(a)dfo-mpo.gc.ca<mailto:Tony.Kess@dfo-mpo.gc.ca>> Subject: HPC stats tool request (Elastic Search) Unclassified / Non classifié Hi Bernie, Some features I would appreciate in elastic search: * Filtering by qos (e.g. a way to filter out low qos jobs) * A bargraph comparing CPU hours requested/allocated vs actual CPU time (to indicate how well users are using the GPSC generally) * There was a discrepancy in Core CPU hours between Satpal and I logged into the same account looking at the same time for some reason * I had ~596,000 hours and Satpal had ~614,000 hours reported. Cheers, Kurtis Clarke Bioinformatics Programmer, BICoE Agriculture and Agri-Food Canada / Government of Canada kurtis.clarke(a)agr.gc.ca<mailto:kurtis.clarke@agr.gc.ca> Programmeur En Bioinformatique, BICoE Agriculture et Agroalimentaire Canada / Gouvernement du Canada kurtis.clarke(a)agr.gc.ca<mailto:kurtis.clarke@agr.gc.ca>

1 year, 1 month

1
0
0 / 0

Re: HPC stats tool request (Elastic Search)

by LeBel, Antoine (SSC/SPC)

Unclassified | Non classifié Hi Bernie, We’ll try to make use of our newly implemented database to generate these on PowerBI. Thanks From: Genswein, Bernie (SSC/SPC) <Bernie.Genswein(a)ssc-spc.gc.ca> Sent: Tuesday, March 5, 2024 3:38 PM To: LeBel, Antoine (SSC/SPC) <Antoine.Lebel(a)ssc-spc.gc.ca> Subject: FW: HPC stats tool request (Elastic Search) Unclassified | Non classifié Hi Antoine, Below is an email requesting additional features for hpc-stats. It’s not something that’s urgent, just wanted to make you aware of the request. I think the 3rd one, the discrepancy, is not an issue for you to worry about right now. I think it’s probably a user thing….. Thanks. Bernie From: Clarke, Kurtis (AAFC/AAC) <kurtis.clarke(a)agr.gc.ca<mailto:kurtis.clarke@agr.gc.ca>> Sent: Wednesday, February 21, 2024 1:30 PM To: Genswein, Bernie (SSC/SPC) <Bernie.Genswein(a)ssc-spc.gc.ca<mailto:Bernie.Genswein@ssc-spc.gc.ca>> Cc: Bilkhu, Satpal (AAFC/AAC) <satpal.bilkhu(a)AGR.GC.CA<mailto:satpal.bilkhu@AGR.GC.CA>>; Kess, Tony <Tony.Kess(a)dfo-mpo.gc.ca<mailto:Tony.Kess@dfo-mpo.gc.ca>> Subject: HPC stats tool request (Elastic Search) Unclassified / Non classifié Hi Bernie, Some features I would appreciate in elastic search: * Filtering by qos (e.g. a way to filter out low qos jobs) * A bargraph comparing CPU hours requested/allocated vs actual CPU time (to indicate how well users are using the GPSC generally) * There was a discrepancy in Core CPU hours between Satpal and I logged into the same account looking at the same time for some reason * I had ~596,000 hours and Satpal had ~614,000 hours reported. Cheers, Kurtis Clarke Bioinformatics Programmer, BICoE Agriculture and Agri-Food Canada / Government of Canada kurtis.clarke(a)agr.gc.ca<mailto:kurtis.clarke@agr.gc.ca> Programmeur En Bioinformatique, BICoE Agriculture et Agroalimentaire Canada / Gouvernement du Canada kurtis.clarke(a)agr.gc.ca<mailto:kurtis.clarke@agr.gc.ca>

1 year, 1 month

1
0
0 / 0

2025

2024

2023

Hpcoatmeal March 2024