Sunny

HPC, big data, and now machine learning. Worked in/led SW eng, global svcs & support, sys integration, and prod dev at Cray, SMI, Linux Networx, SGI, and HPE.

[Sunny @HPC_Sunny]

we do incorporate upstream community /open source code...and also contribute some of it back to the community #gethpcos https://t.co/dwVUjfdXfE

[Sunny @HPC_Sunny]

:-) Since we build on Linux, we already do, Agree with your comment. Beyond the distros.... #gethpcos https://t.co/QBjlBmS6lg

[Sunny @HPC_Sunny]

A lightweight OS means smaller memory footprint, fewer daemons, and fewer system services consuming system resources which are then available to the applications. #gethpcos https://t.co/fWQJ5130ZY

[Sunny @HPC_Sunny]

Martijn: I wonder if you have other observations about HPC operating environments from Bright's system management persepective? #gethpcos https://t.co/Qj8xjwacUN

[Sunny @HPC_Sunny]

Application dependent of course. :-) We have had one user report an INCREASE in performance but we think its due to a memory allocation anomaly. #gethpcos https://t.co/fkFKoxPpmH

[Sunny @HPC_Sunny]

Our users tell us that the gains in flexibility far outweigh the small performance penalty on the majority of applications. #gethpcos https://t.co/hveTz3YuL2

[Sunny @HPC_Sunny]

Yes, but the "abstracttion penalty" as I call it ,is small (generally single digit %) and also generally declining. #gethpcos https://t.co/7hO36eVpwS

[Sunny @HPC_Sunny]

The Cray profiler can analyze 100k MPI ranks. #gethpcos https://t.co/uPRGWwQwPV

[Sunny @HPC_Sunny]

Pay attention to your use of MPI collectives like barrier, reduce, allreduce, and bcast. Ensure you are utliizing libraries that are optimized for HPS. Profile your code using tools in Cray PE. #gethpcos https://t.co/PdPqB1JGLp

[Sunny @HPC_Sunny]

But after that, you need an operating environment that is composed to do this. #gethpcos https://t.co/1ZYB5ckMb6

[Sunny @HPC_Sunny]

At the user level, users can start with the normal things they do to improve process and MPI rank synchronization at the application level. That helps. #gethpcos https://t.co/Hvuly7RSVK

[Sunny @HPC_Sunny]

Cray systems will remain container “agnostic” however and allow use of many container implementations, not just the ones we build on and ship. #gethpcos https://t.co/23PTDZikgI

[Sunny @HPC_Sunny]

Significantly. Today we support Shifter and Docker. We plan to expand that to Singularity. We are doing some fundamental R&D with Docker in regards to HPC user cases. We use containers extensively in our own software implementation. #gethpcos https://t.co/09ZBvXlFTN

[Sunny @HPC_Sunny]

For example, the Cray PE tool set is extensive and we support it across OS releases. It eases porting requirements. We also work directly with 3rd party tool vendors. #gethpcos https://t.co/Hhcbg5Crgi

[Sunny @HPC_Sunny]

Cray is working with Altair and others on the coming mash-up of container, WLM, and orchestration technologies that serve very broad workload requirements. We might throw provisioning in their too. #gethpcos https://t.co/Jr5RCdUvcD

[Sunny @HPC_Sunny]

The resulting collaboration produces better scalabilty, performance, quality, and reliability. #gethpcos https://t.co/E56J8Apw87

[Sunny @HPC_Sunny]

Our engineering teams collaborate to create a seamless integration between CLE and SW like PBS Pro. #gethpcos https://t.co/clLfrHsqKf

[Sunny @HPC_Sunny]

Container use is increasing on HPC workloads. #gethpcos https://t.co/Nn7jwWHJEd

[Sunny @HPC_Sunny]

Yes, we see workloads requirements becoming more diverse and containers being an important part of supporting them. #gethpcos https://t.co/uEu9dYTyPI

[Sunny @HPC_Sunny]

I see Scott from Altair in the crowd. Do you have a comment, Scott? #gethpcos https://t.co/ZhCuiMXPx1